Web-based File Share Part I

Goal
Create a lightweight, easy-to-use file-sharing alternative to FTP for our clients.

Background
During the regular course of a project there are frequent situations we have to provide the client with files (comps, documents, etc.) and the client needs to provide us with files (images, videos, assets, documents, etc.). Typically, this is handled with FTP. Each client has their own account to which they can log in and download/upload files. FTP is a full-featured and tried and true solution to file sharing.

However, in our case there are some instances where asking clients to use FTP is not optimal. There are some clients where due to the security requirements they operate in they cannot download and use and FTP client application. We have clients who are not technically inclined and the time invested in to assisting these clients with their usage.

Nowadays with the internet being ubiquitous our clients all have access to the internet via a web browser. If we could create a simple web-based interface for file sharing then we would be able to provide clients for whom FTP is not a viable option with an alternative.

Another situation that frequently comes up is the need to provide a potential client or vendor with a file. These are individuals for whom passworded access is an unnecessary hassle. Furthermore, these varied file downloaders shouldn’t be able to see each other’s downloads due to privacy concerns. We had been using an online service called yousendit at yousendit.com, however, it had the disadvantage of providing yet another interface for both clients and employees to learn and manage.

Requirements

  • Use the same file structure being used by our FTP service so that either service, the web-based file share, or FTP can be used interchangeably.
  • Clients must be able to use the same username and password that they use for FTP. Ideally, username and password changes and addition would be made one time in one location.
  • Clients must be able to upload files, create directories, and delete files.
  • A public drop box must be made available where employees can upload files and clients can download them without seeing downloads intended for other clients.
  • The dev effort should be simple and quick.
  • The system should use our internal technologies, software, and hardware.

Synopsis
The final Web-based File Share used the following components:

Front-End

  • AJAX Application
  • WebDAV jQuery Plugin

Back-End

  • Mod Perl 2.0
    •  Translate Browser HTTP requests to WebDAV requests
    •  Translate POST in to PUT.
  • Apache 2
    •  Use same file structure as FTP.
    •  Security
    •  Basic Authentication
    •  Use same password file as FTP
    •  Configuration to allow logout

Reasoning
The following is my initial reasoning during the design stage of the Web-based File Share system.

Right away the thought of creating a web-based interface for sharing files led me to think of WebDAV. WebDav is a set of extensions on top of HTTP. Simplistically, if you think of the world wide web, HTTP is the mechanism for reading it, WebDAV allows you to write to it. WebDAV defines a set of verbs (actions) that let’s you make and list collections (create directories and read their contents), put files (upload), and delete files and collections, amongst other things. Those verbs would be able to help fulfill a core requirement.

The Apache web server, our web server of choice, has a WebDAV module. This meant that all I would have to do was enable the module, communicate the appropriate verbs to the web server, and Apache would take care of creating directories, uploading files, etc.

The next question to ask was how to integrate Apache’s ability to do WebDAV with our current FTP setup. Our current FTP server is vsftp, a powerful open source linux solution that comes packaged for Ubuntu, the flavor of linux we use for our servers. In order for both a web-based file share and ftp to work together then the files would have to have permissions such that both Apache and vsftp could read and write to those files and directories. After some investigation it appeared that a configuration setting in vsftp would allow the files to be owned by www-data, which is the user that Apache runs as. Thus, since all the files and directories would be owned by the same user, and both vsftp and Apache could access the files as those users, it became apparent that a web-based file sharing solution using Apache would be able to work in conjunction with our current FTP system.

At that point it seemed possible to integrate file sharing Apache and vsftp. Would it also be possible to use the same authentication mechanism to authenticate users in both Apache and vsftp? The following article http://howto.gumph.org/content/setup-virtual-users-and-directories-in-vsftpd/ suggested that it would indeed be possible.

Since Apache would be taking of the back-end functionality there wouldn’t need to be any back-end application. The only thing needed would be the front-end functionality, or the Web-based part of the Web-based File Share. This could simply be an AJAX application, i.e. an HTML page with Javascript that would communicate to the server in an asynchronous (that’s the A in AJAX) fashion.

The next logical step was to see if there would be a way to connect an AJAX front-end to the Apache’s WebDAV interface. Our preferred Javascript library is jQuery and so I search for a WebDAV jQuery plugin that would take care of the heavy lifting. The only WebDAV plugin available was: http://plugins.jquery.com/project/WebDAV. It appeared that this library did most of what was needed, but would require a few modifications.

So, the initial design investigation suggested that the Web-based File Share would be set up as follows:

The final Web-based File Share used the following components:

Front-End

  • AJAX Application
  • WebDAV jQuery Plugin

Back-End

  • Apache 2
    •  Use same file structure as FTP.
    •  Security
    •  Basic Authentication
    •  Use same password file as FTP

Note that the list above is different than the one under Synopsis. During the development of the File Share there were a few deviations from the original plan. The main problems occurred with the integration WebDAV from the front-end to the back-end and with the login and logout problems associated with different browsers and their ability to handle Basic Authentication.

Stay tuned for Part II for a deeper discussion of some of the development challenges and solutions.

About

Second Story creates enchanting, informative, and entertaining media experiences with innovative technologies that empower connections to ideas.

Tagged with: , , ,
Posted in Technology