CodeWatch

I am pleased to release Deadrop, a secure file upload and download utility. I know there are probably already sites/utilities that provide this but I wanted to build this a) because I could and b) because I trust my stuff more than any cloud provider. Currently, file uploads are limited to 25M based on hard drive space constraints on the server.

Deadrop was built to be secure. To understand the security of the system you must first understand the basics of the architecture. The architecture involves a front-end web server (the only server accessible from the Internet), a backend web services server based on SOAP, and a backend database server. The front-end takes the user submitted file over SSL, encrypts it, and sends it to the web service over SSL along with the encryption IV, file hashes, and other information provided in detail below. The web service server stores this file in a protected directory and stores details about the transaction in the database for future access.

The details stored in the database include:

  • Timestamp at which point the file was stored. This is for some future potential features like deleting the file after X period of time.
  • Original file name.
  • A SHA-256 HMAC of the file using the user provided password as the key.
  • A SHA-256 hash of the file.
  • The IV used in the file encryption process.
  • A hashed version of the provided password.
  • The IV used to create the bcrypt hashed password.
  • The original size of the file.
  • Two fields to customize when the file gets deleted.
  • Two fields to act as counters to determine when to delete the file.

The web server front end create’s an IV for the encryption process using mcrypt. Then, the mcrypt encrypt function is used to encrypt the data (AES-256 bit encryption) using the generated IV and the password provided by the user. This data is then base64 encoded so that it can be sent via SOAP. In addition to the encoded and encrypted data being sent to the web service, the web server also takes an HMAC of the file (SHA-256) using the provided password as the key and a SHA-256 hash of the file. The data, HMAC, Hash, IV, password, name of the file, user supplied options, and original size of the file is then sent to the web service.

The web server then uses shred to securely delete the temporary file created during the upload process. I am using the ext4 filesystem set to data=writeback, but it is on a RAID array so I realize that the file is still accessible for a short period of time. Unfortunately, this temporary file is not what is encrypted, the file content data is what is encrypted and sent to the web service. My research into PHP found that you cannot prevent an uploaded file from hitting your upload directory without patching the PHP source itself (see here and here). To combat this limitation, I am using shred to securely delete the only time the data is stored unencrypted.

The web service receives the data and writes the data for the file to disk using the HMAC as the filename. This ensures that the exact same file can be uploaded to the server with a different password, resulting in a unique file name. The system will prevent uploading an identical file with an identical password. This is where the hash version of the file is used; The system queries the database using the HMAC and if there are any returned results it uses the IV for the password of the matched entry with the newly provided password, and if this matches then the upload is rejected. The password is hashed using bcrypt, an mcrypt generated IV, and a work factor of 13. For a great description on hashing and storing passwords see this article as it provides a good overview of what I am doing and safe storage of passwords in general. The hashed version of the password, along with the associated IV, user supplied options, file HMAC, file Hash, and file IV is then stored in the database.

That kind of concludes the architecture and process for uploading a file. Now, let’s talk about security at the transport layer. The front-end web server is configured like so:

  • TLS is the only supported protocol (TLS 1.0, 1.1, and 1.2).
  • I’m using a 4096 DH param key and a 2048 bit certificate signed with SHA-256.
  • The web server is configured to only support AES-256 bit algorithms with Perfect Forward Secrecy (PFS)
  • The web server is configured to use Strict Transport Security (HSTS) and automatically redirects all requests to SSL as an added security measure.

You can validate the above information by submitting my site to Qualys SSL Labs if you like. The web service server is also configured to only allow TLS and AES-256 bit algorithms. In addition, database traffic to MySQL is encrypted using SSL as well. The file data is also encrypted when in transit inside the SSL encrypted connection to and from the web service server.

Access to a file requires passing the HMAC as the ‘file’ parameter in the Deadrop link. The user will then be prompted for the password. Once the password is provided, the HMAC and password are sent to the web service (again, all over SSL). The HMAC is used to identify the file in the database and then the password is used with the stored IV; if the bcrypted version of the provided password combined with the stored IV matches the hashed version of the password stored in the database, then the file is read back and sent as a SOAP message to the web server (along with the file IV, original file size, and file name). The web server uses this to base64 decode the data, decrypt the file and send it back using the ‘application/octet-stream’ mime type and the original file name. The original file size is required as mcrypt adds extra padding to the file which can be removed by using `fputs($outstream, $data, $filesize)`, only sending back the same size file as was provided. When the file is sent back to the browser, the PHP output stream is used, ensuring the data is not written to disk.

In addition to the transport layer and disk layer encryption, there are two user options when uploading a file that I thought might be useful for sensitive data. The first option determines how many times the file can be downloaded before it is deleted and the second option determines how many consecutive failed attempts to download are allowed before deleting the file. The second option is there to prevent brute force attacks such that after X amount of incorrect passwords the data is wiped. In each case where the limit is reached, the row containing all information in the database concerning the file is deleted and the file itself is shredded.

So far, I have successfully tested this out using several types of files:

  • Adobe PDF
  • Text (Notepad compatible)
  • Excel (.xls, .xlsx, .xlsm, and .xlsb)
  • Word (.doc and .docx)
  • PowerPoint (.ppt and .pptx)
  • Executable (.exe)
  • Compressed (.zip, .tgz, and .bz2)

I welcome any feedback on limitations or flaws in the design (outside of the known issue of the original file being written as a temp file in an unencrypted state as this is a limitation in PHP itself). Or if you just have an idea on how to improve it some way then feel free to comment below.

Note: If you upload a file make sure you keep track of the link to access it as well as the password itself. All data is either hashed, which is one way, or encrypted in a way that prevents even myself from decrypting the data. So if you forget the URL or the password then the file is no longer accessible for all intents and purposes.

UPDATE 2013-01-04: This post was updated to reflect the latest version. The separate SHA-256 hash of the file was not needed in combination with the HMAC. I’m not sure what I was thinking there as I could determine whether the upload was a unique combination of file + password already with the HMAC (duh!). The SHA-256 is no longer taken and that field has been removed from the database.

Tagged with:
 

Leave a Reply

Your email address will not be published. Required fields are marked *