Uploading Files to Amazon S3 Directly from the Web Browser

Amazon S3 is at the time of writing the premier file storage service, and as such an excellent choice for storing files from your Web application. What if I were to tell you these files never need to touch your application servers though?

The thing to be aware of is that the S3 API allows for POST-ing files directly from the user’s Web browser, so with some JavaScript magic you are able to securely upload files from your client side JavaScript code directly to your application’s S3 buckets. The buckets don’t even have to be publicly writeable.

The way this is accomplished is through some cooperation between your application server and your client side JavaScript, although your server can remain oblivious to the files themselves.

Server Side Implementation

Uploading files to S3 from the browser to a non-publicly writeable bucket requires a so-called policy document in order to authenticate the client. It is your application server’s responsibility to generate said document, which purpose is to serve as a temporary security token and to define what the token holder is allowed to do.

Generating the policy document can be a tricky exercise, especially regarding authenticating the POST request with the AWS Signature Version 4 authentication scheme that is currently required by S3. Therefore, you might want to use a library to handle this for you. I chose to use the aws-s3-form Node package for this purpose, which generates the HTML form fields, including policy and AWS signature, that the client must send to S3 with its POST request. In my MuzHack application, I have a REST API method that returns the necessary form fields for a successful POST request to S3 to the client:

let AwsS3Form = require('aws-s3-form')

[...]

// A hapi.js server route
server.route({
  method: ['GET',],
  path: '/api/s3Settings',
  config: {
    auth: 'session',
    handler: (request, reply) => {
      let {key,} = request.query

      let keyPrefix = `u/${request.auth.credentials.username}/`
      let region = process.env.S3_REGION
      let s3Form = new AwsS3Form({
        accessKeyId: process.env.AWS_ACCESS_KEY,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
        region,
        bucket,
        keyPrefix,
        successActionStatus: 200,
      })
      let url = `https://s3.${region}.amazonaws.com/${bucket}/${keyPrefix}${key}`
      let formData = s3Form.create(key)
      reply({
        bucket,
        region,
        url,
        fields: formData.fields,
      })
    },
  },
})

Client Side Implementation

Given the above REST API method for obtaining POST request form fields, implementing the client side upload itself is quite simple. You simply need to obtain S3 metadata from said REST method, then construct a corresponding FormData object and POST it to your S3 bucket URL:

let R = require('ramda')

let ajax = require('./ajax')

class S3Uploader {
  constructor({folder,}) {
    this.folder = folder
  }

  send(file) {
    let key = `${this.folder}/${file.name}`
    return ajax.getJson(`s3Settings`, {key,})
      .then((s3Settings) => {
        let formData = new FormData()
        R.forEach(([key, value,]) => {
          formData.append(key, value)
        }, R.toPairs(s3Settings.fields))
        formData.append('file', file)

        return new Promise((resolve, reject) => {
          let request = new XMLHttpRequest()
          request.onreadystatechange = () => {
            if (request.readyState === XMLHttpRequest.DONE) {
              if (request.status === 200) {
                resolve(s3Settings.url)
              } else {
                reject(request.responseText)
              }
            }
          }

          let url = `https://s3.${s3Settings.region}.amazonaws.com/${s3Settings.bucket}`
          request.open('POST', url, true)
          request.send(formData)
        })
      }, (error) => {
        throw new Error(`Failed to receive S3 settings from server`)
      })
  }
}