Rails Direct Upload to AWS S3 from React Native
Tags: aws, reactnative, rails, howto
Published: 2019-09-24 12:51:54

---
title: Rails Direct Upload to AWS S3 from React Native
published: true
tags: aws, react native, ruby on rails, how-to
canonical_url: https://sethaalexander.com/rails-direct-upload-aws-s3-react-native/
cover_image: https://thepracticaldev.s3.amazonaws.com/i/zlumsyfw5oslqxxj8ate.jpg
---

I recently took on the task of allowing a user of a [React Native](http://facebook.github.io/react-native/) app I’m helping build upload a custom profile picture. It sounded like a relatively simple task when I was estimating it in our sprint planning. However, I still allowed myself some grace since I’d never done such a thing before and put 8 hours on it. Little did I know what was to come.

See, I knew our backend was running [Ruby on Rails](https://rubyonrails.org) (RoR) and I knew that [Active Storage](https://guides.rubyonrails.org/active_storage_overview.html) is now the thing but I didn’t realize the issues I would run into when I threw [Amazon Web Services (AWS) S3](https://aws.amazon.com/s3/) into the mix. I had heard good things bout Active Storage though I hadn’t worked with it any, I know RoR well enough to know that the things they add are intentional and typically well thought out, and I also knew my experience with S3 was while the configuration could be somewhat complex when it comes to IAM roles and things once it was running the way you wanted it _should_ be pretty easy to use. Especially for something that was going to be public.

Early on in my work on this task I was informed by the back end engineer that Active Storage had this pretty neat way of allowing the client application to send files directly to S3 and just sending a reference string to the Rails server. This is preferred because instead of sending the data from the client to the Rails server to Amazon it goes directly from the client to Amazon. Bypassing one step speeds everything up and also saves some load on the server. I thought to myself this was pretty cool. We at [Airship](https://teamairship.com) had done this before with a web app with solid results. I had that code to reference and base my work off of.

#### Where things start to go wrong…

This is where things start to splinter. I start to digest the code from the web app we created:

```js
import axios from "axios";
import SparkMD5 from "spark-md5";

const getUploadInfo = async (file) => {
  const checksum = await createFileChecksum(file);

  return axios.post(
    `${process.env.BASE_URL}/rails/active_storage/direct_uploads`,
    {
      blob: {
        filename: file.name,
        content_type: file.type,
        byte_size: file.size,
        checksum: checksum
      }
    }
  );
};

export const createFileChecksum = async (file) => {
  return new Promise((resolve, reject) => {
    const chunkSize = 2097152; // 2MB
    const chunkCount = Math.ceil(file.size / chunkSize);
    var chunkIndex = 0;

    const md5Buffer = new SparkMD5.ArrayBuffer();
    const fileReader = new FileReader();

    const readNextChunk = () => {
      if (chunkIndex < chunkCount || (chunkIndex === 0 && chunkCount === 0)) {
        const start = chunkIndex * chunkSize;
        const end = Math.min(start + chunkSize, file.size);
        const fileSlice =
          File.prototype.slice ||
          File.prototype.mozSlice ||
          File.prototype.webkitSlice;
        const bytes = fileSlice.call(file, start, end);
        fileReader.readAsArrayBuffer(bytes);
        chunkIndex++;
        return true;
      } else {
        return false;
      }
    };

    fileReader.addEventListener("load", event => {
      md5Buffer.append(event.target.result);

      if (!readNextChunk()) {
        const binaryDigest = md5Buffer.end(true);
        const base64digest = btoa(binaryDigest);
        resolve(base64digest);
      }
    });

    fileReader.addEventListener("error", () =>
      reject(`Error reading ${file.name}`)
    );

    readNextChunk();
  });
};

export const uploadFile = async (file) => {
  const uploadInfo = await getUploadInfo(file);

  await axios.put(uploadInfo.data.direct_upload.url, file, {
    headers: uploadInfo.data.direct_upload.headers
  });

  return uploadInfo.data.signed_id;
};
```

Real quick, `getUploadInfo()` sends the relevant info about the file the the Rails back end and returns what’s needed to direct upload to S3. `createFileChecksum()` is used by `getUploadInfo()` to calculate the base64 encoded md5 checksum of the file being sent. While Amazon does not require this Rails does. Lastly, `uploadFile()` uploads the file to AWS and then returns the `signed_id` that is then sent to Rails so it can associate that file with whatever it is in the back end.

I later realized most of this code came from somewhere else, maybe even the [@rails/activestorage](https://www.npmjs.com/package/@rails/activestorage) package. I found similar code living in a [`file_checksum.js`](https://github.com/rails/rails/blob/master/activestorage/app/javascript/activestorage/file_checksum.js) file in the Rails repository on GitHub. No matter the source of the code, there was an issue. I don’t have access to the `FileReader` api on mobile. I’m working in React Native and not a browser. So now the search commences to doing this exact same thing in React Native.

## All the things that didn’t work

Actually, I’m not going to bore you with everything that didn’t work. I honestly don’t think you care. You probably Googled how to do this and it’s **NOWHERE TO BE FOUND** on the internet. Yet, this direct upload has been a feature in Rails for a while. You might have even landed on the Rails issue [Make ActiveStorage work for API only apps](https://github.com/rails/rails/issues/32208) and a comment there:

> For those on react native, I was able to get direct uploads working using `rn-fetch-blob` for md5 hashing (which is output in hex), then converting its hex output into base64 using `buffer` for calculating the checksum. To lookup the content\_type, I used `react-native-mime-types`, and last but not least, used `rn-fetch-blob` again for calculating the size. Then, just follow the communication guidelines pointed out by @cbothner, and if the files are big, use `rn-fetch-blob` for efficiently uploading the file.
> 
> <cite>– <a href="https://github.com/rails/rails/issues/32208#issuecomment-477688465">Samsinite</a></cite>

So, I tried to follow the above thread and I couldn’t get it to work. Granted, that comment is almost 6 months old and in JavaScript time that’s a lifetime ago. The main issue I ran into is I could not for the life of me get the checksum to match up with what Amazon calculated on their side. I kept getting responses of “The Content-MD5 you specified was invalid”. I tried **MANY** ways of generating the md5 checksum and they all ended up with the same Content-MD5 message being returned from AWS.

So here’s how I ended up getting it to work (why you’re really here):

```js
import axios from "axios";
import Config from "react-native-config";
import RNFetchBlob from "rn-fetch-blob";
import AWS from "aws-sdk/dist/aws-sdk-react-native";
import { Platform } from "react-native";
import { Buffer } from "buffer";
const { fs } = RNFetchBlob;

AWS.config.update({
  accessKeyId: Config.AWS_ACCESS_KEY_ID,
  region: Config.AWS_REGION,
  secretAccessKey: Config.AWS_SECRET_ACCESS_KEY
});
const s3 = new AWS.S3({ apiVersion: "2006-03-01" });

const getUploadInfo = async (fileInfo, file) => {
  const params = {
    Bucket: Config.AWS_BUCKET,
    ContentType: fileInfo.type,
    Key: fileInfo.fileName,
    Body: file
  };
  const psUrl = s3.getSignedUrl("putObject", params);
  const checksum = unescape(psUrl.split("&")[1].split("=")[1]);

  return axios.post(
    `${Config.API_UPLOAD_HOST}/rails/active_storage/direct_uploads`,
    {
      blob: {
        filename: fileInfo.fileName,
        content_type: fileInfo.type,
        byte_size: fileInfo.fileSize,
        checksum: checksum
      }
    }
  );
};

export const uploadFile = async (fileInfo) => {
  const uri =
    Platform.OS === "ios" ? fileInfo.uri.replace("file://", "") : fileInfo.uri;
  const file = await fs
    .readFile(uri, "base64")
    .then(data => new Buffer(data, "base64"));

  const uploadInfo = await getUploadInfo(fileInfo, file);
  const { headers, url } = uploadInfo.data.direct_upload;

  try {
    await axios.put(url, file, { headers: { ...headers } });
  } catch (e) {
    throw e;
  }

  return uploadInfo.data.signed_id;
};
```

This is definitely not the most elegant solution. I haven’t refactored it at all yet either. However, it works. In the world of code that means something. So what in the world is going on here? I’ll walk through it, although, I’ll jump around the file some. First, I setup the `aws-sdk` and a new `s3` instance. I’m using `react-native-config` to manage environment variables here. I initially did this to see if I could get the `signed_id` I needed by just bypassing Rails and uploading directly to AWS, that didn’t work. However, what I noticed when I generated a pre-signed URL for uploading via the `aws-sdk` was that the URL contained and md5 checksum!

#### Back to the code

Okay, the code, walk through it, here we go. I call `uploadFile()` in the response from `react-native-image-picker` on my screen component. That’s where the `fileInfo` argument comes from. I then get the proper URI based on the OS, and read the file with `rn-fetch-blob`. I turn that data into a Buffer because the `aws-sdk` on accepts certain types of files when creating a pre-signed URL. I then pass the `fileInfo` and the `file` along the `getUploadInfo()`. `getUploadInfo()` then creates a pre-signed URL using the `s3` instance we setup earlier and does some hacky string manipulation (needs a refactor) to acquire the checksum. Now, I can use that checksum (which Amazon code created) to get the direct upload URL and headers from Rails. Lastly, I upload the file to AWS and return the `signed_id` which I send along to Rails elsewhere in my code.

Ultimately, this was a pretty frustrating problem to fight against. However, it felt so good when I uploaded a file and saw the user profile image change. I actually got up and ran around my home office with my hands in the air rejoicing. I’m also stoked that I can share this solution and see how others might improve on what I did or figure out better ways to go about this. I’m not convinced this is the best solution to this problem, however, it’s a solution that works.

From my `yarn.lock`:
– react-native v0.60.5
– react-native-image-picker v1.1.0
– rn-fetch-blob v0.10.16
– aws-sdk v2.532.0

The post [Rails Direct Upload to AWS S3 from React Native](https://sethaalexander.com/rails-direct-upload-aws-s3-react-native/) appeared first on [Seth Alexander](https://sethaalexander.com).