Your files go to Cloud : Uploading Images with Express to Cloudinary
Tags: webdev, node, javascript
Published: 2022-11-19 18:25:15

As soon as your website or app reaches a point when you realize you cannot always display static or dummy images among all the dynamic content fetched from your backend, you start looking for ways to integrate image uploads in your project. After looking around on the internet for a while, you begin to hear some names and if you aren't willing to let AWS have your card details, you will fall back to [Cloudinary](https://cloudinary.com). Now, Cloudinary is quite convenient for small project and specially when you are still studying. But last time I tried getting it set up in my project, it wasn't the most "copy-from-docs and go" set up. Along with Cloudinary's own configuration, an important aspect is how you actually bring your files to Cloudinary's API for the upload.

In this one, we will be implementing image uploads to cloudinary wherein we will demonstrate it using a form to set profile pictures for the user. Our focus here would be on how we will handle the file so that it can be uploaded. I will use a node.js backend with authentication already implemented. In the client, I will use a form that handle user's profile information but we mostly care about the profile picture. 

### Preface

I will be going in depth in this one. If you need a to-the-point solution to how you can get image upload set-up with cloudinary you can skip to [In short](#in-short)

### Let's get started

So, I am trying to add a profile picture to the users in my database by uploading them to cloudinary. I would have a form in the front end, I would use a file input to select a file and then it would send this file to the backend where it would be uploaded to cloudinary. Now, these things come with hidden complexities at each steps. The file input gives us a Javascript `File` Object which the database or cloudinary doesn't really like or understand. We need raw image data or maybe a url. Also, JSON wouldn't send images as it is basically just fancy text. Even if we do manage to send these File Objects to our endpoint, this file object would not be enough to upload the image to cloudinary. Let's test each of these using an express server and a mongodb database.

### Situation

I have a user collection in my local mongodb and I have added one user to it manually which looks like the following:

```
blogdb> db.users.find()
[
  {
    _id: ObjectId("629f08b134f91b645470ca97"),
    username: 'suyious',
    name: 'Suyash Kumar',
    email: 'solo@gmail.com',
    password: '$2b$10$eT7nsqWx3jVjlH9xbHOU2.EcowoHlXf8VWMldb2klFKL2CzxSeC5a',
    role: 'user',
    createdAt: ISODate("2022-06-07T08:13:37.436Z"),
    __v: 0,
    avatar: {
      public_id: 'xetra/products/vunlvjcijttmyu6aah1o',
      url: 'https://res.cloudinary.com/djv5txrzp/image/upload/v1667916852/xetra/products/vunlvjcijttmyu6aah1o.jpg'
    }
  }
]

```

This should give an idea of how the user collection looks like. We have an `avatar` field which is an object with a key `url`. Now I would like to change this avatar field and by changing it, I imply sending an image from my frontend to the server which would then get uploaded to cloudinary and the `avatar` field would have it's new `url`. Now we can always manually upload our images to cloudinary and just send it's url through a JSON, but we need that to happen on it's own and we should just need to use the html file input.

### Sending the image from the frontend

I guess even now, you would have started getting a hint that JSON would probably not send our file object to our server. But, let's test that anyway. All we need is an `<input type="file"/>` and an `axios.put('/upload', body)`. When we are sending the image from the frontend, the request body looks like the following in Javascript:

![Browser Javascript console showing our request body](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/04kp7g15v0ic539lnyde.png)

Looks fine for now but as soon as this request reaches the backend, thing begin to look different.

![Server Console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l2ixbseq4u2fc29i3pw9.png)

Clearly, JSON doesn't seem to know what a file exactly is. Guess, it's time to run to stack overflow.

![Naruto run meme](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l1gf2yn46ql47k1twqur.png)

Turns out, apart from `application/json`, there are other content-type for your request body and one of them is [`mutipart/form-data`](https://stackoverflow.com/questions/4526273/what-does-enctype-multipart-form-data-mean). In short, this is what we use when we have files along with text to send to server. Looks like that should solve our problem and the `File` would show up in our backend.

### multipart/form-data

In order to send `multipart/form-data` through an HTMl form, you need `<form enctype='multipart/form-data'>` tag and when using axios, you need `axios.put('/put', body, { "Content-Type": "multipart/form-data" })`. You can also create a `FormData` Object in Javascript and set key/value pairs to construct your request body.

So, now we should just try sending form-data from our frontend similar to before, only this time our request body would be constructed using:

```
const body = new FormData(formElement);
```

The `formElement` is a reference to am html form which can be the name of a form ( or ref.current when using React.useRef)

In the frontend, the request body looks like this:
![Javascript console req.body](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8qv6eeyj0da4re8jia87.png)

But now, we turn to our server only to see an even more obscure output.

![Server Console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ecryo1c359awchrigd1g.png)

We should also try to send the same form-data using postman just in case we are making some mistake in our frontend code.

![Postman Output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iba6yy0c287qpc7l2mz5.png)

Clearly, even postman gives us a similar output. It seems like the backend isn't able to understand what we are throwing at it. Hmm. That doesn't happen often. Or does it? One might begin to remember that we couldn't earlier even read the JSON output if we just threw it at the backend. If that didn't ring a bell, I am talking about the `express.json()` or earlier `bodyparser()` middlewares that we needed to parse our JSON requests. It is fairly obvious that we would also need to parse form-data before our backend can understand it.

Now, there are multiple similar libraries for accomplishing this and the most famous one that I can find is [`multer`](https://www.npmjs.com/package/multer). Naturally, we would start by installing it and including it in our server. We could add the middleware for every route similar to `express.json()` with `express.use()`, but we would want to only add it for the routes that actually need form data.

### multer

So, if we go forward, install multer and add a middleware `multer().single('avatar')` (see [docs](https://www.npmjs.com/package/multer#usage)) to our upload route, we find this in our console:

![Server Console Logs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7un8j26vvgq84vbr3uk6.png)

Looks like our form-data did pass through. But where is the file. Turns out the files are not a part of `req.body`. Instead they are in `req.file`.

![Console log with File](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cwqpjvwxhrcgtiw14cvh.png)

We can finally see our file reach the backend. This looks good. Now all we have to do is throw this file at Cloudinary and it should just work out of the box. Right? Well, we all know our history with throwing things around here.

### Cloudinary

Now, let's install `cloudinary` and set it up with the configurations. You need following environment variables for the config:

```
cloudinary.config({
  cloud_name: process.env.CLOUDINARY_NAME,
  api_key: process.env.CLOUDINARY_API_KEY,
  api_secret: process.env.CLOUDINARY_API_SECRET
})
```

Above is also the code that you need to add to your server for initial configuration of cloudinary. Once this is done, you should be able to talk to your cloudinary account. ( If it wasn't obvious, you need to create an account. It won't charge you or ask for card details. You will get your `api_key` and `api_secret` from the [dashboard](https://cloudinary.com/console)

Now, as from the [docs](https://cloudinary.com/documentation/node_integration#installation_and_setup), we need the following to upload our image:

```
import { v2 as cloudinary } from 'cloudinary'
cloudinary.uploader
  .upload(file, options)
  .then(callback);
```

Once we are set up, we should try to use the file we received from the frontend on the `upload` function provided by the api. I have the following code added to my upload route:
```
cloudinary.uploader.upload(req.file)
  .then((result => {
    console.log(result);
  }))
```

This time, the server gives us a new error. Depending on how your server is set up, the error might show up at different places. As for me, the error comes as a 500 Error Response.

![Javacript Console Output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/72zrpj9ivid3dmp64j37.png)

The error message tells us that cloudinary's API does not accept a Javascript `File` object and instead is asking for some string. If we look into the docs again, we find the following:

![Cloudinary API docs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hf7gx8s5iz6zem5kkwfa.png)

Given above, the `upload` function takes only the allowed types as the file parameter. Out of these, we do not have a local file path in the server as the file comes from the frontend. Neither do we have a file URL or S3 bucket URL. We are left with using a byte-array buffer or a Data URI (Base64 encoded).

![Console log with File](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cwqpjvwxhrcgtiw14cvh.png)

As we can see above, we do seem to be getting a buffer key in the file object received. Let's inspect it's value and see if it can be of our use. If we just plug the buffer value to the `upload` function, the cloudinary api still cannot process it and gives a similar error.

![ Javascript Console ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ixxzj5roh206nmmyb7j8.png)

We could try to process this byte array and make it work with the API but if we look at `multer`'s [documentation](https://www.npmjs.com/package/multer#memorystorage), we notice using the buffer option requires storing the images into memory which could be bad for the server in terms of performance.

[![multer documentation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nitaiafkciogfhtsq52r.png)](https://www.npmjs.com/package/multer#memorystorage)

Now, we are left with either trying to convert the file into a byte-array in the frontend or using the data URI option. Also, we should try to do the computations in the frontend itself as much as possible, now that we are reminded. 

If we dig around a little to find some information about byte-array or Data URI, we come across the [`FileReader`](https://developer.mozilla.org/en-US/docs/Web/API/FileReader) object provided by Javacript's [`File`](https://developer.mozilla.org/en-US/docs/Web/API/File_API) API. This object provides functions for dealing with both byte-arrays and Data URIs.

### FileReader

This `FileReader` object delivers the data using events, as reading from disk may take time. The Syntax for using it is generally of the form:

```
const reader = new FileReader();

reader.onload = () => {
  <your-processing-function>(reader.result);
}

reader.<instance function>(blob);
```

We should try the `readAsDataURL(file)` function to send the image file as a Data URI. Note that the function is `readAsDataURL` and NOT readAsDataURI. The result of this function call returned through `reader.result` can also be used with `<img src=""/>` to preview the image. The following code demonstrate how we would process the file before sending it.

```
const reader = new FileReader();
reader.onload = () => {
  console.log(reader.result);
}
reader.readAsDataURL(input.files[0])

```
Here, `input` may be `e.target` if you are listening to events on a file input or `inputref.current` if inputref is a ref to an input element when using `React.useRef`. The resulting Data URI is returned as a string through `reader.result`. Now, that this is a string, it should work as an argument to cloudinary API's `upload` function.
The result looks like the following:

![Data URI string](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s5isrltxcevinq1eq8gn.png)

It is basically a bunch of gibberish to us but will represent our image file to the server. When we send this new Data URI string to the server, it gets sent as is, that is as a string. But now that it is no longer a file. It is part of `req.body` itself and not of the `req.file`

![Server Console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wq1qs7vkprmnoqd1ga8z.png)![Server Console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oqzsxh74ipg4dhgzb3h2.png)

Now, we could plug this string into cloudinary API's upload function and see if it works:

```
cloudinary.uploader.upload(req.body.avatar)
  .then((result => {
    console.log(result);
  }))

```
This, after all this work, actually works and we get the following result in our server console:

![Server Console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g5bsst4udfwdhc98mbix.png)

We can go to the provided secure url and it is actually uploaded to cloudinary. We can also see this image in our cloudinary [media library](https://cloudinary.com/console/media_library/folders/home). Here, I have used the `folder` option that can be provided in the second argument of the upload function as follows:

```
const result = await cloudinary.uploader
  .upload(req.body.avatar, {
    folder: "/fraise/avatars"
  })

```

![Cloudinary Library](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xp4wx26rktu514txeil0.png)

The API returns an object with a url for the uploaded image as `secure_url`, which can now be stored in the database to update our user profile.

```
const user = await User.findByIdAndUpdate(req.user.id, req.body, {
  new: true,
  runValidators: true,
  useFindAndModify: false
})

```

Here, my server is setting `req.user` while validating authentication, though it is only the providing the `id` of the entry to update in our database. This updates the current user's profile picture.

### In short

In short, all I did all this while was:

- send files as [form-data](#multipartformdata) and not JSON.

here you might need:
```
<form enctype='multipart/form-data'>
```
```
axios.put('/put', body, { "Content-Type": "multipart/form-data" })
```

- receive form-data using [multer](#multer).

you might need to install `multer` and add the middleware:
```
multer().single('name')
```
for other usages, here's the [docs](https://www.npmjs.com/packages/multer#usage)

- setup [cloudinary](#cloudinary) and upload
```
cloudinary.config({
  cloud_name: process.env.CLOUDINARY_NAME,
  api_key: process.env.CLOUDINARY_API_KEY,
  api_secret: process.env.CLOUDINARY_API_SECRET
})
```
```
import { v2 as cloudinary } from 'cloudinary'
cloudinary.uploader
  .upload(path, options)
  .then(callback);
```

- send files as [dataURI](#filereader) for the `path` in `upload()` above:

```
const reader = new FileReader();
reader.onload = () => {
  console.log(reader.result);
}
reader.readAsDataURL(input.files[0])
```

### But this makes me think

Even though we used `multer` to be able to read multipart/form-data in the server, we don't quite seem to have used it for actual file transfer between the frontend and the backend. It makes me wonder if using form-data was for nothing. I did try to use JSON for the same but it didn't quite seem to work. Now, I didn't want to pull apart all that code again. So, I urge you to try it.

Additionally, the data URI seems to be a very long string which might be too long to send back and forth. Tell me if there are other simpler ways to upload the images. I do know of another method, creating a temporary file on the server and using it's file path for the upload. This would require creating and removing files on the server.

I would be really happy if I am pointed to a better direction here, but if my solution was helpful to you, that would make me really happy too.