Open Question: How can I upload files to S3 from Node.js, while having a progress bar and the ability to resize the image?
Tags: node, s3, upload, question
Published: 2018-12-26 05:13:35

---
title: Open Question: How can I upload files to S3 from Node.js, while having a progress bar and the ability to resize the image?
published: true
description: I'm building a web app where image uploading is very important. I don't want to use a 3rd party service to handle this, but I also want to be able to show the user a progress bar and crop/resize the image after it's been uploaded.
tags: node.js, s3, upload, question
cover_image: https://thepracticaldev.s3.amazonaws.com/i/hzvwjoe50u0bdruz2k5h.png
---

Hello Dev.to,

This is an open request for help/suggestions. I'm new to contributing to this community, but a long-time fan. 

I have a problem that I'm hoping you can help with. I'd post this on Stack Overflow, but I'm hoping a conversation here can be more open-ended than it would be there. 

I'm currently building a web app in which image uploading is a core component. I have uploading working and implementing, with a progress bar to show the user the status of the upload. I also have the ability to crop/resize the image after it's uploaded. I'm currently doing this all with the `multer` npm package. 

However, whenever I deploy my server, the PaaS I'm using wipes out all the uploaded files because the new deploy replaces the old one. So, I needed to switch to uploading to a 3rd party service like S3 (or start hosting my web app on Digital Ocean).

Recently, I got uploads to S3 working with `multer` by using the companion package, `multer-s3`. However, with `multer-s3`, I lose the ability to track the progress of the upload. If you look up the `httpUploadProgress` event that's fired by the AWS SDK, you'll [quickly find ](https://github.com/aws/aws-sdk-js/issues/1323) that the event is only fired at the end of an upload -- not multiple times a second as you might expect from an AJAX `progress` event.

So, if I want to keep support file uploading and still have a progress bar, here are my options:

1. Use a 3rd party service like Cloudinary or Transloadit or Uploadcare or Imgix, etc., etc.
2. Upload images directly to S3 using a [Presigned Url](https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObject.html)

Or.... 3. Get rid of the progress bar altogether.

For the life of me, I can't understand why this is such a hard problem. I feel like there *must* be a simple solution I'm just not aware of.

To quickly go through the reasons I don't like my current options:

1. A 3rd party service locks me into their API. Some of these services require jQuery and/or other large front-end libraries in order to upload directly to them (I'd prefer just an endpoint). And, if I choose to send the file directly from my server to avoid these extra client-side dependencies, it's not clear from their docs if most of these services would even return a progress event... However, in the end, the main problem with these services is cost. I don't ever want to pay $20 - $50 per month to handle file uploads if I can do it for $1-$2 on my own (although, at this point, it might be worth it).

2. The second option, uploading directly to S3 from the client using a Presigned URL, would be great in the near term, as I don't really want to modify the images just yet. However, in the near future, I could definitely see converting some of the images to WebP for browsers that support that image format -- or trimming some fat off the larger 2-20MB images.

And, as for the option of removing the progress bar... I think it's a huge usability enhancement to have one! As a user, it's really nice to know when your work is saved and how long it will take. I guess I could just show an infinite loading indicator, but then I worry about the people on a slow connection who might be sitting there with no indication when the stupid indicator is going to stop spinning... just waiting, and waiting...

So, I'm really stuck here. On the one hand, I feel like image uploading with a progress bar (and the ability to modify the image on the back-end after) should be a solved and simple problem. While, on the other hand, I'm strongly considering just biting the bullet and going with something like Cloudinary, even though it will probably double the size of my client-side JavaScript bundle (something I'd *really* love to avoid).

So... Are there any services or packages you know of that I'm missing? Is there anything you know of that will let me upload files in Node.js to an 3rd party storage service like S3, _while_ showing a progress bar to the user, _while_ not adding lots of dependencies to my client-side JS, _while_ giving me the ability to optimize/crop/convert images after they've been uploaded?

Now that I think about it... maybe this *IS* a lot to ask... heh. And, to think, I was incredulous towards a back-end developer *10 years ago* when he told me this was a hard problem... 🙈

Any open source solution would be a huge plus, but an upload service that costs $10/month would be great too.

This is an open discussion and all suggestions are welcome! 😬😁☃️ Happy Holidays!