konklone/bit-voyage — pull request #7 by konklone (2014-10-13 19:16:52+00:00)
Title: Project now supports 100GB files

[![200gb](https://cloud.githubusercontent.com/assets/4592/4618717/daec1c2c-530c-11e4-9c8b-f6392bf37ae9.png)](https://twitter.com/konklone/status/521718929010745344)

https://twitter.com/konklone/status/521718929010745344

This brings in a patch, that I worked out in https://github.com/nathanpeck/s3-upload-stream/issues/20 with the author of s3-upload-stream, to fix an issue where browsers were not properly respecting sliced buffer windows. This was causing browsers to send in more data than the client was expecting, and so MD5 calculations were off, everything was off.

This was coming into effect at smaller sizes, but then vanished -- but reappeared when uploading ~20MB parts (of a 200GB file).

This brings the supported file size of this project from **1GB** to **10GB**!

Next steps are as documented in https://github.com/konklone/bit-voyage/pull/6 - the 1TB level will be difficult, and require a server-side proxy to do it, probably with a websockets transport layer. Sounds fun!