dnanexus/dx-toolkit — pull request #74 by psung (2014-08-11 19:39:54+00:00)
Title: PTFM-12873 Fix file part sizes for large (>~ 160GB) files

* Improper division in mmap-specific computation resulted in too low buffer sizes for platform-external uploads larger than 160GB (or internal uploads larger than 960GB) that are backed by a real file.
* Add assertions to fail-fast the file upload if a similar situation is detected in the future
* Factor the relevant code into a separate method and add regression tests
* Ensure that part size is mmappable even if `dxfile.DEFAULT_BUFFER_SIZE` is not (admittedly unlikely unless user overrides the size)
* For consistency, dividing file size by 10000 now also uses `math.ceil` in the proper way instead of doing integer division and adding 1