uskudnik/amazon-glacier-cmd-interface — pull request #77 by wvmarle (2012-10-13 14:38:26+00:00)
Title: Mmap

Mapping file with mmap to save on memory use. Should allow blocks larger than available RAM.

Note: I have tested the uploads and resumption, both from file and from stdin, and it works for me. I have not done any memory footprint checks yet.

Sending file over stdin still has memory issues; there seems to be a way around this but it's not easy as you have to address the httpconnection directly. And the key issue here is how to get this connection, as authentication is needed.

See also: http://stackoverflow.com/questions/12871362/sending-content-of-large-file-in-parts-over-http-
in-python

And there is this issue of unknown byte size when piping over stdin, and the need of giving a hash of the complete part in advance.