We discussed this in chat, but I wanted to write here for the record that I believe only the VALIDATION_CHUNK_SIZE will be read into memory at a time. The hashlib docs do not state this explicitly, but based on my past studies of hashing algorithms, I believe they are using chaining to calculate the hash, and thus do not need to keep all the chunks that were given to them.

http://docs.python.org/2/library/hashlib.html