boto/boto — pull request #424 by marcacohen (2011-12-05 22:28:31+00:00)
Title: Fix for problem computing checksum when downloading large files on Windows

... this file open". This was due to a cascading sequence of two problems:

1) The checksum was not matching because the local copy of the downloaded file
was opened with mode 'r' instead of 'rb', when computing the checksum. The
solution is to use 'rb', which is portable across all OSes. Per the Python docs
for open():

If mode is omitted, it defaults to 'r'. The default is to use text mode, which
may convert '\n' characters to a platform-specific representation on writing and
back on reading. Thus, when opening a binary file, you should append 'b' to the
mode value to open the file in binary mode, which will improve portability.
(Appending 'b' is useful even on systems that don’t treat binary and text files
differently, where it serves as documentation.) See below for more possible
values of mode.

The boto functions for uploading/downloading files (get_contents_to_filename()
and get_contents_from_filename()) already open the local files with 'wb' and
'rb', respectively so there's a clear established precedent for this behavior.

2) At the time the checksum is computed, the file was still open. If the
checksum fails, it's locally closed and the code attempts to remove the file but
there's an outstanding open from the caller's context, which causes an OS
exception. This was fixed by inserting an fp.close() before calling
check_final_md5().