kylemacfarlane/django-cuddlybuddly-storage-s3 — Issue #3: Large files
Author: kordsmi
Created: 2011-12-22 09:15:51+00:00
State: open

There is patch that fixes a MemoryError during uploading large files (>100M) on python 2.6. HTTPContend receive file objects since version 2.6. Also content length defining fixed.

# src\storage\s3\storage.py
@@ -118,8 +118,7 @@
break
file_pos = content.tell()
content.seek(0)
- content.seek(0, 2)
- content_length = content.tell()
+ content_length = len(content.read())
content.seek(0)
gz_cts = getattr(
settings,
@@ -149,10 +148,10 @@
'Content-Type': content_type,
'Content-Length': str(content_length)
})
- # Httplib in < 2.6 doesn't accept file like objects, and in >= 2.6 it
+ # Httplib in <= 2.6 doesn't accept file like objects, and in >= 2.7 it
# tries to join the content str object with the headers which results in
# encoding problems.
- if sys.version_info[0] == 2 and sys.version_info[1] < 6:
+ if sys.version_info[0] == 2 and sys.version_info[1] < 7:
content_to_send = gz_content.read() if gz_content is not None else content.read()
else:
content_to_send = gz_content if gz_content is not None else content