Can't upload files bigger than 2GB (streaming upload)

Tried this code and it works flawless with files < 2GB but as soon as I'm using files > 2GB it gives me an error.

``` python
e = MultipartEncoder(fields=dict(
            file=(rfbase, open(filename, 'rb'), 'text/plain')
        ))
 m = MultipartEncoderMonitor(e, my_callback)
uploadInfo = requests.request('POST', uploadURL,data=m,headers={'Content-Type': m.content_type})

It gives me the following error:
  File "D:/Upload-Tool/req.py", line 86, in upload
    uploadInfo = requests.request("POST", uploadURL,data=m,headers={'Content-Type': m.content_type})
  File "C:\Python27\lib\site-packages\requests\api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 444, in request
    data = data or {},
  File "C:\Python27\lib\site-packages\requests_toolbelt\multipart\encoder.py", line 292, in __len__
    return len(self.encoder)
  File "C:\Python27\lib\site-packages\requests_toolbelt\multipart\encoder.py", line 96, in __len__
    return self._len or self._calculate_length()
  File "C:\Python27\lib\site-packages\requests_toolbelt\multipart\encoder.py", line 111, in _calculate_length
    ) + boundary_len + 4
  File "C:\Python27\lib\site-packages\requests_toolbelt\multipart\encoder.py", line 110, in <genexpr>
    (boundary_len + len(p) + 4) for p in self.parts
  File "C:\Python27\lib\site-packages\requests_toolbelt\multipart\encoder.py", line 377, in __len__
    return len(self.headers) + super_len(self.body)
  File "C:\Python27\lib\site-packages\requests\utils.py", line 52, in super_len
    return len(o)
OverflowError: long int too large to convert to int
```

Please help if my code has some errors :)

edit: sorry, totally forgot to upgrade the toolbelt :) thx. a lot


That's a bizarre error. It seems like Python 2 doesn't like having len() called on really large files.


Seems like there's literally a requirement that `len()` not return anything that can't fit into an integer:

``` python
>>> class TestClass(object):
...     def __len__(self):
...         return 100000000000000000000000000000000L
... 
>>> c = TestClass()
>>> len(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int
```


I think in principle we could solve this by rearranging the calls in `super_len`, but that puts us at risk of breaking something else.


Hmm, on my system files in Python 2 have no length. So this may actually be a requests-toolbelt bug.


Aha, this is actually a duplicate of sigmavirus24/requests-toolbelt#80. Please upgrade the requests-toolbelt to 0.4.0.


Hello, i have the same beahviour but i don't understand the traceback ...
For information, the file size is 2715536587, i use my own upload handler with a __len__ method.
Requests version: requests==2.13.0

```python
Traceback (most recent call last):
  File "/myapp/tools/__init__.py", line 103, in _wrap_run
    self._real_run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/myapp/tools/__init__.py", line 118, in process_low
    task(*args)
  File "/myapp/propagate.py", line 188, in process_upload
    f.transferred, error = self._upload(f, rdiff=True if changed and self.psync.rsync_mode else False)
  File "/myapp/propagate.py", line 477, in _upload
    up.start()
  File "/myapp/transfer.py", line 290, in start
    read_timeout=30)
  File "/myapp/http.py", line 30, in wrapped
    return f(self, *args, **kwargs)
  File "/myapp/http.py", line 152, in post
    timeout=(self.connect_timeout, read_timeout))
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 535, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 467, in request
    data = data or {},
OverflowError: long int too large to convert to int
```

My guess is that this is happening because of the Boolean check on the data, asking for len and getting the wrong answer. What Python version is this?

Python 2.7.9 (default, Aug 13 2016, 17:56:53)
i checked, the size returned by the len method is : 2715536587L

I tried to fix the evaluation in requests, for the variable data (with is None or not), but i am blocked again by the super len evaluation:

```python
Traceback (most recent call last):
  File "/myapp/transfer.py", line 288, in start
    read_timeout=30)
  File "/myapp/http.py", line 30, in wrapped
    return f(self, *args, **kwargs)
  File "/myapp/http.py", line 152, in post
    timeout=(self.connect_timeout, read_timeout))
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 535, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 474, in request
    prep = self.prepare_request(req)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 407, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 305, in prepare
    self.prepare_body(data, files, json)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 471, in prepare_body
    length = super_len(data)
  File "/usr/local/lib/python2.7/dist-packages/requests/utils.py", line 55, in super_len
    total_length = len(o)
OverflowError: long int too large to convert to int
)
```

Resolved by deleting __len__ method and set len as class attribute