UnicodeDecodeError when using POST with files AND Authentication

Whenever I try to do a Post with an image as file and use Authentification:

```
        r = requests.post(
            url,
            data=data,
            auth=('auth','pass'),
            files={'image': object.image},
        )
```

 I get this error:

UnicodeDecodeError 
'ascii' codec can't decode byte 0xff in position 1141: ordinal not in range(128)

If I remove the authentication it works.

The full Traceback: 

```
Environment:


Request Method: POST
Request URL: http://127.0.0.1:8000/admin/magazines/magazine/71/

Django Version: 1.6.2
Python Version: 2.7.0
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'south',
 'ajax_select',
 'shops',
 'feeds',
 'magazines')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')


Traceback:
File "C:\Python27\lib\site-packages\django\core\handlers\base.py" in get_response
  114.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\Python27\lib\site-packages\django\contrib\admin\options.py" in wrapper
  432.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "C:\Python27\lib\site-packages\django\utils\decorators.py" in _wrapped_view
  99.                     response = view_func(request, *args, **kwargs)
File "C:\Python27\lib\site-packages\django\views\decorators\cache.py" in _wrapped_view_func
  52.         response = view_func(request, *args, **kwargs)
File "C:\Python27\lib\site-packages\django\contrib\admin\sites.py" in inner
  198.             return view(request, *args, **kwargs)
File "C:\Python27\lib\site-packages\django\utils\decorators.py" in _wrapper
  29.             return bound_func(*args, **kwargs)
File "C:\Python27\lib\site-packages\django\utils\decorators.py" in _wrapped_view
  99.                     response = view_func(request, *args, **kwargs)
File "C:\Python27\lib\site-packages\django\utils\decorators.py" in bound_func
  25.                 return func(self, *args2, **kwargs2)
File "C:\Python27\lib\site-packages\django\db\transaction.py" in inner
  339.                 return func(*args, **kwargs)
File "C:\Python27\lib\site-packages\django\contrib\admin\options.py" in change_view
  1230.                 self.save_model(request, new_object, form, True)
File "C:\Python27\lib\site-packages\django\contrib\admin\options.py" in save_model
  860.         obj.save()
File "C:\DEV\projects\stylr\stylr\apps\magazines\models.py" in save
  48.             files={'logo': self.logo, 'image': self.cover},
File "C:\Python27\lib\site-packages\requests\api.py" in post
  88.     return request('post', url, data=data, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py" in request
  44.     return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py" in request
  383.         resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py" in send
  486.         r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py" in send
  330.                     timeout=timeout
File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py" in urlopen
  480.                                                   body=body, headers=headers)
File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py" in _make_request
  285.             conn.request(method, url, **httplib_request_kw)
File "C:\Python27\lib\httplib.py" in request
  946.             self._send_request(method, url, body, headers)
File "C:\Python27\lib\httplib.py" in _send_request
  987.         self.endheaders(body)
File "C:\Python27\lib\httplib.py" in endheaders
  940.         self._send_output(message_body)
File "C:\Python27\lib\httplib.py" in _send_output
  801.             msg += message_body

Exception Type: UnicodeDecodeError at /admin/magazines/magazine/71/
Exception Value: 'ascii' codec can't decode byte 0xff in position 1141: ordinal not in range(128)
```


What version of requests are you using? (`requests.__version __`)


Oh sorry forgot to mention that: I'm using  version 2.2.1


Can you tell me the types of `url`, `data` and `object.image`?


url -> str
data -> dict
object:image -> django.db.models.fields.files.ImageFieldFile


What's in `data`? Anything of type `unicode`?


`str`,`unicode`, `int` , `bool`


Tried to convert all `unicode` from `data`  to `str` but i still get the same error.


@p-clem what's the `filename` of `object.image`, i.e., `object.image.filename`? I'm guessing it has unicode characters which cannot be decoded with `utf-8`.


Yeah `object.image.name` is of type `unicode`


What happens when you do `object.image.name.decode()`?


Nothing changes even with `object.image.name.decode()`. It stays `unicode`
If I do `str(object.image.name)` its converted to `str`, but i still get the error.


Erm, sorry. I meant to say `object.image.name.encode()`


This time their are converted to `str` but the problem still persists


Here how you can reproduce the error:

```
import urllib2
import requests
img = urllib2.urlopen("http://lorempixel.com/300/300").read()
r = requests.post(
            'http://httpbin.org/post',
            data= {},
            auth=('auth','pass'),
            files={'image': img},
        )
```

Result :

```
`Traceback (most recent call last):
  File "<input>", line 8, in <module>
  File "C:\Python27\lib\site-packages\requests\api.py", line 88, in post
    return request('post', url, data=data, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 330, in send
    timeout=timeout
  File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 480, in urlopen
    body=body, headers=headers)
  File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 285, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "C:\Python27\lib\httplib.py", line 946, in request
    self._send_request(method, url, body, headers)
  File "C:\Python27\lib\httplib.py", line 987, in _send_request
    self.endheaders(body)
  File "C:\Python27\lib\httplib.py", line 940, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 801, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 102: ordinal not in range(128)
```


The issue is in the image body:

``` python
import urllib2
img = urllib2.urlopen("http://lorempixel.com/300/300").read().encode()
```

No matter what you try, you cannot safely upload the image it seems. You may have better success writing it to a temporary file and then opening that file with `'rb'`. Alternatively, try this:

``` python
import io
import urllib2
img = io.BytesIO(urllib2.urlopen("http://lorempixel.com/300/300").read())
r = requests.post('https://httpbin.org/post', data={'a': 'b'}, files={'file': ('300x300.jpg', b, 'application/jpeg')})
```

I have no issues doing it that way


Your snippet doesn't work. I get `b is not defined`.

If i change it to :

```
import io
import urllib2
img = io.BytesIO(urllib2.urlopen("http://lorempixel.com/300/300").read())
 r = requests.post('https://httpbin.org/post', data=data, files={'file': (img, 'b', 'application/jpeg')})
```

It works but the file seems to empty. JSON response:

```
`{
    u 'files': {
        u 'file': u 'b'
    },
    u 'origin': u 'XX.XXX.XX.XXX',
    u 'form': {
        u 'status': u '1',
        u 'magazine_id': u '3',
        u 'style': u 'Society&Culture',
        u 'name': u '032c',
        u 'language': u 'en',
        u 'feed_url': u 'http://032c.com/feed/',
        u 'created_at': u '2014-05-14 12:28:55',
        u 'enabled': u 'True',
        u 'updated_at': u '2014-05-16 10:05:56',
        u 'website_url': u 'http://032c.com/'
    },
    u 'url': u 'http://httpbin.org/post',
    u 'args': {},
    u 'headers': {
        u 'X-Request-Id': u 'e3dc6169-b3e6-4a05-a8f5-ffaa2e79f8e6',
        u 'Accept-Encoding': u 'gzip, deflate, compress',
        u 'Content-Length': u '1203',
        u 'Host': u 'httpbin.org',
        u 'Accept': u '*/*',
        u 'User-Agent': u 'python-requests/2.2.1 CPython/2.7.0 Windows/post2008Server',
        u 'Connection': u 'close',
        u 'Content-Type': u 'multipart/form-data; boundary=b4860a97cd344300a2d8dd839b049329'
    },
    u 'json': None,
    u 'data': u ''
}
```

As if img was empty...


The last line in my example was wrong, sorry about that. It should have been:

``` python
r = requests.post('https://httpbin.org/post', data={'a': 'b'}, files={'file': ('300x300.jpg', img, 'application/jpeg')})
```


@sigmavirus24  You forgot to add the authentification. Without it always worked for me but with authentification it didnt.

Anyway I tried to test it out on an other environment and it worked without problems, even on production! So this bug only appears on my work PC which is frustrating but oh well...

Thank you for your help,


For anyone coming across this issue, I think the problem only manifests on Python 2.7.0.

requests.auth._basic_auth_str(), which is used to build the auth header, was generating unicode rather than str, and in Python 2.7.0 unicode header values cause problems in conjunction with a non-ascii binary (str) request payload. In 2.7.1 and later, headers get converted to str in httplib.HTTPConnection.putheader(), since this commit: http://hg.python.org/cpython/rev/e1f6ce836fc5.

The problem will probably go away in requests 2.3.1, since _basic_auth_str() now emits a str (changed since 2.3.0 was released).