Pylons/webob — Issue #18: Accessing request.body after request.params raises UnicodeEncodeError if params contains non-ascii characters.
Author: VasiliyL
Created: 2011-12-26 08:52:56+00:00
State: open

```python
>>> import webob
>>> import urllib
>>> req = webob.Request.blank('/', POST=urllib.urlencode({'test': u'тест'.encode('utf-8')}))
>>> print req.body
test=%D1%82%D0%B5%D1%81%D1%82
>>> print req.params
NestedMultiDict([(u'test', u'\u0442\u0435\u0441\u0442')])
>>> print req.body
Traceback (most recent call last):
File "", line 1, in
File "webob/request.py", line 619, in _body__get
self.make_body_seekable() # we need this to have content_length
File "webob/request.py", line 829, in make_body_seekable
self.copy_body()
File "webob/request.py", line 845, in copy_body
self.body = self.body_file_raw.read()
File "webob/request.py", line 1461, in readinto
data = bytes_(url_encode(self.vars), 'utf8')
File "/usr/lib/python2.7/urllib.py", line 1311, in urlencode
v = quote_plus(str(v))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128)
>>>
```

_urllib.urlencode_ requires _str_, not _unicode_, but accessing _request.params_ sets _body_file_ to _FakeCGIBody_ with _vars_ containing _unicode_ strings. Possible fix - encode all unicode values to utf8 before passing them to _urllib.urlencode_:

```diff
@@ -1455,7 +1455,10 @@ class FakeCGIBody(io.RawIOBase):
if self.content_type.startswith(
'application/x-www-form-urlencoded'):
# TODO: check if bytes_ is necessary
- data = bytes_(url_encode(self.vars), 'utf8')
+ keys = [key.encode('utf-8') for key in self.vars.keys() if isinstance(key, unicode)]
+ values = [value.encode('utf-8') for value in self.vars.values() if isinstance(value, unicode)]
+ vars = dict(zip(keys, values))
+ data = bytes_(url_encode(vars), 'utf8')
self.file = io.BytesIO(data)
elif self.content_type.startswith('multipart/form-data'):
self.file = _encode_multipart(
```

Not sure if this is correct way to fix it though. Also, looks like bytes_ is not needed in this case :)