segmentio/marketo-python — pull request #2 by danriti (2013-08-20 18:49:31+00:00)
Title: Unicode issue in SOAP response

Hello segmentio team, I come in peace!

Last week I noticed the following **error backtrace** for a call made to the `sync_lead` method (I redacted some details from the backtrace since it contained customer data):

```python
[2013-08-08 21:01:11,717: ERROR/PoolWorker-1] Exception UnicodeEncodeError('ascii', u' THE REST REDACTED SINCE IT CONTAINS CUSTOMER DATA', 3785, 3786, 'ordinal not in range(128)'): Traceback (most recent call last):
File "/a/file/on/our/filesystem.py", line 204, in set_marketo_attrs_for_user
client.sync_lead(email=email, attributes=tuple(attributes))
File "/venv/lib/python2.7/site-packages/marketo/__init__.py", line 108, in sync_lead
return sync_lead.unwrap(response)
File "/venv/lib/python2.7/site-packages/marketo/wrapper/sync_lead.py", line 28, in unwrap
root = ET.fromstring(response.text)
File "/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1282, in XML
parser.feed(text)
File "/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1622, in feed
self._parser.Parse(data, 0)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf4' in position 3785: ordinal not in range(128)
```

As I dug deeper, I noticed the error was due to the following **unicode** character appearing in a user's last name in the SOAP response:

`M\xf4ller`

```python
>>> ord('\xf4')
244
>>> unichr(244)
u'\xf4'
>>> print unichr(244)
ô
```

So I decided to dive into your source code to see what was going on! :innocent:

The first culprit was this [call][1]: `root = ET.fromstring(response.text)`. After reading the [docs on ElementTree.fromstring][2], I stumbled upon this [issue][3] on the Python bug tracker titled, "ElementTree.fromstring doesn't work with Unicode."

This issue was **rejected** and here's why:

> This is not a bug because fromstring() unlike some other ElementTree methods is not documented to support unicode strings. Since 2.x is closed for new features, this has to be rejected.

Thus, any string passed into the `fromstring` method should be **explicitly encoded to UTF-8** to prevent this exception from being thrown.

So on this pull request, I've attached the required change, as well as 2 new test cases (all tests pass) that demonstrate the fix.

```bash
(venv)[driti@ubuntu marketo-python]$ python test.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.002s

OK
```

If you have any comments or like to further discuss this pull request, don't be shy! :smile:

[1]: https://github.com/segmentio/marketo-python/blob/master/marketo/wrapper/sync_lead.py#L28
[2]: http://docs.python.org/2/library/xml.etree.elementtree.html#xml.etree.ElementTree.fromstring
[3]: http://bugs.python.org/issue11033