praekelt/vumi-wikipedia — Issue #3: Wikipedia articles with unicode chars in the titles crash the worker
Author: smn
Created: 2012-02-24 11:04:11+00:00
State: open

I searched for "Zundapp" which goes to [Zündapp](http://en.wikipedia.org/wiki/Z%C3%BCndapp)
```

2012-02-24 11:01:53+0000 [WorkerAMQClient,client] Unhandled Error
Traceback (most recent call last):
File "/vol/var/praekelt/vumi/develop/releases/20110623_130621/vumi/ve/lib/python2.6/site-packages/twisted/internet/defer.py", line 1014, in unwindGenerator
return _inlineCallbacks(None, f(*args, **kwargs), Deferred())
File "/vol/var/praekelt/vumi/develop/releases/20110623_130621/vumi/ve/lib/python2.6/site-packages/twisted/internet/defer.py", line 893, in _inlineCallbacks
result = g.send(result)
File "/vol/var/praekelt/vumi/develop/releases/20110623_130621/vumi/ve/src/vumi-wikipedia/vumi_wikipedia/wikipedia.py", line 245, in process_message_content
session['page'], int(msg['content'].strip()) - 1)
File "/vol/var/praekelt/vumi/develop/releases/20110623_130621/vumi/ve/lib/python2.6/site-packages/twisted/internet/defer.py", line 1014, in unwindGenerator
return _inlineCallbacks(None, f(*args, **kwargs), Deferred())
--- ---
File "/vol/var/praekelt/vumi/develop/releases/20110623_130621/vumi/ve/lib/python2.6/site-packages/twisted/internet/defer.py", line 893, in _inlineCallbacks
result = g.send(result)
File "/vol/var/praekelt/vumi/develop/releases/20110623_130621/vumi/ve/src/vumi-wikipedia/vumi_wikipedia/wikipedia.py", line 111, in get_content
'page': page_name.encode('utf-8'),
exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
```