twilio/twilio-python — Issue #28: Unicode Encoding Errors on Validating POST Bodies
Author: jongrall
Created: 2011-10-08 01:41:02+00:00
State: open

I've been using Twilio-python for a little while now to process incoming SMS messages to my server. The other day a user tried sending a Russian character and it resulted in a 500 error. The code blew up in the python hmac routine which is called in the compute_signature() method of the util module.

Here's the end of my stack trace:

File "/base/data/home/apps/s~simplylisted-production/0-5-3.353796244180529246/lib/twilio/util.py", line 46, in validate
return self.compute_signature(uri, params) == signature
File "/base/data/home/apps/s~simplylisted-production/0-5-3.353796244180529246/lib/twilio/util.py", line 33, in compute_signature
computed = base64.encodestring(hmac.new(self.token, s, sha1).digest())
File "/base/python_runtime/python_dist/lib/python2.5/hmac.py", line 121, in new
return HMAC(key, msg, digestmod)
File "/base/python_runtime/python_dist/lib/python2.5/hmac.py", line 71, in __init__
self.update(msg)
File "/base/python_runtime/python_dist/lib/python2.5/hmac.py", line 79, in update
self.inner.update(msg)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 202-203: ordinal not in range(128)

After some quick research, I found that the solution to this problem to prevent the code from blowing up and still ensure that messages pass validation is to add the following line immediately before passing the post body 's' to the hmac method:

s = unicode(s).encode("utf-8")

This correctly converts a unicode string to an ascii bytestring using the utf-8 encoding. I've verified that this fixes the problem.