ArchiveTeam/universal-tracker — pull request #9 by yipdw (2013-11-11 04:19:36+00:00)
Title: Set default external encoding to UTF-8.  #7.

This fix has been running in production for a couple of months now with no evident ill effects, so I'm submitting it for permanent inclusion in the tracker codebase.

The main source of external data in the tracker comes from its Warrior clients and seesaw clients via its Redis instance. Redis is encoding-agnostic, and Warrior/Seesaw processes _usually_ run UTF-8.

I don't believe that this is a total fix; I don't think there's anything stopping someone from sending e.g. a username encoded in UTF-16. But this will solve the common cause of #7, and we can build on top of this work, e.g. enforcing UTF-8 for all Seesaw data returned to the tracker.