stephenmcd/mezzanine — pull request #414 by kennu (2012-09-29 17:07:24+00:00)
Title: TWITTER_STRIP_HIGH_MULTIBYTE setting to strip Emoji

When using MySQL with default UTF-8 collation, tweets containing Emoji or other high-multibyte characters cause this kind of warning messages and corrupted text in the database:

Incorrect string value: '\xF0\x9F\x8C\x82\xF0\x9F...' for column 'text' at row 1

The problem can be solved by switching to utf8mb4, but that causes other problems with index lengths and I couldn't find a way to make Django use it only for a single model. So I added this setting to strip the characters completely when saving new tweets as a quick fix.

Of course, this is not a problem with Mezzanine itself, so I'm not sure if you want this setting. However, it's currently a problem for me as cron keeps sending me these warnings, at least until I can switch to another database.

More info on utf8mb4 here: http://mathiasbynens.be/notes/mysql-utf8mb4