@apollo13 Using the utf8_bin collation seems to work. But the standard collation is case insensitive and you cannot set utf8_bin as the default collation in the settings due to sessions.Session and admin.LogEntry (see https://docs.djangoproject.com/en/dev/ref/databases/#collation-settings).

Which means that I (and everyone who reads this) is now able to solve the problem by manually changing the collation of the taggit tables. However, a lot of people will run into the same problem in the future.