Pita/etherpad-lite — Issue #526: Non english characters are replaced with question marks sometimes (mysql) 
Author: Pita
Created: 2012-03-01 17:00:20+00:00
State: open
Labels: Serious Bug

We found out today, that some non english charcacter doesn't get saved properly to the database. Some of them get replaced by a questionmark, some by two, sometimes it works fine. This only happens if you use a mysql backend

I was able to figure out an example that always fails. Set up an etherpad lite instance with mysql, copy paste this string привет into a pad, restart etherpad lite and reload the pad. You will see that your string got replaced by questionmarks

This results in ueberDB in this sql command

REPLACE INTO `store` VALUES ('pad:test3', '{\"atext\":{\"text\":\"Welcome to Etherpad Lite!\\n\\nThis pad text is syncприветhronized as you type, приветso that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!\\n\\nEtherpad Lite on Github: http://j.mp/ep-lite\\n\\n\",\"attribs\":\"|2+r+l*0+6+m*0+6|4+4e\"},\"pool\":{\"numToAttrib\":{\"0\":[\"author\",\"a.L7J3iR4yRjzJJJVc\"]},\"nextNum\":1},\"head\":2,\"chatHead\":-1,\"publicStatus\":false,\"passwordHash\":null}'),('pad:test3:revs:2', '{\"changeset\":\"Z:6i>6|2=r=l*0+6$привет\",\"meta\":{\"author\":\"a.L7J3iR4yRjzJJJVc\",\"timestamp\":1330620479659}}');

If I paste this sql command into phpmyadmin, the special chars gets replaced with question marks too. It looks like the node mysql driver doesn't escape this chars properly, or the database scheme can not work with such chars

Any help to solve that problem is appreciated