FoundationDB/sql-layer — pull request #1049 by MMcM (2013-11-14 19:34:46+00:00)
Title: Fix issues with codepoints >= U+10000

The basic issue is that these take two char (surrogate) values and all the classic java.lang.String functions are in terms of char indices. We want one SQL CHAR = one Unicode codepoint (like Postgres -- MySQL is a mess).

This branch fixes the obvious places I could think of. I would welcome suggestions for ones I missed. It's pretty hard to find systematically and the author of the code needs to think what an index is supposed to mean, particularly if it comes from SQL statements.

This is an incompatible change that breaks old volumes because it changes the max storage size for CHAR columns. The max for UTF-8 is 4 bytes. (Not 6 because due to limiting the range for compatibility.)

Note that due to the same change you cannot have a compound index with two VARCHAR(255) columns. They just miss fitting in a Persistit Key. I'm not sure basing this DDL-time test on the worst case (4x ASCII -- even when it was 3x) is really what we want, but I didn't change anything.