apache/incubator-flink — pull request #4 by zentol (2014-06-10 19:29:20+00:00)
Title: Serialized String comparison, Unicode support

The StringComparator now works on serialized data.

To this end new string read/write/copy/compare methods were introduced, which use a variable-length encoding for the characters.

key-points:

The most significant bits are written/read first.
The first 2 bits of the character are used to encode the size of the character.
A character is at most 3 Bytes big.

Additionally, the StringSerializer now has full unicode support. i couldn't find a unicode character that uses more than 22 bits, as such 3 Bytes should be sufficient.