JuliaLang/libmojibake — pull request #20 by stevengj (2014-12-12 21:38:47+00:00)
Title: Update graphemes for Unicode 7

This fixes #19, and also makes it much easier to implement grapheme iterators in Julia (JuliaLang/julia#9261) by adding a `bool utf8proc_grapheme_break(int32_t c1, int32_t c2)` function to check for a grapheme break between two codepoints. This allows us to iterate over graphemes in-place, without mapping to a separate string with `0xFF` grapheme separators.

Unfortunately, I had to break backwards compatibility by changing the `utf8proc_property_t` struct to replace the `extend:1` field with a `boundclass:4` field, where the later is now read from Unicode's [GraphemeBreakProperty.txt](http://www.unicode.org/Public/UCD/latest/ucd/auxiliary/GraphemeBreakProperty.txt) file by the updated generator script. I took this opportunity to rearrange the struct to put the bitfields at the end, so that C will not insert alignment padding into the struct; as a consequence, the struct actually got smaller by several bytes.

Once this is merged, I will submit the corresponding patch to the utf8proc folks.

@jiahao, does it look okay to you?