twitter/twitter-cldr-rb — pull request #32 by KL-7 (2012-05-16 22:35:33+00:00)
Title: NFKD implementation

I added NFKD implementation. It ended up being the base class of normalizers hierarchy. I don't like it a bit, as I initially wanted to create some abstract base class for them, but things worked out this way. NFD works absolutely the same way, but skips compatibility decomposition. So it's implemented as a subclass of NFKD with a single overridden method.

During the process I splitted the algorithm into small methods that should make it easier to add NF(K)C algorithms later. Possibly we'll have to revisit the classes hierarchy, but in general it should be quite straightforward (assuming we have composition algorithm implemented) as these forms are build on top of NF(K)D algorithms.

Normalization specs are updated to test both algorithms. I didn't add specific tests for `NFKD#normalize`. It'd be nice to have some simple test cases for it, but I need some valid normalization examples for that. I remember Andrew generated some of them using JRuby and Java Unicode library. Maybe I'll do the same thing a bit later.

Besides that, I moved code points conversion methods into a separate utility module as in my opinion they are not a responsibility of the normalization classes.