sile/gomoku — pull request #1 by keitaf (2011-11-01 18:34:49+00:00)
Title: Handle middle dot ・ as symbol, and add Tagger.parseAsIterable() method

Hi,

I made the following changes. Can you please merge if they look good?

- Modified char.def in IPADIC and regenerated code.bin and code-map.bin so that a middle dot '・' is handled as a symbol.
- In the original IPADIC, a middle dot is registered as Katakana. So unknown words connected by the middle dot (e.g., レディ・ガガ) isn't tokenized. With this change, unknown words with middle dot is tokenized as expected (レディ・ガガ -> /レディ/・/ガガ/)
- Added Tagger.parseAsIterable() method, which returns Iterable as the result of the tokenization.
- parseAsIterable() is more efficient and faster than parse() because it doesn't create String objects.

Thanks,
Keita