tokenmill/ltlangpack — pull request #1 by donatasremeika (2014-09-30 11:26:59+00:00)
Title: Tokenizer training samples with semicolon

Tokens should be split by semicolon. It is the same as comma, just stronger: http://lt.wikipedia.org/wiki/Kabliata%C5%A1kis
Current training data didn't have such samples.