github.com/sugarme/tokenizer (go): Package tokenizer represents a tokenization pipeline.
Keywords: deep-learning|golang-tokenizer|nlp|tokenizer