aleph-alpha-tokenizer (cargo): A fast implementation of a wordpiece-inspired tokenizer
Keywords: tokenizer|nlp