SaulLu
/

bengali-tokenizer

Model card Files Files and versions Community

bengali-tokenizer / tokenizer_config.json

Commit History

tokenizer v2- include normalization discussed with Bengali community

9ff1d40

SaulLu commited on May 11, 2021

add tokenizer

f472ef8

SaulLu commited on May 7, 2021