SaulLu
/

bengali-tokenizer

Model card Files Files and versions Community

bengali-tokenizer

Commit History

set normalized to true

524ccfa

SaulLu commited on May 11, 2021

tokenizer v2- include normalization discussed with Bengali community

9ff1d40

SaulLu commited on May 11, 2021

add tokenizer

f472ef8

SaulLu commited on May 7, 2021

initial commit

565b81d

system HF staff commited on May 6, 2021