spad / tokenizer /tokenizer_config.json

Commit History

add tokenizer with config
4bb4e0f
verified

jadechoghari commited on