AOM3A1B / tokenizer

Commit History

Rename vocab.json to tokenizer/vocab.json
2931a04

megasuta80 commited on

Rename tokenizer_config.json to tokenizer/tokenizer_config.json
26a77fa

megasuta80 commited on

Rename special_tokens_map.json to tokenizer/special_tokens_map.json
d68cfee

megasuta80 commited on

Rename merges.txt to tokenizer/merges.txt
d76a2d7

megasuta80 commited on