gpt2-medium-indonesian / tokenizer.json

Commit History

refactor tokenizer related files with eos token
08d39dc

alvin commited on

fixed mismatched vocab_size between model and tokenizer
eb84e5f

alvin commited on

add tokenizer
8b888a4

cahya commited on