tcftrees's picture
Create README.md
47d7788 verified
The trained BPE tokenziers with various vocabulary sizes, which uses to study how the vocabulary size affects the performance of language models.