Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
Spaces:
yenniejun
/
tokenizers-languages
like
32
Running
App
Files
Files
Community
1
7b528cd
tokenizers-languages
3 contributors
History:
22 commits
yenniejun
Merge branch 'main' of https://github.com/yenniejun/tokenizers-languages into main
7b528cd
over 1 year ago
.github
Create sync_hf_hub.yaml for syncing repo to HF
over 1 year ago
img
Adding images for README
over 1 year ago
.gitattributes
1.48 kB
initial commit
over 1 year ago
LICENSE.txt
20.8 kB
Adding license
over 1 year ago
MassiveDatasetValidationData.csv
25 MB
Updating data file with tokenizer token numbers
over 1 year ago
README.md
1.54 kB
Modifying README
over 1 year ago
app.py
5.81 kB
Rearranging some of the figures
over 1 year ago
requirements.txt
1.09 kB
Adding requirements
over 1 year ago