Add tokenizer.json

#2
by Xenova HF staff - opened

This PR adds the tokenizer.json file so that it can be used with transformers.js.

Linked PR: #1

Xenova changed pull request title from Upload tokenizer.json with huggingface_hub to Add tokenizer.json
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment