Add tokenizer.json
#3
by
jonatanklosko
- opened
Generated with:
from transformers import BlenderbotTokenizerFast
tokenizer = BlenderbotTokenizerFast.from_pretrained("/Users/jonatanklosko/git/hf/blenderbot-400M-distill/")
tokenizer.save_pretrained("/Users/jonatanklosko/git/hf/blenderbot-400M-distill/")
jonatanklosko
changed pull request status to
open
Works for me!
patrickvonplaten
changed pull request status to
merged