Adding tokenizer to the repo (request/suggestion)
#9
by
johnowhitaker
- opened
I know it's just the llama 2 70B tokenizer but saving it in this directory would make it easy for anything that tries to pull the tokenizer from the same place it gets the model. tokenizer.save_pretrained(repo_path) should do it.
It's already in this repo, though. Does it not work for you?
Oops, I had it downloaded from before that was added and didn't check! Apologies, closing
johnowhitaker
changed discussion status to
closed