tokenizer.model
#7
by
etwk
- opened
Hi,
There is no tokenizer.model in this repo, shall we use the one from meta-llama/Meta-Llama-3-70B-Instruct?
Thanks!
Hi @etwk ,
There is a tokenizer.json file that contains the actual tokenizer. Using Huggingface's AutoTokenizer.from_pretrained("Nexusflow/Athene-70B") should load it correctly.
evan-nexusflow
changed discussion status to
closed