Access to the tokenizer

#2
by TheSeriousProgrammer - opened

hi i am excited to try the model but unable to try the same as the tokenizer you use is from llama-7b, and its restricted access
Can you include tokenizer as well?

Thank you for giving back to the community though!!

you can use alternative like tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-hf")

Where do I put that? Thanks.

Sorry just figured out its in python.

Sign up or log in to comment