AutoTokenizer dont working

#3
by Santiagowwi - opened

image.png

@Santiagowwi , use meta-llama/Llama-2-7b-chat-hf as the tokenizer. I was able to make it work with the following tokenizer.

tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-hf")

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")

Thanks!!!!

Sign up or log in to comment