AutoTokenizer dont working
#3
by
Santiagowwi
- opened
@Santiagowwi
, use meta-llama/Llama-2-7b-chat-hf
as the tokenizer. I was able to make it work with the following tokenizer.
tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-hf")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
Thanks!!!!