microsoft

/

Phi-3-mini-4k-instruct

like 717

Thanks for the model!

I encounter the following when loading the tokenizer:

from transformers import AutoTokenizer

checkpoint_path = "microsoft/Phi-3-mini-4k-instruct"
tokenizer = AutoTokenizer.from_pretrained(checkpoint_path)

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

Does this mean I would have to free the embedding layers when doing fine-tuning with LoRA?

Thanks!