Rotary position embeddings not loaded

#39

by cwbc - opened Apr 19

cwbc

Apr 19

When I load the model weights to transformers.LlavaLlamaForCausalLM, it says the rotary position embeddings rotary_emb.inv_freq are loaded from the checkpoint. Does it affect the model performance?

ArthurZ

Meta Llama org Apr 20

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment