upstage

/

SOLAR-10.7B-Instruct-v1.0

like 578

Hi, I faced with some error when I load your model
In your page, we said install transformer 4.35.2 and then run example code.
But after install transformer 4.35.2, and the code return

"""
ValueError: Trying to set a tensor of shape torch.Size([1024, 4096]) in "weight" (which has shape torch.Size([4096, 4096])), this look incorrect.
"""

If I understanding is correct, it mean size of model layer and size of weight is not match

How can I solve this problem?
Below is the all code which I faced that eror

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Upstage/SOLAR-10.7B-Instruct-v1.0")
model = AutoModelForCausalLM.from_pretrained(
"Upstage/SOLAR-10.7B-Instruct-v1.0",
device_map="auto",
torch_dtype=torch.float16,
)