Error loading checkpoint

#11

by EvGUT - opened Jul 19

Jul 19

Hi, I am getting this ValueError: Trying to set a tensor of shape torch.Size([1024, 5120]) in "weight" (which has shape torch.Size([1280, 5120])), this look incorrect.

transformers installed from git rep latest commit in main
Any ideas how to solve this?

Xenova

Jul 19

•

edited Jul 19

Are you sure you have installed transformers from main?

pip install --upgrade git+https://github.com/huggingface/transformers.git

To check:

pip show transformers

EvGUT

Jul 19

Yeah, thank you, seems like problem was in my code
Didn't check that head_dim != hidden_size // num_heads

EvGUT changed discussion status to closed Jul 19

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment