mistralai

/

Mistral-Nemo-Instruct-2407

like 1.1k

Hello Community,
When I want to try out the model and landing it with from pretrained(), I face:

RuntimeError Traceback (most recent call last) Cell In[5], line 1----> 1 model = AutoModelForCausalLM.from_pretrained(model_id) 2 tokenizer = AutoTokenizer.from_pretrained(model_id) File ~/code/genai-ml/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:563, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 561 elif type(config) in cls._model_mapping.keys(): 562 model_class = _get_model_class(config, cls._model_mapping) --> 563 return model_class.from_pretrained( 564 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs 565 ) 566 raise ValueError( 567 f"Unrecognized configuration class {config.class} for this kind of AutoModel: {cls.name}.\n" 568 f"Model type should be one of {', '.join(c.name for c in cls._model_mapping.keys())}." 569 ) File ~/code/genai-ml/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py:3754, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs) 3744 if dtype_orig is not None: 3745 torch.set_default_dtype(dtype_orig) 3747 ( 3748 model, 3749 missing_keys, 3750 unexpected_keys, 3751 mismatched_keys,
...
size mismatch for model.layers.39.self_attn.q_proj.weight: copying a param with shape torch.Size([4096, 5120]) from checkpoint, the shape in current model is torch.Size([5120, 5120]). size mismatch for model.layers.39.self_attn.k_proj.weight: copying a param with shape torch.Size([1024, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]). size mismatch for model.layers.39.self_attn.v_proj.weight: copying a param with shape torch.Size([1024, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]). size mismatch for model.layers.39.self_attn.o_proj.weight: copying a param with shape torch.Size([5120, 4096]) from checkpoint, the shape in current model is torch.Size([5120, 5120]). You may consider adding ignore_mismatched_sizes=True in the model from_pretrained method.

Do you have any idea what is the reason for that?
kindest regards