mistralai

/

Mixtral-8x7B-v0.1

like 1.56k

You did not install transformers==4.36.0 .... pip install -U transformers

thank you Arthur, I updated the transformers via github and it didn't work, the dev version is not up to date

It is working for me with RUN pip install git+https://github.com/huggingface/transformers.git

Thanks ArthurZ, thanks Sacdroid :)

KeyError Traceback (most recent call last)
Cell In[5], line 6
3 model_id = "mistralai/Mixtral-8x7B-v0.1"
4 tokenizer = AutoTokenizer.from_pretrained(model_id)
----> 6 model = AutoModelForCausalLM.from_pretrained(model_id)
8 text = "Hello my name is"
9 inputs = tokenizer(text, return_tensors="pt")

File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:526, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
523 if kwargs.get("quantization_config", None) is not None:
524 _ = kwargs.pop("quantization_config")
--> 526 config, kwargs = AutoConfig.from_pretrained(
527 pretrained_model_name_or_path,
528 return_unused_kwargs=True,
529 trust_remote_code=trust_remote_code,
530 code_revision=code_revision,
531 _commit_hash=commit_hash,
532 **hub_kwargs,
533 **kwargs,
534 )
536 # if torch_dtype=auto was passed here, ensure to pass it on
537 if kwargs_orig.get("torch_dtype", None) == "auto":

File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1064, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
1062 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
1063 elif "model_type" in config_dict:
-> 1064 config_class = CONFIG_MAPPING[config_dict["model_type"]]
1065 return config_class.from_dict(config_dict, **unused_kwargs)
1066 else:
1067 # Fallback: use pattern matching on the string.
1068 # We go from longer names to shorter names to catch roberta before bert (for instance)

File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:761, in _LazyConfigMapping.getitem(self, key)
759 return self._extra_content[key]
760 if key not in self._mapping:
--> 761 raise KeyError(key)
762 value = self._mapping[key]
763 module_name = model_type_to_module_name(key)

KeyError: 'mixtral'

I have tried everything which is mentioned in the above comments but still I'm getting this error

@sanjog21 can you try to re-install transformers in a fresh new env?

Hi,

I am currently trying to load the mistralai/Mixtral-8x7B-v0.1 model using Hugging Face, but I am encountering some issues. I noticed a note stating that "the model cannot (yet) be instantiated with HF." However, I have seen that you have managed to load this model despite this comment.

When I attempt to load the model AutoModelForCausalLM.from_pretrained(model_id), I receive the following error:
OSError: mistralai/Mixtral-8x7B-v0.1 does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.

I would appreciate any guidance on how to properly load this model. Is there a specific method or workaround to load this model using Hugging Face despite the aforementioned comment?

Thank you in advance for your help.

Hi @yardenhoch
Thanks for the comment! I suspect you most likely have an outdated version of transformers that do not support safetensors serialization format. Can you try to upgrade your transformers version? pip install -U transformers this should solve your issue