error while using the model with transformers library
Sorry I am still discovering mlx but I tried to run the quantized version of gemma 2b-it :
model_id = "mlx-community/quantized-gemma-2b-it"
model = AutoModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
and got this error :
running this code causes this issue with the transformers library causes this issue :
AttributeError Traceback (most recent call last)
Cell In[20], line 8
5 model_id = "mlx-community/quantized-gemma-2b-it"
6 # model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0})
7 # okenizer = AutoTokenizer.from_pretrained(model_id, add_eos_token=True)
----> 8 model = AutoModelForCausalLM.from_pretrained(model_id)
9 tokenizer = AutoTokenizer.from_pretrained(model_id)
File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:561, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
559 elif type(config) in cls._model_mapping.keys():
560 model_class = _get_model_class(config, cls._model_mapping)
--> 561 return model_class.from_pretrained(
562 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
563 )
564 raise ValueError(
565 f"Unrecognized configuration class {config.class} for this kind of AutoModel: {cls.name}.\n"
566 f"Model type should be one of {', '.join(c.name for c in cls._model_mapping.keys())}."
567 )
File /opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3287, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
3284 with safe_open(resolved_archive_file, framework="pt") as f:
3285 metadata = f.metadata()
-> 3287 if metadata.get("format") == "pt":
3288 pass
3289 elif metadata.get("format") == "tf":
AttributeError: 'NoneType' object has no attribute 'get'
How to solve it please ?