can't load the llama-3.1-8b-instruct model
Author: deven367
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.43.2
- Platform: Linux-5.3.18-150300.59.90-default-x86_64-with-glibc2.31
- Python version: 3.10.9
- Huggingface_hub version: 0.24.2
- Safetensors version: 0.4.2
- Accelerate version: 0.28.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.2.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA A100-SXM4-40GB

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

To reproduce the issue
```py
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
```


```py
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[6], line 2
      1 tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
----> 2 model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")

File ~/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:523, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    520 if kwargs.get("quantization_config", None) is not None:
    521     _ = kwargs.pop("quantization_config")
--> 523 config, kwargs = AutoConfig.from_pretrained(
    524     pretrained_model_name_or_path,
    525     return_unused_kwargs=True,
    526     trust_remote_code=trust_remote_code,
    527     code_revision=code_revision,
    528     _commit_hash=commit_hash,
    529     **hub_kwargs,
    530     **kwargs,
    531 )
    533 # if torch_dtype=auto was passed here, ensure to pass it on
    534 if kwargs_orig.get("torch_dtype", None) == "auto":

File ~/.local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1147, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
   1141     except KeyError:
   1142         raise ValueError(
   1143             f"The checkpoint you are trying to load has model type `{config_dict['model_type']}` "
   1144             "but Transformers does not recognize this architecture. This could be because of an "
   1145             "issue with the checkpoint, or because your version of Transformers is out of date."
   1146         )
-> 1147     return config_class.from_dict(config_dict, **unused_kwargs)
   1148 else:
   1149     # Fallback: use pattern matching on the string.
   1150     # We go from longer names to shorter names to catch roberta before bert (for instance)
   1151     for pattern in sorted(CONFIG_MAPPING.keys(), key=len, reverse=True):

File ~/.local/lib/python3.10/site-packages/transformers/configuration_utils.py:761, in PretrainedConfig.from_dict(cls, config_dict, **kwargs)
    758 # We remove it from kwargs so that it does not appear in `return_unused_kwargs`.
    759 config_dict["attn_implementation"] = kwargs.pop("attn_implementation", None)
--> 761 config = cls(**config_dict)
    763 if hasattr(config, "pruned_heads"):
    764     config.pruned_heads = {int(key): value for key, value in config.pruned_heads.items()}

File ~/.local/lib/python3.10/site-packages/transformers/models/llama/configuration_llama.py:160, in LlamaConfig.__init__(self, vocab_size, hidden_size, intermediate_size, num_hidden_layers, num_attention_heads, num_key_value_heads, hidden_act, max_position_embeddings, initializer_range, rms_norm_eps, use_cache, pad_token_id, bos_token_id, eos_token_id, pretraining_tp, tie_word_embeddings, rope_theta, rope_scaling, attention_bias, attention_dropout, **kwargs)
    158 self.rope_theta = rope_theta
    159 self.rope_scaling = rope_scaling
--> 160 self._rope_scaling_validation()
    161 self.attention_bias = attention_bias
    162 self.attention_dropout = attention_dropout

File ~/.local/lib/python3.10/site-packages/transformers/models/llama/configuration_llama.py:180, in LlamaConfig._rope_scaling_validation(self)
    177     return
    179 if not isinstance(self.rope_scaling, dict) or len(self.rope_scaling) != 2:
--> 180     raise ValueError(
    181         "`rope_scaling` must be a dictionary with with two fields, `type` and `factor`, "
    182         f"got {self.rope_scaling}"
    183     )
    184 rope_scaling_type = self.rope_scaling.get("type", None)
    185 rope_scaling_factor = self.rope_scaling.get("factor", None)

ValueError: `rope_scaling` must be a dictionary with with two fields, `type` and `factor`, got {'factor': 8.0, 'low_freq_factor': 1.0, 'high_freq_factor': 4.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}

```

### Expected behavior

The model should load as expected and the code should run.

Comments:
just ```pip install --upgrade transformers```

refer to https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/discussions/15


Yes, I did that. I am still running into the same issue. I am running the latest version of `transformers`.


I even updated the `trl`, `vllm` and `accelerate` packages. but even that did not help.

Check out whether you are indeed running the latest version of transformers by `print(transformers.__version__)`.

Thanks @ArthurinRUC @qyy2003, I'm not exactly sure what the issue was, but I was able to run the model after I installed the library in a new virtual env. I will go ahead and close the issue.

i am facing the same issue :-) and have transformers 4.43.3 , with the error you see the print out of the version from transformer...
temporary solution i use currently to at least get the model is downloading everything with git-lfs :-(

the download for lama3-8b works, but when the name is changed to llama3.1 8b it wont work i did copy the name for the model from the URL as i always do.

meta-llama/Meta-Llama-3-8B-Instruct
meta-llama/Meta-Llama-3.1-8B-Instruct

Is it possible that the "dot" in the naming convention makes the transformer lib a little angry? Because other models (not in the list) work even if they are new, but none of them have a "dot" in the name

![image](https://github.com/user-attachments/assets/2712b8a5-3f36-47e0-a4cf-e7767589fcc7)


@snapo Well... It seems like you are using TransformerLens package which has its own supported models here: https://transformerlensorg.github.io/TransformerLens/generated/model_properties_table.html. Currently Llama3.1 is not on the list :)

Ups... :-) did think Transformer library is transformer library... didnt expect they where different. sorry for disturbance :-)