[Bug] KeyError: 'nllb-moe' when trying to load `nllb-moe-54b` model
Author: NanoCode012
Pull request: no

### System Info

- `transformers` version: 4.28.0.dev0
- Platform: Linux-5.4.0-74-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- Huggingface_hub version: 0.13.3
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker from https://github.com/huggingface/transformers/pull/22024

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Following example script on https://huggingface.co/facebook/nllb-moe-54b (but pointing to local git copy),

1. `pip install git+https://github.com/huggingface/transformers.git`
2. `python`
```py
>>> from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("../hub/nllb-moe-54b")
>>> model = AutoModelForSeq2SeqLM.from_pretrained("../hub/nllb-moe-54b")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 920, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
  File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 626, in __getitem__
    raise KeyError(key)
KeyError: 'nllb_moe'
```


Note: The system might not have enough RAM, but this errored immediately after reaching it and does not seem like OOM.

### Expected behavior

It can load model.



Comments:
That's completely right! The `config.model_type` should be `nllb-moe` instead of `nllb_moe`. Will modify this in the checkpoints and in the code. Thanks for reporting! 

@ArthurZucker , hello!

I noticed that and have also attempted that, but I got the same error weirdly. I will try it again later.

It is the config.json right?

Yes the `config.json` was wrong! 

Hello @ArthurZucker , sorry for bothering you again.

I have `git pull` the latest Huggingface repo and still got same error.

```py
>>> tokenizer = AutoTokenizer.from_pretrained("../hub/nllb-moe-54b", use_auth_token=True, src_lang="eng_Latn")
>>> model = AutoModelForSeq2SeqLM.from_pretrained("../hub/nllb-moe-54b")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 920, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
  File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 626, in __getitem__
    raise KeyError(key)
KeyError: 'nllb-moe'
```

Do I need to install from your branch https://github.com/huggingface/transformers/pull/22470?

Edit: Oh, it was just merged 1 min ago.


This is normal! You need to update your config.json file

If you were using a hub model, it would automatically update. The PR fixes the default value but for models that were already downloaded you need to update the config

> If you were using a hub model, it would automatically update. The PR fixes the default value but for models that were already downloaded you need to update the config

Yes, I tried both
1) Updated config.json
2) git pull the downloaded HF repo with the model https://huggingface.co/facebook/nllb-moe-54b/commit/83c96e4658a2e02c182d0ab794229301862791ee (not the transformers).

I'm not sure if it cached the config.json somewhere?

Edit: Will pip install latest transformer from source.

Hm, I have pip install from source and also confirmed that `config.json` got updated.

```bash
Unpacking objects: 100% (3/3), 342 bytes | 0 bytes/s, done.
From https://huggingface.co/facebook/nllb-moe-54b
   59fc265..83c96e4  main       -> origin/main
Updating 59fc265..83c96e4
Fast-forward
 config.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
```

<img width="771" alt="image" src="https://user-images.githubusercontent.com/9899957/228855893-9cd170aa-4359-42f1-9f2e-307bac59ca95.png">



> Hello @ArthurZucker , sorry for bothering you again.
> 
> I have `git pull` the latest Huggingface repo and still got same error.
> 
> ```python
> >>> tokenizer = AutoTokenizer.from_pretrained("../hub/nllb-moe-54b", use_auth_token=True, src_lang="eng_Latn")
> >>> model = AutoModelForSeq2SeqLM.from_pretrained("../hub/nllb-moe-54b")
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained
>     config, kwargs = AutoConfig.from_pretrained(
>   File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 920, in from_pretrained
>     config_class = CONFIG_MAPPING[config_dict["model_type"]]
>   File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 626, in __getitem__
>     raise KeyError(key)
> KeyError: 'nllb-moe'
> ```
> 
> Do I need to install from your branch #22470?
> 
> Edit: Oh, it was just merged 1 min ago.

I just saw. The key error is now `nllb-moe`. It is not the same error as the first post which was `nllb_moe`.

Okay, let me have another look!

> Okay, let me have another look!

Sorry for disturbing. Thank you very much!

So, running this `model = AutoModelForSeq2SeqLM.from_pretrained("hf-internal-testing/random-nllb-moe-2-experts")` definitely worked for me.  

```python 
In [3]: model = AutoModelForSeq2SeqLM.from_pretrained("hf-internal-testing/random-nllb-moe-2-experts")
Downloading (…)lve/main/config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.40k/1.40k [00:00<00:00, 272kB/s]
Downloading (…)model.bin.index.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 91.5k/91.5k [00:00<00:00, 992kB/s]
Downloading (…)00001-of-00002.bin";: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.75G/7.75G [02:04<00:00, 62.0MB/s]
Downloading (…)00002-of-00002.bin";: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9.36G/9.36G [02:17<00:00, 68.0MB/s]
Downloading shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [04:23<00:00, 131.96s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:11<00:00,  5.82s/it]

In [4]: 
```
The issue is most probably related to the config/ the cache! But still will look into it. In the mean time use the model directly 😉 

Hello @ArthurZucker , thank you for info!

> Hello 你好你好@ArthurZucker , thank you for info!，谢谢你的信息！，谢谢你的信息！

Is the problem solved?


> > Hello 你好你好@ArthurZucker , thank you for info!，谢谢你的信息！，谢谢你的信息！
> 
> Is the problem solved?

Hey! I have not tried this yet. I think it could've been fixed. I probably had some caching issue with packages. 

I have not been recently able to get a machine to run this yet.

> > > Hello 你好你好@ArthurZucker , thank you for info!，谢谢你的信息！，谢谢你的信息！
> > 
> > 
> > Is the problem solved?
> 
> Hey! I have not tried this yet. I think it could've been fixed. I probably had some caching issue with packages.
> 
> I have not been recently able to get a machine to run this yet.


I have the same problem, I think changing the config file "nllb-moe" is not the solution, I tried many times, I am not cached, the first time I use


Hey! Really sorry but I can't reproduce this now : https://colab.research.google.com/drive/1uoAKGbkJA4rnZV9Lwg1unOvvEloudcvM?usp=sharing 

This notebook works as expected out of the box.
I am pretty sur it is either:
- you are not using the `main` transformers branch
- your file is not well defined

> Hey! Really sorry but I can't reproduce this now : 嘿！真的很抱歉，但我现在无法重现这个：嘿！真的很抱歉，但我现在无法重现这个：[https://colab.research.google.com/drive/1uoAKGbkJA4rnZV9Lwg1unOvvEloudcvM?usp=sharinghttps://colab.research.google.com/drive/1uoAKGbkJA4rnZV9Lwg1unOvvEloudcvM?usp=sharinghttps://colab.research.google.com/drive/1uoAKGbkJA4rnZV9Lwg1unOvvEloudcvM?usp=sharing](https://colab.research.google.com/drive/1uoAKGbkJA4rnZV9Lwg1unOvvEloudcvM?usp=sharing)
> 
> This notebook works as expected out of the box.此笔记本开箱即用，按预期工作。此笔记本开箱即用，按预期工作。 I am pretty sur it is either: 我很确定它是： 我很确定它是：
> 
> * you are not using the 您没有使用 您没有使用 `main` transformers branch 变压器分支 变压器分支
> * your file is not well defined您的文件未明确定义您的文件未明确定义

Thanks, I'm trying。I see that your model is "hf-internal-testing/random-nllb-moe-2-experts" 。Can you try the "facebook/nllb-moe-54b" model?

Just did, it works the same

> 

OK，Thanks, I'm trying

![image](https://user-images.githubusercontent.com/46487979/233106293-ec1e054e-9ec8-4b7c-bb72-47eaed95889b.png)
I have the same problem. I downloaded it separately and tried to make it work directly, but it still didn't work. Any idea when this will be fixed?

> ![image](https://user-images.githubusercontent.com/46487979/233106293-ec1e054e-9ec8-4b7c-bb72-47eaed95889b.png) I have the same problem. I downloaded it separately and tried to make it work directly, but it still didn't work. Any idea when this will be fixed? 我有同样的问题。我单独下载了它并试图让它直接工作，但它仍然不起作用。知道什么时候会解决这个问题吗？ 我有同样的问题。我单独下载了它并试图让它直接工作，但它仍然不起作用。知道什么时候会解决这个问题吗？

me too

are you sure that you are on the latest release of transformers? 
`pip install --upgrade transformers`

> are you sure that you are on the latest release of transformers? `pip install --upgrade transformers`

Wow, I had forgotten about this, but after trying it, I ran it and it works fine, thank you very much.