Use Distilbert to run language model, encounter error  "Unrecognized configuration class "  
Author: OleNet
Pull request: no


- `transformers` version: 4.6.1
- Platform: centos 7.5
- Python version: 3.7
- PyTorch version (GPU?): 1.10
- Using GPU in script?: v100
- Using distributed or parallel set-up in script?: no

## To reproduce
```
python3 run_clm.py \
    --model_name_or_path distilbert-base-uncased \
    --dataset_name wikitext \
    --dataset_config_name wikitext-2-raw-v1 \
    --do_train \
    --do_eval \
    --output_dir ./tmp/test-clm
```

```
[INFO|tokenization_utils_base.py:1717] 2021-06-09 17:11:05,729 >> loading file https://huggingface.co/distilbert-base-uncased/resolve/main/vocab.txt from cache at /home/work/liujiaxiang/.cache/huggingface/transformers/0e1bbfda7f63a99bb52e3915dcf10c3c92122b827d92eb2d34ce94ee79ba486c.d789d64ebfe299b0e416afc4a169632f903f693095b4629a7ea271d5a0cf2c99
[INFO|tokenization_utils_base.py:1717] 2021-06-09 17:11:05,730 >> loading file https://huggingface.co/distilbert-base-uncased/resolve/main/tokenizer.json from cache at /home/work/liujiaxiang/.cache/huggingface/transformers/75abb59d7a06f4f640158a9bfcde005264e59e8d566781ab1415b139d2e4c603.7f2721073f19841be16f41b0a70b600ca6b880c8f3df6f3535cbc704371bdfa4
[INFO|tokenization_utils_base.py:1717] 2021-06-09 17:11:05,730 >> loading file https://huggingface.co/distilbert-base-uncased/resolve/main/added_tokens.json from cache at None
[INFO|tokenization_utils_base.py:1717] 2021-06-09 17:11:05,730 >> loading file https://huggingface.co/distilbert-base-uncased/resolve/main/special_tokens_map.json from cache at None
[INFO|tokenization_utils_base.py:1717] 2021-06-09 17:11:05,730 >> loading file https://huggingface.co/distilbert-base-uncased/resolve/main/tokenizer_config.json from cache at /home/work/liujiaxiang/.cache/huggingface/transformers/8c8624b8ac8aa99c60c912161f8332de003484428c47906d7ff7eb7f73eecdbb.20430bd8e10ef77a7d2977accefe796051e01bc2fc4aa146bc862997a1a15e79
Traceback (most recent call last):
  File "run_clm.py", line 536, in <module>
    main()
  File "run_clm.py", line 322, in main
    use_auth_token=True if model_args.use_auth_token else None,
  File "/ssd2/liujiaxiang/workfiles/transformer_invariant_bigger/transformers/src/transformers/models/auto/auto_factory.py", line 397, in from_pretrained
    f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
ValueError: Unrecognized configuration class <class 'transformers.models.distilbert.configuration_distilbert.DistilBertConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of RoFormerConfig, BigBirdPegasusConfig, GPTNeoConfig, BigBirdConfig, CamembertConfig, XLMRobertaConfig, RobertaConfig, BertConfig, OpenAIGPTConfig, GPT2Config, TransfoXLConfig, XLNetConfig, XLMConfig, CTRLConfig, ReformerConfig, BertGenerationConfig, XLMProphetNetConfig, ProphetNetConfig, BartConfig, MBartConfig, PegasusConfig, MarianConfig, BlenderbotConfig, BlenderbotSmallConfig, MegatronBertConfig.
```

Comments:
As printed by the error, DistilBERT is not supported by `AutoModelForCausalLM`, since it's an encoder-only model. Please use one of the supported models to perform autoregressive (i.e. left-to-right) language modeling.

Why the  [README](https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling#robertabertdistilbert-and-masked-language-modeling) have gave a example of distilbert ? 

The example you refer to is `run_mlm.py` (mlm is short for masked language modeling). However, the script you're using above is `run_clm.py` (clm is short for causal language modeling, also called autoregressive language modeling). DistilBERT only supports mlm, not clm. 

Ah! yes, you are right!
My fault !
Thanks for your answer.