run_clm.py AttributeError: 'NoneType' object has no attribute 'keys'
Author: mukhtar-algezoli
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 4.2.0dev0
- Platform: Linux-4.19.0-13-cloud-amd64-x86_64-with-debian-10.7
- Python version: 3.7.8
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.4.0 (False)
- Using GPU in script?: No (using TPU)
- Using distributed or parallel set-up in script?: don't know

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 Trainer: @sgugger
 examples/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): gpt2

The problem arises when using:
* [ *] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ *] an official GLUE/SQUaD task: pre-training on wikitext dataset
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.create a tpu
2.install transformers and dataset
3.run the code
```ruby
python3 run_clm.py \
    --model_name_or_path gpt2 \
    --dataset_name wikitext \
    --dataset_config_name wikitext-2-raw-v1 \
    --do_train \
    --do_eval \
    --output_dir /tmp/test-clm
```

error message:
```ruby
Traceback (most recent call last):
  File "run_clm.py", line 51, in <module>
    MODEL_CONFIG_CLASSES = list(MODEL_FOR_CAUSAL_LM_MAPPING.keys())
AttributeError: 'NoneType' object has no attribute 'keys'
```

## Expected behavior
To start pretraining on the TPU




Comments:
Hey @mukhtar-algezoli,

It seems like you are using tensorflow, but the script `run_clm.py` only works for PyTorch -> I'm afraid you have to use PyTorch in order to use `run_clm.py`.

@patrickvonplaten thank you.

hi @patrickvonplaten ,
if i want use tensorflow to pretrain bert model with transformers , are there some examples like run_mlm.py ?

https://github.com/huggingface/transformers/tree/master/examples/tensorflow/language-modeling

use tensorflow examples get the same error, How can i fix it?

@Rocketknight1 for TF examples :-)

@hrdxwandg This is an odd issue! Can you please install the most recent version of transformers (e.g. with `pip install --upgrade transformers`) and then try running the following in a console or notebook and tell me what you see?

```
from transformers import MODEL_FOR_CAUSAL_LM_MAPPING

print(MODEL_FOR_CAUSAL_LM_MAPPING)
```

> print(MODEL_FOR_CAUSAL_LM_MAPPING)

OrderedDict([(<class 'transformers.models.roformer.configuration_roformer.RoFormerConfig'>, <class 'transformers.models.roformer.modeling_roformer.RoFormerForCausalLM'>), (<class 'transformers.models.bigbird_pegasus.configuration_bigbird_pegasus.BigBirdPegasusConfig'>, <class 'transformers.models.bigbird_pegasus.modeling_bigbird_pegasus.BigBirdPegasusForCausalLM'>), (<class 'transformers.models.gpt_neo.configuration_gpt_neo.GPTNeoConfig'>, <class 'transformers.models.gpt_neo.modeling_gpt_neo.GPTNeoForCausalLM'>), (<class 'transformers.models.big_bird.configuration_big_bird.BigBirdConfig'>, <class 'transformers.models.big_bird.modeling_big_bird.BigBirdForCausalLM'>), (<class 'transformers.models.camembert.configuration_camembert.CamembertConfig'>, <class 'transformers.models.camembert.modeling_camembert.CamembertForCausalLM'>), (<class 'transformers.models.xlm_roberta.configuration_xlm_roberta.XLMRobertaConfig'>, <class 'transformers.models.xlm_roberta.modeling_xlm_roberta.XLMRobertaForCausalLM'>), (<class 'transformers.models.roberta.configuration_roberta.RobertaConfig'>, <class 'transformers.models.roberta.modeling_roberta.RobertaForCausalLM'>), (<class 'transformers.models.bert.configuration_bert.BertConfig'>, <class 'transformers.models.bert.modeling_bert.BertLMHeadModel'>), (<class 'transformers.models.openai.configuration_openai.OpenAIGPTConfig'>, <class 'transformers.models.openai.modeling_openai.OpenAIGPTLMHeadModel'>), (<class 'transformers.models.gpt2.configuration_gpt2.GPT2Config'>, <class 'transformers.models.gpt2.modeling_gpt2.GPT2LMHeadModel'>), (<class 'transformers.models.transfo_xl.configuration_transfo_xl.TransfoXLConfig'>, <class 'transformers.models.transfo_xl.modeling_transfo_xl.TransfoXLLMHeadModel'>), (<class 'transformers.models.xlnet.configuration_xlnet.XLNetConfig'>, <class 'transformers.models.xlnet.modeling_xlnet.XLNetLMHeadModel'>), (<class 'transformers.models.xlm.configuration_xlm.XLMConfig'>, <class 'transformers.models.xlm.modeling_xlm.XLMWithLMHeadModel'>), (<class 'transformers.models.ctrl.configuration_ctrl.CTRLConfig'>, <class 'transformers.models.ctrl.modeling_ctrl.CTRLLMHeadModel'>), (<class 'transformers.models.reformer.configuration_reformer.ReformerConfig'>, <class 'transformers.models.reformer.modeling_reformer.ReformerModelWithLMHead'>), (<class 'transformers.models.bert_generation.configuration_bert_generation.BertGenerationConfig'>, <class 'transformers.models.bert_generation.modeling_bert_generation.BertGenerationDecoder'>), (<class 'transformers.models.xlm_prophetnet.configuration_xlm_prophetnet.XLMProphetNetConfig'>, <class 'transformers.models.xlm_prophetnet.modeling_xlm_prophetnet.XLMProphetNetForCausalLM'>), (<class 'transformers.models.prophetnet.configuration_prophetnet.ProphetNetConfig'>, <class 'transformers.models.prophetnet.modeling_prophetnet.ProphetNetForCausalLM'>), (<class 'transformers.models.bart.configuration_bart.BartConfig'>, <class 'transformers.models.bart.modeling_bart.BartForCausalLM'>), (<class 'transformers.models.mbart.configuration_mbart.MBartConfig'>, <class 'transformers.models.mbart.modeling_mbart.MBartForCausalLM'>), (<class 'transformers.models.pegasus.configuration_pegasus.PegasusConfig'>, <class 'transformers.models.pegasus.modeling_pegasus.PegasusForCausalLM'>), (<class 'transformers.models.marian.configuration_marian.MarianConfig'>, <class 'transformers.models.marian.modeling_marian.MarianForCausalLM'>), (<class 'transformers.models.blenderbot.configuration_blenderbot.BlenderbotConfig'>, <class 'transformers.models.blenderbot.modeling_blenderbot.BlenderbotForCausalLM'>), (<class 'transformers.models.blenderbot_small.configuration_blenderbot_small.BlenderbotSmallConfig'>, <class 'transformers.models.blenderbot_small.modeling_blenderbot_small.BlenderbotSmallForCausalLM'>), (<class 'transformers.models.megatron_bert.configuration_megatron_bert.MegatronBertConfig'>, <class 'transformers.models.megatron_bert.modeling_megatron_bert.MegatronBertForCausalLM'>)])

That's extremely strange - the error above said that `MODEL_FOR_CAUSAL_LM_MAPPING` was `None`, but your code there clearly shows that it's importing the dict fine.

Are you sure you're getting the same error as the commenter above?

> That's extremely strange - the error above said that `MODEL_FOR_CAUSAL_LM_MAPPING` was `None`, but your code there clearly shows that it's importing the dict fine.
> 
> Are you sure you're getting the same error as the commenter above?

sorry, I don't check clearly. My error msg is below:
Traceback (most recent call last):
  File "run_mlm_tf.py", line 63, in <module>
    MODEL_CONFIG_CLASSES = list(MODEL_FOR_MASKED_LM_MAPPING.keys())
AttributeError: 'NoneType' object has no attribute 'keys'

I use your method and recheck:
from transformers import MODEL_FOR_MASKED_LM_MAPPING
print(MODEL_FOR_MASKED_LM_MAPPING)

the output is None

Hi, sorry! I lost track of this issue because we marked it as closed, but I see we didn't resolve @hrdxwandg's problem. Are you still encountering the same error?

@Rocketknight1 ; sorry to bump this, but I am getting the same error with the latest transformers version