ImportError: cannot import name 'DataCollatorForLanguageModeling'_File "run_language_modeling.py"
Author: Kittisaksam
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
_https://huggingface.co/blog/how-to-train_
I followed everything in this colab without changing anything.
And this is the problem I encountered.

**Traceback (most recent call last):
  File "run_language_modeling.py", line 29, in <module>
    from transformers import (
ImportError: cannot import name 'DataCollatorForLanguageModeling'
CPU times: user 21.5 ms, sys: 18 ms, total: 39.5 ms
Wall time: 4.52 s**

How can I fix this problem?

Thank you for your kindness and support
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->


Comments:
Unfortunatelly I have the same error

Traceback (most recent call last):
  File "run_language_modeling.py", line 29, in <module>
    from transformers import (
ImportError: cannot import name 'DataCollatorForLanguageModeling' from 'transformers' (E:\PycharmProjects\Ancient_BERT\venv\lib\site-packages\transformers\__init__.py)

> Unfortunatelly I have the same error
> 
> Traceback (most recent call last):
> File "run_language_modeling.py", line 29, in
> from transformers import (
> ImportError: cannot import name 'DataCollatorForLanguageModeling' from 'transformers' (E:\PycharmProjects\Ancient_BERT\venv\lib\site-packages\transformers__init__.py)

I tried to do this (instead of !pip install transformers)

!git clone https://github.com/huggingface/transformers
cd transformers
pip install .

And I get the following error:

**Traceback (most recent call last):
File "run_language_modeling.py", line 280, in
main()
File "run_language_modeling.py", line 225, in main
if training_args.do_train
File "run_language_modeling.py", line 122, in get_dataset
tokenizer=tokenizer, file_path=file_path, block_size=args.block_size, local_rank=local_rank
File "/usr/local/lib/python3.6/dist-packages/transformers/data/datasets/language_modeling.py", line 84, in init
assert os.path.isfile(file_path)
AssertionError**

I think it should be about **tokenizers-0.7.0.**
But I now still don't know how to fix it.

Both PyCharm running and example in Colab script has the same problem:
https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb

Traceback (most recent call last):
  File "run_language_modeling.py", line 29, in <module>
    from transformers import (
ImportError: cannot import name 'DataCollatorForLanguageModeling'
CPU times: user 36.2 ms, sys: 22.9 ms, total: 59.1 ms
Wall time: 15.2 s

I change in Colab installation command to: 
!pip install git+https://github.com/huggingface/transformers

and its running :), now only to solve problem with out of memory

> !pip install git+https://github.com/huggingface/transformers

You have to change batch size.

cmd =	"""
  python run_language_modeling.py
  --train_data_file ./oscar.eo.txt
  --output_dir ./EsperBERTo-small-v1
	--model_type roberta
	--mlm
	--config_name ./EsperBERTo
	--tokenizer_name ./EsperBERTo
	--do_train
	--line_by_line
	--learning_rate 1e-4
	--num_train_epochs 1
	--save_total_limit 2
	--save_steps 2000
	**--per_gpu_train_batch_size 4**
	--seed 42
""".replace("\n", " ")

And Thank you for your help.

Maybe somone had error (error occurs after reaching save step value=2000):

Iteration:  16% 1997/12500 [10:38<49:42,  3.52it/s]
Iteration:  16% 1998/12500 [10:39<47:03,  3.72it/s]
                                
{"learning_rate": 8.4e-05, "loss": 7.684231301307678, "step": 2000}
Epoch:   0% 0/1 [10:39<?, ?it/s]
Iteration:  16% 1999/12500 [10:39<49:56,  3.50it/s]04/23/2020 11:55:42 - INFO - transformers.trainer -   Saving model checkpoint to ./EsperBERTo-small-v1/checkpoint-2000
04/23/2020 11:55:42 - INFO - transformers.configuration_utils -   Configuration saved in ./EsperBERTo-small-v1/checkpoint-2000/config.json
04/23/2020 11:55:42 - INFO - transformers.modeling_utils -   Model weights saved in ./EsperBERTo-small-v1/checkpoint-2000/pytorch_model.bin
Traceback (most recent call last):
  File "run_language_modeling.py", line 280, in <module>
    main()
  File "run_language_modeling.py", line 254, in main
    trainer.train(model_path=model_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 363, in train
    self._rotate_checkpoints()
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 458, in _rotate_checkpoints
    checkpoints_sorted = self._sorted_checkpoints(use_mtime=use_mtime)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 443, in _sorted_checkpoints
    regex_match = re.match(".*{}-([0-9]+)".format(checkpoint_prefix), path)
  File "/usr/lib/python3.6/re.py", line 172, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or bytes-like object

Epoch:   0% 0/1 [10:40<?, ?it/s]
Iteration:  16% 1999/12500 [10:40<56:04,  3.12it/s]
CPU times: user 2.78 s, sys: 928 ms, total: 3.71 s
Wall time: 11min 33s


My configuration is:
config = {
	"architectures": [
		"RobertaForMaskedLM"
	],
	"attention_probs_dropout_prob": 0.1,
	"hidden_act": "gelu",
	"hidden_dropout_prob": 0.1,
	"hidden_size": 768,
	"initializer_range": 0.02,
	"intermediate_size": 3072,
	"layer_norm_eps": 1e-05,
	"max_position_embeddings": 514,
	"model_type": "roberta",
	"num_attention_heads": 12,
	"num_hidden_layers": 6,
	"type_vocab_size": 1,
	"vocab_size": 52000
}
cmd = """
python run_language_modeling.py
--train_data_file ./oscar.eo.txt
--output_dir ./EsperBERTo-small-v1
--model_type roberta
--mlm
--config_name ./EsperBERTo
--tokenizer_name ./EsperBERTo
--do_train
--line_by_line
--learning_rate 1e-4
--num_train_epochs 1
--save_total_limit 2
--save_steps 2000
--per_gpu_train_batch_size 4  
--seed 42
""".replace("\n", " ")


Closing in favor of #3920 

How to solve this? Why is this closed?