run_mlm.py: error: argument
Author: Shafi2016
Pull request: no

I just found today that transformers/examples/language-modeling/run_language_modeling.py"  has now been changed to transformers/examples/language-modeling. Everything fine for me when was using run_language_modeling.py. But with the new change, I am getting the error that, **run_mlm.py: error: argument --mlm_probability: expected one argument** I could not find the reason despite spending many hours on it.

!python "/content/transformers/examples/language-modeling/run_mlm.py" \
    --output_dir "/content/drive/My Drive/Ottawa_citit" \
    --model_type roberta \
    --model_name_or_path roberta-base \
    --do_train \
    --per_gpu_train_batch_size 16 \
    --seed 42 \
    --train_data_file "/content/input_text.txt" \
    --block_size 512 \
    --line_by_line \
    --learning_rate 6e-4 \
    --num_train_epochs 4 \
    --save_total_limit 2 \
    --run_name high_mlm_prob \
    --save_steps 200 \
    --mlm\
    --weight_decay 0.01 \
    --mlm_probability 0.15 

Thanks!!

Comments:
Note that the old script is still available [here](https://github.com/huggingface/transformers/blob/master/examples/contrib/legacy/run_language_modeling.py).

The new one expects slightly different arguments:
- you should remove the `mlm` flag now, since the script has been split in several others
- `train_data_file` is now `train_file`
- there is no `model_type` argument anymore as this is not necessary when we have the model checkpoint
- `block_size` has been renamed `max_seq_length`

Thank you so much!!!

You saved my day 

I am getting the following error while running the below code. Could anyone please help me with this?

!python "/content/transformers/examples/language-modeling/run_mlm.py" \
--output_dir "/content/drive/MyDrive/Bert_models/test-mlm" \
--model_name_or_path "/content/bert-base-uncased" \
--do_train \
--do_eval \
--train_file "/content/train.txt" \
--validation_file "/content/test.txt" \


Error:
[INFO|modeling_utils.py:1022] 2020-12-26 10:18:28,002 >> loading weights file /content/bert-base-uncased/pytorch_model.bin
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py", line 1035, in from_pretrained
    state_dict = torch.load(resolved_archive_file, map_location="cpu")
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 764, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/content/transformers/examples/language-modeling/run_mlm.py", line 420, in <module>
    main()
  File "/content/transformers/examples/language-modeling/run_mlm.py", line 264, in main
    cache_dir=model_args.cache_dir,
  File "/usr/local/lib/python3.6/dist-packages/transformers/models/auto/modeling_auto.py", line 1092, in from_pretrained
    pretrained_model_name_or_path, *model_args, config=config, **kwargs
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py", line 1038, in from_pretrained
    f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
OSError: Unable to load weights from pytorch checkpoint file for '/content/bert-base-uncased' at '/content/bert-base-uncased/pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.