Bug Transformers 2.3.0 - ValueError: invalid literal for int() with base 10: 'pytorch'
Author: calusbr
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): BERT

Language I am using the model on (English, Chinese....): PT-BR (Multilingual)

The problem arise when using:
* [ ] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:

CUDA_VISIBLE_DEVICES=2,3 nohup python /home/lucasrodrigues/code/transformers-2.3.0/examples/run_lm_finetuning.py \
--output_dir=/home/lucasrodrigues/train/transformers/output/multi/250k/ \
--model_type=bert \
--model_name_or_path=/home/lucasrodrigues/train/transformers/model-multi-pytorch/ \
--tokenizer_name=/home/lucasrodrigues/train/transformers/model-multi-pytorch/ \
--config_name=/home/lucasrodrigues/train/transformers/model-multi-pytorch/ \
--block_size=510 \
--do_lower_case \
--train_data_file=/home/lucasrodrigues/datasets/nilc/initial/initial_corpus_train.txt \
--eval_data_file=/home/lucasrodrigues/datasets/nilc/initial/initial_corpus_train.txt \
--do_train \
--do_eval \
--evaluate_during_training \
--logging_steps=50 \
--save_steps=50 \
--per_gpu_train_batch_size=2 \
--per_gpu_eval_batch_size=2 \
--mlm \
> logs/transformers_multi_250k.txt & 

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior
Traceback (most recent call last):
  File "/home/lucasrodrigues/code/transformers-2.3.0/examples/run_lm_finetuning.py", line 713, in <module>
    main()
  File "/home/lucasrodrigues/code/transformers-2.3.0/examples/run_lm_finetuning.py", line 663, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "/home/lucasrodrigues/code/transformers-2.3.0/examples/run_lm_finetuning.py", line 268, in train
    global_step = int(args.model_name_or_path.split("-")[-1].split("/")[0])
ValueError: invalid literal for int() with base 10: 'pytorch'

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: Ubuntu 
* Python version: 3.6
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch): 
* Using GPU ? 4x GeForce GTX 1080 Ti
* Distributed or parallel setup ? No
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

I can't execute the code, could anyone help?

Comments:
Hi, thank you for raising this issue. Could you please let me know if 27c1b656cca75efa0cc414d3bf4e6aacf24829de fixed this issue by trying the updated script?

Hello, to solve this problem I added my checkpoint to a folder that has the same Transformer output.

**new folder -> chekpoint-0**

Folders:
|
chekpoint-0
| vocab.txt
| pytorch_model.bin
| config.json

global_step = int(args.model_name_or_path.split("-")[-1].split("/")[0])

**Result:
global_step = 0**

> Hi, thank you for raising this issue. Could you please let me know if [`27c1b65`](https://github.com/huggingface/transformers/commit/27c1b656cca75efa0cc414d3bf4e6aacf24829de) fixed this issue by trying the updated script?

@LysandreJik, your commit fixed the issue for me, thanks!

> Hi, thank you for raising this issue. Could you please let me know if [27c1b65](https://github.com/huggingface/transformers/commit/27c1b656cca75efa0cc414d3bf4e6aacf24829de) fixed this issue by trying the updated script?

I think this modification is a terrible one since some people maybe download pytorch-pretrained-models like pytorch-model.bin alone in a dir, but when use this "global_step = int(args.model_name_or_path.split("-")[-1].split("/")[0])" as a number for global_step, what does it mean?@LysandreJik  like [this_issue](https://github.com/huggingface/transformers/issues/2258) said. Whoever add "global_step = int(args.model_name_or_path.split("-")[-1].split("/")[0])" could you please fix this bug? I remember there are nothing about it before.

@severinsimmler, I agree with @zysNLP, this introduces a bug when you try to use a lm that wasn't from checkpoint folder. I used `run_langauge_modeling.py` to output a lm, which I then feed into `run_glue.py`. This pipeline no longer works because `run_glue.py` is trying to parse a global step number from a folder that doesn't have one. Renaming my folder to checkpoint-0 and then feeding it into `run_glue.py` shouldn't have to be done, surely `args.model_name_or_path.split("-")[-1].split("/")[0]` can be modified slightly; so that, it returns `0` instead of `""`; so that, models in non-checkpoint folders can be added.

@stefan-it solution in https://github.com/huggingface/transformers/issues/2258 fixes the issue. This or something similar should be added to all the example scripts.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.