run_clm.py Early stopping with ^C
Author: Clickative
Pull request: no

- `transformers` version: 4.0.1
- Platform: Colab
- Python version: 3.6.9
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger

## Information

Model I am using: GPT2

The problem arises when using:
`run_clm.py`

## To reproduce
`!python ./transformers/examples/language-modeling/run_clm.py \
    --model_name_or_path ./GPT2_PRETRAINED_LOCAL \
    --dataset_name bookcorpusopen \
    --dataset_config_name plain_text \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --block_size 128 \
    --gradient_accumulation_steps 1 \
    --overwrite_output_dir \
    --do_train \
    --do_eval \
    --num_train_epochs 20 \
    --save_steps 50000 \
    --save_total_limit 1 \
    --output_dir ./GPT2-trained-save`

Output:
`2020-12-13 20:02:51.391764: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
12/13/2020 20:02:53 - WARNING - __main__ -   Process rank: -1, device: cpu, n_gpu: 0distributed training: False, 16-bits training: False
12/13/2020 20:02:53 - INFO - __main__ -   Training/evaluation parameters TrainingArguments(output_dir='./RPT-trained-save', overwrite_output_dir=True, do_train=True, do_eval=True, do_predict=False, evaluation_strategy=<EvaluationStrategy.NO: 'no'>, prediction_loss_only=False, per_device_train_batch_size=2, per_device_eval_batch_size=2, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=1, eval_accumulation_steps=None, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=20.0, max_steps=-1, warmup_steps=0, logging_dir='runs/Dec13_20-02-53_7d34d2e22bee', logging_first_step=False, logging_steps=500, save_steps=50000, save_total_limit=1, no_cuda=False, seed=42, fp16=False, fp16_opt_level='O1', local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=500, dataloader_num_workers=0, past_index=-1, run_name='./RPT-trained-save', disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=None)
Downloading: 4.00kB [00:00, 2.73MB/s]       
Downloading: 2.10kB [00:00, 1.47MB/s]       
Downloading and preparing dataset book_corpus_open/plain_text (download: 2.24 GiB, generated: 6.19 GiB, post-processed: Unknown size, total: 8.43 GiB) to /root/.cache/huggingface/datasets/book_corpus_open/plain_text/1.0.0/5cc3e4620a202388e77500f913b37532be8b036287436f3365e066671a1bd97e...
Downloading: 100% 2.40G/2.40G [02:41<00:00, 14.9MB/s]
9990 examples [01:04, 149.50 examples/s]^C`

The ^C automatically appears and the script stops.

## Expected behavior
The training process takes place as normal.


Comments:
^C means you have hit Ctrl + C on your machine and stops the command running. You should re-run the command without hitting Ctrl + C.

Yup I am aware that ^C is a halt command. I am running this on colab and I have tried to run this 5-7 times now, not hitting Ctrl+C once. For some reason it appears itself and halts the execution.  

There might be something in colab that aborts bash command after some time then, or it happens when the session disconnects. But there is absolutely nothing in the script that triggers a cancel like this, so there is nothing we can do to fix this.

Note that the scripts are not meant to be run on Colab, we have [notebook versions](https://github.com/huggingface/notebooks/tree/master/examples) of them for that.

I think I have figured out the issue. This is happening because the dataset is large and when the full thing is loaded, colab crashes.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.