Error training GPT-2 from scratch on Hindi
Author: parthplc
Pull request: no

I was trying to retrain GPT-2 from scratch. I was able able to train a tokenizer but was facing issues while running run_language_modeling.py.
```
2020-09-29 06:00:14.450718: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
/usr/local/lib/python3.6/dist-packages/transformers/training_args.py:299: FutureWarning: The `evaluate_during_training` argument is deprecated in favor of `evaluation_strategy` (which has more options)
  FutureWarning,
09/29/2020 06:00:16 - WARNING - __main__ -   Process rank: -1, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: False
09/29/2020 06:00:16 - INFO - __main__ -   Training/evaluation parameters TrainingArguments(output_dir='/content/data', overwrite_output_dir=True, do_train=True, do_eval=True, do_predict=False, evaluate_during_training=True, evaluation_strategy=<EvaluationStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=2, per_device_eval_batch_size=2, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=1, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=5.0, max_steps=-1, warmup_steps=0, logging_dir='runs/Sep29_06-00-16_fcba31604e1d', logging_first_step=False, logging_steps=500, save_steps=500, save_total_limit=2, no_cuda=False, seed=108, fp16=False, fp16_opt_level='O1', local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=500, dataloader_num_workers=0, past_index=-1, run_name=None, disable_tqdm=False, remove_unused_columns=True, label_names=None)
Traceback (most recent call last):
  File "/content/transformers/examples/language-modeling/run_language_modeling.py", line 313, in <module>
    main()
  File "/content/transformers/examples/language-modeling/run_language_modeling.py", line 205, in main
    tokenizer = AutoTokenizer.from_pretrained(model_args.tokenizer_name, cache_dir=model_args.cache_dir)
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_auto.py", line 251, in from_pretrained
    return tokenizer_class_py.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py", line 1428, in from_pretrained
    return cls._from_pretrained(*inputs, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py", line 1575, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_albert.py", line 155, in __init__
    self.sp_model.Load(vocab_file)
  File "/usr/local/lib/python3.6/dist-packages/sentencepiece.py", line 367, in Load
    return self.LoadFromFile(model_file)
  File "/usr/local/lib/python3.6/dist-packages/sentencepiece.py", line 177, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
TypeError: not a string
```
Here is the link to my colab file. https://colab.research.google.com/drive/1rWHwWCB_U_rTOnfGyXdgZ9Kb4HpbNdVs?usp=sharing

Comments:
You seem to have a `"model_type": "albert"` in your config.json which should be a `gpt2`.

Also, I would suggest using the Trainer directly instead of shelling out to `run_language_modeling.py`, as described in https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb

Finally, this question's probably better suited to the [Forum](http://discuss.huggingface.co/), please ask over there!