run_squad.py not working on 3.1.0 version
Author: deepanshu650
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Linux-4.15.0-91-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): 2.1.0 (False)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: True

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->@LysandreJik @sshleifer

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
 run_squad.py is working with 2.9.1 but when I update it to 3.1.0 it gives error

* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQUaD 
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. !pip install transformers==3.1.0
It works when using 2.9.1 but not with this
2.
!mkdir dataset \
&& cd dataset \
&& wget https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json \
&& wget https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json
3.
!export SQUAD_DIR=/content/dataset \
&& python transformers/examples/run_squad.py \
  --model_type bert \
  --model_name_or_path bert-base-uncased \
  --do_train \
  --do_eval \
  --do_lower_case \
  --train_file $SQUAD_DIR/train-v2.0.json \
  --predict_file $SQUAD_DIR/dev-v2.0.json \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 1.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /content/model_output \
  --save_steps 1000 \
  --threads 4 \
  --version_2_with_negative 

Error message is: 
2020-09-07 19:33:26.850641: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
  File "transformers/examples/run_squad.py", line 74, in <module>
    (),
  File "transformers/examples/run_squad.py", line 73, in <genexpr>
    (tuple(conf.pretrained_config_archive_map.keys()) for conf in (BertConfig, RobertaConfig, XLNetConfig, XLMConfig)),
AttributeError: type object 'BertConfig' has no attribute 'pretrained_config_archive_map'

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. --> It should start training without error just like older versions


Comments:
Indeed, it seems it hasn't been up to date. Did you try running `run_squad_trainer.py`? It should be more up to date.

@sgugger we should probably deprecate `run_squad.py` now that we have a Trainer-based SQuAD script.

The missing part was the eval which shouldn't be too hard to add (https://github.com/huggingface/transformers/pull/4829#issuecomment-645994130)

And then we can rename files as in https://github.com/huggingface/transformers/pull/5582

> Indeed, it seems it hasn't been up to date. Did you try running `run_squad_trainer.py`? It should be more up to date.
> 
> @sgugger we should probably deprecate `run_squad.py` now that we have a Trainer-based SQuAD script.

No run_squad_trainer.py also gives error in 3.1.0
`!python transformers/examples/question-answering/run_squad_trainer.py --help
`
Error is 
`python3: can't open file 'transformers/examples/question-answering/run_squad_trainer.py': [Errno 2] No such file or directory`

@deepanshu650 the file exists, it's [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad_trainer.py). Are you sure you cloned the v3.1.0 repo?

Yes it's working , initially I was working on copy of someone's notebook which was installing v2.3.0 and I had to pip install v3.1.0, so after changing to new notebook it starts running.
But it doesn't give evaluation results(which `run_squad.py` does) as `run_squad_trainer.py` not calling trainer.evaluate() though it makes eval_dataset .
So how do I evaluate . Thanks for replying.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.