Hard-coded force_download in run_squad forces expensive community download 
Author: mfeblowitz
Pull request: no
Labels: name: Ex: Question Answering

name: Usage

# 🐛 Bug

## Information
Using a community-registered model (albert, squad2) I noticed that there's no real caching going on during predict/evaluate. In an application that invokes run_squad dozens to hundreds of times, this adds significantly to processing time.

This is due to at least one of the two force_download hard-codings in the run_squad.py script. It would be best to promote the force_download option into the run_squad arguments and let the user override. 

I have tested by manually modifying the force_download to be False and caching does occur (I haven't tested dirty cache refetch).

Model I am using (Bert, XLNet ...): Community-submitted Albert v2 xxlarge fine-tuned for SQuAD2 on Torch, https://huggingface.co/mfeb/albert-xxlarge-v2-squad2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: 
* [ ] my own task or dataset: (give details below)

## To reproduce
Use run_squad.py more than once on a community-installed model and see the fetch go to a different /tmp copy for each invocation.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Expect force_download to be overridable at run_squad invocation, especially for community-registered models.

<!-- A clear and concise description of what you would expect to happen. -->
Update run_squad.py to add a force_download argument (default of your choosing) and use the result in the two places force_download is hard coded.

Better might be for the default value to be determined from the nature of the model location/type (e.g., no forced download for non-local models).
 
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: v2.2.1
- Platform: ubuntu
- Python version: 3.7.7
- PyTorch version (GPU?): gpu 1.3.1
- Tensorflow version (GPU?): gpu 2.0.0
- Using GPU in script?: Y
- Using distributed or parallel set-up in script?: N


Comments:
Hi, thanks for the well-formulated question! Are you using the latest examples? When I look at the current branch, there is no force_downloading (any more) - it has been commented out:

https://github.com/huggingface/transformers/blob/a08652772791fdaeed6f263b1a99926ca64be5dc/examples/question-answering/run_squad.py#L790

https://github.com/huggingface/transformers/blob/a08652772791fdaeed6f263b1a99926ca64be5dc/examples/question-answering/run_squad.py#L815

Yes - I saw that. I have been using v2.2.1, since upgrading has broken at least one of the tasks I'm performing. Until I can debug, I guess I can limp along by sed-replacing the True in the two instances of force_download.

Alright. In that case, I'm closing this since it's already "fixed" in the recent versions.