HTTPS request to model repo despite local_files_only=T
Author: asheetal
Pull request: no

### System Info

- `transformers` version: 4.9.2
- Platform: Linux-5.4.0-109-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.10.0a0+git36449ea (True)
- Tensorflow version (GPU?): 2.4.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: none


### Who can help?

@mrm8488 
I am trying to to Q&A in a very large text so I chunk the text and do Q&A in smaller chunks. 
I have a pipeline using this model embedded in a function that I call in a loop.
```
  my_guru <- function(my_sents, my_topk, my_question) {
    sequence_length <- lengths(gregexpr("\\W+", my_sents)) + 1
    print(sequence_length)
    assert("Sequence length < 4096", sequence_length < 4096)
    text <- reticulate::import("tensorflow_text")
    transformers <- reticulate::import("transformers")
    torch <- reticulate::import("torch")
    
    model <- transformers$AutoModelForQuestionAnswering$from_pretrained("mrm8488/longformer-base-4096-finetuned-squadv2",
                                                                        #low_cpu_mem_usage=FALSE,
                                                                        local_files_only=T)
    tokenizer <- transformers$AutoTokenizer$from_pretrained("mrm8488/longformer-base-4096-finetuned-squadv2",
                                                            truncation = FALSE,
                                                            padding='max_length',
                                                            local_files_only=T)
    guru <- transformers$pipeline("question-answering", 
                                  model=model, 
                                  tokenizer=tokenizer,
                                  device=0L)
    
    answers <- guru(context = my_sents, question = my_question, top_k = my_topk)
    rm(tokenizer)
    gc()
    torch$cuda$empty_cache()
    return(answers)
  }
````

Since I am using local_files_only=T. I should expect it to run over the days and complete without going to internet. However after looping a few thousand times, It generates an error and crashes out

```
/usr/local/lib/python3.8/dist-packages/transformers/pipelines/question_answering.py:316: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:198.)
  fw_args = {k: torch.tensor(v, device=self.device) for (k, v) in fw_args.items()}
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/mrm8488/longformer-base-4096-finetuned-squadv2
```



### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

set local files to TRUE
repeat the pipeline over thousands of times

### Expected behavior

The script should not send a request to huggingface when local files is set to TRUE

Comments:
You need to upgrade your version of Transformers as it's pretty old and many bugs with the cache system have been fixed since then.

Thank you. Upgrade fixes the problem