Problem with pretrained Transformers in Offline mode 
Author: marianpascalau
Pull request: no
Labels: name: Model on the Hub

### System Info

For me, the usage of pretrained transformers in Offline mode is critical. Originally, the Module was OK when the needed files are available in "huggingface_hub" cache folder. 

But now it is not working anymore,, whatever the status of my cached folder is, the library performs at least one "HEAD" request which I don't want.

Is there a way to address this situation? And make the program conform to Online/Offline mode described in huggingface_hub documentation?

Regards, Marian


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

set HF_HUB_OFFLINE=1

model_checkpoint = "Davlan/bert-base-multilingual-cased-ner-hrl"
var tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)

"""
this will cause:
  File "python-3.11.9-embed-amd64\Lib\site-packages\transformers\models\auto\tokenization_auto.py", line 804, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\transformers\models\auto\tokenization_auto.py", line 637, in get_tokenizer_config
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\transformers\utils\hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
                    ^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\huggingface_hub\utils\_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\huggingface_hub\file_download.py", line 1261, in hf_hub_download
    metadata = get_hf_file_metadata(
               ^^^^^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\huggingface_hub\utils\_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\huggingface_hub\file_download.py", line 1674, in get_hf_file_metadata
    r = _request_wrapper(
        ^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\huggingface_hub\file_download.py", line 369, in _request_wrapper
    response = _request_wrapper(
               ^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\huggingface_hub\file_download.py", line 392, in _request_wrapper
    response = get_session().request(method=method, url=url, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python-3.11.9-embed-amd64\Lib\site-packages\requests\adapters.py", line 513, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Davlan/bert-base-multilingual-cased-ner-hrl/resolve/main/tokenizer_config.json (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002A44F5A1250>: Failed to establish a new connection: [WinError 10061] Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte')))
"""

### Expected behavior

The expection should not be raised. According to huggingface_hub documentation, the HF_HUB_OFFLINE should disable communication. Especially, by fact that the model files are precached by me into a local folder. 

Comments:
Here is how I set a custom huggingface_hub cache location:

rem set HF_HOME=%~dp0.cache\huggingface
set HF_HOME=PATH_TO_A_CACHE_FOLDER\.cache\huggingface
set HF_HUB_CACHE=%HF_HOME%\hub
set HF_ASSETS_CACHE=%HF_HOME%\assets


Hi @marianpascalau, could you share your running environment - run `transformers-cli env` in the terminal and copy-paste the output - as well the last version of transformers where this did work for you?

Hi, here is it
transformers-cli env

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.40.0
- Platform: Windows-10-10.0.22000-SP0
- Python version: 3.11.9
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.2
- Accelerate version: 0.28.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.2.2+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

But I do not think that it will help. Because it is easy to follow the callstack and to see that the request is made anyway, regarding if environment-settings are set or not,. And this must be a feature introduced with the newest version.

@amyeroberts ,
I am also facing the similar issue while running inference from cached model with transformers version: 4.40.0  for following models:
- Helsinki-NLP/opus-mt-en-fr
- cardiffnlp/twitter-roberta-base-sentiment-latest

It is trying to reach `huggingface.co`  over the network even if the model is available in hugging face cache. Here is the error:
```
    model = AutoModelForSeq2SeqLM.from_pretrained(model_path)
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3370, in from_pretrained
    if not has_file(pretrained_model_name_or_path, safe_weights_name, **has_file_kwargs):
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/transformers/utils/hub.py", line 627, in has_file
    r = requests.head(url, headers=headers, allow_redirects=False, proxies=proxies, timeout=10)
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/requests/api.py", line 100, in head
    return request("head", url, **kwargs)
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/Users/sukumar.burra/.env/lib/python3.10/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Helsinki-NLP/opus-mt-en-fr/resolve/main/model.safetensors (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x16f486740>: Failed to resolve 'huggingface.co' ([Errno 8] nodename nor servname provided, or not known)"))
```
I do not see this issue with previous versions of transformers. The version `4.39.3` still works.

I noticed, this issue is only observed with models which do not have `safetensors` version of the model in hugging face repo.

Is this something intentional or due to issue in latest `transformers version: 4.40.0`  ? 
Also, Is there any workaround for this ?

Hi @sukumarburra, can you open a new issue? This appears to be separate from the one in this description i.e. that the offline mode is not respected 

cc @Wauplin as it looks like a possible hub issue -> hf_hub_download not respecting `HF_HUB_OFFLINE` 

> Hi @sukumarburra, can you open a new issue? This appears to be separate from the one in this description i.e. that the offline mode is not respected

The error seems to be different but my use case is also same. I am trying to load pre-trained models from huggingface cache for offline inference. While loading the model it's trying to reach huggingface network for specific models.
I opened https://github.com/huggingface/transformers/issues/30469 

Hi @marianpascalau, that's strange, it shouldn't try to make the HEAD call :confused: How do you set `HF_HUB_OFFLINE` exactly? 

Can you save
```py
from transformers import AutoTokenizer
 
model_checkpoint = "Davlan/bert-base-multilingual-cased-ner-hrl"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
```
in a script.py file. And then run

```
HF_HUB_OFFLINE=1 python script.py 
```

That should work as expected. If you still get the error, please let me know. It might be that you are setting `HF_HUB_OFFLINE` env var at runtime after `huggingface_hub` evaluated it.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hallo,
sorry I was busy and focused on a different project. 

@Wauplin 
I do not understand your question. I have provided the call-stack which goes until the web request is issued : hub.py", line 398,
And the environment variable to set before python is executed.

To solve this problem I have forced my pip requirements to download the previous "transformers" version (available on that day).
In few hours I will test once more (with the code provided by you).

@marianpascalau Thanks for getting back to this. With [this test](https://github.com/huggingface/transformers/issues/30345#issuecomment-2076677651) I just want to be sure that the environment variable is set correctly and understood by the codebase. Since I don't have an explanation on why this might be happening, I prefer to isolate the problem. So if you have some time, I'd be interested to know what it outputs. Thanks in advance! 

Hi there,
Sorry, I cannot replicate the original problem anymore. But I have a new one which seems to introduce the same problem:

Here my code (which also shows that HF_HUB_OFFLINE environment variable is set):
```
import os
from transformers import AutoTokenizer, AutoModelForTokenClassification
from huggingface_hub.constants import HF_HUB_OFFLINE

value = os.getenv("HF_HUB_OFFLINE")
if value:
    print("getenv(\"HF_HUB_OFFLINE\")=%s" % value)
if HF_HUB_OFFLINE:
    print("HF_HUB_OFFLINE=%r" % HF_HUB_OFFLINE)

model_checkpoint = "Davlan/bert-base-multilingual-cased-ner-hrl"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model     = AutoModelForTokenClassification.from_pretrained(model_checkpoint)

```

The error happens on the "AutoModelForTokenClassification.from_pretrained" line in transformers\modeling_utils.py

I have compared the old (which works as expected) and the new version and found the following difference:

The new Version has an if statement (on line 3393 in the new Version) "if resolved_archive_file is not None:
                        if filename in [WEIGHTS_NAME, WEIGHTS_INDEX_NAME]:" which is followed later by "if not has_file(pretrained_model_name_or_path, safe_weights_name, **has_file_kwargs):" ... and here the problem begins!

While old version is jumping direct in the "else" position (relative to the new code). 

The function "has_file" is part of transformers library and is doing a "head" request ... probably in order to check if the file is valid. And this test is wrong, if the library is set to work OFFLINE, then the request must be successfull if the file is available (with no extra check). The old code is probably not particullary better, but it does not perform that extra "has_file" check. 


Thanks for the detailed answer @marianpascalau and investigating the source code! 

The direct HEAD call is clearly an oversight and should not be performed if offline mode is enabled. I just pushed a PR (https://github.com/huggingface/transformers/pull/31016) that should be this.

I am also facing this new connection error having latest version of transformers. 

When i I am setting HF_HUB_OFFLINE=1 
I am getting this error "You cannot infer task automatically within `pipeline` when using offline mode"

Not sure what's happening, 

When i I am setting HF_HUB_OFFLINE=1
I am getting this error "You cannot infer task automatically within pipeline when using offline mode"

I have also done this, have you found any fix?

Appreciate any help

> I am also facing this new connection error having latest version of transformers.
> 
> When i I am setting HF_HUB_OFFLINE=1 I am getting this error "You cannot infer task automatically within `pipeline` when using offline mode"
> 
> Not sure what's happening,

and

> When i I am setting HF_HUB_OFFLINE=1
> I am getting this error "You cannot infer task automatically within pipeline when using offline mode"
>
> I have also done this, have you found any fix?
> 
> Appreciate any help

How do you declare the pipeline? 
In my case, I was missing the 'task="text-classification"', after that it worked in OFFLINE Mode :)

```
sentiment_pipeline = pipeline(
        task="text-classification", model=email_config.sentiment_analysis.model
    )
```