Manual download of a pytorch_model.bin results in a zip file
Author: gandroz
Pull request: no

I'm behind a firewall so that I cannot download models from a python script. However, I am able to download files directly from the hub, for example here https://huggingface.co/facebook/wav2vec2-base-100h.
The probleme I have is that the download of the `pytorch_model.bin` file results in a `.zip` file. I dont know what to do with this zip file and its content does not help either. I tried to simply rename it to `pytorch_model.bin` but of course I got errors when loading this pre_trained model.

So my question is how to download models and use them offline afterwards ?

Comments:
is `git clone`ing the repo and then moving it around locally an option for you?

PS: FWIW a `pytorch_model.bin` file most of the times _is_ a zip file IIRC

`git` is not an option as it is unavailable on my machine and I am not allowed to install it.
And I suspected that the `pytorch_model.bin` were most of the time a `.zip` file, but simply renaming the downloaded file does not work here

I can maybe suggest to try to use [`huggingface_hub`](https://github.com/huggingface/huggingface_hub)'s `snapshot_download` method: https://github.com/huggingface/huggingface_hub/tree/main/src/huggingface_hub#snapshot_download

Thanks for the suggestion Julien.
In the mean time, I tried to download the model on another machine (that has proper access to internet so that I was able to load the model directly from the hub) and save it locally, then I transfered it to my problematic machine. I tried to load the model from this file but faces the same issue, so it might not be related to the file itself but to something else ...

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in nti(s)
    186             s = nts(s, "ascii", "strict")
--> 187             n = int(s.strip() or "0", 8)
    188         except ValueError:

ValueError: invalid literal for int() with base 8: 'build_te'

During handling of the above exception, another exception occurred:

InvalidHeaderError                        Traceback (most recent call last)
~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in next(self)
   2288             try:
-> 2289                 tarinfo = self.tarinfo.fromtarfile(self)
   2290             except EOFHeaderError as e:

~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in fromtarfile(cls, tarfile)
   1094         buf = tarfile.fileobj.read(BLOCKSIZE)
-> 1095         obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
   1096         obj.offset = tarfile.fileobj.tell() - BLOCKSIZE

~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in frombuf(cls, buf, encoding, errors)
   1036
-> 1037         chksum = nti(buf[148:156])
   1038         if chksum not in calc_chksums(buf):
 

~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in nti(s)
    188         except ValueError:
--> 189             raise InvalidHeaderError("invalid header")
    190     return n


InvalidHeaderError: invalid header

During handling of the above exception, another exception occurred:

ReadError                                 Traceback (most recent call last)
~\AppData\Local\conda\conda\envs\nlp\lib\site-packages\torch\serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
    555         try:
--> 556             return legacy_load(f)
    557         except tarfile.TarError:

~\AppData\Local\conda\conda\envs\nlp\lib\site-packages\torch\serialization.py in legacy_load(f)
    466
--> 467         with closing(tarfile.open(fileobj=f, mode='r:', format=tarfile.PAX_FORMAT)) as tar, \
    468                 mkdtemp() as tmpdir:

~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in open(cls, name, mode, fileobj, bufsize, **kwargs)
   1592                 raise CompressionError("unknown compression type %r" % comptype)
-> 1593             return func(name, filemode, fileobj, **kwargs)
   1594

~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in taropen(cls, name, mode, fileobj, **kwargs)
   1622             raise ValueError("mode must be 'r', 'a', 'w' or 'x'")
-> 1623         return cls(name, mode, fileobj, **kwargs)
   1624

~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in __init__(self, name, mode, fileobj, format, tarinfo, dereference, ignore_zeros, encoding, errors, pax_headers, debug, errorlevel, copybufsize)
   1485                 self.firstmember = None
-> 1486                 self.firstmember = self.next()
   1487

~\AppData\Local\conda\conda\envs\nlp\lib\tarfile.py in next(self)
   2300                 elif self.offset == 0:
-> 2301                     raise ReadError(str(e))
   2302             except EmptyHeaderError:

ReadError: invalid header

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
~\AppData\Local\conda\conda\envs\nlp\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1204                 try:
-> 1205                     state_dict = torch.load(resolved_archive_file, map_location="cpu")
   1206                 except Exception:

~\AppData\Local\conda\conda\envs\nlp\lib\site-packages\torch\serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    386     try:
--> 387         return _load(f, map_location, pickle_module, **pickle_load_args)
   388     finally:

~\AppData\Local\conda\conda\envs\nlp\lib\site-packages\torch\serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
    559                 # .zip is used for torch.jit.save and will throw an un-pickling error here
--> 560                 raise RuntimeError("{} is a zip archive (did you mean to use torch.jit.load()?)".format(f.name))
    561             # if not a tarfile, reset file offset and proceed

RuntimeError: C:\Users\xxxx\src\stt\model\wav2vec2-large-xlsr-53-french\pytorch_model.bin is a zip archive (did you mean to use torch.jit.load()?)

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
~\AppData\Local\Temp\1/ipykernel_12020/1473353111.py in <module>
      1 # Initialize the model
      2 # model = Wav2Vec2ForCTC.from_pretrained('facebook/wav2vec2-large-xlsr-53-french')
----> 3 model = Wav2Vec2ForCTC.from_pretrained(model_path)


~\AppData\Local\conda\conda\envs\nlp\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1206                 except Exception:
   1207                     raise OSError(
-> 1208                         f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
   1209                         f"at '{resolved_archive_file}'"
   1210                         "If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. "

 

OSError: Unable to load weights from pytorch checkpoint file for 'C:\Users\xxxx\src\stt\model\wav2vec2-large-xlsr-53-french' at 'C:\Users\xxxx\src\stt\model\wav2vec2-large-xlsr-53-french\pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
```

Could it be due to the use of a "very" old version of pytorch ?

@gandroz could you share your environment information? You can do so by pasting the result of `transformers-cli env` in your terminal.

I think this may very well originate from the new zip-file system that `torch` uses, which was introduced in PyTorch 1.6.0. 

I dug a little further, and as you mentionned, pytorch moved from pickle serialization to zip from version 1.6.0 but still supports leagacy one. However, old pytorch version are not forward-compatible of course. My pytorch version is quite old indeed, version 1.1.0 ouch !

```
- `transformers` version: 4.6.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.7.10
- PyTorch version (GPU?): 1.1.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

You could do the conversion by loading the model in PyTorch v1.6.0, saving it using the old serialization system, and then loading it in 1.1.0

Good try, but the `gelu` activation function was added in the 1.5.0 version of pytorch so that my obsolete installed pytorch version is not able to construct the model. I could try to add it directly in code.... lol

Ah it's very possible that model only works with more recent versions of PyTorch indeed. We should do a better job of documenting which models are compatible with which version (which is on the roadmap).

Is there something blocking you from updating to a more recent version of PyTorch?


> Is there something blocking you from updating to a more recent version of PyTorch?

Yes, security.... I must install packages from intern mirror in which pytorch for Windows in only in version 1.1
I can confirm that there is no problem with the last release.