Loading PyTorch model from TF checkpoint

spacemeerkat: <p>I am trying to load a pretrained model from the HuggingFace repository (<a href="https://huggingface.co/cross-encoder/ms-marco-TinyBERT-L-6" rel="noopener nofollow ugc">this model</a>), but when I attempt to instantiate the model I get an error referring to loading a PyTorch model from a TensorFlow checkpoint.</p>
<p>There are similar posts (e.g. <a href="https://github.com/huggingface/transformers/issues/4336" rel="noopener nofollow ugc">[1]</a>, <a href="https://discuss.huggingface.co/t/how-to-load-weights-which-was-trained-older-version-of-pytorch/1267" rel="noopener nofollow ugc">[2]</a>) from others experiencing this problem in the past but I can’t for the life of me see how they are resolved. Some attempt to match the <code>sentence-transformers</code> and <code>torch</code> package versions to when the models were trained but I have no idea how to find the versions to down/upgrade my packages. Others simply upgrade their <code>torch</code> package to 1.8.0 which I’m using anyway.</p>
<h3><a name="edit-1" class="anchor" href="#edit-1"></a>Edit:</h3>
<p>Please note that I am explicitly trying to load the model from it’s repository files rather than use the recommended way of loading these models which is to simply pass the model name e.g. <code>model = CrossEncoder('ms-marco-TinyBERT-L-6')</code>, which triggers an automatic http download of the model I believe.</p>
<hr>
<p>Here is my workflow:</p>
<h3><a name="clone-the-model-repository-and-get-the-required-packages-2" class="anchor" href="#clone-the-model-repository-and-get-the-required-packages-2"></a>Clone the model repository and get the required packages:</h3>
<pre><code class="lang-auto">$git clone https://huggingface.co/cross-encoder/ms-marco-TinyBERT-L-6
$conda install -c conda-forge sentence-transformers
</code></pre>
<h3><a name="load-the-model-in-a-python-script-3" class="anchor" href="#load-the-model-in-a-python-script-3"></a>Load the model in a python script:</h3>
<pre><code class="lang-auto">### Import packages
from sentence_transformers.cross_encoder import CrossEncoder

### Setup paths
model_path = 'ms-marco-TinyBERT-L-6'

### Instantiate model
model = CrossEncoder(model_path)

</code></pre>
<h3><a name="the-resulting-error-4" class="anchor" href="#the-resulting-error-4"></a>The resulting error:</h3>
<blockquote>
<p>Traceback (most recent call last):<br>
File “/home/userx/anaconda3/envs/NLP/lib/python3.6/site-packages/transformers/modeling_utils.py”, line 1205, in from_pretrained<br>
state_dict = torch.load(resolved_archive_file, map_location=“cpu”)<br>
File “/home/userx/anaconda3/envs/NLP/lib/python3.6/site-packages/torch/serialization.py”, line 593, in load<br>
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)<br>
File “/home/userx/anaconda3/envs/NLP/lib/python3.6/site-packages/torch/serialization.py”, line 762, in _legacy_load<br>
magic_number = pickle_module.load(f, **pickle_load_args)<br>
_pickle.UnpicklingError: invalid load key, ‘v’.</p>
<p>During handling of the above exception, another exception occurred:</p>
<p>Traceback (most recent call last):<br>
File “test.py”, line 8, in <br>
model = CrossEncoder(model_path)<br>
File “/home/userx/anaconda3/envs/NLP/lib/python3.6/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py”, line 47, in <strong>init</strong><br>
self.model = AutoModelForSequenceClassification.from_pretrained(model_name, config=self.config)<br>
File “/home/userx/anaconda3/envs/NLP/lib/python3.6/site-packages/transformers/models/auto/auto_factory.py”, line 381, in from_pretrained<br>
return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)<br>
File “/home/userx/anaconda3/envs/NLP/lib/python3.6/site-packages/transformers/modeling_utils.py”, line 1208, in from_pretrained<br>
f"Unable to load weights from pytorch checkpoint file for ‘{pretrained_model_name_or_path}’ "<br>
OSError: Unable to load weights from pytorch checkpoint file for ‘ms-marco-TinyBERT-L-6’ at 'ms-marco-TinyBERT-L-6/pytorch_model.bin’If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.</p>
</blockquote>
<p>Any help on this would be greatly appreciated, thank you!</p>
<h3><a name="extra-info-that-may-be-useful-5" class="anchor" href="#extra-info-that-may-be-useful-5"></a>Extra info that may be useful</h3>
<p>sentence-transformers=0.4.1<br>
pytorch=1.8.0</p>

ptrblck: <aside class="quote no-group" data-username="spacemeerkat" data-post="1" data-topic="122110">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spacemeerkat/48/4442_2.png" class="avatar"> spacemeerkat:</div>
<blockquote>
<p>Please note that I am explicitly trying to load the model from it’s repository files rather than use the recommended way of loading these models which is to simply pass the model name e.g. <code>model = CrossEncoder('ms-marco-TinyBERT-L-6')</code>, which triggers an automatic http download of the model I believe.</p>
</blockquote>
</aside>
<p>This sounds as if you are digging into the internal loading used by HuggingFace, so I think you would get a better answer in their <a href="https://discuss.huggingface.co/">discussion board</a>.</p>