Unable to load model state_dict using torch.utils.model_zoo.load_url()

benihime91: <p>Hello !<br>
I save a PyTorch model state dict saved in github. Whenever I try to load  <code>.pth</code> file using  <code>torch.utils.model_zoo.load_url()</code>, I get this error :</p>
<pre><code class="lang-bash">File "/Users/ayushman/Desktop/retinanet_pet_detector/utils.py", line 17, in get_model
    state_dict = model_zoo.load_url(url, map_location="cpu", progress=True)
  File "/Users/ayushman/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/hub.py", line 504, in load_state_dict_from_url
    raise RuntimeError('Only one file(not dir) is allowed in the zipfile')
RuntimeError: Only one file(not dir) is allowed in the zipfile
</code></pre>
<p>My <code>.pth</code> file is saved as a zip file with the following structure:</p>
<pre><code class="lang-auto">resnet34-retinanet-weights.zip
  |- resnet34-retinanet-weights.pth
</code></pre>
<p>I can see the zip file is get’s downloaded without any problem.  And according to the <a href="https://pytorch.org/docs/stable/model_zoo.html" rel="nofollow noopener">docs</a> the downloaded file can be a zip file.<br>
And also I manualy tried downloading and loading in the checkpoint file <code>resnet34-retinanet-weights.pth</code> it works .</p>
<p>This my code bdw:</p>
<pre><code class="lang-auto">url = "https://github.com/benihime91/retinanet_pet_detector/releases/download/retinanet_v1.0/resnet34-retinanet-weights.zip"

def get_model():
    model = Retinanet(num_classes=37, backbone_kind="resnet34")
    state_dict = model_zoo.load_url(url, map_location="cpu", progress=True)
    model.load_state_dict(state_dict)
    return model
</code></pre>
<p>I even tried:</p>
<pre><code class="lang-auto">def get_model():
    model = Retinanet(num_classes=37, backbone_kind="resnet34")
    state_dict = torch.hub.load_state_dict_from_url(
        url, map_location="cpu", progress=True
    )
    model.load_state_dict(state_dict)
    return model
</code></pre>
<p>Same error !</p>

Henry_Chibueze: <p>I suggest u save ur model state dict as a “pth.tar” file instead of compressing the .pth file to a zip file<br>
Also I think u can get away with renaming the pth ext to pth.tar and not zipping it</p>

benihime91: <p><a class="mention" href="/u/henry_chibueze">@Henry_Chibueze</a><br>
So my file should look like this right <code>random_name.pth.tar</code> . Is this what you mean ?<br>
I’ll give it a try…</p>

benihime91: <p><a class="mention" href="/u/henry_chibueze">@Henry_Chibueze</a><br>
Didn’t work same error:</p>
<pre><code class="lang-bash">Downloading: "https://github.com/benihime91/retinanet_pet_detector/releases/download/retinanet_v1.0/resnet34-retinanet-weights.pth.tar" to /Users/ayushman/.cache/torch/checkpoints/resnet34-retinanet-weights.pth.tar
100%|██████████| 117M/117M [01:36&lt;00:00, 1.27MB/s]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
 in 
      1 url = 'https://github.com/benihime91/retinanet_pet_detector/releases/download/retinanet_v1.0/resnet34-retinanet-weights.pth.tar'
----&gt; 2 state_dict = torch.hub.load_state_dict_from_url(url, map_location='cpu')

~/opt/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/hub.py in load_state_dict_from_url(url, model_dir, map_location, progress, check_hash)
    502             members = cached_zipfile.infolist()
    503             if len(members) != 1:
--&gt; 504                 raise RuntimeError('Only one file(not dir) is allowed in the zipfile')
    505             cached_zipfile.extractall(model_dir)
    506             extraced_name = members[0].filename

RuntimeError: Only one file(not dir) is allowed in the zipfile
</code></pre>

Henry_Chibueze: <p>What is ur pytorch version? is it v1.6</p>

benihime91: <p>my pytorch version is 1.5.1</p>

Henry_Chibueze: <p>If that’s the case I’ve I think u should set the zip file serialization to false in torch.save( ) and re-save ur model again.</p>
<p>Let’s hope this works</p>

benihime91: <p>zip file serialization <code>_use_new_zipfile_serialization</code> is set to False by deafult in  <code>torch.save( )</code>. I did not set it as True.</p>

Henry_Chibueze: <p>Could it be possible then that the torch.hub.load_state_dict( ) is using the new serialization mode to load the model state?<br>
If what I’m thinking is the case u might want to upgrade to v1.6</p>
<p>The error is obviously a serialization error b4 u update to v1.6 try setting the serialization to true in the torch.save() and re-save ur model let’s see if it’s the torch.hub.load… that is causing the issue</p>

benihime91: <p>Maybe that’s what I trained in Google Collab and I just noticed collab uses torch version1.6 where _use_new_zipfile_serialization is True by default</p>

Henry_Chibueze: <p>Ok cool it’s nice to be of help <img src="https://discuss.pytorch.org/images/emoji/apple/ok_hand/2.png?v=9" title=":ok_hand:t2:" class="emoji" alt=":ok_hand:t2:"></p>

Jaideep_Valani: <p><a class="mention" href="/u/benihime91">@benihime91</a> how can we fix this issue . I try to use pretrained weights,so i dont have control.</p>