Error loading trained model

username: Fish
cooked: <p>Hi, I followed the pytorch imagenet code to train a binary model, which has been saved as “model_best.pth.tar”. And I perform the following options to load the model.</p>
<pre><code>	vgg = models.vgg19(pretrained=False)
	vgg.load_state_dict(torch.load('model_best.pth.tar'))
	cnn = vgg.features
	cnn = cnn.cuda()
	model = nn.Sequential()
	model = model.cuda()
</code></pre>
<p>But it gives me the error as “KeyError: 'unexpected key “arch” in state_dict”. I am not sure why “load_state_dict” cannot load the model as indicated. It appears that the original imagenet code saves “args.arch”.  Thanks for the help.</p>

username: ptrblck
cooked: <p>Probably you are loading a checkpoint dict with different key value pairs, e.g. the <code>state_dict</code>, best accuracy etc.<br>
Could you print the result of <code>torch.load('model...')</code>?</p>

username: Fish
cooked: <p>Thanks for your reply. I followed the imagenet code where the checkpoint is saved.</p>
<pre><code>    save_checkpoint({
        'epoch': epoch + 1,
        'arch': args.arch,
        'state_dict': model.state_dict(),
        'best_prec1': best_prec1,
        'optimizer' : optimizer.state_dict(),
    }, is_best)
</code></pre>
<p>So the following should be the correct way to load the model.<br>
cnn = models.vgg19()<br>
cnn.features = torch.nn.DataParallel(cnn.features)<br>
cnn.cuda()<br>
checkpoint = torch.load(‘models/checkpoint.pth.tar’)<br>
cnn.load_state_dict(checkpoint[‘state_dict’])</p>
<p>In this case, the loading works fine. But if I want to iterate the CNN,</p>
<p>for i,layer in enumerate(list(cnn)):</p>
<p>it gives me TypeError: ‘VGG’ object is not iterable. So I guess I am not loading the model the way I want?</p>

username: ptrblck
cooked: <p>Do you want to see all layers?<br>
If so, you could iterate the <code>.children()</code> or <code>.modules()</code>.<br>
I’m answering on my mobile now, so I cannot check it, but I think you cannot iterate a model like this.</p>

username: Fish
cooked: <p>It appears that the model keys are also inconsistent.</p>
<p>When you use the following codes:<br>
cnn = models.vgg19()<br>
cnn_state_dict = cnn.state_dict()<br>
print(cnn_state_dict)<br>
It gives you “features.0.weight”</p>
<p>And if you load a previously trained model,<br>
checkpoint = torch.load(‘models/checkpoint.pth.tar’)<br>
print(checkpoint[‘state_dict’].keys())<br>
Then it will give you “KeyError: ‘unexpected key “features.module.0.weight” in state_dict’”. What are the solutions for this one?</p>

username: deJQK
cooked: <p>I am facing the same error, for vgg 13 and 19. These two websites talks about this problem, but I have not yet tested on it. I am not sure about if this applies to vgg 13 also. BTW, vgg11 and 16 does not have this problem.</p>
<aside class="quote quote-modified" data-post="1" data-topic="1719">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ecolss/48/5045_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/upgrading-torchvision-module-makes-old-model-useless/1719">Upgrading torchvision module makes old model useless?</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    Once upon a time, I downloaded the <a href="https://github.com/jcjohnson/pytorch-vgg" rel="noopener nofollow ugc">vgg models by jonhson</a> and loaded with torchvision.models.vgg16/19 without a problem. 
Lately, I pip upgraded torchvision modules, and the same old vgg models fail to load, error goes like: 
KeyError: 'unexpected key "classifier.1.weight" in state_dict' 
Then I check stat_dict of current upgraded torchvision.models.vgg19, goes like this: 
In [5]: v19.state_dict().keys() 
Out[5]: odict_keys([‘features.0.weight’, ‘features.0.bias’, ‘features.2.weight’, ‘features.2…
  </blockquote>
</aside>
<aside class="onebox githubissue" data-onebox-src="https://github.com/jcjohnson/pytorch-vgg/issues/3">
  <header class="source">

      <a href="https://github.com/jcjohnson/pytorch-vgg/issues/3" target="_blank" rel="noopener nofollow ugc">github.com/jcjohnson/pytorch-vgg</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/jcjohnson/pytorch-vgg/issues/3" target="_blank" rel="noopener nofollow ugc">vgg model checkpoint needs a change of classifier weight names</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2017-04-08" data-time="21:26:19" data-timezone="UTC">09:26PM - 08 Apr 17 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/soumith" target="_blank" rel="noopener nofollow ugc">
          <img alt="soumith" src="https://avatars.githubusercontent.com/u/1310570?v=4" class="onebox-avatar-inline" width="20" height="20">
          soumith
        </a>
      </div>
    </div>

    <div class="labels">
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">See https://discuss.pytorch.org/t/upgrading-torchvision-module-makes-old-model-u<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">seless/1719

The reason this changed was because of https://github.com/pytorch/vision/pull/107 where Sam realized that he put dropout in the wrong location.

So the state_dict needs the names changed appropriately.</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


username: deJQK
cooked: <p>Today I retried the code, and there is no such bug. I do not know why. My code is something like this:</p>
<pre><code class="lang-auto">    model_list = ['vgg11_bn', 'vgg13_bn', 'vgg16_bn', 'vgg19_bn']
    for mdl_name in model_list:
        print(mdl_name)
        mdl_path = os.path.join(mdl_result_path, mdl_name)
        print(mdl_path)
        try:
            mdl_file = GetLatestFile(mdl_path)
            print('{} is loaded.'.format(mdl_file))
        except:
            print('The result for {} does not exist.'.format(mdl_name))
            continue

        mdl = pretrainedmodels.__dict__[mdl_name](num_classes=1000, pretrained='imagenet')

        ...

        mdl.load_state_dict(torch.load(mdl_file))
</code></pre>
<p>When there is the bug, it seems that the ‘<code>try except</code>’ part is not executed correctly, and I guess the mdl is still something from previous iteration (e.g. vgg11_bn). The output is as follows:</p>
<pre><code class="lang-auto">vgg13_bn
~/results/vgg13_bn
Traceback (most recent call last):
  File "InterValid.py", line 119, in &lt;module&gt;
    if __name__ == '__main__':
  File "InterValid.py", line 87, in main

  File "~/.conda/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 721, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for VGG:
</code></pre>
<p>I change the model list to [‘vgg13_bn’], and now it works fine.</p>
<p>I have no idea why the <code>try except</code> is not executed correctly …</p>

username: stoneyang
cooked: <p>Hi <a class="mention" href="/u/fish">@Fish</a>, how did you solve this problem? I’ve encountered one somewhat similar to yours. Could u please take a look at <a href="https://discuss.pytorch.org/t/missing-keys-unexpected-keys-in-state-dict-when-loading-self-trained-model/22379">it</a>? And suggestion would be great. Thanks!</p>

username: Nowrin_Akter_Surovi
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/c/c398cdefd82d14283e0cae745bba8901464e2860.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/c398cdefd82d14283e0cae745bba8901464e2860" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/c/c398cdefd82d14283e0cae745bba8901464e2860_2_690x113.jpeg" alt="Capture" data-base62-sha1="rUkuiJ1vxS1gIdVtNiEWKg6xiKs" width="690" height="113" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/c/c398cdefd82d14283e0cae745bba8901464e2860_2_690x113.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/c/c398cdefd82d14283e0cae745bba8901464e2860_2_1035x169.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/c/c398cdefd82d14283e0cae745bba8901464e2860.jpeg 2x" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">1253×207 33.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> i got those errors. I can load torch.load(my  model)</p>