Load model from 1.5.1 and save/use it on 1.7.1

FiReTiTi: <p>Hi,<br>
I have an old model saved from PyTorch 1.5.1 using <code>torch.save(model, "MyModel.pt")</code>.<br>
When I try to use it on PyTorch 1.7.1, I get the following error:</p>
<pre><code>torch.nn.modules.module.ModuleAttributeError: 'FrozenBatchNorm2d' object has no attribute 'eps'
</code></pre>
<p>Unfortunately, I don’t know the architecture used to create the model, so I cannot recreate the same in 1.7.1 and copy the weights.</p>
<p>What are my options?<br>
Thank you in advance.</p>

ptrblck: <aside class="quote no-group" data-username="FiReTiTi" data-post="1" data-topic="109750">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/firetiti/48/28042_2.png" class="avatar"> FiReTiTi:</div>
<blockquote>
<p>Unfortunately, I don’t know the architecture used to create the model, so I cannot recreate the same in 1.7.1 and copy the weights.</p>
</blockquote>
</aside>
<p>If you’ve stored the “complete” model, you would still need the model class definition and all source files, wouldn’t you, or did you use a pretrained model somehow?</p>

FiReTiTi: <p>It’s Mask R-CNN model from torchvision, but I know that I tuned many parameters, and I don’t remember which ones.</p>

ptrblck: <p>Where did you tune these parameters? Directly in the source file (in torchvision) or in another script?<br>
I don’t think that <code>torch.save(model, path)</code> will store all source files with it, and would thus try to load them in <code>torch.load</code>.<br>
Note that I’m speculating a lot here, as I’m not using this workflow, since it can break in many ways.</p>

FiReTiTi: <p>The parameters were mainly for the RPN.<br>
I can load the old model in 1.7.1, but when I use it for inference I get the error I cited above. If the sources were store within the <code>.pt</code> file, I would likely not have this error.</p>

ptrblck: <p>I don’t think the sources are stored (which is the reason the loading could break) and, if I’m not mistaken, it’s a known pickle limitation.<br>
Since you already can load the model, you could try to set the missing <code>eps</code> to <code>0</code>, which seems to be the default based on <a href="https://github.com/pytorch/vision/pull/2190">this PR</a>.</p>

FiReTiTi: <p>Thanks.</p>
<p>So if I understand what you suggest, what I have to do is:</p>
<ul>
<li>Load the old model.</li>
<li>Find the FrozenBatchNorm2d layers</li>
<li>Create a FrozenBatchNorm2d layers with PyTorch 1.7.1 with the old eps parameter.</li>
<li>Copy the old layers weights/biases in the new layers</li>
<li>Replace the old layers by the new one.</li>
</ul>
<p>Am I correct?<br>
If yes, I have two questions:</p>
<ul>
<li>How do I replace one layer by another one?</li>
<li>How do I copy the weights and bias form one layer into the new one?</li>
</ul>

ptrblck: <p>I don’t think you have to replace the <code>FrozenBatchNorm2d</code> layers and could try to add the missing <code>eps</code> attribute directly to the already used layers.<br>
If I understand your issue correctly, the error is only raised in the <code>forward</code> pass when the <code>eps</code> attribute is used, not while loading the <code>state_dict</code>. If so, then adding the <code>eps</code> before executing the forward pass might work.</p>

FiReTiTi: <p>Thanks.<br>
But if the model is already created with a missing parameter/argument, how can I add it?</p>

ptrblck: <p>You can add attributes by directly assigning them:</p>
<pre><code class="lang-python">model.frozen_bn1.eps = 0.
</code></pre>

FiReTiTi: <p>Thanks, I’ll try it.</p>

FiReTiTi: <p>It worked, thanks a lot.</p>
<pre><code>from torchvision.ops import misc
model = torch.load("...")
for name, layer in model.named_modules():
   if isinstance(layer, misc.FrozenBatchNorm2d):
       layer.eps = 0.
torch.save(model, "...")</code></pre>