How to save a model and load to a model with diff. #of output

username: ms_K
cooked: <p>Hello.</p>
<p>As a project, we created and trained an Alexnet model to train with CIFAR-10.</p>
<p>Our next project is to conduct transfer learning with the model we trained using CIFAR-10.<br>
Problem is, the dataset we are using for transfer learning requires 100 output nodes.<br>
Therefore, if we define a model with 100 output nodes and try to load parameters from previous model, error occurs as below.</p>
<blockquote>
<p>Error(s) in loading state_dict for AlexNet2:<br>
size mismatch for fc.4.weight: copying a param with shape torch.Size([10, 4096]) from checkpoint, the shape in current model is torch.Size([100, 4096]).<br>
size mismatch for fc.4.bias: copying a param with shape torch.Size([10]) from checkpoint, the shape in current model is torch.Size([100]).</p>
</blockquote>
<p>How should I deal with this problem?<br>
I’m guessing I should only save and load parameters excluding the last layer but not sure about it.</p>

username: alex.veuthey
cooked: <p>Use the following: <code>model.load_state_dict(ckpt['model'], strict=False)</code>. This will ignore errors when keys are not matching (i.e. not found in the destination <strong>OR</strong> the source state dict).</p>
<p>Be careful about using this option, as it might hide other errors! Although I think the new default is printing the non-matching keys in the most recent version(s) of PyTorch…</p>

username: ms_K
cooked: <p>THX for the reply!<br>
However, same error occurs although I used strict=False.</p>
<blockquote>
<p>PATH = ‘/gdrive/My Drive/AIproject/epoch80.pt’<br>
model.load_state_dict(torch.load(PATH), strict=False)</p>
</blockquote>
<p>code below is structure of my Model</p>
<blockquote>
<p>AlexNet2(<br>
(layer1): Sequential(<br>
(0): Conv2d(3, 96, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))<br>
(1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): LeakyReLU(negative_slope=0.01, inplace=True)<br>
(3): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(4): Conv2d(96, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(5): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(6): LeakyReLU(negative_slope=0.01, inplace=True)<br>
(7): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(8): Conv2d(256, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(9): LeakyReLU(negative_slope=0.01, inplace=True)<br>
(10): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(11): LeakyReLU(negative_slope=0.01, inplace=True)<br>
(12): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(13): LeakyReLU(negative_slope=0.01, inplace=True)<br>
(14): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
)<br>
(fc): Sequential(<br>
(0): Linear(in_features=2304, out_features=4096, bias=True)<br>
(1): Dropout(p=0.4, inplace=False)<br>
(2): Linear(in_features=4096, out_features=4096, bias=True)<br>
(3): Dropout(p=0.4, inplace=False)<br>
(4): Linear(in_features=4096, out_features=100, bias=True)<br>
)<br>
)</p>
</blockquote>
<p>model I trained previously has almost same structure but it had 10 out_features at the last Linear array</p>

username: alex.veuthey
cooked: <p>Oh right, this is probably because <code>strict=False</code> only checks for missing keys, not different sizes for the same keys.</p>
<p>What you could do is removing the <code>fc</code> from your destination model, and adding it again after loading the weights…</p>

username: ms_K
cooked: <p>Got it I’ll try with your advice.<br>
thx:)</p>
<p>******And it worked!<br>
delete last linear layer =&gt; load =&gt; add linear layer<br>
You saved my day</p>