[SOLVED]State_dict and items?

username: Petro_Key
cooked: <p>Hi, thank you always for your help.<br>
I am in trouble in understanding the following code:</p>
<p>filtered_weight_dict = {k: v for k, v in weight_dict.items() if k in model.state_dict()}</p>
<p>Here, weight_dict and “model” is two models. weight_dict was loaded by torch.load from a pretrained model path and the “model” is resnet18.<br>
I cannot understand two points:<br>
(1) why those two models should be compared to store<br>
and<br>
(2) what the items mean in general?</p>
<p>Below is shown weight_dict.items outputs:<br>
encoder.conv1.weight<br>
encoder.bn1.weight<br>
encoder.bn1.bias<br>
encoder.bn1.running_mean<br>
encoder.bn1.running_var<br>
encoder.bn1.num_batches_tracked<br>
encoder.layer1.0.conv1.weight<br>
encoder.layer1.0.bn1.weight<br>
encoder.layer1.0.bn1.bias<br>
encoder.layer1.0.bn1.running_mean<br>
encoder.layer1.0.bn1.running_var<br>
encoder.layer1.0.bn1.num_batches_tracked<br>
encoder.layer1.0.conv2.weight<br>
encoder.layer1.0.bn2.weight<br>
encoder.layer1.0.bn2.bias<br>
encoder.layer1.0.bn2.running_mean<br>
encoder.layer1.0.bn2.running_var<br>
encoder.layer1.0.bn2.num_batches_tracked<br>
encoder.layer1.1.conv1.weight<br>
encoder.layer1.1.bn1.weight<br>
encoder.layer1.1.bn1.bias<br>
encoder.layer1.1.bn1.running_mean<br>
encoder.layer1.1.bn1.running_var<br>
encoder.layer1.1.bn1.num_batches_tracked<br>
encoder.layer1.1.conv2.weight<br>
encoder.layer1.1.bn2.weight<br>
encoder.layer1.1.bn2.bias<br>
encoder.layer1.1.bn2.running_mean<br>
encoder.layer1.1.bn2.running_var<br>
encoder.layer1.1.bn2.num_batches_tracked<br>
encoder.layer2.0.conv1.weight<br>
encoder.layer2.0.bn1.weight<br>
encoder.layer2.0.bn1.bias<br>
encoder.layer2.0.bn1.running_mean<br>
encoder.layer2.0.bn1.running_var<br>
encoder.layer2.0.bn1.num_batches_tracked<br>
encoder.layer2.0.conv2.weight<br>
encoder.layer2.0.bn2.weight<br>
encoder.layer2.0.bn2.bias<br>
encoder.layer2.0.bn2.running_mean<br>
encoder.layer2.0.bn2.running_var<br>
encoder.layer2.0.bn2.num_batches_tracked<br>
encoder.layer2.0.downsample.0.weight<br>
encoder.layer2.0.downsample.1.weight<br>
encoder.layer2.0.downsample.1.bias<br>
encoder.layer2.0.downsample.1.running_mean<br>
encoder.layer2.0.downsample.1.running_var<br>
encoder.layer2.0.downsample.1.num_batches_tracked<br>
encoder.layer2.1.conv1.weight<br>
encoder.layer2.1.bn1.weight<br>
encoder.layer2.1.bn1.bias<br>
encoder.layer2.1.bn1.running_mean<br>
encoder.layer2.1.bn1.running_var<br>
encoder.layer2.1.bn1.num_batches_tracked<br>
encoder.layer2.1.conv2.weight<br>
encoder.layer2.1.bn2.weight<br>
encoder.layer2.1.bn2.bias<br>
encoder.layer2.1.bn2.running_mean<br>
encoder.layer2.1.bn2.running_var<br>
encoder.layer2.1.bn2.num_batches_tracked<br>
encoder.layer3.0.conv1.weight<br>
encoder.layer3.0.bn1.weight<br>
encoder.layer3.0.bn1.bias<br>
encoder.layer3.0.bn1.running_mean<br>
encoder.layer3.0.bn1.running_var<br>
encoder.layer3.0.bn1.num_batches_tracked<br>
encoder.layer3.0.conv2.weight<br>
encoder.layer3.0.bn2.weight<br>
encoder.layer3.0.bn2.bias<br>
encoder.layer3.0.bn2.running_mean<br>
encoder.layer3.0.bn2.running_var<br>
encoder.layer3.0.bn2.num_batches_tracked<br>
encoder.layer3.0.downsample.0.weight<br>
encoder.layer3.0.downsample.1.weight<br>
encoder.layer3.0.downsample.1.bias<br>
encoder.layer3.0.downsample.1.running_mean<br>
encoder.layer3.0.downsample.1.running_var<br>
encoder.layer3.0.downsample.1.num_batches_tracked<br>
encoder.layer3.1.conv1.weight<br>
encoder.layer3.1.bn1.weight<br>
encoder.layer3.1.bn1.bias<br>
encoder.layer3.1.bn1.running_mean<br>
encoder.layer3.1.bn1.running_var<br>
encoder.layer3.1.bn1.num_batches_tracked<br>
encoder.layer3.1.conv2.weight<br>
encoder.layer3.1.bn2.weight<br>
encoder.layer3.1.bn2.bias<br>
encoder.layer3.1.bn2.running_mean<br>
encoder.layer3.1.bn2.running_var<br>
encoder.layer3.1.bn2.num_batches_tracked<br>
encoder.layer4.0.conv1.weight<br>
encoder.layer4.0.bn1.weight<br>
encoder.layer4.0.bn1.bias<br>
encoder.layer4.0.bn1.running_mean<br>
encoder.layer4.0.bn1.running_var<br>
encoder.layer4.0.bn1.num_batches_tracked<br>
encoder.layer4.0.conv2.weight<br>
encoder.layer4.0.bn2.weight<br>
encoder.layer4.0.bn2.bias<br>
encoder.layer4.0.bn2.running_mean<br>
encoder.layer4.0.bn2.running_var<br>
encoder.layer4.0.bn2.num_batches_tracked<br>
encoder.layer4.0.downsample.0.weight<br>
encoder.layer4.0.downsample.1.weight<br>
encoder.layer4.0.downsample.1.bias<br>
encoder.layer4.0.downsample.1.running_mean<br>
encoder.layer4.0.downsample.1.running_var<br>
encoder.layer4.0.downsample.1.num_batches_tracked<br>
encoder.layer4.1.conv1.weight<br>
encoder.layer4.1.bn1.weight<br>
encoder.layer4.1.bn1.bias<br>
encoder.layer4.1.bn1.running_mean<br>
encoder.layer4.1.bn1.running_var<br>
encoder.layer4.1.bn1.num_batches_tracked<br>
encoder.layer4.1.conv2.weight<br>
encoder.layer4.1.bn2.weight<br>
encoder.layer4.1.bn2.bias<br>
encoder.layer4.1.bn2.running_mean<br>
encoder.layer4.1.bn2.running_var<br>
encoder.layer4.1.bn2.num_batches_tracked<br>
encoder.fc.weight<br>
encoder.fc.bias</p>
<p>Thank you in advance.</p>

username: Oli
cooked: <p>Hi Petro,</p>
<p>I’m not sure if I understand your first question but I’ll give it a go.</p>
<p>If one doesn’t want to train from scratch every time, but rather use some saved state/weights they would have to load the weights from a file. In this code they load the model state into the <code>weight_dicts</code> variable. When you iterate over it, the k (key) represent the module/layer name and v (value) its corresponding weights. <code>items()</code> is a function you can call on dictionaries to get the keys and values.</p>
<p>By checking if the key from the loaded model exists in the existing model, we could filter the weights to only include the layers that match, and then later load the pre-trained model. Typically, you’d like to check so the keys match to avoid crashes caused by size differences in the fully connected layer at the very end of the network.</p>

username: Petro_Key
cooked: <p>Thank you for your kind reply, Oli.<br>
I see that the code tries to check and gather the appropriate layers from the existing pretrained model to be included in our model.</p>
<p>So it seems that e.g. if the loaded model have excluded batch normalization then our model will also be or like that.<br>
It is quite a nice and helpful function!</p>
<p>Thank you very much:)</p>