Is it possible to lookup a module by parameter name for selective pre-trained parameter loading?

himat: <p>I have a network that I built which has the same structure and parameter names as alexnet, except that I have my own custom layers for some layers. And I want to load pre-trained alexnet parameters for only SOME layers. The main issue is that I want to choose which pretrained parameters to load based on what class the layer is, but I can’t figure out a way to cross-lookup the layer module with the parameter name.</p>
<p>I want to be able to do something like this:</p>
<h3>Option 1</h3>
<pre><code class="lang-auto">pretrained_state_dict = torchvision.models.alexnet(pretrained=True).state_dict()
my_model_state_dict = mymodel.state_dict()

for param_name, pt_param in pretrained_state_dict.items():
    if type(mymodel.modules().lookup_by_param_name_somehow(param_name)) == mycustomclass.Conv2d:
        curr_model_state_dict[param_name].copy_(pt_param)

</code></pre>
<h3>Option 2</h3>
<p>Alternatively, the converse would also work if it is possible somehow</p>
<pre><code class="lang-auto">for layer in mymodel.modules():
    if type(layer) == mycustomclass.Conv2d:
        param_names = layer.get_param_names_somehow()
        # layer.state_dict.keys() only gives "odict_keys(['weight', 'bias'])", and not the full parameter names
        for param_name in param_names:
            curr_model_state_dict[param_name].copy_(pretrained_state_dict[param_name])

</code></pre>
<hr>
<p>The state_dict names are all only string names with the tensor parameters, so they have no relation to the actual layer class, like so:</p>
<pre><code class="lang-auto">features.0.weight   torch.Size([64, 3, 11, 11])
features.0.bias   torch.Size([64])
features.3.weight   torch.Size([192, 64, 5, 5])
etc.
</code></pre>
<p>And the .modules() list contains all the layer classes which is good, but you can’t get the parameter names from it:</p>
<pre><code class="lang-auto">Conv2d(3, 64, something=4, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))  type:  &lt;class 'mycustomclass.Conv2d'&gt;
ReLU(inplace)  type:  &lt;class 'torch.nn.modules.activation.ReLU'&gt;
MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)  type:  &lt;class 'torch.nn.modules.pooling.MaxPool2d'&gt;
etc.
</code></pre>
<p>Am I just missing some function that can do this, or is it more tricky?</p>

MariosOreo: <p>Hi,</p>
<p>Did you have a try on <code>model.named_parameters()</code>? It will return the name and value of parameters used in the model.</p>

himat: <p>Yes, I’ve tried that already.<br>
<code>model.named_parameters()</code> doesn’t give anything different from <code>model.state_dict()</code><br>
I still get a tuple containing for example: (“features.0.weight”, tensor).</p>
<p>I need the actual layer type in order to check what class it is from.</p>

MariosOreo: <p>What about <code>named_children</code>, it returns the layer name and the module type.</p>

himat: <p>I found the <code>named_modules</code> function, which provides exactly what I’m looking for. Thanks everyone!</p>

Brando_Miranda: <p>this is good too: <a href="https://discuss.pytorch.org/t/how-to-access-to-a-layer-by-module-name/83797/6" class="inline-onebox">How to access to a layer by module name? - #6 by klory</a></p>

Bill_Collins: <p>Can you show a snippet of how it solved your problem? I’m trying to train a pretrained googlenet from torchvision but I can’t because I can’t figure out how to access the parameters to pass to an optimizer.<br>
named_modules() gives access to the modules, but not their parameters.</p>