How to give Pytorch layer a name?

username: yxchng
cooked: <p>I want to give Pytorch layers names so that I can easily select them. Right now, I am selecting them based on index, which means I have to be sure of where they are. Is it possible to just give them names so that I don’t have to keep checking for their indices?</p>

username: antspy
cooked: <p>Updates on this? It would be nice to have the possibility to assign names to layers, instead of having a name automatically assigned. Isn’t it just a matter of modifying nn.Module base class to include a “name” parameter? Or are you worried about possible name conflicts in case the assigned name is not unique?</p>

username: smth
cooked: <p>you can easily do:</p>
<pre><code class="lang-auto">self.my_name = nn.Linear(...)
</code></pre>
<p>what is the problem with this approach?</p>

username: antspy
cooked: <p>Good point - nothing is wrong with it. The only thing would be when using nn.ModuleList and you want to give a name to each module inside the list, but I agree it’s probably not worth it to implement a (possibly problematic) naming system just for this case.</p>

username: salihkaragoz
cooked: <p>Sorry but what did you mention with self ?<br>
Can i give name like other frameworks?</p>
<p><code>nn.Conv2d(name="test_layer")</code></p>

username: smth
cooked: <p>self is the parent module.</p>
<pre><code class="lang-auto">class MyNet(nn.Module):
    def __init__(self):
        super(MyNet, self).__init__()
        self.conv1 = nn.Conv2d(...)
</code></pre>

username: drscotthawley
cooked: <p><a class="mention" href="/u/smth">@smth</a> The problem with that approach is that later (in some other routine) if you want to use the name, the ‘self.my_name’ is long gone.</p>
<p>If there were a name= keyword argument (like Keras supports), then it would make it easier to track, e.g.,  <em>which</em> Conv2d layer out of however many I’m using, to output weights for visualization.</p>
<p>Instead we get  TypeError: <strong>init</strong>() got an unexpected keyword argument ‘name’</p>

username: josmi9966
cooked: <p>Yes, in PyTorch the name is a property of the container, not the contained layer, so if the same layer A<br>
is part of two other layers B and C, that same layer A could have two different names in layers B and C.<br>
This is not very helpful, I think, and I would agree that allowing layers to have identifying names which are part of the layer would be very useful.</p>

username: Roulbac
cooked: <p>I use a function that gives me a dict of named Conv2d, Con2dTranspose and Batchnorm2D layers.<br>
I use it to visualize weights/gradients of my layers.<br>
Hope this is useful:</p>
<pre><code class="lang-auto">def get_named_layers(net):
    conv2d_idx = 0
    convT2d_idx = 0
    linear_idx = 0
    batchnorm2d_idx = 0
    named_layers = {}
    for mod in net.modules():
        if isinstance(mod, torch.nn.Conv2d):
            layer_name = 'Conv2d{}_{}-{}'.format(
                conv2d_idx, mod.in_channels, mod.out_channels
            )
            named_layers[layer_name] = mod
            conv2d_idx += 1
        elif isinstance(mod, torch.nn.ConvTranspose2d):
            layer_name = 'ConvT2d{}_{}-{}'.format(
                conv2d_idx, mod.in_channels, mod.out_channels
            )
            named_layers[layer_name] = mod
            convT2d_idx += 1
        elif isinstance(mod, torch.nn.BatchNorm2d):
            layer_name = 'BatchNorm2D{}_{}'.format(
                batchnorm2d_idx, mod.num_features)
            named_layers[layer_name] = mod
            batchnorm2d_idx += 1
        elif isinstance(mod, torch.nn.Linear):
            layer_name = 'Linear{}_{}-{}'.format(
                linear_idx, mod.in_features, mod.out_features
            )
            named_layers[layer_name] = mod
            linear_idx += 1
    return named_layers
</code></pre>

username: tymokvo
cooked: <p>Couldn’t you do this?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/e571a9e340ddf4c945e10a669cfbabfece453aae.png" data-download-href="https://discuss.pytorch.org/uploads/default/e571a9e340ddf4c945e10a669cfbabfece453aae" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e571a9e340ddf4c945e10a669cfbabfece453aae.png" alt="image" data-base62-sha1="wJKRyVeVFldbSey7T5sn3eUUFwa" width="690" height="154" data-dominant-color="080808"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1715×383 17.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Using <code>nn.Sequential</code> with an <code>OrderedDict</code> from <code>collections</code> is a good way to organized named <code>Modules</code> ahead of time and then construct the actual nn graph at run time.</p>

username: Joker
cooked: <p>Yes the doc says that we can use  ’ add_module(name=‘layer1’, module)’  in Sequential, but what if   we can add name like  ‘con2d(name=‘layer1’,in_channel=10)’ when we add this layer ,and  we  can  call   this layer  like   ‘model_Layer1=model[‘layer1’]’         That sounds very cool</p>

username: ptrblck
cooked: <p>In a custom model, you can call the layer using it’s attribute name:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = nn.Conv2d(3, 6 ,3, 1, 1)
        
    def forward(self, x):
        x = self.conv1(x)
        return x

model = MyModel()
model.conv1(torch.randn(1, 3, 24, 24))
</code></pre>

username: AtenaNguyen
cooked: <p>Currently, I have to define the  initial</p>
<blockquote>
<p>self.conv1 = nn.Conv2d(3, 24, kernel_size=1, bias=False)<br>
self.bn1 = nn.BatchNorm2d(24)<br>
self.conv2 = nn.Conv2d(mid_planes, out_planes, kernel_size=1, groups=groups, bias=False)</p>
</blockquote>
<p>and corresponding forward pass as</p>
<blockquote>
<p>out = F.relu(self.bn1(self.conv1(x)))<br>
out = F.conv2(out)<br>
out = out + x</p>
</blockquote>
<p>Currently, I have 2 convolutions, what happen if I want to have a flexible number of convolution layers in my network?. Let’s say 15 convolutions layers.</p>
<p>Is there any easy way to do it, rather than hand naming as conv1, …, conv15.</p>

username: tymokvo
cooked: <p>As noted above, you can create an <code>nn.Sequential</code> from an <code>OrderedDict</code> with keys as module names (<code>conv0</code>, <code>conv1</code>, …, <code>conv14</code>) and <code>nn.Conv</code> layers as the values.</p>
<p>However, if you want to make a more complicated forward pass with conditions on how to dispatch to the various convolutional layers, or make residual connections, etc. then your module must define that behavior.</p>
<p>You could mix the approaches where you construct the various layers from a collection, and then still refer to them by their name in the forward method:</p>
<pre><code class="lang-auto">class AGoodModule(nn.Module):
    def __init__(self):
        super(AGoodModule, self).__init__()

        self.main = nn.Sequential()

        layers = [
            ("conv{}".format(n), nn.Conv2d(4, 4, (3, 3), 1, 1)) for n in range(3)
        ]

        [self.main.add_module(n, l) for n, l in layers]
    
    def forward(self, x):
        x0 = self.conv0(x)
        x1 = self.conv1(x0) + x0
        x2 = self.conv2(x1) + x1
        return x2

</code></pre>

username: Joker
cooked: <p>Yes,there is ,but  you  must   be  very   familiar  to   the  convolution  output_channel  and  input  channel, there  are  two  choice  :<br>
1: (sequential   or  modulelist).add     written    in    a  for  loop<br>
2:  you must  have  a look  about   resnet    and   hourglass,   they   are  very  classics   about  how  define  a  network</p>

username: ABHIMANYU_ABHIMANYU
cooked: <p>Thanks so much for this:grinning:</p>

username: Antoine
cooked: <p>To both use a Sequential object and access a layer with a string, you could also do something like this:</p>
<p><strong>Method 1</strong></p>
<pre><code class="lang-auto">class Model(nn.Module):
    def __init__(self, channels):
        super(Model, self).__init__()

        self.conv_block = nn.Sequential()
        for k in range(len(channels)-1):
            self.conv_block.add_module(
                'conv{}'.format(k),
                nn.Conv2d(channels[k], channels[k+1], kernel_size=3, padding=0)
            )
</code></pre>
<p>As it defines attributes, you could access the layer using the <code>__getattr__</code> function:</p>
<pre><code class="lang-auto">channels = [1, 2, 4, 8]
model = Model(channels)
model.conv_block.__getattr__('conv1')
</code></pre>
<p>To get the list of all names, you could either extract them from <code>dir(model.conv_block)</code> (by selecting the ‘conv{}’, ‘linear{}’ etc.) or create an attribute <code>self.layer_names</code>.</p>
<p><strong>Edit</strong>: Actually this may be cleaner (I realize this is very similar to a previous answer using <code>ModuleDict</code>). We can easily extend it when you have several blocks:<br>
<strong>Method 2</strong></p>
<pre><code class="lang-auto">class Model(nn.Module):

    def __init__(self, channels):
        super(Model, self).__init__()

        self.conv_block = nn.Sequential()
        self.layer2id = dict()
        self._layer_idx = self._conv_idx = 0

        for k in range(len(channels)-1):
            self.layer2id['conv{}'.format(self._conv_idx)] = self._layer_idx
            self.conv_block.add_module(
                'conv{}'.format(self._conv_idx),
                nn.Conv2d(channels[k], channels[k+1], kernel_size=3, padding=0)
            )
            self._layer_idx += 1
            self._conv_idx += 1

    def get_layer(self, name):
        return self.conv_block[self.layer2id[name]]

channels = [1, 2, 4, 8]
model = Model(channels)
model.get_layer('conv1')
</code></pre>

username: Wangaroo
cooked: <p>Use <code>nn.Sequential().add_module</code> or <code>OrderedDict</code>.</p>
<pre><code class="lang-python"># OrderedDict
self.conv5 = nn.Sequential(OrderedDict({
            'conv': nn.Conv2d(out_channels[2], out_channels[3], 1),
            'bn': nn.BatchNorm2d(out_channels[3]),
            'relu': nn.ReLU(inplace=True),
        }))
</code></pre>
<pre><code class="lang-python"># add_module
self.conv5 = nn.Sequential()
self.conv5.add_module('conv', nn.Conv2d(out_channels[2], out_channels[3], 1))
self.conv5.add_module('bn', nn.BatchNorm2d(out_channels[3]))
self.conv5.add_module('relu', nn.ReLU(inplace=True))
</code></pre>
<p>Then you can use <code>net.conv5.conv</code> to track this conv.</p>

username: Noam_Salomonski
cooked: <p>A full answer to this is given <a href="https://stackoverflow.com/questions/66152766/how-to-assign-a-name-for-a-pytorch-layer/66162559#66162559" rel="noopener nofollow ugc">in stackoferflow</a></p>