Unable to assign nn.Module class to self.encoder in another nn.Module class

username: William_Justin
cooked: <p>In the code shown below, I would like to create an <code>Encoder</code> class and then assign it into <code>self.encoder</code> in <code>ComponentEmbedding</code> class. However, when i try to access <code>ComponentEmbedding().encoder</code>, the value is still <code>None</code>. Anyone help?</p>
<p>Note: I need to preserve <code>encoder = None</code>, in case <code>create_encoder = False</code>.</p>
<p>Code:</p>
<pre><code class="lang-auto">import torch.nn as nn

class Encoder(nn.Module):
    def __init__(self):
        super().__init__()
        pass
    
    def forward(self, x):
        pass

class ComponentEmbedding(nn.Module):
    
    encoder = None
    
    def __init__(self, create_encoder=True):
        super().__init__()
        if create_encoder:
            self.init_encoder()
        
    def forward(self, x):
        pass
    
    def init_encoder(self):
        self.encoder = Encoder()

print(ComponentEmbedding().encoder)
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">None
</code></pre>
<p>Expected Output:</p>
<pre><code class="lang-auto">Encoder()
</code></pre>

username: Matias_Vasquez
cooked: <p>This should work</p>
<pre><code class="lang-python">import torch.nn as nn

class Encoder(nn.Module):
    def __init__(self):
        super().__init__()
        pass
    
    def forward(self, x):
        pass

class ComponentEmbedding(nn.Module):
    def __init__(self, create_encoder=True):
        super().__init__()
        self.encoder = None
        if create_encoder:
            self.init_encoder()
        
    def forward(self, x):
        pass
    
    def init_encoder(self):
        self.encoder = Encoder()

print(ComponentEmbedding().encoder)
</code></pre>

username: William_Justin
cooked: <p>Great, it’s working! Yet, I’m curious why doesn’t it work if we declare <code>encoder = None</code> outside <code>def</code>. I do also found out that if I declare <code>Encoder()</code> as normal class (without <code>nn.Module</code>), its works.</p>
<pre><code class="lang-auto">import torch.nn as nn

class Encoder():
    def __init__(self):
        super().__init__()
        pass
    
    def forward(self, x):
        pass

class ComponentEmbedding(nn.Module):
    
    encoder = None
    
    def __init__(self, create_encoder=True):
        super().__init__()
        if create_encoder:
            self.init_encoder()
        
    def forward(self, x):
        pass
    
    def init_encoder(self):
        self.encoder = Encoder()

print(ComponentEmbedding().encoder)
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">&lt;__main__.Encoder object at 0x7f9eaddad3d0&gt;
</code></pre>

username: Matias_Vasquez
cooked: <h2>
<a name="encoder-none-1" class="anchor" href="#encoder-none-1"></a>encoder = None</h2>
<p>If you define it outside <code>__init__</code>, then it is a class variable. What this means is that you can define many objects of this class and then by doing something like this ↓ you will modify this variable for every object created.</p>
<pre><code class="lang-python">class SimpleClass():
    var = 5
    def __init__(self):
        pass

simple1 = SimpleClass()
simple2 = SimpleClass()

print(simple1.var)
print(simple2.var)

SimpleClass.var = 3

print(simple1.var)
print(simple2.var)

# Output:
#5
#5
#3
#3
</code></pre>
<p><a href="https://pynative.com/python-class-variables/#:~:text=In%20Python,%20Class%20variables%20are,all%20objects%20of%20the%20class." rel="noopener nofollow ugc">Here</a> you can read a little bit more on class variables and standard practices.</p>
<p>But if you really want to use it like this, then you would still have to either initialize it for this object inside <code>__init__</code> and it would be basically the same as my previous solution, only with the extra class variable definition OR using it as a class variable and modifying this variable for EVERY object</p>
<pre><code class="lang-python">def init_encoder(self):
    ComponentEmbedding.encoder = Encoder()
</code></pre>
<h2>
<a name="encoder-without-nnmodule-2" class="anchor" href="#encoder-without-nnmodule-2"></a>Encoder() without nn.Module</h2>
<p>This works because you are creating an object of the type <code>Encoder</code>. However, it does not have all the functionality that a <code>nn.Module</code> has. It is only a simple class with an <code>__init__</code> and <code>forward</code> method, but this does not mean that it will function the same.</p>
<p>If you need the functionality of <code>nn.Module</code> then it should inherit from it.</p>
<p>You can try this by attempting to use a standard <code>nn.Module</code> method that you have still not implemented such as <code>eval</code>.</p>
<pre><code class="lang-python">class Encoder(torch.nn.Module):
    def __init__(self):
        super().__init__()
        pass
    
    def forward(self, x):
        pass

enc = Encoder()

enc.eval() # WIthout torch.nn.Module this will result in error
</code></pre>

username: William_Justin
cooked: <p>Great explanations, thank you so much <img src="https://discuss.pytorch.org/images/emoji/apple/heart.png?v=12" title=":heart:" class="emoji" alt=":heart:" loading="lazy" width="20" height="20"></p>

username: InnovArul
cooked: <aside class="quote no-group" data-username="William_Justin" data-post="3" data-topic="151168">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/william_justin/48/49414_2.png" class="avatar"> William_Justin:</div>
<blockquote>
<p>Yet, I’m curious why doesn’t it work if we declare <code>encoder = None</code> outside <code>def</code>.</p>
</blockquote>
</aside>
<p>TL;DR: Do not use class attributes inside <code>nn.Module</code>. At least, not with the same name as instance attributes.</p>
<p>To answer your curiosity, I am just extending <a class="mention" href="/u/matias_vasquez">@Matias_Vasquez</a>’s answer a bit more.</p>
<p>PyTorch’s <code>nn.Module</code> does have its own internal structure under the hood to store the <code>module</code>/<code>parameter</code>/<code>buffer</code> attributes. Whenever an attribute is assigned, the <a href="https://github.com/pytorch/pytorch/blob/417676720e3bf9e77b70eeaff5595f4c80a990e6/torch/nn/modules/module.py#L1210-L1255"><code>_setattr__</code></a> method of <code>nn.Module</code> is called and the particular attribute value is stored under the appropriate internal dictionary (<code>_parameters</code>, <code>_modules</code>, <code>_buffers</code>) depending on its type.</p>
<p>In your case, the instance attribute <code>encoder</code> that you create inside <code>init_encoder()</code> method will be stored under <code>nn.Module</code>’s <code>_modules</code> dictionary.</p>
<p>Now, when you access <code>ComponentEmbedding().encoder</code>, it will be searched according to <a href="https://stackoverflow.com/a/32295861">attribute lookup order</a> of Python. When it looks in the class <code>__dict__</code> first, it will find the <code>encoder</code> attribute with value <code>None</code> and its returned. Thus the class attribute <code>encoder</code> does not give any chance for <code>nn.Module</code> to call its own <code>__getattr__</code>.</p>
<p><a href="https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/#object-attribute-lookup">Additional reference</a> on metaclasses, attribute lookup order.</p>

username: William_Justin
cooked: <p>Great explanations! solved my curiosity, thank you <img src="https://discuss.pytorch.org/images/emoji/apple/heart.png?v=12" title=":heart:" class="emoji" alt=":heart:" loading="lazy" width="20" height="20"></p>