Why cannot nn.Module use `__setattr__` to set non-pytoch objects?

username: Igo312
cooked: <p>Recentelly I find <code>nn.Module</code> cannot set some objects as its attributes.for example</p>
<pre data-code-wrap="python"><code class="lang-python">class Example(nn.Module):
    def set(self):
        # seq is instance of nn.Sequential
        list = [seq1,seq2,seq3]
        self.__setattr__('exp',list)
</code></pre>
<p>when using <code>__getattr__</code>, it will return</p>
<blockquote>
<p>AttributeError: ‘Example’ object has no attribute ‘exp’</p>
</blockquote>
<p>I figure it out by replacing <code>list</code> with <code>nn.ModuleList</code> but what is the purpose?</p>

username: ptrblck
cooked: <p>Using an <code>nn.ModuleList</code> will make sure that all parameters are properly transferred to the device, if you are using <code>model.to()</code> or a data parallel approach.<br>
The attribute should still be found using a plain <code>list</code> object, but should yield a device mismatch error.<br>
Are you trying to create these modules during a forward pass and data parallel?<br>
If so, note that these changes would be applied to each model copy on the device, not the main model.</p>

username: Igo312
cooked: <p>Thank for your answering and the hint. But what still confuse me is why cannot post a <code>list</code> into model.Any reason for this?</p>

username: ptrblck
cooked: <p>I would assume the <code>list</code> should also be registered as an attribute, but should yield a device mismatch error. I still don’t know, why the attribute cannot be found at all.<br>
Are you registering it after wrapping the model into a data parallel wrapper?</p>

username: Igo312
cooked: <p>Sorry for misunderstanding your previous advice before. But no, I register the attribute before warpping the model into data parallel. Here’s my whole script</p>
<pre><code class="lang-python">import torch
import torch.nn as nn

class Example(nn.Module):
    def set(self):
        list = [1,2,3]
        self.__setattr__('exp',list)

    def set_torch(self):
        conv1 = nn.Conv2d(128,256,3)
        self.__setattr__('exp1',nn.ModuleList([conv1,conv1,conv1]))

example = Example()
example.set()
example.set_torch()
example.__getattr__('exp')
example.__getattr__('exp1')
</code></pre>
<p>and I get</p>
<pre><code class="lang-python">&gt;&gt;example.__getattr__('exp')
Traceback (most recent call last):
  File "E:\pycharm\PyCharm Community Edition 2020.1.3\plugins\python-ce\helpers\pydev\_pydevd_bundle\pydevd_exec2.py", line 3, in Exec
    exec(exp, global_vars, local_vars)
  File "&lt;input&gt;", line 1, in &lt;module&gt;
  File "C:\Users\acer\AppData\Roaming\Python\Python36\site-packages\torch\nn\modules\module.py", line 576, in __getattr__
    type(self).__name__, name))
AttributeError: 'Example' object has no attribute 'exp'

&gt;&gt;example.__getattr__('exp1')
ModuleList(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
  (1): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
  (2): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
)
</code></pre>

username: ptrblck
cooked: <p>Thanks for the code. It seems you can directly access the attribute via <code>example.exp</code>, but <code>example.__getattr_</code> calls into this <a href="https://github.com/pytorch/pytorch/blob/3846e35a5536a6a2b6a7b6782eaa274c9624b211/torch/nn/modules/module.py#L782-L796">derived method</a>, which checks for parameters, buffers, and modules.</p>