What is the base class that only models inherit from

mohammedi-haroune: <p>I’m working on supporting automatic model detection/logging for PyTorch models for our Machine learning platform <a href="https://iko.ai" rel="noopener nofollow ugc">https://iko.ai</a> and I want to know what is the base class for models that only models inherit from? i.e. I’m looking for the class X that passes this condition: If a class Y inherits from class X, Y is a PyTorch model.</p>
<p>Is it <code>torch.nn.modules.module.Module</code> ?</p>

Henry_Chibueze: <p>The base class is the Module class in torch.nn</p>
<pre><code class="lang-auto">Import torch
Import torch.nn as nn

class NNmodel(nn.Modules):
    def __init__(self, .....):
        super(NNmodel, self).__init__()
</code></pre>

mohammedi-haroune: <p>All the classes that inherits from <code>torch.nn.Module</code> are models  ?</p>

Henry_Chibueze: <p>Yah<br>
They are neural network models<br>
Any neural network model made with pytorch always inherits from the Modules class.</p>

mohammedi-haroune: <p>What about <code>torch.nn.CrossEntropyLoss</code> class, it inherits from <code>torch.nn.Module</code> but it’s not a model, right ?</p>

Henry_Chibueze: <p>It’s a loss function</p>
<p>What I meant was that to build a neural network model in pytorch, u have to inherit from the <code>torch.nn.Module</code> class</p>

mohammedi-haroune: <p>Yeah, but I’m looking for the base class for models that <strong>only</strong> models inherit from. Is there such a class? Or any condition that only models fulfills?</p>

Henry_Chibueze: <p>The only class that models inherit from in pytorch is the torch.nn.Module class</p>

mohammedi-haroune: <p>OK, Thanks.</p>
<p>Do you see any other way to distinguish model classes from non-models classes? a common attribute/method?</p>

Henry_Chibueze: <p>I don’t Know if I understand ur question but if u mean distinguishing the class of an implemented model architecture from one that is not a model then the only difference is that model classes in pytorch inherit from <code>torch.nn.Module</code>.</p>
<p>If u are referring to ones that make up a NN model architecture like <code>torch.nn.Linear</code>, <code>torch.nn.Conv2D</code> etc and are trying to differentiate them from things like loss functions eg: <code>torch.nn.CrossEntropy</code> or <code>torch.nn.NLLLoss</code> etc then I guess u just different them by there names, by the names u know which can constitute a model architecture and which is a loss function</p>

mohammedi-haroune: <p>I’m referring to your second guess: how can I differentiate between classes that constitute an NN model and those which implement losses …</p>
<p>I want to write a python function to do that for me and clearly basing on class names is not a good way to do it.</p>
<p>Anyway, thank you very much for your help.</p>

Henry_Chibueze: <p>Hmmmm🤔</p>
<p>You want to write a python function that differentiate these the model constituents’ class from the loss classes?</p>

mohammedi-haroune: <p>Yup. The function is something like for now</p>
<pre><code class="lang-python">def is_pytorch_model(obj):
    return isinstance(obj, torch.nn.Module)
</code></pre>
<p>It, clearly, consider losses and other <code>torch.nn.Module</code> subclasses as NN models which is not true.</p>

Henry_Chibueze: <p>Hmmm🤔</p>
<p>Have U tried differentiating them by the kind of values they return ?<br>
The output of let’s say <code>torch.nn.Conv2D</code> is different from what <code>torch.nn.CrossEntropy</code> outputs shape wise and all…<img src="https://discuss.pytorch.org/images/emoji/apple/man_shrugging.png?v=9" title=":man_shrugging:" class="emoji" alt=":man_shrugging:"></p>

mohammedi-haroune: <p>I can’t do that because I’m doing a static analysis of the code using <code>astroid</code>.</p>
<p>For losses, there is actually a couple of attributes that could differentiate them from the other classes</p>
<pre><code class="lang-python">
class Net(nn.Module):
    # define nn
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(4, 100)
        self.fc2 = nn.Linear(100, 100)
        self.fc3 = nn.Linear(100, 3)
        self.softmax = nn.Softmax(dim=1)
​
    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        x = self.fc3(x)
        x = self.softmax(x)
        return x

net = Net()
criterion = nn.CrossEntropyLoss()
set(dir(criterion)) - set(dir(net))

&gt;&gt;&gt; {'__constants__', 'ignore_index', 'reduction', 'weight'}
</code></pre>
<p>What are all the other things that should inherit from <code>torch.nn.Module</code>?</p>

Henry_Chibueze: <p>Well to my current knowledge only the model constituents and loss functions inherit from <code>torch.nn.Module</code> class</p>

KFrank: <p>Hello Haroune and Henry!</p>
<aside class="quote no-group" data-username="mohammedi-haroune" data-post="11" data-topic="101759" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/dc4da7/48.png" class="avatar"> mohammedi-haroune:</div>
<blockquote>
<p>how can I differentiate between classes that constitute an NN model and those which implement losses</p>
</blockquote>
</aside>
<p>I don’t think that there is a simple, one-stop-shopping way of doing<br>
this that is completely reliable.  Note that activations also inherit<br>
from <code>Module</code>.</p>
<p>Losses, however, (appear to) inherit (possibly indirectly)  from<br>
<code>torch.nn.modules.loss._Loss</code> (which, in turn, inherits from<br>
<code>Module</code>).  You could look for that.</p>
<p>You might consider counting <code>parameters()</code> on the theory that any<br>
self-respecting model has parameters (although some activation<br>
functions also have parameters).</p>
<p>Also, you have to be clear about what you mean by a “model.”<br>
Would you consider a single <code>Linear</code> to be a model?  Arguably<br>
it would be.  How about a single <code>Softmax()</code>.  This seems more<br>
of a stretch, but how does it behave any differently than a<br>
<code>Linear</code>?  (One difference is that it doesn’t have any parameters.)</p>
<p>This script illustrates some of these points:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)
model_linear = torch.nn.Linear (3, 5)
model_sequential = torch.nn.Sequential ((torch.nn.Linear (4, 6)))
loss_mse = torch.nn.MSELoss()
loss_ce = torch.nn.CrossEntropyLoss()
act_softmax = torch.nn.Softmax()
act_prelu = torch.nn.PReLU()
print (sum (1 for _ in model_linear.parameters()))       # count parameters
print (sum (1 for _ in model_sequential.parameters()))   # count parameters
print (sum (1 for _ in loss_mse.parameters()))           # count parameters
print (sum (1 for _ in loss_ce.parameters()))            # count parameters
print (sum (1 for _ in act_softmax.parameters()))        # count parameters
print (sum (1 for _ in act_prelu.parameters()))          # count parameters
print (model_linear.__class__.__bases__)                 # immediate superclass
print (model_sequential.__class__.__bases__)             # immediate superclass
print (loss_mse.__class__.__bases__)                     # immediate superclass
print (loss_ce.__class__.__bases__)                      # immediate superclass
print (act_softmax.__class__.__bases__)                  # immediate superclass
print (act_prelu.__class__.__bases__)                    # immediate superclass
print (model_linear.__class__.__mro__)                   # full class hierarchy
print (model_sequential.__class__.__mro__)               # full class hierarchy
print (loss_mse.__class__.__mro__)                       # full class hierarchy
print (loss_ce.__class__.__mro__)                        # full class hierarchy
print (act_softmax.__class__.__mro__)                    # full class hierarchy
print (act_prelu.__class__.__mro__)                      # full class hierarchy
</code></pre>
<p>Here is the output:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
1.6.0
&gt;&gt;&gt; model_linear = torch.nn.Linear (3, 5)
&gt;&gt;&gt; model_sequential = torch.nn.Sequential ((torch.nn.Linear (4, 6)))
&gt;&gt;&gt; loss_mse = torch.nn.MSELoss()
&gt;&gt;&gt; loss_ce = torch.nn.CrossEntropyLoss()
&gt;&gt;&gt; act_softmax = torch.nn.Softmax()
&gt;&gt;&gt; act_prelu = torch.nn.PReLU()
&gt;&gt;&gt; print (sum (1 for _ in model_linear.parameters()))       # count parameters
2
&gt;&gt;&gt; print (sum (1 for _ in model_sequential.parameters()))   # count parameters
2
&gt;&gt;&gt; print (sum (1 for _ in loss_mse.parameters()))           # count parameters
0
&gt;&gt;&gt; print (sum (1 for _ in loss_ce.parameters()))            # count parameters
0
&gt;&gt;&gt; print (sum (1 for _ in act_softmax.parameters()))        # count parameters
0
&gt;&gt;&gt; print (sum (1 for _ in act_prelu.parameters()))          # count parameters
1
&gt;&gt;&gt; print (model_linear.__class__.__bases__)                 # immediate superclass
(&lt;class 'torch.nn.modules.module.Module'&gt;,)
&gt;&gt;&gt; print (model_sequential.__class__.__bases__)             # immediate superclass
(&lt;class 'torch.nn.modules.module.Module'&gt;,)
&gt;&gt;&gt; print (loss_mse.__class__.__bases__)                     # immediate superclass
(&lt;class 'torch.nn.modules.loss._Loss'&gt;,)
&gt;&gt;&gt; print (loss_ce.__class__.__bases__)                      # immediate superclass
(&lt;class 'torch.nn.modules.loss._WeightedLoss'&gt;,)
&gt;&gt;&gt; print (act_softmax.__class__.__bases__)                  # immediate superclass
(&lt;class 'torch.nn.modules.module.Module'&gt;,)
&gt;&gt;&gt; print (act_prelu.__class__.__bases__)                    # immediate superclass
(&lt;class 'torch.nn.modules.module.Module'&gt;,)
&gt;&gt;&gt; print (model_linear.__class__.__mro__)                   # full class hierarchy
(&lt;class 'torch.nn.modules.linear.Linear'&gt;, &lt;class 'torch.nn.modules.module.Module'&gt;, &lt;class 'object'&gt;)
&gt;&gt;&gt; print (model_sequential.__class__.__mro__)               # full class hierarchy
(&lt;class 'torch.nn.modules.container.Sequential'&gt;, &lt;class 'torch.nn.modules.module.Module'&gt;, &lt;class 'object'&gt;)
&gt;&gt;&gt; print (loss_mse.__class__.__mro__)                       # full class hierarchy
(&lt;class 'torch.nn.modules.loss.MSELoss'&gt;, &lt;class 'torch.nn.modules.loss._Loss'&gt;, &lt;class 'torch.nn.modules.module.Module'&gt;, &lt;class 'object'&gt;)
&gt;&gt;&gt; print (loss_ce.__class__.__mro__)                        # full class hierarchy
(&lt;class 'torch.nn.modules.loss.CrossEntropyLoss'&gt;, &lt;class 'torch.nn.modules.loss._WeightedLoss'&gt;, &lt;class 'torch.nn.modules.loss._Loss'&gt;, &lt;class 'torch.nn.modules.module.Module'&gt;, &lt;class 'object'&gt;)
&gt;&gt;&gt; print (act_softmax.__class__.__mro__)                    # full class hierarchy
(&lt;class 'torch.nn.modules.activation.Softmax'&gt;, &lt;class 'torch.nn.modules.module.Module'&gt;, &lt;class 'object'&gt;)
&gt;&gt;&gt; print (act_prelu.__class__.__mro__)                      # full class hierarchy
(&lt;class 'torch.nn.modules.activation.PReLU'&gt;, &lt;class 'torch.nn.modules.module.Module'&gt;, &lt;class 'object'&gt;)
</code></pre>
<p>Best.</p>
<p>K. Frank</p>