Backpropogation on custom function in pytorch model class

username: sanchit2843
cooked: <p>I am new to programming and unable to figure out something. Thanks for help in advance.<br>
So the question is<br>
when we make a class for model</p>
<pre><code class="lang-auto">class model(nn.Module):
    def __init__(self,):
         super(model, self).__init__()
         self.featureextract = fext
         self.classifier = classifier
    def forward(self, input):
        x = self.classifier(input)
        return x
   def feature_extract(self,input):
        x = self.featureextract(input)
        return x
</code></pre>
<p>So in such a case where I have two functions, how the backpropagation will happen, will the parameters of model contain only the forward part or both. And in case only both, how can I make such a model where the features can be extracted separately, classifier can be made separately and then backpropagation happens on the complete model.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="sanchit2843" data-post="1" data-topic="53887">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sanchit2843/48/12858_2.png" class="avatar"> sanchit2843:</div>
<blockquote>
<p>will the parameters of model contain only the forward part or both</p>
</blockquote>
</aside>
<p>The parameters of the model are not related to the definition of <code>forward</code> or some other functions, but are defined inside <code>__init__</code>.</p>
<aside class="quote no-group" data-username="sanchit2843" data-post="1" data-topic="53887">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sanchit2843/48/12858_2.png" class="avatar"> sanchit2843:</div>
<blockquote>
<p>how can I make such a model where the features can be extracted separately, classifier can be made separately and then backpropagation happens on the complete model.</p>
</blockquote>
</aside>
<p>The computation graph will be created dynamically based on the forward execution.<br>
I.e. you don’t even necessarily need an <code>nn.Module</code> to wrap your parameters, but could also work directly with some parameters:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(1, 1)
w = nn.Parameter(torch.randn(1, 1))

y = x * w
y.backward()
print(w.grad)
</code></pre>
<p>This is of course not the recommended way to write your complicated models, but might give you some information about the work flow of the computation graph and Autograd.</p>
<p>The disadvantage of your current approach is, that you should usually call the model directly:</p>
<pre data-code-wrap="python"><code class="lang-python">output = model(input)
</code></pre>
<p>instead of the <code>forward</code> method or some other methods:</p>
<pre data-code-wrap="python"><code class="lang-python">output1 = model.forward(input)
output2 = model.feature_extraction(input)
</code></pre>
<p>The former approach will call the internal <code>__call__</code> method, which will e.g. register hooks (if necessary) and then call <code>forward</code>.<br>
A possible workaround would be to define <code>forward</code> with a flag to decide which path to chose:</p>
<pre data-code-wrap="python"><code class="lang-python">class model(nn.Module):
    def __init__(self,):
         super(model, self).__init__()
         self.featureextract = fext
         self.classifier = classifier
    def forward(self, input, path='class'):
        if path == 'class':
            x = self.classifier(input)
        else:
            x = self.feature_extractor(input)
        return x

   def feature_extract(self,input):
        x = self.featureextract(input)
        return x
</code></pre>

username: sanchit2843
cooked: <p>Thanks a lot sir I got it now.</p>