Do we always need to define the forward function for a subclass of nn.Module?

entslscheia: <p>A common practice of developing neural models is to define a subclass of <code>nn.Module</code> with both <code>__init__</code> function and <code>forward</code> function inside it. Then we can create an instance <code>model</code> of the class we defined and call <code>forward</code> function directly use <code>model(...)</code>. I have never questioned this before, until I found a PyTorch project on GitHub that the author doesn’t really define <code>forward</code> function, instead, the author create his own function to compute the loss based on the modules defined inside <code>__init__</code>. I don’t understand why he does that. In principle, are we always recommended using <code>forward</code> function to define the computation we want our model to do? If we don’t use <code>forward</code> and instead define our own function and then call the function explicitly, will there be any problem or disadvantage?</p>

ptrblck: <p>If you define an <code>nn.Module</code>, you are usually storing some submodules, parameters, buffers or other arguments in its <code>__init__</code> method and write the actual forward logic in its <code>forward</code> method.<br>
This is a convenient method as <code>nn.Module.__call__</code> will register hooks etc. and call finally into the <code>forward</code> method.<br>
However, you don’t need to use this approach and could completely write your model in a functional way.</p>
<p>I’ve written some more information in <a href="https://discuss.pytorch.org/t/beginner-should-relu-sigmoid-be-called-in-the-init-method/18689/6">this thread</a> which might be interesting to you.</p>

entslscheia: <p>Thanks!<br>
But if we define the behavior of the neural model in our own function instead of defining <code>forward</code> and call <code>__call__</code>, do we need to take care of the hooks in our own function? Since <code>__call__</code> will take care of all the hooks. Specifically, if we define a function called <code>func</code>, and then call <code>model.func</code> to do the computation, we will need to take care of hooks inside <code>func</code> by ourselves right?<br>
Actually, I don’t quite get the necessity for the hooks. What are they essentially for? I suppose the hooks do not really affect the forward pass and back propagation. They are only used to do some auxiliary work such as visualizing the neural nets, e.t.c. In another words, our neural models can still work properly even we don’t register the hooks, e.g., we define the computation using our own entry point instead of <code>forward</code>.</p>

ptrblck: <p>Yes, your model might work without the <code>forward</code> usage.<br>
However, utility functions such as <code>nn.DataParallel</code> rely on the <code>__call__</code> method and thus on the implementation of <code>forward</code>.<br>
Also, if other users would like to use your model, they would have to reimplement the forward pass in the <code>forward</code>, if they want to use hooks.</p>
<p>Hooks can be used for debugging or visualizations. But you could also easily add auxiliary losses to your model using forward hooks or some gradient regularization using <code>register_hook</code>.</p>
<p>I don’t see much benefit in using custom names.</p>