Understand nn Module

username: yuqli
cooked: <p>Apologize for the naiveness of the question but I’ve been struggling for days trying to understand the Pytorch tutorial examples.</p>
<p>Specially the following codes are from this link: <a href="http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#sphx-glr-beginner-blitz-neural-networks-tutorial-py" rel="nofollow noopener">http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#sphx-glr-beginner-blitz-neural-networks-tutorial-py</a></p>
<p>My quesion is :</p>
<ul>
<li>What does <strong>“def forward”</strong> do? When and how is the function called?</li>
<li>In the feedforward function, what does “<strong>x = x.view(-1, self.num_flat_features(x))</strong>” do?</li>
</ul>
<p>Thanks! Also, I might be missing some pieces in programming so I have difficulties understanding the codes, and it would be helpful if you could kindly point it out!</p>
<p>[codes start from here]</p>
<pre><code>import torch
from torch.autograd import Variable  
import torch.nn as nn
import torch.nn.functional as F


class Net(nn.Module):

def __init__(self):
    super(Net, self).__init__()
    # 1 input image channel, 6 output channels, 5x5 square convolution
    # kernel
    self.conv1 = nn.Conv2d(1, 6, 5)
    self.conv2 = nn.Conv2d(6, 16, 5)
    # an affine operation: y = Wx + b
    self.fc1 = nn.Linear(16 * 5 * 5, 120)
    self.fc2 = nn.Linear(120, 84)
    self.fc3 = nn.Linear(84, 10)

def forward(self, x):
    # Max pooling over a (2, 2) window
    x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
    # If the size is a square you can only specify a single number
    x = F.max_pool2d(F.relu(self.conv2(x)), 2)
    x = x.view(-1, self.num_flat_features(x))
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x

def num_flat_features(self, x):
    size = x.size()[1:]  # all dimensions except the batch dimension
    num_features = 1
    for s in size:
        num_features *= s
    return num_features


net = Net()
print(net)</code></pre>

username: richard
cooked: <p>The <code>forward</code> function defines how to get the output of the neural net. In particular, it is called when you apply the neural net to an input <code>Variable</code>:</p>
<pre><code class="lang-auto">net = Net()
net(input)  # calls net.forward(input)
</code></pre>
<p>The <a href="http://pytorch.org/docs/master/tensors.html#torch.Tensor.view" rel="nofollow noopener">view function</a> takes a Tensor and reshapes it.<br>
In particular, here <code>x</code> is being resized to a matrix that is <code>-1</code> by <code>self.num_flat_features(x)</code>. The <code>-1</code> isn’t actually<br>
<code>-1</code>, it autoinfers the amount necessary (see <a href="http://pytorch.org/docs/master/tensors.html#torch.Tensor.view" rel="nofollow noopener">docs</a>)<br>
<code>x = x.view(-1, self.num_flat_features(x))</code></p>
<p>What this is doing here exactly is changing <code>x</code> to a vector with <code>self.num_flat_features(x)</code> elements (the size of the first dimension is inferred to be 1.)</p>

username: yuqli
cooked: <p>Hi Richard,</p>
<p>Apologize for the late reply. With more experience I now understand your explanations better. Thanks!</p>
<p>Yuqiong</p>

username: jman4162
cooked: <p>What exactly are the flat features being defined in num_flat_features?</p>

username: yuqli
cooked: <p>So the code goes like:</p>
<pre><code class="lang-auto">def num_flat_features(self, x):
    size = x.size()[1:]  # all dimensions except the batch dimension
    num_features = 1
    for s in size:
        num_features *= s
    return num_features
</code></pre>
<p><code>x.size()[1:]</code> would return a tuple of all dimensions except the batch. e.g. if <code>x</code> is a 25x3x32x32 tensor (an image), then <code>size</code> would be <code>3x32x32</code> and thus <code>num_features</code> would be <code>3x32x32 = 3072</code>. So it’s the total number of pixels in that image. In other words <code>flat_features</code> are “flattened features”.</p>

username: apivovarov
cooked: <p>Looks like it flattens all dimensions except of the first one. It is similar to <code>torch.flatten(x, 1)</code>.</p>