Detailed explanation on nn creation

tomera: <p>Hi, I started to learn PyTorch after doing Machine learning course in Coursera.<br>
Reading the docs I find that Im unfamiliar with the terms and the logic of the nn creation.</p>
<p>For example this code:</p>
<pre><code class="lang-auto">    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x
</code></pre>
<p>What is relu? why we need to call it twice? why we need to use x.view after this? why we need to call fc1/2/3? I dont get the intuition of this forward function.<br>
I fond example of a “simple” nn design with a forward function like this which is how I was thought nn is built:</p>
<pre><code class="lang-auto"> #activation function ==&gt; S(x) = 1/1+e^(-x)
    def sigmoid(self, x, deriv=False):
        if deriv == True:
            return x * (1 - x)
        return 1 / (1 + np.exp(-x))

    # data will flow through the neural network.
    def feed_forward(self):
        self.hidden = self.sigmoid(np.dot(self.inputs, self.weights))
</code></pre>
<p>I also dont understand the init function:</p>
<pre><code class="lang-auto">    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
</code></pre>
<p>I tried to go for the docs but Im still clueless about the logic flow of the init function. Why we need t occall Conv2d and then MaxPool2d and then Linear.</p>
<p>What channels means? why do we need channels? I know the simple design on input layer, hidden layer and output layer.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/d/8d57e4d8774c1c2be35d6718fc332c9988549797.jpeg" alt="Capture" data-base62-sha1="kanDnJheVBL9yjHAnbfQkimIJFR" width="688" height="414"></p>
<p>The simple code for nn as I learned it is found here:<br>
<a href="https://towardsdatascience.com/inroduction-to-neural-networks-in-python-7e0b422e6c24" class="onebox" target="_blank" rel="noopener nofollow ugc">https://towardsdatascience.com/inroduction-to-neural-networks-in-python-7e0b422e6c24</a></p>
<p>Does PyTorch have a detailed explanation for each line of the nn design, explaining to a total newbie in PyTorch.</p>
<p>Thanks.</p>

G.M: <p>ReLU:  activation function; usually, we apply an activation after each layer.<br>
<code>.view</code>: This is for changing the shape of tensors without copying it.<br>
Linear layer: it is also called Fully Connected layer. In Pytorch a FC takes in an input that’s 2D.<br>
fc1/2/3: That’s just variable names.<br>
“Calling maxpool2d after Conv2d”: Take a look at the common structure of a Conv net.<br>
channels: Channels are originally used for describing colors of an image, for example RGB images have 3 channels. Here, “channel” represents a dimension of the tensor, it is usually the second dimension of an input tensor for Conv2d.</p>
<p>Taking a look at this <a href="https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html" rel="nofollow noopener">page</a> would probably get u some help.</p>

tomera: <p>Thanks for answering, Im afraid I still dont understand enough, the docs are not detailed enough for me.<br>
I looked in the NN tutorial:</p>
<pre><code class="lang-auto">def __init__(self):
        super(Net, self).__init__()
        # 1 input image channel, 6 output channels, 3x3 square convolution
        # kernel
        self.conv1 = nn.Conv2d(1, 6, 3)
        self.conv2 = nn.Conv2d(6, 16, 3)
        # an affine operation: y = Wx + b
        self.fc1 = nn.Linear(16 * 6 * 6, 120)  # 6*6 from image dimension
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
</code></pre>
<p>Im still not sure what Conv2d does. From the docs:</p>
<blockquote>
<p>Applies a 2D convolution over an input signal composed of several input planes.</p>
</blockquote>
<p>Conv2d takes high dimension and put it in a 2D matrix? but why we want do it? and why the choosed the input to be (6, 16, 3) in the second line of  <code>self.conv2 = nn.Conv2d(6, 16, 3)</code>. In addition what output channel means for Conv2d? Why we need to define here output channel while we already do it in the <code>nn.Linear</code></p>
<p>In this code lines:</p>
<pre><code class="lang-auto">        self.fc1 = nn.Linear(16 * 6 * 6, 120)  # 6*6 from image dimension
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
</code></pre>
<p>Why they choose the hidden layer units to be with 120 and 84 units?</p>
<p>In the forward function:</p>
<pre><code class="lang-auto">def forward(self, x):
        # Max pooling over a (2, 2) window
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
        # If the size is a square you can only specify a single number
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)
        x = x.view(-1, self.num_flat_features(x))
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x
</code></pre>
<p>Im not sure why we need the max_pool2d function, from the docs:</p>
<blockquote>
<p>Applies a 2D max pooling over an input signal composed of several input planes.</p>
</blockquote>
<p>What is it max pooling and why do we need it?</p>
<p>Thanks.</p>