How do you pick your dimensions for a neural net

adomakor412: <p>I’ve been struggling with this patch of code where I cannot seem to get the size of my matrices right. This one is based on a batch size of 4.</p>
<p><a href="https://github.com/adomakor412/NERTO_CIMSS_GOES-R/blob/a35f882fe15f68cb6df22bc5c28d503ca18a88b3/AUC_model.ipynb" rel="noopener nofollow ugc">https://github.com/adomakor412/NERTO_CIMSS_GOES-R/blob/a35f882fe15f68cb6df22bc5c28d503ca18a88b3/AUC_model.ipynb</a></p>
<p>If you have trouble rendering the code use <a href="https://nbviewer.jupyter.org/" rel="noopener nofollow ugc">https://nbviewer.jupyter.org/</a></p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 2)
#         super(Net, self).__init__()
#         self.conv1 = nn.Conv2d(3, 32, 5)
#         self.pool = nn.MaxPool2d(2, 2)
#         self.conv2 = nn.Conv2d(32, 64, 5)
#         self.fc1 = nn.Linear(64*9*9, 1024)
#         self.fc2 = nn.Linear(1024, 7)
    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1) # flatten all dimensions except batch
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
#         x = self.pool(F.relu(self.conv1(x)))
#         x = self.pool(F.relu(self.conv2(x)))
#         x = x.view(x.size(0), -1)
#         #x = x.view(-1, 64)
#         #x = F.relu(self.fc1(x))
#         #x = F.relu(self.fc2(x))
#         #x = self.fc2(x)
        return x
net = Net()
</code></pre>
<p>And when try the net() this way, I get an index out of range error:</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
#         super().__init__()
#         self.conv1 = nn.Conv2d(3, 6, 5)
#         self.pool = nn.MaxPool2d(2, 2)
#         self.conv2 = nn.Conv2d(6, 16, 5)
#         self.fc1 = nn.Linear(16 * 5 * 5, 120)
#         self.fc2 = nn.Linear(120, 84)
#         self.fc3 = nn.Linear(84, 2)
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(32, 64, 5)
        self.fc1 = nn.Linear(64*9*9, 1024)
        self.fc2 = nn.Linear(1024, 7)
    def forward(self, x):
#         x = self.pool(F.relu(self.conv1(x)))
#         x = self.pool(F.relu(self.conv2(x)))
#         x = torch.flatten(x, 1) # flatten all dimensions except batch
#         x = F.relu(self.fc1(x))
#         x = F.relu(self.fc2(x))
#         x = self.fc3(x)
#         return x
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(x.size(0), -1)
        #x = x.view(-1, 64)
        #x = F.relu(self.fc1(x))
        #x = F.relu(self.fc2(x))
        #x = self.fc2(x)
        return x
</code></pre>

adomakor412: <aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/adomakor412/NERTO_CIMSS_GOES-R/blob/d7097d8850d8d764c1508bbe3d0e92d34a2c18bd/AUC_model.ipynb" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/adomakor412/NERTO_CIMSS_GOES-R/blob/d7097d8850d8d764c1508bbe3d0e92d34a2c18bd/AUC_model.ipynb" target="_blank" rel="noopener nofollow ugc">adomakor412/NERTO_CIMSS_GOES-R/blob/d7097d8850d8d764c1508bbe3d0e92d34a2c18bd/AUC_model.ipynb</a></h4>
<pre><code class="lang-ipynb">{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "large-stamp",
   "metadata": {},
   "source": [
    "# AI model for training (shark fin) artifacts on GOES-R\n",
    "\n",
    "[How to Train an Image Classifier in PyTorch and use it to Perform Basic Inference on Single Images](https://towardsdatascience.com/how-to-train-an-image-classifier-in-pytorch-and-use-it-to-perform-basic-inference-on-single-images-99465a1e9bf5)\n",
    "\n",
    "[TRAINING A CLASSIFIER](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)\n",
    "\n",
    "[VISUALIZING MODELS, DATA, AND TRAINING WITH TENSORBOARD–Area Under Curve](https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html?highlight=area%20under%20curve)\n",
    "\n",
    "\n",
    "[PRECISION-RECALL Curves](https://www.scikit-yb.org/en/latest/api/classifier/prcurve.html)"
   ]
  },
  {
</code></pre>

  This file has been truncated. <a href="https://github.com/adomakor412/NERTO_CIMSS_GOES-R/blob/d7097d8850d8d764c1508bbe3d0e92d34a2c18bd/AUC_model.ipynb" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


adomakor412: <p>There are only two classes in my target label.</p>

ptrblck: <p>Your initial model works fine for an input shape of <code>[batch_size, 3, 32, 32]</code>:</p>
<pre><code class="lang-python">class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 2)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1) # flatten all dimensions except batch
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()
x = torch.randn(3, 3, 32, 32)
out = net(x)
print(out.shape)
&gt; torch.Size([3, 2])
</code></pre>
<p>What error are you getting and could you post an executable code snippet, which would reproduce it?</p>

adomakor412: <p>How did you figure out my initial model would work for an input shape of [batch_size, 3, 32, 32]?</p>

adomakor412: <p>Here is the error and snippet of code when attempted batch size of 4:</p>
<pre><code class="lang-auto">running_loss = 0.0
for epoch in range(1):  # loop over the dataset multiple times
    
    if len(list(enumerate(trainloader, 0))) &lt; 10:
            print('Batch size too small (&lt;10) for PR-curve')
            
    for i, data in enumerate(trainloader, 0):
        
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if i % 10 == 9:    # every 1000 mini-batches...
#         if i % 1000 == 999:    # every 1000 mini-batches...

            # ...log the running loss
            writer.add_scalar('training loss',
                            running_loss / 1000,
                            epoch * len(trainloader) + i)

            # ...log a Matplotlib Figure showing the model's predictions on a
            # random mini-batch
            writer.add_figure('predictions vs. actuals',
                            plot_classes_preds(net, inputs, labels),
                            global_step=epoch * len(trainloader) + i)
            running_loss = 0.0
        
print('Finished Training')
</code></pre>
<blockquote>
<h2>0 index of predictions with length 4: 49254<br>
1 index of predictions with length 4: 79865<br>
2 index of predictions with length 4: 44043<br>
3 index of predictions with length 4: 43242</h2>
<p>IndexError                                Traceback (most recent call last)<br>
 in <br>
31             # random mini-batch<br>
32             writer.add_figure(‘predictions vs. actuals’,<br>
—&gt; 33                             plot_classes_preds(net, inputs, labels),<br>
34                             global_step=epoch * len(trainloader) + i)<br>
35             running_loss = 0.0</p>
<p> in plot_classes_preds(net, images, labels)<br>
31         matplotlib_imshow(images[idx], one_channel=True)<br>
32         ax.set_title(“{0}, {1:.1f}%\n(label: {2})”.format(<br>
—&gt; 33             classes[preds[idx]],<br>
34             probs[idx] * 100.0,<br>
35             classes[labels[idx]]),</p>
<p>IndexError: list index out of range</p>
</blockquote>

adomakor412: <p>So for the LeNet model this is how the convolution is designed for 32x32 image sizes:</p>
<p><a href="https://stackoverflow.com/questions/57822568/deep-learning-with-pytorch-understanding-the-neural-network-example" rel="noopener nofollow ugc">Deep learning with Pytorch: understanding the neural network example</a></p>

ptrblck: <aside class="quote no-group" data-username="adomakor412" data-post="5" data-topic="120682" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/eada6e/48.png" class="avatar"> adomakor412:</div>
<blockquote>
<p>How did you figure out my initial model would work for an input shape of [batch_size, 3, 32, 32]?</p>
</blockquote>
</aside>
<p>It was a guess based on your model definition.</p>
<p>The indexing error is raised in <code>plot_classes_preds</code>, where <code>classes[preds[idx]]</code> uses an invalid index.<br>
Could you check the shape of <code>preds</code> and the value of <code>idx</code> as well as the shape of <code>classes</code> and the value of <code>preds[idx]</code> as one of them is using an invalid indexing?</p>