FashionMNIST- CNN architecture and have a poblem with the train-loss which show : nan

flora1: <p>We decided to implement the CNN architecte which looks like this</p>
<h1><a name="implement-the-mnistnet-network-architecture-1" class="anchor" href="#implement-the-mnistnet-network-architecture-1"></a>implement the MNISTNet network architecture</h1>
<p>class FashionMNISTNet(nn.Module):</p>
<pre><code># define the class constructor
def __init__(self):
    
    # call super class constructor
    super(FashionMNISTNet, self).__init__()
    
    # specify convolution layer 1
    self.layer_1 = nn.Sequential(
        nn.Conv2d(in_channels=1, out_channels=32, kernel_size=3, padding=1),
        nn.BatchNorm2d(32),
        nn.ReLU(),
        nn.MaxPool2d(kernel_size=2, stride=2)
    )

    #specify convultion layer 2
    self.layer_2 = nn.Sequential(
        nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3),
        nn.BatchNorm2d(64),
        nn.ReLU(),
        nn.MaxPool2d(2)
    )

    self.linear1 = nn.Linear(64*6*6, 600)

    self.drop = nn.Dropout2d(0.25)

    self.linear2 = nn.Linear(600, 120)
    
    self.linear3 = nn.Linear(120, 10)
    
    # add a softmax to the last layer
    # self.logsoftmax = nn.LogSoftmax(dim=1) # the softmax
    
# define network forward pass
def forward(self, images):
    
    x = self.layer_1(images)

    x = self.layer_2(x)

    x = x.view(x.size(0), -1)

    x = self.linear1(x)

    x = self.drop(x)

    x = self.linear2(x)

    x = self.linear3(x)

    # define layer 3 forward pass
    # x = self.logsoftmax(self.linear3(x))
    
    # return forward pass result
    return x
</code></pre>
<p>When we try to run the Network Training with the same code (with the exception of mini_batch_size = 100), all the train_epoch_loss display “nan”:</p>
<p>and the error show this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2cbbbebd7c1763b9f7d22b32360508cd0d381654.png" data-download-href="https://discuss.pytorch.org/uploads/default/2cbbbebd7c1763b9f7d22b32360508cd0d381654" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2cbbbebd7c1763b9f7d22b32360508cd0d381654.png" alt="image" data-base62-sha1="6nJfxB6yBQaytkWFH47cHeb2aeE" width="690" height="205" data-dominant-color="3B3C3C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1478×441 28 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Does anyone know how to fix this error?</p>
<p>Thank you very much!</p>

pchandrasekaran: <p>You’ll need to reshape/unsqueeze your inputs as <code>[10000, 1, 28, 28]</code>. You need to this this because the network expects inputs of shape <code>[_, channel, height, width]</code>.</p>

flora1: <p>Thanks! On which line should I implement this?</p>

pchandrasekaran: <p>You could do it in your forward function as <code>images=images.unsqueeze(dim=1)</code>. However, it would be better if you include this step in your data preprocessing pipeline.</p>

flora1: <aside class="quote no-group" data-username="pchandrasekaran" data-post="4" data-topic="122398">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pchandrasekaran/48/29605_2.png" class="avatar"> pchandrasekaran:</div>
<blockquote>
<p>images=images.unsqueeze(dim=1)</p>
</blockquote>
</aside>
<p>You mean here :<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/9/2917e6a19e561f14e8dae213dc7698288ed46c13.png" data-download-href="https://discuss.pytorch.org/uploads/default/2917e6a19e561f14e8dae213dc7698288ed46c13" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/9/2917e6a19e561f14e8dae213dc7698288ed46c13.png" alt="image" data-base62-sha1="5RwMAGLb8Lkmts8uWubHzK2mk7x" width="648" height="500" data-dominant-color="F3F4F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">776×598 11.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>It does not seem to work, could you be more precise?<br>
I am fairyl new to the coding world. <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>

pchandrasekaran: <pre><code class="lang-auto">def forward(self, images):
    images=images.unsqueeze(dim=1)
    x = self.layer_1(images)
    x = self.layer_2(x)
    x = x.view(x.size(0), -1)
    .
    .
    .
</code></pre>
<p>You need to perform the step before you begin passing it into the network.</p>

flora1: <p>Thanks. now this work:</p>
<h1><a name="define-network-forward-pass-1" class="anchor" href="#define-network-forward-pass-1"></a>define network forward pass</h1>
<pre><code>def forward(self, images):

    images=images.unsqueeze(dim=1)

         

    x = self.layer_1(images)

    x = self.layer_2(x)

    x = x.view(x.size(0), -1)

    

    x = self.linear1(x)

    x = self.drop(x)

    x = self.linear2(x)

    x = self.linear3(x)

    

    # define layer 3 forward pass

    # x = self.logsoftmax(self.linear3(x))

    

    # return forward pass result

    return x
</code></pre>
<p>But it still leads to an issue here:</p>
<h1><a name="init-collection-of-training-epoch-losses-2" class="anchor" href="#init-collection-of-training-epoch-losses-2"></a>init collection of training epoch losses</h1>
<p>train_epoch_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<h1><a name="set-the-model-in-training-mode-3" class="anchor" href="#set-the-model-in-training-mode-3"></a>set the model in training mode</h1>
<p>model.train()</p>
<h1><a name="train-the-mnistnet-model-4" class="anchor" href="#train-the-mnistnet-model-4"></a>train the MNISTNet model</h1>
<p>for epoch in range(num_epochs):</p>
<pre><code># init collection of mini-batch losses

train_mini_batch_losses = []



# iterate over all-mini batches

for i, (images, labels) in enumerate(fashion_mnist_train_dataloader):

    

    # push mini-batch data to computation device

    images = images.to(device)

    labels = labels.to(device)

    # run forward pass through the network

    outputs = model(images)

            

    # reset graph gradients

    model.zero_grad()

    

    # determine classification loss

    loss = nll_loss(outputs, labels)

    

    # run backward pass

    loss.backward()

    

    # update network paramaters

    optimizer.step()

    

    # collect mini-batch reconstruction loss

    train_mini_batch_losses.append(loss.data.item())



# determine mean min-batch loss of epoch

train_epoch_loss = np.mean(train_mini_batch_losses)



# print epoch loss

now = datetime.utcnow().strftime("%Y%m%d-%H:%M:%S")

print('[LOG {}] epoch: {} train-loss: {}'.format(str(now), str(epoch), str(train_epoch_loss)))



# set filename of actual model

model_name = 'fashion_mnist_model_epoch_{}.pth'.format(str(epoch))

# save current model to GDrive models directory

torch.save(model.state_dict(), os.path.join(models_directory, model_name))

        

# determine mean min-batch loss of epoch

train_epoch_losses.append(train_epoch_loss)
</code></pre>
<p>with this error message:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/f/2f5612cc03b2fd227ed48df99945c4ca9edb3814.png" data-download-href="https://discuss.pytorch.org/uploads/default/2f5612cc03b2fd227ed48df99945c4ca9edb3814" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f5612cc03b2fd227ed48df99945c4ca9edb3814_2_690x211.png" alt="image" data-base62-sha1="6KKQRirhVrU5Ik4CxjyO9LSrb00" width="690" height="211" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f5612cc03b2fd227ed48df99945c4ca9edb3814_2_690x211.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f5612cc03b2fd227ed48df99945c4ca9edb3814_2_1035x316.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f5612cc03b2fd227ed48df99945c4ca9edb3814_2_1380x422.png 2x" data-dominant-color="F9F8FA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1467×450 27.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>