Is there any hope for this neural network?

PirateHunterZoro: <p>Hello, everyone!<br>
So I am currently working on creating a neural network which I hope to train to classify ten different types of plant pictures, which I took, cropped, and compressed myself. Everything in the code physically works, but I just cannot seem to get the network to reach any reasonable degree of accuracy. Any recommendations on my hyper parameters? Number of convolutions, layers, sizes, etc.? Should I use dropout? Batch normalization? How many epochs of training, and what learning rate? Adam optimization, or just regular gradient descent? Are my images simply too big and there’s no hope? Thanks! Attached is an example of one of my images - they are all colored, 1000x1000, and I have around 800 of them. I’d appreciate any tips people could give me! Attached is a screenshot of the results of the most recent training (which took my computer all day since I do not have a GPU <img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=12" title=":sob:" class="emoji" alt=":sob:" loading="lazy" width="20" height="20">). It’s not pretty… any tips anyone has would be much appreciated!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/2/f20588fd0d6afed4d1d5140e87a8e5e631063d60.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f20588fd0d6afed4d1d5140e87a8e5e631063d60" title="Screen Shot 2022-06-15 at 6.31.03PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f20588fd0d6afed4d1d5140e87a8e5e631063d60_2_690x463.jpeg" alt="Screen Shot 2022-06-15 at 6.31.03PM" data-base62-sha1="yx1pdCErkA7Lr2lxbZoxWPZgIw0" width="690" height="463" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f20588fd0d6afed4d1d5140e87a8e5e631063d60_2_690x463.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f20588fd0d6afed4d1d5140e87a8e5e631063d60_2_1035x694.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f20588fd0d6afed4d1d5140e87a8e5e631063d60_2_1380x926.jpeg 2x" data-dominant-color="EDEBEB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2022-06-15 at 6.31.03PM</span><span class="informations">1920×1289 109 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here’s an example of one of my images:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/6/26c8d6cd1d366a3c990d9a36f81166818f271a62.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/26c8d6cd1d366a3c990d9a36f81166818f271a62" title="IMG_1360"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/26c8d6cd1d366a3c990d9a36f81166818f271a62_2_500x500.jpeg" alt="IMG_1360" data-base62-sha1="5x6rfNC8VURx7zNmdJIgskXwCaK" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/26c8d6cd1d366a3c990d9a36f81166818f271a62_2_500x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/26c8d6cd1d366a3c990d9a36f81166818f271a62_2_750x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/26c8d6cd1d366a3c990d9a36f81166818f271a62_2_1000x1000.jpeg 2x" data-dominant-color="738B4D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">IMG_1360</span><span class="informations">1000×1000 206 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also, once again everything in my code does physically work. But you all may have some suggestions on my network hyper parameters, so I’ll just include my code for my CNN class below:</p>
<hr>
<pre><code class="lang-auto">class CNN(nn.Module):
    
    
    def __init__(self, layers=[100,50,10], convolutions=[(3,5,1,4),(1, 5, 1, 4),(1, 5, 1, None)]):
        
        super(CNN, self).__init__()
        # so that we can now call neural network modules
        
        self.initialized = False
        
        self.dims = layers
        
        self.convs = []
        for index, convolution in enumerate(convolutions):
            if index &gt; 0:
                assert convolutions[index-1][2] == convolutions[index][0], f"Invalid input and output channel matching at convolutions {index-1} and {index}."
            
            if convolution[3] != None:
                conv_pair = (nn.Conv2d(in_channels=convolution[0], out_channels=convolution[2], kernel_size=convolution[1]), nn.MaxPool2d(convolution[3]))
            else:
                conv_pair = (nn.Conv2d(in_channels=convolution[0], out_channels=convolution[2], kernel_size=convolution[1]), None)
            self.convs.append(conv_pair)
            # all of the convolutions that go at the start of our network
        
        self.linears = nn.ModuleList()
        for this_dim, next_dim in zip(layers, layers[1:]):
            self.linears.append(nn.Linear(this_dim, next_dim))
            # all of the weights and biases
            
        self.binary = layers[-1]==1
        # binary or multiclass output?
        
        
    def forward(self, x):
        
        x = x.type(torch.FloatTensor)
        # convert x to float
        
        for conv in self.convs:
            convolution = conv[0]
            maxpool = conv[1]
            x = convolution(x)
            if maxpool != None:
                x = maxpool(x)
        # run all of the convolution on x
        
        # now we need to reshape x into one vector
        num_pixels = 1
        for num in x.shape[1:]:
            # I don't care about the NUMBER of images; only the number of pixels PER image
            num_pixels *= num
        
        x = x.view(-1, num_pixels)
        # roll out into one vector
        
        if not self.initialized:
            # then we need to get PyTorch to count the dimensions
            if self.dims[0] != x.shape[-1]:
                # then we need to create a linear function to force the fit
                new_linear = nn.Linear(x.shape[-1], self.dims[0])
                self.linears.insert(0, new_linear)
            self.initialized = True
            # now we ARE initialized - the network has seen some images
        
        for index, linear in enumerate(self.linears):
            if index == len(self.linears) - 1:
                # we ARE on the last layer
                if self.binary:
                    x = sigmoid(linear(x))
                    # run the linear function and apply sigmoid for the binary output
                else:
                    x = linear(x)
                    # otherwise for multi-class there is no activation on the output layer
            else:
                # we are NOT on the last layer
                x = relu(linear(x))
                # apply relu activation
        
        return x
        # return x
        
    
    def show_progression(self, x):
        # visually show the feedforward behavior of our network
        
        assert len(x.shape)==4 and x.shape[0]==1, "show_progression method will only deal with one image at once - if you think you only have one, wrap it in an outer tensor."
        assert self.initialized, "Network has not yet seen any images."
        
        print('Original Image:')
        plt.imshow(x[0].permute(1, 2, 0))
        plt.show()
        print(x.shape)
        # show the image
        
        x = x.type(torch.FloatTensor)
        # convert x to float
        
        for index, conv in enumerate(self.convs):
            
            print('\n')
            
            print(f'Convolution {index}:')
            convolution = conv[0]
            x = convolution(x)
            # show the image 
            
            permuted = x.permute(1,0,2,3)
            # turn it into the dimensions necessary in order to show the image via matplotlib.pyplot
            for i in range(permuted.shape[0]):
                # sort of think of this as how many 'colors'/output channels
                plt.imshow(permuted.detach()[i].permute(1,2,0))
                plt.show()
                print(x.shape)
            print('\n')
            
            maxpool = conv[1]
            if maxpool != None:
                x = maxpool(x)
                permuted = x.permute(1,0,2,3)
                # turn it into the dimensions necessary in order to show the image via matplotlib.pyplot
                print(f'Max Pool {index}:')
                for i in range(permuted.shape[0]):
                    # sort of think of this as how many 'colors'/output channels
                    plt.imshow(permuted.detach()[i].permute(1,2,0))
                    plt.show()
                    print(x.shape)
                print('\n')
                
        # run all of the convolutions and maxpools on x
        
        # now we need to reshape x into one vector
        num_pixels = 1
        for num in x.shape[1:]:
            # I don't care about the NUMBER of images; only the number of pixels PER image
            num_pixels *= num
        
        x = x.view(x.shape[0], num_pixels)
        # roll out into one vector (or a collection of vectors if multiple images)
        
        print('Unrolled Vector:')
        print(x.shape)
        
        if not self.initialized:
            # then we need to get PyTorch to count the dimensions
            if self.dims[0] != x.shape[-1]:
                # then we need to create a linear function to force the fit
                new_linear = nn.Linear(x.shape[-1], self.dims[0])
                self.linears.insert(0, new_linear)
            self.initialized = True
            # now we ARE initialized - the network has seen some images
        
        for index, linear in enumerate(self.linears):
            if index == len(self.linears) - 1:
                # we ARE on the last layer
                if self.binary:
                    x = sigmoid(linear(x))
                    # run the linear function and apply sigmoid for the binary output
                else:
                    x = linear(x)
                    # otherwise for multi-class there is no activation on the output layer
            else:
                # we are NOT on the last layer
                x = relu(linear(x))
                # apply relu activation
            
            print('\n')
            print(f'Hidden Layer {index}:')
            print(x.detach().tolist())
            print(x.shape)
</code></pre>
<p>Thanks again everyone!</p>

ptrblck: <p>I don’t think <code>self.convs</code> are trained, as you’ve assigned them to a plain Python <code>list</code> and they are thus not properly registered as submodules. Their parameters would thus also not show up in <code>model.parameters()</code> and they are most likely also not passed to the optimizer (unless you’ve manually passed them).<br>
Use an <code>nn.ModuleList</code> (as done for the linear layers) and they should be trained.</p>
<p>Also, generally you could try to scale down your use case and make sure your model is able to overfit a small dataset (e.g. just 10 samples) first by playing around with the hyperparameters.</p>

PirateHunterZoro: <p>Thank you so much! That means I’ve been doing convolutional layers wrong all semester!!! <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=12" title=":joy:" class="emoji" alt=":joy:" loading="lazy" width="20" height="20"><br>
My network actually makes progress in training now, though still does very poorly. I will follow your advice and make sure the model can overfit on a small training set, and work from there!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/7/376aa871e787398ccbe455e4e29f788c0668780a.png" data-download-href="https://discuss.pytorch.org/uploads/default/376aa871e787398ccbe455e4e29f788c0668780a" title="Screen Shot 2022-06-18 at 5.35.39PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/376aa871e787398ccbe455e4e29f788c0668780a_2_690x372.png" alt="Screen Shot 2022-06-18 at 5.35.39PM" data-base62-sha1="7UeM1pnzAPuKnfbhfmIEgtO6KNA" width="690" height="372" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/376aa871e787398ccbe455e4e29f788c0668780a_2_690x372.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/376aa871e787398ccbe455e4e29f788c0668780a_2_1035x558.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/376aa871e787398ccbe455e4e29f788c0668780a_2_1380x744.png 2x" data-dominant-color="FAF9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2022-06-18 at 5.35.39PM</span><span class="informations">2098×1134 217 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>