The problem CNN

randino: <p>When I tested with Softmax the result of fully-Connected and softmax :</p>
<pre><code class="lang-auto">with torch.no_grad():    
        output = model((val_x.float()),(val_x1.float()))
        val_y = val_y.to(device)    
        softmax = torch.exp(output).cpu() 
        print("softmax",softmax)      
        prob = list(softmax.numpy())
        print("softmax de test", prob)
        predictions = np.argmax(prob, axis=1)
        print("prediction test", predictions)
       print('Validation accuracy test: {:.4f}%'.format(float(accuracy_score(val_y, predictions)) * 100))
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/0/40149eaa0aa79b6946897dd8be724f432d12adbe.png" data-download-href="https://discuss.pytorch.org/uploads/default/40149eaa0aa79b6946897dd8be724f432d12adbe" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/0/40149eaa0aa79b6946897dd8be724f432d12adbe.png" alt="image" data-base62-sha1="98SJCmVKRHF0oiYECRjYMbZr1BI" width="690" height="130" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1034×195 7.02 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/b/2bf553325c8610c2884a7922186a93f4bedc280f.png" alt="image" data-base62-sha1="6gS8wTCR8AjHm5AORsTyyFVVZoz" width="292" height="292"></p>
<p>but the values of softmax is not between[0,1] where is the problem?? the accuracy is 70,52%</p>

ptrblck: <p>Based on the output of “Fully-Connected” I would guess your model returns logits, so you should call <code>F.softmax</code> on the output to get the probabilities.</p>

randino: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  it’s the same</p>
<pre><code class="lang-auto">with torch.no_grad():    
        output = model((val_x.float()),(val_x1.float()))
        val_y = val_y.to(device)    
        softmax = torch.exp(output).cpu() 
        softmax= F.softmax(softmax, dim = 1)  
        print("softmax",softmax)      
        prob = list(softmax.numpy())
        print("softmax de test", prob)
        predictions = np.argmax(prob, axis=1)
       print('Validation accuracy test: {:.4f}%'.format(float(accuracy_score(val_y, predictions)) * 100))
</code></pre>
<p>the values of softmax is not between [0,1]</p>

ptrblck: <p>How did you check the min. and max. values of the softmax?<br>
If you have used:</p>
<pre><code class="lang-python">softmax= F.softmax(softmax, dim = 1) 
print(softmax.min(), softmax.max())
</code></pre>
<p>and the values are still out-of-bounds, please post an executable code snippet to reproduce it.</p>

randino: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  why the results of softmax are in two columns?<br>
like that :<br>
softmax de test [array([0.35895655, 0.6410435 ], dtype=float32), array([0.5, 0.5], dtype=float32), array([0.38063172, 0.61936826], dtype=float32), array([0.30245292, 0.6975471 ], dtype=float32), array([0.00381802, 0.99618196], dtype=float32),…, array([0.5064734 , 0.49352652], dtype=float32).</p>

ptrblck: <p>The softmax operation normalizes the specified <code>dim</code> so that its sum equals to <code>1.</code> and the results can be interpreted as a probability.<br>
In your case the two values correspond to the probability for <code>class0</code> and <code>class1</code> and their sum is <code>1</code>.<br>
Take a look at the <a href="https://en.wikipedia.org/wiki/Softmax_function">Softmax Wikipedia article</a> (or any other resource) for more information.</p>

randino: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  thanks for your reply.</p>
<p>just an information, when I tested the network with 8 convolutional layers, and 2 pooling.</p>
<p>the results are in this form :</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/a/0ae5d0d73793346614699b48394d86b1ac610fde.png" alt="image" data-base62-sha1="1yp9taMV9nhkgyYYOjxr7wa6pbw" width="238" height="467"><br>
I haven’t finished all the values<br>
so i can’t display the results as an image??</p>

ptrblck: <p>I think visualizing tensors and arrays was already discussed in <a href="https://discuss.pytorch.org/t/problem-cnn-picture/133945">this thread</a>.</p>
<p>I don’t know what shape the tensor in the current screenshot has, but as already described you will be able to visualize tensors using <code>plt.imshow</code> as long as they have a valid image shape.<br>
I’m also unsure why the values are again negative, but assume you are not using the <code>softmax</code> operation anymore.</p>

randino: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  the values that I transmitted to you, are just the results of the convolution layers.<br>
I want to substruct the results of the convolutional layers of x and x1.</p>
<pre><code class="lang-auto"> out = self.cnn1(x)
        out = self.batchnorm1(out)
        out = self.relu(out)
        out = self.maxpool1(out)
        out = self.cnn2(out)
        out = self.batchnorm2(out)
        out = self.relu(out)
        out = self.maxpool2(out)
        out = self.cnn3(out)
        out = self.batchnorm3(out)
        out = self.relu(out)
        out = self.cnn4(out)
        out = self.batchnorm4(out)
        out = self.relu(out)
        out = self.cnn5(out)
        out = self.batchnorm5(out)
        out = self.relu(out)
       
        out1 = self.cnn1(x2)
        out1 = self.batchnorm1(out1)
        out1 = self.relu(out1)
        out1 = self.maxpool1(out1)
        out1 = self.cnn2(out1)
        out1 = self.batchnorm2(out1)
        out1 = self.relu(out1)
        out1 = self.maxpool2(out1)
        out1 = self.cnn3(out1)
        out1 = self.batchnorm3(out1)
        out1 = self.relu(out1)
        out1 = self.cnn4(out1)
        out1 = self.batchnorm4(out1)
        out1 = self.relu(out1)
        out1 = self.cnn5(out1)
        out1 = self.batchnorm5(out1)
        out1 = self.relu(out1)
     
        out2 = out - out1
       print(out2)
</code></pre>