What should be the output of the Sigmoid layer

shrbrh: <p>I have used the Sigmoid layer as the output layer for the discriminator of a GAN model. The discriminator is supposed to classify fake and real images. The output of the Sigmoid layer produces a tensor of shape</p>
<pre><code class="lang-auto">torch.Size([1, 1, 16, 16])
</code></pre>
<p>Here is an example of the output:</p>
<pre><code class="lang-auto">tensor([[[[0.9162, 0.9692, 0.9791, 0.9853, 0.9897, 0.9900, 0.9909, 0.9912,
           0.9914, 0.9888, 0.9846, 0.9822, 0.9871, 0.9891, 0.9783, 0.8980],
          [0.9801, 0.9961, 0.9977, 0.9984, 0.9989, 0.9991, 0.9992, 0.9992,
           0.9992, 0.9991, 0.9987, 0.9986, 0.9989, 0.9990, 0.9976, 0.9671],
          [0.9874, 0.9984, 0.9991, 0.9993, 0.9994, 0.9994, 0.9996, 0.9996,
           0.9995, 0.9994, 0.9994, 0.9994, 0.9994, 0.9995, 0.9988, 0.9797],
          [0.9878, 0.9987, 0.9993, 0.9994, 0.9993, 0.9994, 0.9995, 0.9995,
           0.9994, 0.9994, 0.9993, 0.9992, 0.9992, 0.9993, 0.9988, 0.9821],
          [0.9880, 0.9987, 0.9994, 0.9994, 0.9994, 0.9994, 0.9994, 0.9994,
           0.9994, 0.9993, 0.9990, 0.9988, 0.9988, 0.9991, 0.9986, 0.9830],
          [0.9885, 0.9988, 0.9994, 0.9994, 0.9994, 0.9994, 0.9993, 0.9993,
           0.9993, 0.9993, 0.9991, 0.9988, 0.9986, 0.9989, 0.9986, 0.9832],
          [0.9893, 0.9989, 0.9994, 0.9994, 0.9993, 0.9992, 0.9991, 0.9992,
           0.9993, 0.9992, 0.9993, 0.9990, 0.9990, 0.9991, 0.9986, 0.9835],
          [0.9892, 0.9990, 0.9994, 0.9993, 0.9991, 0.9990, 0.9990, 0.9991,
           0.9991, 0.9991, 0.9993, 0.9993, 0.9992, 0.9993, 0.9987, 0.9823],
          [0.9893, 0.9991, 0.9995, 0.9992, 0.9991, 0.9991, 0.9992, 0.9992,
           0.9989, 0.9989, 0.9991, 0.9992, 0.9993, 0.9992, 0.9982, 0.9788],
          [0.9902, 0.9993, 0.9996, 0.9994, 0.9990, 0.9991, 0.9994, 0.9995,
           0.9992, 0.9991, 0.9991, 0.9992, 0.9991, 0.9990, 0.9977, 0.9770],
          [0.9906, 0.9993, 0.9997, 0.9995, 0.9993, 0.9992, 0.9995, 0.9996,
           0.9996, 0.9994, 0.9994, 0.9994, 0.9994, 0.9993, 0.9982, 0.9789],
          [0.9903, 0.9993, 0.9997, 0.9996, 0.9994, 0.9993, 0.9995, 0.9997,
           0.9997, 0.9996, 0.9995, 0.9996, 0.9996, 0.9994, 0.9985, 0.9816],
          [0.9882, 0.9989, 0.9995, 0.9995, 0.9994, 0.9991, 0.9992, 0.9995,
           0.9996, 0.9996, 0.9996, 0.9996, 0.9996, 0.9996, 0.9990, 0.9846],
          [0.9877, 0.9986, 0.9992, 0.9992, 0.9991, 0.9989, 0.9990, 0.9992,
           0.9995, 0.9996, 0.9996, 0.9996, 0.9996, 0.9996, 0.9992, 0.9866],
          [0.9826, 0.9974, 0.9983, 0.9976, 0.9976, 0.9971, 0.9974, 0.9980,
           0.9989, 0.9992, 0.9992, 0.9991, 0.9992, 0.9991, 0.9982, 0.9778],
          [0.9283, 0.9793, 0.9815, 0.9768, 0.9749, 0.9744, 0.9762, 0.9795,
           0.9845, 0.9877, 0.9886, 0.9877, 0.9883, 0.9881, 0.9800, 0.9083]]]]
</code></pre>
<p>Shouldn’t the output be a single probability value ranging between 0 and 1 instead of a tensor?</p>

J_Johnson: <p><code>nn.Sigmoid</code> applies the sigmoid function to all inputs separately.</p>
<aside class="onebox wikipedia" data-onebox-src="https://en.m.wikipedia.org/wiki/Sigmoid_function">
  <header class="source">

      <a href="https://en.m.wikipedia.org/wiki/Sigmoid_function" target="_blank" rel="noopener nofollow ugc">en.m.wikipedia.org</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:320/213;"><img src="//upload.wikimedia.org/wikipedia/commons/thumb/8/88/Logistic-curve.svg/320px-Logistic-curve.svg.png" class="thumbnail" width="320" height="213"></div>

<h3><a href="https://en.m.wikipedia.org/wiki/Sigmoid_function" target="_blank" rel="noopener nofollow ugc">Sigmoid function</a></h3>

<p>
				Pages for logged out editors learn more

			 

 A sigmoid function is a mathematical function having a characteristic "S"-shaped curve or sigmoid curve.
 A common example of a sigmoid function is the logistic function shown in the first figure and defined by the formula:
</p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>If you want to get one value out of your model, you could either down sample it once more, or use:</p>
<pre><code class="lang-auto">x=x.flatten(start_dim=2).mean(dim=2)
</code></pre>

shrbrh: <p><a class="mention" href="/u/j_johnson">@J_Johnson</a>  I am a bit confused about the dim part. What is the significance of start_dim and dim here and why do you set them to 2 specifically?<br>
Will it create any discrepancies with the actual values?</p>

J_Johnson: <p><code>start_dim</code> just tells the function what to flatten and what to leave in separate dimensions.  For example, in this case you might have a size [16, 3, 32, 32] —&gt; [16, 3, 32*32].</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.flatten.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.flatten.html</a></p>
<p><code>dim</code> in the <code>.mean()</code> function tells it what dimension to take the mean of. For example, we want the mean of each channel, vs. the mean of the entire batch.</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.mean.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.mean.html</a></p>
<p>Both functions together are flattening the output channels and getting the mean of each channel.</p>