How to stack adaptiveavgpool2D and softmax?

Mona_Jalal: <p>Is this code correct or what arguments should I pass to the adaptiveavgpool2D and softmax? I suspect it is wrong since it is giving me lower than expected accuracy</p>
<pre><code class="lang-auto">self.layer_attend1 =  nn.Sequential(nn.Conv2d(64, 64, stride=2, padding=1, kernel_size=3),
                                    nn.AdaptiveAvgPool2d(1),
                                    nn.Softmax(1))
</code></pre>
<p>Mainly I think the args to nn.Softmax is wrong here. Given the code is running and throws no error I find it hard to debug.</p>

ptrblck: <p>The docs describe each input argument (<a href="https://pytorch.org/docs/stable/generated/torch.nn.AdaptiveAvgPool2d.html"><code>nn.AdaptiveAvgPool2d</code></a>, <a href="https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html"><code>nn.Softmax</code></a>) so you can see that the former is using the argument as the <code>output_size</code> while the latter uses it as the <code>dim</code> argument.<br>
In case you are unsure what these arguments do, write a small code snippet to check its usage, e.g. via:</p>
<pre><code class="lang-python">pool = nn.AdaptiveAvgPool2d(1)
x = torch.randn(2, 3, 4, 4)
out = pool(x)
print(out.shape)
&gt; torch.Size([2, 3, 1, 1]) # returns an output with the specified output_size


sm = nn.Softmax(1)
out = sm(x) # applies to softmax in the specified dimension
print(out.sum(1))
&gt; tensor([[[1.0000, 1.0000, 1.0000, 1.0000],
           [1.0000, 1.0000, 1.0000, 1.0000],
           [1.0000, 1.0000, 1.0000, 1.0000],
           [1.0000, 1.0000, 1.0000, 1.0000]],

          [[1.0000, 1.0000, 1.0000, 1.0000],
           [1.0000, 1.0000, 1.0000, 1.0000],
           [1.0000, 1.0000, 1.0000, 1.0000],
           [1.0000, 1.0000, 1.0000, 1.0000]]])
</code></pre>

Mona_Jalal: <p>yeah i have done that and there wasn’t anything problematic though i think i should use a relu between adaptiveavgpool2d and softmax<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/9/79e326fc9e36d978f937193420d54a030a8e7fdf.png" data-download-href="https://discuss.pytorch.org/uploads/default/79e326fc9e36d978f937193420d54a030a8e7fdf" title="Screen Shot 2021-09-01 at 3.30.13 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/9/79e326fc9e36d978f937193420d54a030a8e7fdf_2_690x310.png" alt="Screen Shot 2021-09-01 at 3.30.13 PM" data-base62-sha1="hogrCmXCjhrcgNSGRfVQHxWyMsD" width="690" height="310" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/9/79e326fc9e36d978f937193420d54a030a8e7fdf_2_690x310.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/9/79e326fc9e36d978f937193420d54a030a8e7fdf_2_1035x465.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/9/79e326fc9e36d978f937193420d54a030a8e7fdf_2_1380x620.png 2x" data-dominant-color="292929"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2021-09-01 at 3.30.13 PM</span><span class="informations">1500×676 58.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>