Maxpool2d edge value is negative at output

username: Arun_Vishwanathan
cooked: <p>I have a simple network consisting of a conv2d following by maxpool2d.</p>
<p>My maxpool2d has padding of (2,2) and stride is (1,1) and kernel_size is (5,5) and as per the documentation <code>If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides</code></p>
<p>Now when I print my input to pool this is how it looks (2,4,7,7 is the input size)</p>
<pre><code class="lang-auto">          [ 0.0489,  1.6420,  0.2287, -1.0550, -1.1927, -0.8047,  0.0784],
          [ 3.1435, -1.5051, -1.0993, -1.2291,  1.9592,  0.9374,  1.0278],
          [ 4.1902, -1.6815, -1.4005,  0.3315,  0.4801,  0.6345,  2.0060],
          [-0.3247, -2.6899, -1.1491,  3.6877,  3.4306,  2.6198,  0.1040],
          [-0.5162,  0.6600,  2.8160,  5.4908,  3.5650,  2.4575,  1.6125],
          [-5.6328, -0.9014,  1.5804,  2.9726,  2.2904,  0.5041,  3.1346]],

         [[-0.1914, -2.8677,  0.0729,  2.4558,  4.0144,  1.8283,  0.5931],
          [ 0.1440,  0.3167, -0.2808,  0.9678,  3.0057,  0.9184,  0.6368],
          [ 3.9118, -1.1746,  0.4501, -1.8484,  0.5270,  2.2912,  1.9351],
          [ 2.6868,  0.3502,  1.0592,  1.5951, -0.4269,  0.0184,  1.6321],
          [-0.4935, -0.2281,  1.2582,  5.1018,  2.0337,  4.3594,  1.0276],
          [ 2.2973,  0.4393,  1.2802,  2.7683,  3.5030,  0.4795,  3.1873],
          [-1.6443,  0.5248,  1.1553,  2.5217,  1.1599,  5.2611, -0.7326]],

         [[-3.9278, -3.7870, -1.1932,  1.3942,  4.4397,  1.5615, -0.1242],
          [-1.4697, -2.1559, -0.3979, -0.3652,  1.0954,  0.1458, -0.3445],
          [ 3.2035, -3.2459, -1.2674, -1.8574,  0.8088, -0.2128,  1.6267],
          [ 3.3607, -1.1613, -1.5472,  0.2017,  0.0222,  0.2070,  2.4390],
          [ 0.2863, -1.8395, -1.5054,  2.4502,  3.5512,  3.7417, -0.4721],
          [-0.5731,  0.1661,  1.1135,  4.8935, -0.7547,  1.6938,  1.6246],
          [-3.2584,  0.2666,  2.7086,  2.9518,  2.2872,  2.5348,  0.3620]],

         [[-4.0493, -4.8643, -0.7076,  2.3171,  5.8105,  2.9847, -2.6179],
          [-2.4079,  0.6828, -0.6392, -0.5281,  1.1452,  1.7984, -0.5071],
          [ 1.6854, -1.5713, -2.0388, -2.2313,  1.1435,  2.6450,  1.4343],
          [ 3.2826, -0.0465, -0.6639,  0.1978, -0.4772, -0.7204,  1.9651],
          [-0.2371, -2.9784, -1.3908,  2.1289,  4.9562,  4.0141, -0.4528],
          [ 1.6566, -1.1490,  0.1557,  3.4228,  4.1824,  1.4596,  2.2735],
          [-4.5200, -0.5847,  1.4765,  2.0553,  3.2786,  2.6815, -0.7931]]],


        [[[ 2.1042,  3.3742,  0.5247,  0.8450,  0.2137, -0.1025,  1.3134],
          [ 1.0647,  0.8145,  2.5763,  1.5895,  0.5657, -2.8018, -1.0151],
          [-1.0696,  1.5710,  0.5461,  0.1239,  1.1746,  0.8900,  0.8370],
          [ 3.2110,  3.5812,  0.6116, -0.2446,  0.8808, -0.7343, -2.4390],
          [ 1.7637,  3.0690,  4.1770,  1.4025,  0.7210, -3.2826, -3.2330],
          [ 5.7765,  6.8915,  4.2196,  2.6918, -0.9390, -2.1477, -2.2693],
          [ 3.8918,  0.8171,  4.3219,  1.7632, -3.7690, -4.8244, -4.2342]],

         [[ 2.2697,  2.8911,  3.0222, -0.1710,  1.7580,  0.7359,  2.5586],
          [-0.8067,  1.7193,  0.8415,  0.9246,  0.9927,  0.9173, -0.7195],
          [ 0.8351,  1.7259,  2.4022,  2.6045,  2.1432,  0.5825,  1.1809],
          [ 2.3973,  2.6272,  1.1421,  1.1446,  2.9422, -0.9794, -1.4130],
          [ 1.9513,  3.1972,  3.0604,  0.2871, -0.1983, -0.5607, -1.0176],
          [ 3.8872,  3.6584,  3.3600,  2.3792,  0.1604, -2.4171, -0.4698],
          [ 2.5361,  4.2509,  3.3117,  3.2684, -0.5432, -2.7799, -1.8438]],

         [[ 0.5719,  3.5698,  4.4450,  0.6703, -0.7591, -0.3706,  0.6389],
          [-1.2735,  1.0070,  0.0426,  1.1367,  0.1103, -1.4961, -2.4907],
          [ 2.8407,  3.4860,  0.5163,  0.9328,  1.9300,  1.5186, -1.0641],
          [ 1.7709,  1.3212,  1.1954,  1.3916,  0.5054, -1.4010, -1.6475],
          [ 3.0165,  2.8974,  2.8397, -0.0489,  0.5612, -3.9587, -2.1370],
          [ 4.8533,  6.0433,  4.3979,  2.2355,  0.6366, -2.7814, -1.8583],
          [ 2.5606,  3.4099,  4.3756,  4.0021, -2.2347, -2.7669, -3.4700]],

         [[ 2.5396,  3.8363,  2.3450,  0.6624, -0.2863,  0.3438,  1.9313],
          [ 0.1720,  1.0229,  2.3096,  0.9799, -1.3438, -2.1497, -2.2450],
          [ 0.2471,  1.9793,  1.8274,  0.7245,  0.3894,  0.7552,  1.4527],
          [ 3.4289,  2.2729, -0.5627,  0.5417,  1.3856, -1.4114, -1.7625],
          [ 2.4366,  4.1636,  2.5470,  0.7687, -0.3675, -2.0235, -2.7498],
          [ 6.1177,  6.5956,  3.6907,  2.4760, -0.5046, -2.4810, -1.7314],
          [ 3.7165,  3.0131,  4.2347,  2.3551, -1.5218, -3.7080, -4.3321]]]],
</code></pre>
<p>and the output of pool is also (2,4,7,7) :</p>
<pre><code class="lang-auto">          [ 4.1902,  4.1902,  5.4995,  5.4995,  5.4995,  5.4995,  5.4995],
          [ 4.1902,  4.1902,  5.4995,  5.4995,  5.4995,  5.4995,  5.4995],
          [ 4.1902,  5.4908,  5.4908,  5.4908,  5.4908,  5.4908,  3.5650],
          [ 4.1902,  5.4908,  5.4908,  5.4908,  5.4908,  5.4908,  3.5650],
          [ 4.1902,  5.4908,  5.4908,  5.4908,  5.4908,  5.4908,  3.5650],
          [ 2.8160,  5.4908,  5.4908,  5.4908,  5.4908,  5.4908,  3.5650]],

         [[ 3.9118,  3.9118,  4.0144,  4.0144,  4.0144,  4.0144,  4.0144],
          [ 3.9118,  3.9118,  4.0144,  4.0144,  4.0144,  4.0144,  4.0144],
          [ 3.9118,  5.1018,  5.1018,  5.1018,  5.1018,  5.1018,  4.3594],
          [ 3.9118,  5.1018,  5.1018,  5.1018,  5.1018,  5.1018,  4.3594],
          [ 3.9118,  5.1018,  5.1018,  5.2611,  5.2611,  5.2611,  5.2611],
          [ 2.6868,  5.1018,  5.1018,  5.2611,  5.2611,  5.2611,  5.2611],
          [ 2.2973,  5.1018,  5.1018,  5.2611,  5.2611,  5.2611,  5.2611]],

         [[ 3.2035,  3.2035,  4.4397,  4.4397,  4.4397,  4.4397,  4.4397],
          [ 3.3607,  3.3607,  4.4397,  4.4397,  4.4397,  4.4397,  4.4397],
          [ 3.3607,  3.3607,  4.4397,  4.4397,  4.4397,  4.4397,  4.4397],
          [ 3.3607,  4.8935,  4.8935,  4.8935,  4.8935,  4.8935,  3.7417],
          [ 3.3607,  4.8935,  4.8935,  4.8935,  4.8935,  4.8935,  3.7417],
          [ 3.3607,  4.8935,  4.8935,  4.8935,  4.8935,  4.8935,  3.7417],
          [ 2.7086,  4.8935,  4.8935,  4.8935,  4.8935,  4.8935,  3.7417]],

         [[ 1.6854,  2.3171,  5.8105,  5.8105,  5.8105,  5.8105,  5.8105],
          [ 3.2826,  3.2826,  5.8105,  5.8105,  5.8105,  5.8105,  5.8105],
          [ 3.2826,  3.2826,  5.8105,  5.8105,  5.8105,  5.8105,  5.8105],
          [ 3.2826,  3.4228,  4.9562,  4.9562,  4.9562,  4.9562,  4.9562],
          [ 3.2826,  3.4228,  4.9562,  4.9562,  4.9562,  4.9562,  4.9562],
          [ 3.2826,  3.4228,  4.9562,  4.9562,  4.9562,  4.9562,  4.9562],
          [ 1.6566,  3.4228,  4.9562,  4.9562,  4.9562,  4.9562,  4.9562]]],


        [[[ 3.3742,  3.3742,  3.3742,  3.3742,  2.5763,  1.5895,  1.3134],
          [ 3.5812,  3.5812,  3.5812,  3.5812,  2.5763,  1.5895,  1.3134],
          [ 4.1770,  4.1770,  4.1770,  4.1770,  4.1770,  1.5895,  1.3134],
          [ 6.8915,  6.8915,  6.8915,  6.8915,  4.2196,  2.6918,  1.1746],
          [ 6.8915,  6.8915,  6.8915,  6.8915,  4.3219,  2.6918,  1.1746],
          [ 6.8915,  6.8915,  6.8915,  6.8915,  4.3219,  2.6918,  0.8808],
          [ 6.8915,  6.8915,  6.8915,  6.8915,  4.3219,  2.6918,  0.7210]],

         [[ 3.0222,  3.0222,  3.0222,  3.0222,  3.0222,  2.6045,  2.5586],
          [ 3.0222,  3.0222,  3.0222,  3.0222,  3.0222,  2.9422,  2.9422],
          [ 3.1972,  3.1972,  3.1972,  3.1972,  3.0604,  2.9422,  2.9422],
          [ 3.8872,  3.8872,  3.8872,  3.6584,  3.3600,  2.9422,  2.9422],
          [ 4.2509,  4.2509,  4.2509,  4.2509,  3.3600,  3.2684,  2.9422],
          [ 4.2509,  4.2509,  4.2509,  4.2509,  3.3600,  3.2684,  2.9422],
          [ 4.2509,  4.2509,  4.2509,  4.2509,  3.3600,  3.2684,  0.1604]],

         [[ 4.4450,  4.4450,  4.4450,  4.4450,  4.4450,  1.9300,  1.9300],
          [ 4.4450,  4.4450,  4.4450,  4.4450,  4.4450,  1.9300,  1.9300],
          [ 4.4450,  4.4450,  4.4450,  4.4450,  4.4450,  1.9300,  1.9300],
          [ 6.0433,  6.0433,  6.0433,  6.0433,  4.3979,  2.2355,  1.9300],
          [ 6.0433,  6.0433,  6.0433,  6.0433,  4.3979,  4.0021,  1.9300],
          [ 6.0433,  6.0433,  6.0433,  6.0433,  4.3979,  4.0021,  0.6366],
          [ 6.0433,  6.0433,  6.0433,  6.0433,  4.3979,  4.0021,  0.6366]],

         [[ 3.8363,  3.8363,  3.8363,  3.8363,  2.3450,  1.9313,  1.9313],
          [ 3.8363,  3.8363,  3.8363,  3.8363,  2.3450,  1.9313,  1.9313],
          [ 4.1636,  4.1636,  4.1636,  4.1636,  2.5470,  1.9313,  1.9313],
          [ 6.5956,  6.5956,  6.5956,  6.5956,  3.6907,  2.4760,  1.4527],
          [ 6.5956,  6.5956,  6.5956,  6.5956,  4.2347,  2.4760,  1.4527],
          [ 6.5956,  6.5956,  6.5956,  6.5956,  4.2347,  2.4760,  1.3856],
          [ 6.5956,  6.5956,  6.5956,  6.5956,  4.2347,  2.4760, -0.3675]]]],

</code></pre>
<p>Could someone please explain to me why the last value is negative the -0.3675?? Should it not be 0 as the max as it is zero padded? The last block contains all negative values with zero padding in the pool kernel</p>

username: albanD
cooked: <p>What is the padding size you use? Could you give a small code sample that reproduces this?</p>

username: Arun_Vishwanathan
cooked: <p>I don’t have a reproduction code to share unfortunately. Padding size is (2,2) and kernel size is (5,5) , rest are all defaults. Input coming in is (2,4,7,7). Maybe this can be reproduced if you initialize all the input values to pool to be negative at the bottom right corner that contains the kernel for pooling with zero padding as you see in my example above.</p>

username: albanD
cooked: <p>In the cpu implementation we have, they are actually ignored. <a href="https://github.com/pytorch/pytorch/blob/a40a19ccabde0981c021a94ee81cd1f3ceb9f97a/aten/src/ATen/native/DilatedMaxPool2d.cpp#L44-L45">here</a> is the implementation. But possibly, the cuda code does something different.</p>
<p>Could you open an issue on github stating that in the best case the doc is incorrect and in the worst case our implementations across devices are not consistent.<br>
A simple repro to put there:</p>
<pre><code class="lang-python">import torch
inp = -torch.ones(2, 4, 7, 7)

print(torch.nn.MaxPool2d(5, padding=2)(inp))
</code></pre>

username: Arun_Vishwanathan
cooked: <p>Thanks so much for confirming this! I have not checked the cuda code yet. To clarify, there is no option today to configure at the user API level how to use the padding, correct?</p>

username: albanD
cooked: <p>No there is no configuration for how the padding is used.</p>