Output of nn.MaxUnpool2d is not unique resp. gives different results after each run

username: Fridolin
cooked: <p>Hi all,</p>
<p>I am trying to implement a U-NET. I stumbled upon a problem using MaxUnpool2d. The solution is not unique, hence, I get different solutions each time I run it.</p>
<p>I reproduced the error here:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

x = torch.load("x.pt")
indices = torch.load("indices.pt")

unpool = nn.MaxUnpool2d(kernel_size=(3, 3), stride=(1, 1), padding=(0, 0))


###############################################

output_1 = unpool(x,indices)
output_2 = unpool(x,indices)


############################################
print("Sum of Output_1:")
print(sum(sum(sum(sum(output_1)))))
print("---------------------")
print("Sum of Output_2:")
print(sum(sum(sum(sum(output_2)))))

</code></pre>
<p>GitHub repository:<br>
<a href="https://github.com/fridolinvii/MaxUnpool2d.git" rel="nofollow noopener">https://github.com/fridolinvii/MaxUnpool2d.git</a></p>
<p>I’d be thankful for anyone, who has an idea, what might be wrong.</p>
<p>Edit: My problem is, that <strong>output_1</strong> is different from  <strong>output_2</strong></p>

username: albanD
cooked: <p>What do you mean by “the solution is not unique” ?<br>
Multiple indices point to the same output?</p>

username: Fridolin
cooked: <p>I mean that if I do the following:</p>
<aside class="quote no-group" data-username="Fridolin" data-post="1" data-topic="88555">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/5fc32e/48.png" class="avatar"> Fridolin:</div>
<blockquote>
<pre><code class="lang-auto">output_1 = unpool(x,indices)
output_2 = unpool(x,indices)
</code></pre>
</blockquote>
</aside>
<p>output_1 is different to output_2</p>

username: albanD
cooked: <p>How did you get the indices? Are they overlapping by any chance?</p>

username: Fridolin
cooked: <p>Thank you for your reply.<br>
I apply</p>
<ol>
<li>Conv2D</li>
<li>maxpool2D</li>
<li>Cov2D</li>
<li>[x,indices] = maxpool2D     &lt;-got these indices</li>
<li>multiple FC layers</li>
<li>y = RNN (convert the output to the size of x)</li>
<li>output_1 = unpool(y,indices)</li>
</ol>
<p>How can I check, If I have overlapping indices?<br>
If you want, I can give an exemplary script.</p>

username: albanD
cooked: <p>Hi,</p>
<p>In the case where the maxpool2d kernels in the forward overlap, the inpooling can be ambiguous as two values need to be written at the same place.<br>
This function is non-deterministic in this case. You can read more about this <a href="https://pytorch.org/docs/stable/notes/randomness.html">here</a>.</p>

username: Fridolin
cooked: <p>Thank you for the link.</p>
<p>I tried to add:</p>
<pre><code class="lang-auto">torch.manual_seed(0)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
</code></pre>
<p>But I still get two different outputs.</p>
<p>If I understand it correctly, the problem might be</p>
<blockquote>
<p>floating-point addition is not perfectly associative for floating-point operands</p>
</blockquote>
<p>Hence, the non-deterministic function?</p>

username: albanD
cooked: <p>Hi,</p>
<p>Yes the seeding won’t change anything for the determinism.<br>
Unfortunately, the only options I can see here is to change the pooling to avoid overlapping kernel.</p>

username: Fridolin
cooked: <p>Hi,<br>
Thank you for your help.<br>
So, when I understand you correctly, The output maxpool2d has overlapping indices, and when I do unpool, these lead to different results.</p>

username: albanD
cooked: <p>Yes it can.<br>
When during the pooling, a value was used twice (because kernels overlap).<br>
In the unpooling layer, you want to write two different values back at that spot.<br>
And for speed reasons, we don’t check all the Tensors for overlaps and so you might end up with one value or the other depending on the runs.</p>