akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Conv2D on ARM device wrong result

username: octavianmm
cooked: <p>Hello,<br>
I’m getting different results for a Conv2D operation on ARM and x86_64 devices. Basically, the output tensors are identical in some parts, but have major blocks of data different in other parts. A quick view of the differences I got: <a href="https://github.com/octavianmm/torch_nn_functional_conv2d_problem/blob/main/results/difference.png" class="inline-onebox" rel="noopener nofollow ugc">torch_nn_functional_conv2d_problem/difference.png at main · octavianmm/torch_nn_functional_conv2d_problem · GitHub</a><br>
A minimal working example that replicates this problem can be found here: <a href="https://github.com/octavianmm/torch_nn_functional_conv2d_problem" class="inline-onebox" rel="noopener nofollow ugc">GitHub - octavianmm/torch_nn_functional_conv2d_problem: Different output on ARM and x86_64 architectures for torch.nn.functional.conv2d</a><br>
Also, the repo contains the two tensors obtained on ARM and Intel platforms using the same code, so the differences can be easily inspected.<br>
I’ve discovered this problem when debugging why the same CNN works as expected in inference on x86_64 machines, but gives bad results (always predicting the same class) on ARM devices.</p>

username: ptrblck
cooked: <p>Which PyTorch version are you using?<br>
<a href="https://github.com/pytorch/pytorch/pull/49273">A NEON issue</a> was fixed some time ago, so could you update to the latest release in case you are using an older build?</p>

username: octavianmm
cooked: <p>I’ve tested this on two ARM devices, with pytorch version 1.4.0 and 1.7.0, respectively, and the issue was present in both cases.<br>
Do you know starting with which pytorch version was this issue fixed?<br>
Thanks.</p>
<p>UPDATE: I’ve just tried pytorch 1.8.0 on the NVIDIA Jetson nano, got the same wrong result.</p>

username: ptrblck
cooked: <p>Thanks for the update. Could you create an issue on <a href="https://github.com/pytorch/pytorch/issues">GitHub</a> with all necessary information to reproduce and debug this issue further, please?</p>

username: octavianmm
cooked: <p>Thanks for the tip. I’ve created the issue here: <a href="https://github.com/pytorch/pytorch/issues/55781" class="inline-onebox" rel="noopener nofollow ugc">Conv2D operation on ARM architecture gives wrong result · Issue #55781 · pytorch/pytorch · GitHub</a></p>
Using getattr with torch.nn.Parameter

username: Sam_Lerman
cooked: <p>I ran something like this within a class that extends <code>Module</code>:</p>
<p><code>getattr(self, "example", Parameter(torch.Tensor([size])).to(self.device))</code></p>
<p>Which should set a default class <code>__dict__</code> value to the key “example.”</p>
<p>But when I print <code>__dict__.keys()</code>, “example” isn’t added.</p>
<p>However, it is added as expected when I change the above to a non-Parameter:</p>
<p><code>getattr(self, "example", torch.Tensor([size]).to(self.device))</code></p>
<p>How can I set default params from <code>Parameter</code> to a class attribute in <code>__dict__</code>?</p>
Pytorch equivalent of Tensorflow Recommenders

username: Vishal_Ahuja
cooked: <p>I came across Tensorflow Recommenders (<a href="https://www.tensorflow.org/recommenders" class="inline-onebox" rel="noopener nofollow ugc">TensorFlow Recommenders</a>) today, and would like to know if we have something similar in the pytorch eco-system.</p>
<p>Sincerely,<br>
Vishal</p>
Mode collapse when training GAN

username: Yolkandwhite
cooked: <p>Hello, everyone.</p>
<p>This is my first time training GANs and I don’t have any insights about how to train GANs.</p>
<p>the model(PGGAN right now) is having mode collapse problem right now.</p>
<p>Should I stop training and start all over again when I observe mode collapse issues??</p>
<p>Or is there any possibilities that model can get over those issues by itself?</p>
State_dict() round trip fails

username: zyzhu2000
cooked: <p>I tried the following:</p>
<pre><code class="lang-auto">torch.save(net.state_dict(), f"file")
weights = torch.load('file1', map_location=device)

net1 = Net()
net1.load_state_dict(weights)

net2 = Net()
net2.load_state_dict(weights)

torch.save(net1.state_dict(), f"file1")
torch.save(net2.state_dict(), f"file2")
</code></pre>
<p>Then I find that file1 and file2 are different. How can it be? My goal is to find out if the weights of a net has changed since last time.</p>

username: ptrblck
cooked: <p>Are you seeing different parameters in <code>file1</code> and <code>file2</code> after reloading them?<br>
If so, could you post the model architecture so that we could reproduce this issue?</p>

username: zyzhu2000
cooked: <p>I am seeing the same parameters. In fact, I figured out. <code>torch.save</code> appears to add some randomness to the byte representation it writes out, similar to salting.</p>