Inference result is different between x86 and android libpytorch

username: Jonson
cooked: <p>Suppose I have a pretrained model weight, and if I export model with the following steps, I found that the inference result is different between x86 and android libpytorch。</p>
<pre><code class="lang-python">....
model = Conv1d()
w = torch.load("weight.pt")
model.conv.weight.data = w
jit_model = torch.jit.script(model)
torch.jit.save(jit_model, "conv_2.pt")
...
</code></pre>
<p>Then load and inference with the <code>conv_2.pt</code> on x86 and android separately, and I found the two results are different.</p>
<p>for x86 pytorch output is:</p>
<pre><code class="lang-auto">tensor([[ 0.1662,  0.0207,  0.1688,  0.1911,  0.1231, -0.1205,  0.1114,  0.1749,
          0.2523,  0.0050],
        [-0.0863, -0.0477, -0.0551, -0.0679, -0.0117, -0.0239, -0.0428,  0.0200,
         -0.0516, -0.0071]])
</code></pre>
<p>while for android libpytorch output is:</p>
<pre><code class="lang-auto">(1,.,.) =
 Columns 1 to 9  0.0366 -0.0032  0.0441  0.0488  0.0381 -0.0448  0.0277  0.0619  0.0724
  0.1012  0.0999  0.0257  0.0421 -0.0487  0.1312  0.0322 -0.1556 -0.0342

Columns 10 to 10  0.0001
  0.0137
[ CPUFloatType{1,2,10} ]
</code></pre>
<p>I’m sure that the inputs are the same.</p>
<p>But if I change the code into:</p>
<pre><code class="lang-python">model = Conv1d()
w = torch.load("weight.pt")
new_w = torch.flatten(torch.normal(0, 1, w.shape))
org_w = torch.flatten(w)
for idx, a in enumerate(org_w):
    new_w[idx] = a
new_w = new_w.reshape(w.shape)
model.conv.weight.data = new_w
jit_model = torch.jit.script(model)
torch.jit.save(jit_model, "conv_2.pt")
</code></pre>
<p>Then load and inference with the <code>conv_2.pt</code> on x86 and android separately, and the two results are the same.</p>
<p>Codes to reproduce the bug: <a href="https://www.dropbox.com/s/vldy1veaqgihk2j/pytorch_bug_report.zip?dl=0" rel="noopener nofollow ugc">https://www.dropbox.com/s/vldy1veaqgihk2j/pytorch_bug_report.zip?dl=0</a></p>

username: ptrblck
cooked: <p><code>new_w</code> doesn’t seem to be used in the second code snippet so the difference seems to be the <code>.data</code> assignment.<br>
Could you remove the <code>.data</code> usage and assign the weight via:</p>
<pre><code class="lang-python">with torch.no_grad():
    model.conv.weight = nn.Parameter(w)
</code></pre>
<p>and rerun the test?</p>

username: Jonson
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="99455">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>new_w</code> doesn’t seem to be used in the second code snippet so the difference seems to be the <code>.data</code> assignment.</p>
</blockquote>
</aside>
<p>This is a mistake, and I had re-edit the code snippet.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="99455">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Could you remove the <code>.data</code> usage and assign the weight via:</p>
<pre><code class="lang-auto">with torch.no_grad():
    model.conv.weight = nn.Parameter(w)
</code></pre>
</blockquote>
</aside>
<p>And I tried this assignment method, but the results are different still.</p>

username: sklationd
cooked: <p>Did you solve this problem?<br>
I think I encounter almost same situation.</p>

username: Jonson
cooked: <p>yep, you can follow this code snippet as a temporary solution:</p>
<pre><code class="lang-auto">model = Conv1d()
w = torch.load("weight.pt")
new_w = torch.flatten(torch.normal(0, 1, w.shape))
org_w = torch.flatten(w)
for idx, a in enumerate(org_w):
    new_w[idx] = a
new_w = new_w.reshape(w.shape)
model.conv.weight.data = new_w
jit_model = torch.jit.script(model)
torch.jit.save(jit_model, "conv_2.pt")
</code></pre>