Multiplying with identity matrix doesnt give same input matrix. Can anyone explain why?

username: Junaid
cooked: <p>I created a single linear layer model with identity matrix as the weights. But by passing the random input values to the model. the output and input values are not same after 3 to 4 decimal points of float type . But incase of integers the values are same.</p>
<p><strong>Output:</strong><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/9/6/969234425d85a3982551756771e89952c7f2aa32.png" alt="image" data-base62-sha1="lu0RpyCH2EKPWtAhZw34aneH4ky" width="636" height="469"></p>
<p>The input and output have to be same. But you can see that its not the same when i print them.<br>
<strong>Code used:</strong></p>
<pre><code class="lang-auto">from multiprocessing import reduction

import torch

from torchvision import datasets, models, transforms

from torchsummary import summary

import os

import argparse

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

channel=1

length=1

bredth=2

torch.set_printoptions(threshold=10000,precision=16)

size_=channel*length*bredth

tinymodel = torch.nn.Sequential(torch.nn.Linear(size_, size_, bias=False))

print('The model:----------')

print(tinymodel)

tinymodel.to(device)

weights=torch.eye(size_)

print('Identity matrix:-----------')

print(weights)

weights=weights.int()

print(weights)

print(type(weights))

print(weights.shape)
print('\n\nModel params:-------')
for param in tinymodel.parameters():
    print(param.shape)

with torch.no_grad():
    tinymodel[0].weight.data.copy_(weights)

for param in tinymodel.parameters():
    print(param.shape)
    print(param)
tinymodel.to(device)
input_val=torch.rand([channel, length, bredth], dtype=torch.float, requires_grad=False).cuda()

input_val=torch.reshape(input_val,(-1,))
print("\nInput")
print(input_val.shape)
print(input_val)

# input_val=torch.Tensor([234,244]).to(device)                  // Testing with integer inputs while size_=2
result=tinymodel(input_val)
print("\nOutput")
print(result.shape)
print(result)

loss=torch.nn.MSELoss(reduction='sum')
loss_val=loss(result,input_val)
print("/n",loss_val)

</code></pre>
<p>Can anyone explain why is this happening?</p>

username: KFrank
cooked: <p>Hi Junaid!</p>
<aside class="quote no-group" data-username="Junaid" data-post="1" data-topic="158213" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/junaid/48/52021_2.png" class="avatar"> Junaid:</div>
<blockquote>
<p>the output and input values are not same after 3 to 4 decimal points of float type</p>
</blockquote>
</aside>
<p>This is probably the “TF32 bug.”  This post has some details:</p>
<aside class="quote quote-modified" data-post="3" data-topic="140947">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/considerable-absolute-error-in-torch-matmul/140947/3">Considerable absolute error in torch.matmul</a> 
  </div>
  <blockquote>
    Hi Koramajin! 

Could this be the <a href="https://pytorch.org/docs/1.10.1/notes/cuda.html#tensorfloat-32-tf32-on-ampere-devices" rel="noopener nofollow ugc">TF32</a> “bug?” 
What specific gpu are you using and does the issue go away if you set 
torch.backends.cuda.matmul.allow_tf32 = True? 
Edit: 
The above should read 
torch.backends.cuda.matmul.allow_tf32 = False

True is the default for allow_tf32.  The suggestion is to turn off TF32 
which is done by setting allow_tf32 to False. 
Best. 
K. Frank
  </blockquote>
</aside>

<p>Do you get the right answer when running on your cpu?  What gpu are<br>
you using?  Does <code>torch.backends.cuda.matmul.allow_tf32 = False</code><br>
fix your issue?</p>
<p>As an aside, please don’t post screenshots of textual information.  Doing<br>
so breaks accessibility, searchability, and copy-paste.</p>
<p>Best.</p>
<p>K. Frank</p>