[Torchvision]Saving pytorch model as file issue

username: adle
cooked: <p>Hi everyone,</p>
<p>I got a bit of confuse in saving the model, as suggested, I did something like this:</p>
<blockquote>
<p>torch.save(model.state_dict(), PATH_1)</p>
</blockquote>
<p>Rerun the code, save as different file</p>
<blockquote>
<p>torch.save(model.state_dict(), PATH_2)</p>
</blockquote>
<p>When I use <strong>cmp</strong> command to pair between the 2 binary files, there are differences between them. It is because storing float numbers is slightly different in 2 files, even though they are still referencing to the same values.</p>
<p>My requirement for the project is to have the identical binary files everytime we store the same model. So this is quite troublesome.<br>
Do you have any solution or work around to deal with this?</p>
<p>Thanks a lot for your support,</p>
<p>Cheers,</p>

username: ptrblck
cooked: <p>If you are rerunning the code, are you following the <a href="https://pytorch.org/docs/stable/notes/randomness.html" rel="nofollow noopener">reproducibility docs</a>?</p>
<p>If you store the <code>model.state_dict()</code> in two different files, both will contain the same binary representation.</p>
<p>However, if you don’t use deterministic operations, your model will of course differ.</p>

username: adle
cooked: <p>Hi,</p>
<p>Thank you for your response. When mentioning rerunning the code, I meant I already have the *.pt file, so I just try to load the model and save again with different file.<br>
I noticed that if I load the model and save in different files at the same code, they will be identical, but if I did it in 2 different runs the files will be different.</p>
<p>I don’t know much about deterministic operations, would you refer me some links/readings so I can take a bit research on it?</p>
<p>Thanks again and have a nice day,<br>
Regards,</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="adle" data-post="3" data-topic="53302">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/b9bd4f/48.png" class="avatar"> adle:</div>
<blockquote>
<p>I noticed that if I load the model and save in different files at the same code, they will be identical, but if I did it in 2 different runs the files will be different.</p>
</blockquote>
</aside>
<p>You are right and I’m not sure, where these different bits come from.</p>
<aside class="quote no-group" data-username="adle" data-post="3" data-topic="53302">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/b9bd4f/48.png" class="avatar"> adle:</div>
<blockquote>
<p>I don’t know much about deterministic operations, would you refer me some links/readings so I can take a bit research on it?</p>
</blockquote>
</aside>
<p>The linked docs gives you a good overview about reproducibility etc.<br>
However, even setting the same seed in both scripts will yield different binary representations and I think it might be related to the pickling itself.</p>

username: adle
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="53302">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>The linked docs gives you a good overview about reproducibility etc.<br>
However, even setting the same seed in both scripts will yield different binary representations and I think it might be related to the pickling itself.</p>
</blockquote>
</aside>
<p>Hi,</p>
<p>Thanks for your response. Yes I was also thinking about the pickling but I am struggling on how to precise (to some amount of precision after floating point) the storing on the binary file. If I understand properly, it is because of storing float in the file. For instance, 5.002 might be stored 5.002000000000001 in file 1 and 5.002000000000002 in file 2.</p>
<p>Would be nice if you know any function to indicate torch.save with precision level, or maybe it’s not there at all.</p>
<p>Regards,</p>