`torch.linspace` in Pytorch 1.4 and 1.5 produce different results cause noise in image compression

CharlieKoo: <p>I am learning the code in [<a href="https://github.com/fab-jul/L3C-PyTorch" class="inline-onebox" rel="noopener nofollow ugc">GitHub - fab-jul/L3C-PyTorch: PyTorch Implementation of the CVPR'19 Paper "Practical Full Resolution Learned Lossless Image Compression"</a>], and I am facing an issue because I want to run it on Nvidia A100. However, A100 only supports CUDA 11, not CUDA 10.1, which is required for installing PyTorch 1.4 or lower.</p>
<p>This code runs correctly on PyTorch 1.1-1.4 with CUDA 10.1, but it does not work correctly on PyTorch versions higher than 1.5. When I use PyTorch &gt; 1.5 to train this code, it introduces noise in the generated images.</p>
<p>During my debugging process, I discovered that PyTorch 1.5 updated <code>torch.linspace</code>, and PyTorch 1.6 updated <code>torch.nn.Conv2d</code>.</p>

ptrblck: <p>No, you won’t be able to use CUDA 11 in PyTorch 1.4 as 1.4.0 was released before CUDA 11 came out and is thus not compatible. You could of course try to cherry-pick needed changes into a custom 1.4 branch but this sounds like a lot of unnecessary work.<br>
I would recommend updating to the latest PyTorch release instead.</p>

CharlieKoo: <p>Yes, I already updated my PyTorch, but it introduced a lot of noise. I’m unsure how to resolve this issue. As I mentioned earlier, I noticed that <code>torch.linspace</code> behaves differently in PyTorch versions 1.4 and 1.5.</p>

ptrblck: <p>Could you post a code snippet showing the different behaviors, please?</p>

CharlieKoo: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/0/304a54e9ba06a56a3556138f39ce9f63d39bf4c8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/304a54e9ba06a56a3556138f39ce9f63d39bf4c8" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/0/304a54e9ba06a56a3556138f39ce9f63d39bf4c8_2_539x499.jpeg" alt="image" data-base62-sha1="6TcaKUVjrjNrBFkU2MtCgxIs83K" width="539" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/0/304a54e9ba06a56a3556138f39ce9f63d39bf4c8_2_539x499.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/3/0/304a54e9ba06a56a3556138f39ce9f63d39bf4c8_2_808x748.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/0/304a54e9ba06a56a3556138f39ce9f63d39bf4c8.jpeg 2x" data-dominant-color="2E2C2F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1057×979 153 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>As you can see, the results of <code>test_code</code> differ between PyTorch versions 1.5 and 1.4. I believe this discrepancy might be due to a precision issue(<code>double and float</code>). To investigate further, I attempted to use <code>torch.linspace(-1., 1., 25, dtype=torch.double).float() </code>in PyTorch 1.4, and I obtained the same output as in version 1.5.</p>
<p>Based on this observation, I don’t think the PyTorch version is the reason why L3C cannot be trained in higher versions. To explore this further, I ran the L3C code with PyTorch 1.4(I use <code>dtype=torch.double).float()</code> ) and noticed the introduction of noise in the results.</p>

ptrblck: <p>I also don’t think these expected small numerical differences caused by the limited floating point precision are to blame for any real issues.</p>

CharlieKoo: <p>To ensure that I’m not running the code in the wrong virtual environment, I’m trying every possibility. I will post the images obtained from running the code under different versions and precision later. Thank you for your time.</p>

CharlieKoo: <p>Hi, here are the results:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f994ec198d7ccf27b14b39f1485b448250ce2210.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f994ec198d7ccf27b14b39f1485b448250ce2210" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f994ec198d7ccf27b14b39f1485b448250ce2210_2_690x307.jpeg" alt="image" data-base62-sha1="zBTXbzm7BXE8va5HMqIB3zGTFYs" width="690" height="307" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f994ec198d7ccf27b14b39f1485b448250ce2210_2_690x307.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/9/f994ec198d7ccf27b14b39f1485b448250ce2210_2_1035x460.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/9/f994ec198d7ccf27b14b39f1485b448250ce2210.jpeg 2x" data-dominant-color="AAA69B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1379×614 126 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>In PyTorch 1.4, the quality is the same as in test.png. However, when I used <code>double.float</code> in PyTorch 1.4, it introduced noise. In PyTorch 1.5, regardless of whether <code>double.float</code> was used, it still introduced noise.</p>