torchvision.transforms.GaussianBlur occasionally returns its input unchanged, and doesn't behave deterministically

Brian_Krause: <p>I was trying to implement a few versions of local image normalization, all involving some variation of a Gaussian blur, then subtracting that from the original image. I kept getting odd results such as occasional images filled with all 0s or all -1s or similar. After some investigation, I was able to narrow it down to a minimal example to reproduce the bug.</p>
<p>It turns out that torchvision.transforms.GaussianBlur will occasionally return a tensor identical to its input, seemingly at random. Not only that, but it doesn’t appear to behave deterministicly, even with torch.use_deterministic_algorithms(True) set.</p>
<p>I’ve made a <a href="https://colab.research.google.com/drive/1tlnawsmFHEK3_e-dYQjfpd_ivMlS665y#scrollTo=Od5lCGOdHHI7" rel="noopener nofollow ugc">colab notebook</a> to demonstrate the issue. It initializes an image tensor, clones it 64 times, and runs it through a simplified local normalization function. This function just blurs the image using torchvision.transforms.GaussianBlur, and subtracts the original form it. Not only does the output of this operation vary by several orders of magnitude, even when using deterministic algorithms, the output is occasionally 0, meaning that applying a gaussian(img) == img approximately 2/64 times. This behavior persists on both CPU and GPU.</p>
<p>This very much looks like a bug to me, but perhaps someone who knows Pytorch’s internals better than me can explain something I’m missing.</p>
<p>The code from the notebook is reproduced below:</p>
<pre><code class="lang-auto">import torch
import torchvision
import matplotlib
import matplotlib.pyplot as plt
torch.use_deterministic_algorithms(True)

print(torch.__version__)

def localNorm(img):

	img = img.float()
	#img = img.to("cuda") ##Bug exists on both CPU and GPU
	blur = torchvision.transforms.GaussianBlur(9)
	img_b = blur(img)
	
	#assert(not (img_b == img).all()) ##Optional assert to catch the bug here, rather than comparing the sum to 0 later

	img_b = img_b - img ##subtract blurred image from the original

	return img_b.cpu().detach()

##three different ways of initializing the image. Two random, and one deterministic. All exhibit the bug

#r = torch.rand(1,2160,2560)

#r = torch.normal(0,1,(1,2160,2560))

r = torch.linspace(0,2160*2560,2160*2560)#*(3.141592653589)
r = torch.cos(r)
r = torch.unflatten(r,0,(1,2160,2560))

##intit a tensor 'r' representing an image, and clone it 64 times
dataset = [torch.clone(r) for i in range(64)]

fig, axarr = plt.subplots(8,8)

means = []

##run localNorm on the identical image 64 times, and observe differing results, with the occasional identical (zero sum of differences) result
for i,img in enumerate(dataset):
	norm = localNorm(img)[0]
	axarr[i//8,i%8].imshow(norm)#, cmap='gray')
	mean = torch.sum(norm)
	means.append(mean)
	print(mean.item(), (mean==0).item())

plt.gca().set_axis_off()
plt.subplots_adjust(top = 1, bottom = 0, right = 1, left = 0, hspace = 0, wspace = 0)
plt.margins(0,0)
for ax in fig.axes:
	ax.axis('off')

plt.show()

plt.plot(means)
plt.show()
</code></pre>

KFrank: <p>Hi Brian!</p>
<aside class="quote no-group" data-username="Brian_Krause" data-post="1" data-topic="198542" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brian_krause/48/68330_2.png" class="avatar"> Brian_Krause:</div>
<blockquote>
<p>It turns out that torchvision.transforms.GaussianBlur will occasionally return a tensor identical to its input, seemingly at random. Not only that, but it doesn’t appear to behave deterministicly, even with torch.use_deterministic_algorithms(True) set.</p>
</blockquote>
</aside>
<p>By design, an instance of <code>GaussianBlur</code> will use a randomly chosen<br>
value of <code>sigma</code> to generate its gaussian kernel every time it is applied<br>
to an image.  This is true regardless of whether or not you have set<br>
<code>torch.use_deterministic_algorithms (True)</code>.</p>
<p>When your instance of <code>GaussianBlur</code> chooses a sufficiently small value<br>
for <code>sigma</code>, the kernel it generates can be very close to no blur at all.  This<br>
is the effect you are seeing.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">	blur = torchvision.transforms.GaussianBlur(9)
	img_b = blur(img)
</code></pre>
</blockquote>
</aside>
<p>Even though you run this in a loop, you instantiate the same “value” of<br>
<code>blur</code> every time the first line runs – there is no randomness yet.  Then<br>
every time you run the second line, <code>blur</code> (the <code>GaussianBlur</code> instance)<br>
generates a random value of <code>sigma</code>, uses that <code>sigma</code> to create the kernel,<br>
and uses that kernel to blur the image.  Sometimes the generated <code>sigma</code><br>
is sufficiently small that, up to numerical precision, the image doesn’t get<br>
blurred.</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; import torchvision
&gt;&gt;&gt;
&gt;&gt;&gt; print (torch.__version__)
2.2.1
&gt;&gt;&gt; print (torchvision.__version__)
0.17.1
&gt;&gt;&gt;
&gt;&gt;&gt; _ = torch.manual_seed (2024)
&gt;&gt;&gt;
&gt;&gt;&gt; delt = torch.zeros (1, 11, 11)                                 # make a "delta-function" image with one central spot
&gt;&gt;&gt; delt[0, 5, 5] = 1.0
&gt;&gt;&gt; delt[0, 3:8, 3:8]                                              # look at just the center of delt
tensor([[0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.],
        [0., 0., 1., 0., 0.],
        [0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.]])
&gt;&gt;&gt;
&gt;&gt;&gt; blurA = torchvision.transforms.GaussianBlur (9)                # instantiate a GaussianBlur
&gt;&gt;&gt; blurA.sigma                                                    # sigma is the default range of (0.1, 2.0)
(0.1, 2.0)
&gt;&gt;&gt; delt_blurA = blurA (delt)                                      # blur delt with a kernel with a random choice of sigma
&gt;&gt;&gt; delt_blurA[0, 3:8, 3:8]                                        # look at just the center of the kernel
tensor([[0.0050, 0.0170, 0.0255, 0.0170, 0.0050],
        [0.0170, 0.0574, 0.0861, 0.0574, 0.0170],
        [0.0255, 0.0861, 0.1291, 0.0861, 0.0255],
        [0.0170, 0.0574, 0.0861, 0.0574, 0.0170],
        [0.0050, 0.0170, 0.0255, 0.0170, 0.0050]])
&gt;&gt;&gt;
&gt;&gt;&gt; blurB = torchvision.transforms.GaussianBlur (9, sigma = 0.1)   # instantiate a GaussianBlur with fixed sigma
&gt;&gt;&gt; blurB.sigma                                                    # sigma is the fixed value of 0.1
(0.1, 0.1)
&gt;&gt;&gt; delt_blurB = blurB (delt)                                      # blur delt with sigma = 0.1
&gt;&gt;&gt; delt_blurB[0, 3:8, 3:8]                                        # kernel is essentially no blur
tensor([[0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00],
        [0.0000e+00, 3.7835e-44, 1.9287e-22, 3.7835e-44, 0.0000e+00],
        [0.0000e+00, 1.9287e-22, 1.0000e+00, 1.9287e-22, 0.0000e+00],
        [0.0000e+00, 3.7835e-44, 1.9287e-22, 3.7835e-44, 0.0000e+00],
        [0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00]])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>