Pytorch color jitter

username: Arjun_Gupta
cooked: <p>From the documentation: “brightness_factor is chosen uniformly from [max(0, 1 - brightness), 1 + brightness]”</p>
<p>brightness by default is set to 0. This means that the brightness factor is chosen uniformly from [1, 1] meaning that brightness factor=1. The other parameters (contrast, saturation, hue) also seem to be constant under the default arguments. Does this mean that if color jitter is applied to the same image twice, the output will be the same?</p>
<p>If not, is there a way to perform the same color jitter twice on a pair of images? Thanks!</p>

username: ptrblck
cooked: <p>You could use the <code>staticmethod</code> <code>get_params</code> to apply the same “random” transformation via:</p>
<pre><code class="lang-python">img = transforms.ToPILImage()(torch.randn(3, 224, 224))

color_jitter = transforms.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1, hue=0.1)
transform = transforms.ColorJitter.get_params(
    color_jitter.brightness, color_jitter.contrast, color_jitter.saturation,
    color_jitter.hue)

img_trans1 = transform(img)
img_trans2 = transform(img)

print((np.array(img_trans1) == np.array(img_trans2)).all())
&gt; True
</code></pre>
<p>or alternatively use the funtional API directly via:</p>
<pre><code class="lang-python">import torchvision.transforms.functional as TF

img = TF.adjust_brightness(img, brightness_factor)
img = TF.adjust_contrast(img, contrast_factor)
img = TF.adjust_saturation(img, saturation_factor)
img = TF.adjust_hue(img, hue_factor)
</code></pre>

username: Arjun_Gupta
cooked: <p>Thanks for your response!</p>
<p>I tried <code>brightness=1, contrast=1, saturation=1, hue=0</code> in both the methods you suggested, which should theoretically return the original image (looking at the comments in the adjust_brightness function in <a href="https://github.com/pytorch/vision/blob/master/torchvision/transforms/functional.py" rel="nofollow noopener">source</a> and the functions below it) but both methods do not, while they do return the same image.</p>
<p>I am using the following to display the output image:</p>
<pre><code class="lang-auto">plt.imshow(img)
fig = plt.gcf()
fig.set_size_inches(14, 10)
plt.show()
</code></pre>
<p>Could there be an issue here? Or did I misunderstand something?</p>

username: ptrblck
cooked: <p>The passed arguments will be used to determine the new factors using the posted uniform distribution in the range <code>[max(0, 1 - arg), 1 + arg]</code>, where <code>arg</code> is <code>brightness, contrast, saturation</code>.<br>
For <code>hue</code>, <code>[-hue, +hue]</code> will be used.</p>
<p>If you want to get the original image without any transformation, you should set all arguments to <code>0.0</code>.</p>

username: Arjun_Gupta
cooked: <p>When I set all arguments to 0.0, I still do not get the same output.</p>
<p>I thought I was not converting the output images to tensors (<code>transforms.ToTensor()</code>) and normalizing them (<code>transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])</code>) to get them in exactly the same format as the original image displayed, but when I do this, I still do not get the original output.</p>
<p>Any ideas?</p>

username: ptrblck
cooked: <p>That’s weird, as I get the exact same outputs:</p>
<pre><code class="lang-python">img = transforms.ToPILImage()(torch.randn(3, 224, 224))

color_jitter = transforms.ColorJitter(brightness=0.0, contrast=0., saturation=0., hue=0.)
transform = transforms.ColorJitter.get_params(
    color_jitter.brightness, color_jitter.contrast, color_jitter.saturation,
    color_jitter.hue)

img_trans1 = transform(img)
img_trans2 = transform(img)

print((np.array(img) == np.array(img_trans1)).all())
&gt; True
print((np.array(img) == np.array(img_trans2)).all())
&gt; True
print((np.array(img_trans1) == np.array(img_trans1)).all())
&gt; True
</code></pre>
<p>Could you check your code for other transformations, which could change the output?</p>

username: Arjun_Gupta
cooked: <p>Interesting.</p>
<p>I’m not sure what I’m doing wrong:</p>
<p>I have a data loader, which does the following transformation:</p>
<pre><code class="lang-auto">def imageNetTransformPIL(size=224):
    return transforms.Compose([
        transforms.Resize(size),
        transforms.CenterCrop(size),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
</code></pre>
<p>I get a torch tensor of images (called <code>k</code>; <code>k.shape = (batch_size, 3, 224, 224)</code>) from my data loader and display it using the following code:</p>
<pre><code class="lang-auto">plt.imshow(k[0].permute(1, 2, 0))
fig = plt.gcf()
fig.set_size_inches(14, 10)
plt.show()
</code></pre>
<p>This displays the image as expected.</p>
<p>I then use your code and display the images at various points:</p>
<pre><code class="lang-auto">img = transforms.ToPILImage()(k[0])

color_jitter = transforms.ColorJitter(brightness=0, contrast=0, saturation=0, hue=0)
transform = transforms.ColorJitter.get_params(
    color_jitter.brightness, color_jitter.contrast, color_jitter.saturation,
    color_jitter.hue)
img = transform(img)

plt.imshow(img)
fig = plt.gcf()
fig.set_size_inches(14, 10)
plt.show()

transform_tensor = transforms.ToTensor()
img = transform_tensor(img)

transform_normalize = transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
img = transform_normalize(img)

plt.imshow(img.permute(1, 2, 0))
fig = plt.gcf()
fig.set_size_inches(14, 10)
plt.show()
</code></pre>
<p>Surprisingly, even the first image displayed is different (same structure but wildly different colors), suggesting that it is the method by which that I am displaying these images which is different. This led me to try turning the output into a torch tensor and displaying the image again, but this is also a very different image (same structure, very different colors).</p>
<p>Do you have any ideas what I might be doing wrong? Thanks a bunch for your help thus far.</p>

username: Arjun_Gupta
cooked: <p>What’s odd is that even if I comment out the color jittering, I don’t get the same output. Seems like it’s not a problem with your provided solution but rather the way in which I am displaying these images.</p>
<p>What is the right way to convert the PIL Image back into a torch tensor for both viewing purposes as well as to use in the <code>forward()</code> function?</p>

username: ptrblck
cooked: <p><code>ToPILImage()</code> will not reconstruct the original image, if you’ve normalized it as seen here:</p>
<pre><code class="lang-python">img = transforms.ToPILImage()(torch.randn(3, 224, 224))

norm = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) # comment this line
])


x_norm = norm(img)
img_reconstructed = transforms.ToPILImage()(x_norm)
print((np.array(img_reconstructed) == np.array(img)).all())
&gt; False
</code></pre>
<p>If you comment out the mentioned line it’ll work.</p>
<p>Alternatively, you could also “unnormalize” the image, but this would most likely yield small rounding errors, so that some pixel in the <code>uint8</code> format might be off by 1:</p>
<pre><code class="lang-python">x_norm = norm(img)
x = x_norm * torch.tensor([0.229, 0.224, 0.225])[:, None, None] + torch.tensor([0.485, 0.456, 0.406])[:, None, None]

img_reconstructed = transforms.ToPILImage()(x)
print((np.array(img_reconstructed).astype(np.long) - np.array(img).astype(np.long)))
</code></pre>

username: Arjun_Gupta
cooked: <p>Unfortunately, I have to use the original transformation I referenced.</p>
<p>Is there <code>norm()</code> function in your code <code>torch.norm()</code>? I just added:</p>
<pre><code class="lang-auto">x_norm = norm(img)
x = x_norm * torch.tensor([0.229, 0.224, 0.225])[:, None, None] + torch.tensor([0.485, 0.456, 0.406])[:, None, None]
</code></pre>
<p>in my code before <code>ToPILImage()</code> yet this yields an image which is just entirely blue. Sorry for all this back and forth. I’m still unsure why I’m not getting the expected output…</p>

username: ptrblck
cooked: <p><code>norm</code> is defined as the <code>ToTensor</code> and <code>Normalize</code> transformation in my code.<br>
Could you post your current code you are using, which creates these blue images, so that I could take another look, please?</p>

username: Arjun_Gupta
cooked: <p><code>torch.norm()</code> was the issue. Thank you so much for your help! Very much appreciated. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Robbert_D
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,</p>
<p>Your code snippet worked fine for me until I upgraded to PyTorch 1.8.1.<br>
Now I get a type error: <code>TypeError: 'tuple' object is not callable</code></p>
<p>Any suggestions?</p>

username: ptrblck
cooked: <p>Could you post an executable code snippet to reproduce this issue in the latest PyTorch and <code>torchvision</code> release, please?</p>

username: Robbert_D
cooked: <pre><code class="lang-auto">import torch
import PIL
from torchvision import transforms

img = transforms.ToPILImage()(torch.randn(3, 224, 224))

color_jitter = transforms.ColorJitter(brightness=0.0, contrast=0., saturation=0., hue=0.)
transform = transforms.ColorJitter.get_params(
    color_jitter.brightness, color_jitter.contrast, color_jitter.saturation,
    color_jitter.hue)

img_trans1 = transform(img)
img_trans2 = transform(img)
</code></pre>
<p>using pytorch 1.8.1 and torchvision 0.9.1<br>
build: py3.8_cuda11.1_cudnn8.0.5_0</p>

username: ptrblck
cooked: <p>Thanks for the code. The <code>get_params</code> method will return a <code>tuple</code> with the applied parameters for each transformation as well as their order and thus cannot be used as a function call.</p>
<p>From the docs:</p>
<blockquote>
<p>Returns:	<br>
The parameters used to apply the randomized transform along with their random order.</p>
</blockquote>
<p>If you want to apply the transformation in the image, call it via <code>out = color_jitter(img)</code>.</p>

username: Robbert_D
cooked: <p>Hi ptrblck,</p>
<p>I understand that color_jitter(img) will apply the transformation to the img.<br>
However, I want to use the same exact transformation and apply it to multiple images (different sizes so can’t batch them together). This used to work with the code snippet you provided. Since torch 1.9.1   <code>transforms.ColorJitter.get_params</code> seems to return order and values of the transformations instead of the lambda functions as in previous torch versions.<br>
I guess a workaround would be to take the values of <code>transforms.ColorJitter.get_params</code> and use them in functions like <code>torchvision.transforms.functional.adjust_brightness</code> but this seems like a roundabout way of doing things.</p>
<p>Thank you for your help so far.</p>

username: ptrblck
cooked: <p>The behavior was changed in <a href="https://github.com/pytorch/vision/pull/3001">this PR</a> and is BC-breaking.<br>
I think returning the parameters instead of transformations sounds reasonable given the <code>get_params</code> name and the behavior of other transformations (which also return the parameters, not transformations).<br>
I’m not sure, why it was implemented in the previous way.</p>
<p>Yes, you could use the functional API to apply the parameters and could reuse the <a href="https://github.com/pytorch/vision/blob/2f40a483d73018ae6e1488a484c5927f2b309969/torchvision/transforms/transforms.py#L1053-L1085">old implementation</a>.</p>

username: Robbert_D
cooked: <p>Ahh, I didn’t think to just go get the code in the previous release.<br>
And I agree that the naming conventions make more sense now.<br>
Thank you for the suggestion!</p>