Grayscale to RGB transform

Deeply: <p>Some of the images I have in the dataset are gray-scale, thus, I need to convert them to RGB, by replicating the gray-scale to each band. I am using a transforms.lambda  to do that, based on torch.cat. However, this seems to not give the expected results<br>
Example: Let xx be some image of size 28x28, then,</p>
<pre><code class="lang-auto">In [67]: xx.shape
Out[67]: torch.Size([28, 28])

In [68]: y =torch.cat([xx,xx,xx],0)

In [69]: y.shape
Out[69]: torch.Size([84, 28])
</code></pre>
<p>I expected the size to be [28, 28, 3]. Any suggestions how to resole this?</p>
<p>Here’s the transform I am trying:</p>
<pre><code class="lang-auto">transform = transforms.Compose([
                transforms.ToTensor(), 
                transforms.Lambda(lambda x: torch.cat([x, x, x], 0)),
                transforms.Normalize(mean, std),
                 ])
</code></pre>

ptrblck: <p>While loading your images, you could use <code>Image.open(path).convert('RGB')</code> on all images.<br>
If you are using <code>ImageFolder</code>, this functionality should be already there using the default <code>loader</code>.<br>
Alternatively, you could repeat the values:</p>
<pre><code class="lang-auto">x = torch.randn(28, 28)
x.unsqueeze_(0)
x = x.repeat(3, 1, 1)
x.shape
&gt; torch.Size([3, 28, 28])
</code></pre>

Deeply: <p>I am using it with with MNIST, and I am using  datasets.MNIST dataloader.<br>
Not sure however how to call the conversion <code>Image.open(path).convert('RGB')</code>, as it is already there as you noted.</p>

ptrblck: <p>The MNIST dataset doesn’t convert the images to RGB, but to a grayscale image. Have a look at <a href="https://github.com/pytorch/vision/blob/master/torchvision/datasets/mnist.py#L84">this line of code</a>.</p>
<p>I assume you are using the MNIST data with another color image set?<br>
If so, you could check in <code>__getitem__</code>, if it’s already a color image, and if not use my second approach to convert it.</p>

karandwivedi42: <p>If you replace <code>y =torch.cat([xx,xx,xx],0)</code> with <code>y =torch.stack([xx,xx,xx],2)</code> it works <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<pre><code class="lang-python">import torch
xx = torch.rand(28,28)
y =torch.stack([xx,xx,xx],0)
print(xx.shape)
print(y.shape)
print(torch.norm(y[:,:,0] - xx))
</code></pre>

Deeply: <p>Thanks Karan. For some reason, the statement that get things done was the one that ptrblck suggested:</p>
<p><code>transforms.Lambda(lambda x: x.repeat(3, 1, 1) )</code></p>

neighthan: <p>You can also convert a 2D grayscale image to a 3D RGB one by doing:</p>
<pre><code class="lang-auto">img = img.view(width, height, 1).expand(-1, -1, 3)
</code></pre>
<p>Calling <code>.repeat</code> will actually replicate the image data (taking 3x the memory of the original image) whereas <code>.expand</code> will behave as if the data is replicated without actually doing so. Thus <code>.expand</code> is probably better unless you want to change the channels independently of each other.</p>

Deeply: <p>That’s nice!<br>
How about speed/performance, Repeat vs Expand?</p>

John_Grabner: <p>A few of my files are grayscale, but most are jpeg RGB.<br>
Why does the following not work?</p>
<blockquote>
<p>im = PIL.Image.open(img_path)<br>
im.convert(“RGB”)<br>
im_torch = torchvision.transforms.ToTensor()(im)</p>
</blockquote>
<p>Just like the suggestion above, I need to add</p>
<blockquote>
<p>if im_torch.shape[0]==1:<br>
print(f"im_torch.shape={im_torch.shape}“) # im_torch.shape=torch.Size([1, 4077, 4819])<br>
im_torch = im_torch.expand(3,-1,-1)<br>
print(f"im_torch.shape={im_torch.shape}”) # im_torch.shape=torch.Size([3, 4077, 4819])</p>
</blockquote>
<p>notice the output of the first print statement is</p>
<blockquote>
<p>im_torch.shape=torch.Size([1, 4077, 4819])</p>
</blockquote>
<p>So does im.convert(“RGB”) not convert the file?</p>

Deeply: <p>You missed storing the converted image:</p>
<p><code>im = im.convert('RGB')</code></p>
<p><strong>Else, you can try this</strong>:</p>
<pre><code class="lang-auto">class NoneTransform(object):
    ''' Does nothing to the image. To be used instead of None '''
    
    def __call__(self, image):       
        return image
    

im = PIL.Image.open(img_path)

T = transforms.Compose([
            transforms.ToTensor(),            
            transforms.Lambda(lambda x: x.repeat(3, 1, 1))  if im.mode!='RGB'  else NoneTransform()                 
            ])    

im_torch = T(im)
</code></pre>
<p>NB. As far as I remember, <code>None</code> would not work if used instead of NoneTransform()</p>

John_Grabner: <p>Thank you. How silly of me.</p>

Deeply: <p>No worries. This happens to everyone. We are used to OOP, and thus, we expect that <code>im.convert('RGB')</code> does the job.</p>

torch_bearer: <p>How does converting gray scale to rgb work?</p>
<p>Does it duplicate the channel?</p>

Deeply: <p>Depending on what you want to do. Above the channels are replicated. If you want to to colorize grayscale images, then you need to use some <a href="https://deepai.org/machine-learning-model/colorizer" rel="noopener nofollow ugc">colorization algorithms</a>.</p>

wogong: <p>transforms.Grayscale(3)</p>

MaKaNu: <p>I don’t now if this is something wrong with pillow. But I recognized, that using the <code>convert</code> method from pillow it looses all information from the loaded int32 grayscale image and sets all values to 255. I can confirm that the entropy of the image was definitely higher before I converted the image to “RGB”. Stacking the image by hand is working but results in problems for the image transformations I want to apply.</p>

ptrblck: <p>I guess you are converting the image array from <code>int32</code> to <code>uint8</code>, so the clipping would be expected.<br>
From the <code>mode</code> <a href="https://pillow.readthedocs.io/en/stable/handbook/concepts.html#concept-modes">docs</a>:</p>
<blockquote>
<p><code>RGB</code> (3x8-bit pixels, true color)</p>
</blockquote>

MaKaNu: <p>yes you are correct, any Idea how to convert from int32 to uint8 without clipping? because my images are always get loaded as int32. The mode of the images is set to “I” which results from the docs as int32 pixels.</p>

Sammo1: <aside class="quote no-group quote-modified" data-username="Deeply" data-post="3" data-topic="18315">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/deeply/48/3840_2.png" class="avatar"> Deeply:</div>
<blockquote>
<p>.convert(‘RGB’)</p>
</blockquote>
</aside>
<p>excuse me will the result be the same. I mean if i used <code>convert('RGB')</code> or repeat the values of grayscale will be the same</p>

ptrblck: <aside class="quote no-group" data-username="MaKaNu" data-post="18" data-topic="18315">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/makanu/48/48166_2.png" class="avatar"> MaKaNu:</div>
<blockquote>
<p>yes you are correct, any Idea how to convert from int32 to uint8 without clipping?</p>
</blockquote>
</aside>
<p>That won’t be possible as <code>int32</code> is using 32 bits and has a wider range than<code>uint8</code> using 8 bits.<br>
Unless you make sure the original <code>int32</code> image doesn’t have values <code>&lt;0</code> and <code>&gt;255</code> you would clip them.</p>
<aside class="quote no-group" data-username="Sammo1" data-post="19" data-topic="18315">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sammo1/48/46342_2.png" class="avatar"> Sammo1:</div>
<blockquote>
<p>if i used <code>convert('RGB')</code> or repeat the values of grayscale will be the same</p>
</blockquote>
</aside>
<p>Yes, this should be the case:</p>
<pre data-code-wrap="python"><code class="lang-python">img = transforms.ToPILImage()(torch.randint(0, 256, (224, 224)).byte())
img_rgb = img.convert("RGB")

img_rgb_arr = np.array(img_rgb)
img_arr = np.array(img)

for c in range(img_rgb_arr.shape[2]):
    print(c, np.all(img_arr == img_rgb_arr[:, :, c]))
# 0 True
# 1 True
# 2 True

print((torch.from_numpy(img_rgb_arr) == torch.from_numpy(img_arr).unsqueeze(2).expand(-1, -1, 3)).all())
# tensor(True)
</code></pre>