Question about transforms impact on images

Geoffrey_Payne: <p>I load my pickle file as follows;</p>
<blockquote>
<p>with open(os.path.join(“.”, picklePath), “br”) as fh:<br>
data = pickle.load(fh)<br>
and from that I set my datasets for images and labels;<br>
X_tr = data[‘training_data’][0]<br>
y_tr = data[‘training_data’][1].ravel()<br>
X_v = data[‘validation_data’][0]<br>
y_v = data[‘validation_data’][1].ravel()<br>
X_t = data[‘test_data’][0]<br>
y_t = data[‘test_data’][1].ravel()</p>
</blockquote>
<p>I take a view of the first image and it looks as expected, so I create this transform (note that the resize is commented out)</p>
<blockquote>
<pre><code> _transforms = transforms.Compose([
        transforms.ToPILImage(),
        #transforms.Resize((ImageSize.width, ImageSize.height)),
        transforms.Grayscale(num_output_channels=1),
        transforms.ToTensor()
    ])
</code></pre>
</blockquote>
<p>I then use this transform to create a custom dataset in which the labels and the images are aligned</p>
<blockquote>
<pre><code>    train_dataset = EmotionDatasetTrain(X_tr, y_tr, transform=_transforms)
    test_dataset = EmotionDatasetTest(X_t, y_t, transform=_transforms)
    val_dataset = EmotionDatasetVal(X_v, y_v, transform=_transforms)
</code></pre>
</blockquote>
<p>And the custom dataset is defined as follows (similar classes exists for Test and Validation)</p>
<blockquote>
<p>class EmotionDataSet():<br>
def <strong>init</strong>(self, transform):<br>
self.transform = transform<br>
self.classes = [“neutral”, “anger”, “contempt”, “disgust”, “fear”, “happy”, “sadness”, “surprise”]</p>
<pre><code>def get_x_y(self, idx, _X, _Y):
    if torch.is_tensor(idx):
        idx = idx.tolist()

    y = _Y[idx].item()
    
    if self.transform:
        x = self.transform(_X[idx].reshape(ImageSize.width, ImageSize.height))
    return x, y
</code></pre>
<p>class EmotionDatasetTrain(EmotionDataSet, Dataset):</p>
<pre><code>def __init__(self, X_Train, Y_Train, transform=None):
    super().__init__(transform)
    self.X_Train = X_Train
    self.Y_Train = Y_Train

def __len__(self):
    return len(self.X_Train)

def __getitem__(self, idx):
    x, y = super().get_x_y(idx, self.X_Train, self.Y_Train)
    return x, y
</code></pre>
</blockquote>
<p>And I have a dataloader;</p>
<blockquote>
<pre><code>  train_loader = torch.utils.data.DataLoader(
        train_dataset,
        batch_size=Settings.batchSize,
        num_workers=0,
        shuffle=True
    )
</code></pre>
</blockquote>
<p>My problem is that although the images are read in correctly and can be displayed via Matlibplot, after the transform the images deteriorate considerably when viewed via Matlibplot.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/f/2f3c33cbf7ca37979462fa62a77e4314b0691ae3.png" alt="After Face" data-base62-sha1="6JRqhTzZ9ruH90vlHeFdi71xH4n" width="640" height="476"><br>
The image is mostly purple with yellow outlines of the face. The resolution had gone completely, the faces were barely recognisable. When I run them through the CNN the loss function value barely changes after each epoch and given how crude the images are that is hardly surprising. I notice the following.<br>
1/ I was hoping the Resize transform would convert an array of 1000 float numbers into a 100 * 100 matrix. It does not, so I use a reshape instead. I clearly misunderstand the resize transform.<br>
2/ I expected the grayscale transform to turn a RGB image into a single channel grayscale image. I thought what I would see was a black and white image of the colour photo with the same resolution. Instead this was the transform - it would appear - that radically changed the image as described earlier.</p>
<p>I am new to all this, can someone explain what is happening and how I can fix this?</p>

ptrblck: <p>I guess the <code>ToPILImage</code> transformation (and later the <code>ToTensor</code>) might try to normalize your numpy arrays, which might result in a clamped output.<br>
Could you try to pass a single input array to <a href="https://github.com/pytorch/vision/blob/d481f2d88f16c6a6c1f143629876bf536072921b/torchvision/transforms/functional.py#L192-L276"><code>torchvision.transforms.functional.to_pil_image()</code></a> as well as <code>to_tensor()</code> and check the output values?</p>

Geoffrey_Payne: <p>The problem was that although MatlibPlot displayed an image with a green tinge to it, it was actually grayscale already and didn’t need transforming.</p>