Float32 error when using iter(next())

Jan_Spiegel: <p>Hi all,</p>
<p>I have translated the brain MRI scans here:</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://figshare.com/articles/dataset/brain_tumor_dataset/1512427">
  <header class="source">
      <img src="https://websitev3-p-eu.figstatic.com/assets-v3/bee2b12a367b114cc0f33f2f24c15a70b76227db/static/media/favicon-16.658d1490.png" class="site-icon" width="" height="">

      <a href="https://figshare.com/articles/dataset/brain_tumor_dataset/1512427" target="_blank" rel="noopener nofollow ugc">figshare</a>
  </header>

  <article class="onebox-body">
    

<h3><a href="https://figshare.com/articles/dataset/brain_tumor_dataset/1512427" target="_blank" rel="noopener nofollow ugc">brain tumor dataset</a></h3>

  <p>This brain tumor dataset contains 3064 T1-weighted contrast-inhanced images with three kinds of brain tumor. Detailed information of the dataset can be found in readme file.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>into a pickle dataset containing 3,064 OpenCV RGB images. My pickle dataset is here:</p>
<p><a href="https://drive.google.com/file/d/143O--CA_hrnGrWOzBy02hQBfbx7OvMBI/view?usp=sharing" class="onebox" target="_blank" rel="noopener nofollow ugc">https://drive.google.com/file/d/143O--CA_hrnGrWOzBy02hQBfbx7OvMBI/view?usp=sharing</a></p>
<p>I am trying to load this data into a dataloader object to run some pytorch image classification models on this, here is my code:</p>
<pre><code class="lang-auto">training_data = pickle.load(open('/content/drive/My Drive/MSc Applied Stats/Thesis/baseline_models/Cheng2015/new_dataset/training_data.pickle', 'rb'))

Xt = []
yt = []
features = None
labels = None
label = []

for features, labels, masks, borders in training_data:
  Xt.append(features)
  yt.append(labels)

mn = [0.485, 0.456, 0.406]

st = [0.229, 0.224, 0.225]

</code></pre>
<p>I define  my dataloader object like this</p>
<pre><code class="lang-auto">class BrainTumorDataset(Dataset):
  def __init__(self, images, labels):
    # images
    self.X = images
    # labels
    self.y = labels
    
    # Transformation for converting original image array to an image and then convert it to a tensor
    self.transform = transforms.Compose([
        
        transforms.ToPILImage(),
                                         
    # transforms.ToTensor() does min-max normalisation of pixel values into range 0-1.
        transforms.ToTensor(),
    #alternatively can standardise so pixel values have similar distribution to imagenet data, for transfer learning models
        transforms.Normalize(mean=mn, std=st)
    ])


    # Transformation for converting original image array to an image, rotate it randomly between -90 degrees and 90 degrees, and then convert it to a tensor
    self.transform2 = transforms.Compose([
        transforms.ToPILImage(),
        transforms.RandomRotation(90),
    # transforms.ToTensor() does min-max normalisation of pixel values into range 0-1.
        transforms.ToTensor(),
    #also need to standardise so pixel values have similar distribution to imagenet data, for transfer learning models
        transforms.Normalize(mean=mn, std=st)
    ])


    # Transformation for converting original image array to an image, rotate it randomly between -180 degrees and 180 degrees, and then convert it to a tensor
    self.transform4 = transforms.Compose([
        transforms.ToPILImage(),
        transforms.RandomRotation(180),    
    # transforms.ToTensor() does min-max normalisation of pixel values into range 0-1.    
        transforms.ToTensor(),
    
    #alternatively need to standardise so pixel values have similar distribution to imagenet data, for transfer learning models
        transforms.Normalize(mean=mn, std=st)
                       
    ])


    # Transformation for converting original image array to an image, random vertical flip, and then convert it to a tensor
    self.transform7 = transforms.Compose([
        transforms.ToPILImage(),
        transforms.RandomVerticalFlip(p=0.99),
    # transforms.ToTensor() does min-max normalisation of pixel values into range 0-1.
        transforms.ToTensor(),
        #alternatively can standardise so pixel values have similar distribution to imagenet data, for transfer learning models
        transforms.Normalize(mean=mn, std=st)
    ])

  def __len__(self):
    # return length of image samples
    return len(self.X)

  def __getitem__(self, idx):
    # perform transformations on one instance of X
    # Original image as a tensor
    data = self.transform(self.X[idx])
    #data = data..astype('float64')
    print(data)
    # Augmented image at 90 degrees as a tensor
    aug90 = self.transform2(self.X[idx])

    # Augmented image at 180 degrees as a tensor
    aug180 = self.transform4(self.X[idx])

    # Augmented image vertical flip as a tensor
    augVF = self.transform7(self.X[idx])      
    
    # store the transformed images in a list
    new_batch = [data, aug90, aug180, augVF]

    # one-hot encode the labels
    labels = torch.zeros(4, dtype=torch.float32)
    labels[int(self.y[idx])] = 1.0

    new_labels = [labels, labels, labels, labels]

    # 8 augmented images and corresponding labels per sample will be returned
    return (torch.stack(new_labels), torch.stack(new_batch))
</code></pre>
<p>Creating the dataloader like this works</p>
<pre><code class="lang-auto"># 70 % training, 15% validating, 15% testing
X_train, X_test, y_train, y_test = train_test_split(Xt, yt, test_size=0.3, shuffle=True, random_state= 42)  # 70% training, 30% testing
X_valid, X_test, y_valid, y_test = train_test_split(X_test, y_test, test_size=0.5, shuffle=True, random_state = 42)  # split testing set into 50% validation , 50% testing 


train_set = BrainTumorDataset(X_train, y_train)
valid_set = BrainTumorDataset(X_valid, y_valid)
test_set = BrainTumorDataset(X_test, y_test)

train_gen = DataLoader(train_set, batch_size=4, shuffle=True, pin_memory=True, num_workers=4)
valid_gen = DataLoader(valid_set, batch_size=4, shuffle=True, pin_memory=True, num_workers=4)
test_gen = DataLoader(test_set, batch_size=10, shuffle=True, pin_memory=True, num_workers=4)

</code></pre>
<p>But when I try to inspect say the train_gen object I get a strange error message</p>
<pre><code class="lang-auto">next(iter(train_gen))[1]
</code></pre>
<pre><code class="lang-auto">---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

&lt;ipython-input-112-6664f9e249ff&gt; in &lt;module&gt;
----&gt; 1 next(iter(train_gen))[1]

3 frames

/usr/local/lib/python3.9/dist-packages/torch/_utils.py in reraise(self)
    541             # instantiate since we don't know how to
    542             raise RuntimeError(msg) from None
--&gt; 543         raise exception
    544 
    545 

TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.9/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.9/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "&lt;ipython-input-102-00a3581dfcf5&gt;", line 61, in __getitem__
    data = self.transform(self.X[idx])
  File "/usr/local/lib/python3.9/dist-packages/torchvision/transforms/transforms.py", line 95, in __call__
    img = t(img)
  File "/usr/local/lib/python3.9/dist-packages/torchvision/transforms/transforms.py", line 227, in __call__
    return F.to_pil_image(pic, self.mode)
  File "/usr/local/lib/python3.9/dist-packages/torchvision/transforms/functional.py", line 332, in to_pil_image
    raise TypeError(f"Input type {npimg.dtype} is not supported")
TypeError: Input type float32 is not supported

</code></pre>
<p>My understanding is that float32 is the standard data type here, so why does iter(next()) not work?</p>

ptrblck: <p>The issue is raised by <code>ToPILImage</code> as you are passing a numpy array in an unsupported <code>dtype</code> to it:</p>
<pre data-code-wrap="python"><code class="lang-python">transforms = transforms.ToPILImage()

# works
x = torch.randint(0, 256, (3, 224, 224)).byte()
img = transforms(x)

# works
x = torch.randn(3, 224, 224)
img = transforms(x)

# fails
x = np.random.randn(224, 224, 3)
img = transforms(x)
# TypeError: Input type float64 is not supported
</code></pre>
<aside class="quote no-group" data-username="Jan_Spiegel" data-post="1" data-topic="175677">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jan_spiegel/48/58713_2.png" class="avatar"> Jan_Spiegel:</div>
<blockquote>
<p>My understanding is that float32 is the standard data type here, so why does iter(next()) not work?</p>
</blockquote>
</aside>
<p>That’s correct for PyTorch tensors, which also work, but the transformation requires “standard” numpy image formats as seen <a href="https://github.com/pytorch/vision/blob/2b25d67925df9741ba2a75a07bc3046302969e87/torchvision/transforms/functional.py#L244-L337">here</a>.</p>

Jan_Spiegel: <p>Thank you!</p>
<p>Transposing my data gave me the error below (which is clearer than what I got before), but your link helped!</p>
<p>Adding mode=‘RGB’ to the ToPILImage call resolved the issue.</p>
<p>(what I get when transposing my data into [C,H,W])</p>
<pre><code class="lang-auto">ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.9/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.9/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "&lt;ipython-input-169-4b1060b9c7d1&gt;", line 61, in __getitem__
    data = self.transform(self.X[idx])
  File "/usr/local/lib/python3.9/dist-packages/torchvision/transforms/transforms.py", line 95, in __call__
    img = t(img)
  File "/usr/local/lib/python3.9/dist-packages/torchvision/transforms/transforms.py", line 227, in __call__
    return F.to_pil_image(pic, self.mode)
  File "/usr/local/lib/python3.9/dist-packages/torchvision/transforms/functional.py", line 283, in to_pil_image
    raise ValueError(f"pic should not have &gt; 4 channels. Got {pic.shape[-1]} channels.")
ValueError: pic should not have &gt; 4 channels. Got 512 channels.
</code></pre>