Trouble using transforms.FiveCrop()/TenCrop()

c_t: <p>Hello,</p>
<p>I am trying to increase my CNN’s performance and thus i decided to “play” with some transformations in order to see how they affect my model.  I read that FiveCrop() and TenCrop() might help because they generate extra data to train on.  However, when i try to train the model, using one of the transformations mentioned above, i get the following error:</p>
<p>TypeError: pic should be PIL Image or ndarray. Got &lt;class ‘tuple’&gt;</p>
<p>In the documentation of those transformations, it only states a note for the test procedure, any idea how to fix this?</p>
<p>Thanks in advance!</p>

ptrblck: <p>Could you post the code snippet where you try to apply this transformation?<br>
Apparently you are passing a <code>tuple</code>, while the method expects a <code>PIL.Image</code>.</p>

Ridzy619: <pre><code class="lang-auto">trans = transforms.Compose([transforms.Resize(256),
                            transforms.TenCrop(224), # this is a list of PIL Images
                            transforms.Lambda(lambda crops: torch.stack([transforms.ToTensor()(crop) for crop in crops])),
                            transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
                           ])
test_trans = transforms.Compose([transforms.Resize(256),
                            transforms.CenterCrop(224),
                            #transforms.Lambda(lambda crops: torch.stack([ToTensor()(crop) for crop in crops])),
                            transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
                           ])
train = datasets.ImageFolder('/data/dog_images/train', transform = trans)
test = datasets.ImageFolder('/data/dog_images/test', transform = test_trans)
valid = datasets.ImageFolder('/data/dog_images/valid', transform = test_trans)

trainloader = torch.utils.data.DataLoader(train, shuffle = True, batch_size = 16, drop_last = True)
testloader = torch.utils.data.DataLoader(test, shuffle = False, batch_size = 16, drop_last = True)
validloader = torch.utils.data.DataLoader(valid, shuffle = False, batch_size = 16, drop_last = True)

loaders_scratch = {'train': trainloader,
                  'valid': validloader,
                  'test': testloader}
</code></pre>
<p>So, it appears the tencrop part of the transforms is stopping the Normalize part of it.</p>

Ridzy619: <pre><code class="lang-auto">trans = transforms.Compose([transforms.Resize(256),
                            transforms.TenCrop(224), # this is a list of PIL Images
                            transforms.Lambda(lambda crops: torch.stack([transforms.ToTensor()(crop) for crop in crops])),
                            transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
                           ])
test_trans = transforms.Compose([transforms.Resize(256),
                            transforms.CenterCrop(224),
                            #transforms.Lambda(lambda crops: torch.stack([ToTensor()(crop) for crop in crops])),
                            transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
                           ])
train = datasets.ImageFolder('/data/dog_images/train', transform = trans)
test = datasets.ImageFolder('/data/dog_images/test', transform = test_trans)
valid = datasets.ImageFolder('/data/dog_images/valid', transform = test_trans)

trainloader = torch.utils.data.DataLoader(train, shuffle = True, batch_size = 16, drop_last = True)
testloader = torch.utils.data.DataLoader(test, shuffle = False, batch_size = 16, drop_last = True)
validloader = torch.utils.data.DataLoader(valid, shuffle = False, batch_size = 16, drop_last = True)

loaders_scratch = {'train': trainloader,
                  'valid': validloader,
                  'test': testloader}
</code></pre>
<p>Please help with a way around the transforms.Normalize as it is returning error</p>

ptrblck: <p>I assume the error is raised, since you are passing multiple inputs returned by the <code>TenCrop</code> transformation?<br>
If that’s the case, you could add the <code>Normalize</code> operation to the <code>transforms.Lambda</code> call as:</p>
<pre><code class="lang-python">transforms.Lambda(lambda crops: torch.stack([transforms.Normalize(mean=..., std=...)(transforms.ToTensor()(crop)) for crop in crops]))
</code></pre>
<p>or alternatively add another <code>transforms.Lambda</code> as:</p>
<pre><code class="lang-python">trans = transforms.Compose([transforms.Resize(256),
                            transforms.TenCrop(224), # this is a list of PIL Images
                            transforms.Lambda(lambda crops: torch.stack([transforms.ToTensor()(crop) for crop in crops])),
                            transforms.Lambda(lambda tensors:
                                torch.stack([transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])(t) for t in tensors]))
                           ])
</code></pre>

Ridzy619: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a> . I am yet to try it but I’m positive it would work. Why I didn’t of this solution beats my imagination.</p>

Ridzy619: <aside class="quote no-group" data-username="ptrblck" data-post="5" data-topic="32059">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>torch.stack([transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])(t) for t in tensors]))</p>
</blockquote>
</aside>
<p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a>, it worked.</p>
<p>I have another question: Applying <code>TenCrop</code> transforms and using batch (say 16) loading creates a batch with the dimension <code>torch.Size([16, 10, 3, 224, 224])</code>. This means I have 160 images when I reshape it to BCWH, ten of each. I think will be to large a batch size to use and it will also make the batch distribution unrepresentative of the entire dataset.</p>
<p>Please help.<br>
`</p>

ptrblck: <p>If the total batch size is too large, you could either decrease the <code>batch_size</code> or use <code>FiveCrop</code>.<br>
<code>FiveCrop</code> and <code>TenCrop</code> will increase the samples in the batch, but that’s of course its purpose.</p>

Pranjol_Sen_Gupta: <p>I am working with ChestXray data. I have found a model on Github.<br>
Link <a href="https://github.com/arnoweng/CheXNet" class="inline-onebox" rel="noopener nofollow ugc">GitHub - arnoweng/CheXNet: A pytorch reimplementation of CheXNet</a><br>
I have downloaded the pre-trained model and loaded it in CUDA-based GPU. Now I want to test the code by using an image dataset. I believe the ChestXrayDataSet function is working properly. and I am using ChestXrayDataSet for test_dataset generation. Then I have used the DataLoader of PyTorch for creating the test_loader variable. Up to this everything works fine.<br>
But when I’m using the enumerate in test_loader it is creating an error. I believe there is a problem with my lambda function in the test_dataset variable or the problem lies in the ChestXrayDataSet function. I have added the ChestXrayDataSet function and the other two images with errors!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/2/f25e0a2f921a2b361fa00db56ab26910b53803be.png" data-download-href="https://discuss.pytorch.org/uploads/default/f25e0a2f921a2b361fa00db56ab26910b53803be" title="tan2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f25e0a2f921a2b361fa00db56ab26910b53803be_2_470x500.png" alt="tan2" data-base62-sha1="yA51GqqAP221E9hbenz785FloLQ" width="470" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f25e0a2f921a2b361fa00db56ab26910b53803be_2_470x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f25e0a2f921a2b361fa00db56ab26910b53803be_2_705x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f25e0a2f921a2b361fa00db56ab26910b53803be_2_940x1000.png 2x" data-dominant-color="F6F7F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">tan2</span><span class="informations">1586×1687 255 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Pranjol_Sen_Gupta: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/d/3dcbd577f66cf5dac313d6ae689103952ce1603a.png" data-download-href="https://discuss.pytorch.org/uploads/default/3dcbd577f66cf5dac313d6ae689103952ce1603a" title="tan3"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/d/3dcbd577f66cf5dac313d6ae689103952ce1603a_2_690x466.png" alt="tan3" data-base62-sha1="8OFPWf0rnEIevmjbxInUcwkb4hA" width="690" height="466" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/d/3dcbd577f66cf5dac313d6ae689103952ce1603a_2_690x466.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/d/3dcbd577f66cf5dac313d6ae689103952ce1603a_2_1035x699.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/d/3dcbd577f66cf5dac313d6ae689103952ce1603a_2_1380x932.png 2x" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">tan3</span><span class="informations">1704×1153 101 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Pranjol_Sen_Gupta: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/0/c01470829022ab4ff71bb69f0d83c827d1f247c7.png" data-download-href="https://discuss.pytorch.org/uploads/default/c01470829022ab4ff71bb69f0d83c827d1f247c7" title="tan1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/0/c01470829022ab4ff71bb69f0d83c827d1f247c7_2_690x408.png" alt="tan1" data-base62-sha1="rpdsSi116lLfIvO1QUPk9dasooT" width="690" height="408" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/0/c01470829022ab4ff71bb69f0d83c827d1f247c7_2_690x408.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/0/c01470829022ab4ff71bb69f0d83c827d1f247c7_2_1035x612.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/0/c01470829022ab4ff71bb69f0d83c827d1f247c7_2_1380x816.png 2x" data-dominant-color="F0F1F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">tan1</span><span class="informations">2550×1510 220 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>I don’t know exactly what’s causing the error, but try to replace the <code>lambda</code> operations with custom transformations implemented as classes. You could use any <a href="https://github.com/pytorch/vision/blob/3926c905b74af680b181fb06cca8ab7d871813af/torchvision/transforms/transforms.py"><code>torchvision.transforms</code></a> as the boilerplate.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>