Is transforms.RandomResizedCrop used for Data Augmentation?

xiaodongww: <p>In the past, I thought  <code>transforms.RandomResizedCrop</code> is used for data augmentation because it will random scale the image and crop it, and then resize it to the demanded size. And the data augmentation part in my code is usually as follows:</p>
<pre><code class="lang-auto">normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                 std=[0.229, 0.224, 0.225])

train_transform = transforms.Compose([transforms.RandomHorizontalFlip(), 
                                      transforms.RandomResizedCrop(224), 
                                      transforms.ToTensor(),
                                      normalize])
</code></pre>
<p>But I have just check the mannual and it says that</p>
<blockquote>
<p><a href="http://pytorch.org/docs/master/torchvision/transforms.html#torchvision.transforms.RandomResizedCrop" rel="noopener nofollow ugc">A crop of random size (default: of 0.08 to 1.0) of the original size</a></p>
</blockquote>
<p>According to the comment, it may only get a very  very small part of the original image after cropping  considering that the scale range from 0.08 to 1.0. In that way, it will be harmful for training.</p>
<p>Do I misunderstand this function? Is it used for data augmentation?<br>
And is it necessary to specifically assign the scale parameter as follows if I want to use it for data augmentation?</p>
<pre><code class="lang-auto">normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                 std=[0.229, 0.224, 0.225])

train_transform = transforms.Compose([transforms.RandomHorizontalFlip(), 
                                      transforms.RandomResizedCrop(224, scale=(0.7, 1.0), 
                                      transforms.ToTensor(),
                                      normalize])
</code></pre>
<p>Any suggestion will be helpful.<br>
Thank you!</p>

ptrblck: <p>You can use it for data augmentation.<br>
It’s used for example in the <a href="https://github.com/pytorch/examples/blob/master/imagenet/main.py#L122">ImageNet training</a>.</p>
<p>The scale might seem a little odd, but have a look at the Inception paper <a href="https://arxiv.org/pdf/1409.4842.pdf">Going deeper with convolutions</a>, section 6:</p>
<blockquote>
<p>Still, one prescription that was verified to work very well after the competition includes sampling of various sized patches of the image whose size is distributed evenly between 8% and 100% of the image area and whose aspect ratio is chosen randomly between 3/4 and 4/3.</p>
</blockquote>
<p>So they used indeed this scaling to train their InceptionNet.</p>

Amlan107: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> !<br>
I have been reading this InceptionV1 paper, I understand how they need RandomResizedCrop function, but I didn’t know about the original size of the image, from which they sample the crop and resize to 224x224. Is it 256xN or Nx256 ( 256 is the smallest scaled side )?</p>

ptrblck: <p>If you re using <code>transforms.Resize</code> and pass an <code>int</code> value to the <code>size</code> argument, the smaller edge will be matched to this number.<br>
From the <a href="https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Resize">docs</a>:</p>
<blockquote>
<p><strong>size</strong>  ( <em>sequence</em>  <em>or</em>  <a href="https://docs.python.org/3/library/functions.html#int"> <em>int</em> </a>) – Desired output size. If size is a sequence like (h, w), output size will be matched to this. If size is an int, smaller edge of the image will be matched to this number. i.e, if height &gt; width, then image will be rescaled to (size * height / width, size)</p>
</blockquote>
<p>If I understand your question correctly, this approach would use both possible resizing strategies depending which edge is smaller.</p>
<p>Let me know, if I misunderstood the question.</p>

Amlan107: <p>Thanks a lot, I got it now.</p>

aswamy: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Regarding various patches, what is the center of the random patches? Is the center also random?</p>
<p>And aspect ratio range would be used while computing the patches, right? not for the final image size. Because the final image’s aspect ratio is inferred from the size parameter mentioned?</p>

ptrblck: <aside class="quote no-group" data-username="aswamy" data-post="6" data-topic="16716">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/d2c977/48.png" class="avatar"> aswamy:</div>
<blockquote>
<p>Regarding various patches, what is the center of the random patches? Is the center also random?</p>
</blockquote>
</aside>
<p>Yes, the top left pixel location as well as the height and width will be randomly sampled as seen <a href="https://github.com/pytorch/vision/blob/a884cb7bcc03e57a969c1a572b7af85a95b4421c/torchvision/transforms/transforms.py#L792-L798">here</a>.</p>
<p>The <code>aspect_ratio</code> is used to create the <code>w</code> and <code>h</code> values and there is a fallback to a central crop <a href="https://github.com/pytorch/vision/blob/a884cb7bcc03e57a969c1a572b7af85a95b4421c/torchvision/transforms/transforms.py#L800-L810">here</a>.</p>

aswamy: <p>Thanks for confirming <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>