Cropping a minibatch of images, each image a bit differently

jeff: <p>I have a tensor named <code>input</code> with dimensions 64x21x21. It is a minibatch of 64 images, each 21x21 pixels. I’d like to crop each image down to 11x11 pixels. So the output tensor I want would have dimensions 64x11x11.</p>
<p>I’d like to crop each image around a different “center pixel.” The center pixels are given by a 2-dimensional long tensor named <code>center</code> with dimensions 64x2. For image i, <code>center[i][0]</code> gives the row index and <code>center[i][1]</code> gives the column index for the pixel that should be at the center in the output. We can assume that the center pixel is always at least 5 pixels away from the border.</p>
<p>Is there an efficient way to do this in pytorch (on the gpu)?</p>
<p>[NB: I posted this <a href="https://stackoverflow.com/questions/48235916/cropping-a-minibatch-of-images-in-pytorch-each-image-differently" rel="nofollow noopener">on stackexchange earlier</a>, but I suspect I’m more likely to get a answer here.]</p>

smth: <p>it’s a bit more advanced usage, but you can do this efficiently on the GPU using the <code>grid_sample</code> method. It implements warping, given a 2D flow-field.</p>
<p><a href="http://pytorch.org/docs/0.3.0/nn.html?highlight=grid_sample#torch.nn.functional.grid_sample" class="onebox" target="_blank">http://pytorch.org/docs/0.3.0/nn.html?highlight=grid_sample#torch.nn.functional.grid_sample</a></p>

jeff: <p>Fantastic! I’d almost given up on finding an efficient way to do it. Thanks a lot!</p>

alwynmathew: <p>What do you mean by <code>flow-field</code>?</p>
<p><code>&gt;&gt;&gt; output = F.grid_sample(input, flow-field)</code></p>
<p>According to doc:</p>
<blockquote>
<p>input (Variable) – input batch of images (N x C x IH x IW)<br>
grid (Variable) – flow-field of size (N x OH x OW x 2)</p>
</blockquote>
<p>Why does <code>flow-field</code> have size (N x OH x OW x 2)?<br>
Does the <code>output</code> of <a href="http://pytorch.org/docs/0.3.0/nn.html?highlight=grid_sample#torch.nn.functional.grid_sample" rel="noopener nofollow ugc">grid_sample</a> have same size as <code>input</code>?</p>

kulikovv: <p>Hope that could help</p>
<pre><code class="lang-auto">def build_grid(source_size,target_size):
    k = float(target_size)/float(source_size)
    direct = torch.linspace(0,k,target_size).unsqueeze(0).repeat(target_size,1).unsqueeze(-1)
    full = torch.cat([direct,direct.transpose(1,0)],dim=2).unsqueeze(0)
    return full.cuda()

def random_crop_grid(x,grid):
    delta = x.size(2)-grid.size(1)
    grid = grid.repeat(x.size(0),1,1,1).cuda()
    #Add random shifts by x
    grid[:,:,:,0] = grid[:,:,:,0]+ torch.FloatTensor(x.size(0)).cuda().random_(0, delta).unsqueeze(-1).unsqueeze(-1).expand(-1, grid.size(1), grid.size(2)) /x.size(2)
    #Add random shifts by y
    grid[:,:,:,1] = grid[:,:,:,1]+ torch.FloatTensor(x.size(0)).cuda().random_(0, delta).unsqueeze(-1).unsqueeze(-1).expand(-1, grid.size(1), grid.size(2)) /x.size(2)
    return grid
    
#We want to crop a 80x80 image randomly for our batch
#Building central crop of 80 pixel size 
grid_source = build_grid(batch.size(2),80)
#Make radom shift for each batch
grid_shifted = random_crop_grid(batch,grid_source)
#Sample using grid sample
sampled_batch = F.grid_sample(batch, grid_shifted)
</code></pre>

tsauri: <p>direct = torch.linspace(0,k,target_size).unsqueeze(0).repeat(target_size,1).unsqueeze(-1)<br>
is wrong, I got zoomed in image.</p>
<p>range should be in range [-1,1]<br>
the correct way<br>
direct = torch.linspace(<strong>-k</strong>,k,target_size).unsqueeze(0).repeat(target_size,1).unsqueeze(-1)</p>