First non zero value in an image dim

username: bluesky314
cooked: <p>I have a stack of MRI images that look like <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/1/1b4ca3a052e932937e93bacd3c6eedf1f6a24cad.png" data-download-href="https://discuss.pytorch.org/uploads/default/1b4ca3a052e932937e93bacd3c6eedf1f6a24cad" title="10%20AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/1/1b4ca3a052e932937e93bacd3c6eedf1f6a24cad_2_528x500.png" alt="10%20AM" data-base62-sha1="3Tv5BknFiObahb4PDPKvsH0HaQB" width="528" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/1/1b4ca3a052e932937e93bacd3c6eedf1f6a24cad_2_528x500.png, https://discuss.pytorch.org/uploads/default/original/2X/1/1b4ca3a052e932937e93bacd3c6eedf1f6a24cad.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/1/1b4ca3a052e932937e93bacd3c6eedf1f6a24cad.png 2x" data-dominant-color="6B5B83"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">10%20AM</span><span class="informations">660×624 50 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>say in shape [train_size, h,w,d]</p>
<p>There is alot of black space making my convolution very heavy so I want to trim the images from the minimum index of the first non-zero pixel value from all the 4 corners. i.e identify the maximum distance I can trim my images from the four corners without losing any non-zero values pixel from any of the images in whole training size. This is only for the height and width. How Could I go about doing this?</p>

username: ptrblck
cooked: <p>This code should work for a <code>[batch, channel, height, width]</code>-shaped tensor:</p>
<pre><code class="lang-python">x = torch.zeros(1, 3, 24, 24)
x[0, :, 5:10, 6:11] = torch.randn(1, 3, 5, 5)

non_zero = (x!=0).nonzero()  # Contains non-zero indices for all 4 dims
h_min = non_zero[:, 2].min()
h_max = non_zero[:, 2].max()
w_min = non_zero[:, 3].min()
w_max = non_zero[:, 3].max()
</code></pre>

username: jphoward
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> this solution gives the min and max nonzero cell across <em>all</em> images in the batch, which is unlikely to be the desired behavior.</p>
<p>I’m currently trying to solve the same problem, and so far haven’t found anything that can be done without a loop.</p>

username: ptrblck
cooked: <p>That’s right and I think it would be the only way to keep the batch intact, wouldn’t it?<br>
If you crop each image separately, you could end up with variable spatial sizes, which would disallow creating a batch afterwards.</p>
<p>Let me know, if I’m missing something.</p>

username: jphoward
cooked: <p>Yup <a class="mention" href="/u/ptrblck">@ptrblck</a> you are missing something <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
<p>I’ve figured out how to do this now so I can show you! Here’s how I did the batchwise nonzero thing:</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/fastai/fastai_dev/blob/master/dev/local/medical/imaging.py#L180" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/fastai/fastai_dev/blob/master/dev/local/medical/imaging.py#L180" target="_blank" rel="nofollow noopener">fastai/fastai_dev/blob/master/dev/local/medical/imaging.py#L180</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="170" style="counter-reset: li-counter 169 ;">
<li>    c = x.sum(dim).nonzero().cpu()</li>
<li>    idxs,vals = torch.unique(c[:,0],return_counts=True)</li>
<li>    vs = torch.split_with_sizes(c[:,1],tuple(vals))</li>
<li>    d = {k.item():v for k,v in zip(idxs,vs)}</li>
<li>    default_u = tensor([0,x.shape[-1]-1])</li>
<li>    b = [d.get(o,default_u) for o in range(x.shape[0])]</li>
<li>    b = [tensor([o.min(),o.max()]) for o in b]</li>
<li>    return torch.stack(b)</li>
<li>
</li>
<li>#Cell</li>
<li class="selected">def mask2bbox(mask):</li>
<li>    no_batch = mask.dim()==2</li>
<li>    if no_batch: mask = mask[None]</li>
<li>    bb1 = _px_bounds(mask,-1).t()</li>
<li>    bb2 = _px_bounds(mask,-2).t()</li>
<li>    res = torch.stack([bb1,bb2],dim=1).to(mask.device)</li>
<li>    return res[...,0] if no_batch else res</li>
<li>
</li>
<li>#Cell</li>
<li>def _bbs2sizes(crops, init_sz, use_square=True):</li>
<li>    bb = crops.flip(1)</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>And here’s how I then use that to create cropped zoomed sections from a batch:</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/fastai/fastai_dev/blob/master/dev/local/medical/imaging.py#L199" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/fastai/fastai_dev/blob/master/dev/local/medical/imaging.py#L199" target="_blank" rel="nofollow noopener">fastai/fastai_dev/blob/master/dev/local/medical/imaging.py#L199</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="189" style="counter-reset: li-counter 188 ;">
<li>def _bbs2sizes(crops, init_sz, use_square=True):</li>
<li>    bb = crops.flip(1)</li>
<li>    szs = (bb[1]-bb[0])</li>
<li>    if use_square: szs = szs.max(0)[0][None].repeat((2,1))</li>
<li>    overs = (szs+bb[0])&gt;init_sz</li>
<li>    bb[0][overs] = init_sz-szs[overs]</li>
<li>    lows = (bb[0]/float(init_sz))</li>
<li>    return lows,szs/float(init_sz)</li>
<li>
</li>
<li>#Cell</li>
<li class="selected">def crop_resize(x, crops, new_sz):</li>
<li>    # NB assumes square inputs. Not tested for non-square anythings!</li>
<li>    bs = x.shape[0]</li>
<li>    lows,szs = _bbs2sizes(crops, x.shape[-1])</li>
<li>    if not isinstance(new_sz,(list,tuple)): new_sz = (new_sz,new_sz)</li>
<li>    id_mat = tensor([[1.,0,0],[0,1,0]])[None].repeat((bs,1,1)).to(x.device)</li>
<li>    with warnings.catch_warnings():</li>
<li>        warnings.filterwarnings('ignore', category=UserWarning)</li>
<li>        sp = F.affine_grid(id_mat, (bs,1,*new_sz))+1.</li>
<li>        grid = sp*unsqueeze(szs.t(),1,n=2)+unsqueeze(lows.t()*2.,1,n=2)</li>
<li>        return F.grid_sample(x.unsqueeze(1), grid-1)</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>As you see, the trick is to use grid_sample to resize them all to the same size after the crop. This is a nice way to do things like RandomResizeCrop on the GPU.</p>

username: ptrblck
cooked: <p>Thanks for sharing! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
Did you know what speedup to expect, as I assume it should be way faster than the loop.</p>

username: jphoward
cooked: <p>Yeah my preprocessing overall is 150x faster on CUDA, and I think that’s nearly all in these two methods.</p>

username: tshadley
cooked: <p>Updated github link to this solution:<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/fastai/fastai2/blob/f9231256e2a8372949123bda36e44cb0e1493aa2/fastai2/medical/imaging.py#L198" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/fastai/fastai2/blob/f9231256e2a8372949123bda36e44cb0e1493aa2/fastai2/medical/imaging.py#L198" target="_blank" rel="noopener nofollow ugc">fastai/fastai2/blob/f9231256e2a8372949123bda36e44cb0e1493aa2/fastai2/medical/imaging.py#L198</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="188" style="counter-reset: li-counter 187 ;">
<li>    p = x.windowed(*window)</li>
<li>    if remove_max: p[p==1] = 0</li>
<li>    return gauss_blur2d(p, s=sigma*x.shape[-1])&gt;thresh</li>
<li>
</li><li># Cell</li>
<li>@patch</li>
<li>def mask_from_blur(x:DcmDataset, window, sigma=0.3, thresh=0.05, remove_max=True):</li>
<li>    return to_device(x.scaled_px).mask_from_blur(window, sigma, thresh, remove_max=remove_max)</li>
<li>
</li><li># Cell</li>
<li class="selected">def _px_bounds(x, dim):</li>
<li>    c = x.sum(dim).nonzero().cpu()</li>
<li>    idxs,vals = torch.unique(c[:,0],return_counts=True)</li>
<li>    vs = torch.split_with_sizes(c[:,1],tuple(vals))</li>
<li>    d = {k.item():v for k,v in zip(idxs,vs)}</li>
<li>    default_u = tensor([0,x.shape[-1]-1])</li>
<li>    b = [d.get(o,default_u) for o in range(x.shape[0])]</li>
<li>    b = [tensor([o.min(),o.max()]) for o in b]</li>
<li>    return torch.stack(b)</li>
<li>
</li><li># Cell</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>