Extracting patches using sliding window?

username: farah_jabeen
cooked: <pre><code class="lang-auto">Hi 

Does the following code describe that data stored in numpy_array in. pkl format the author of the code try to extract patches using the sliding window function, I have following queries if someone can answer?
1- As this data is numpy_array and extracted patches are in numpy_array as well I want to ask is that possible to create images from it?
2- The data is of low dose CT images so I think the author extracted 3D patches from it?
3-To recreate it will be a 3D image (axial/Sagittal/Coronal)?
4-Size of extracted patches are 64*64?


Code:

import os

import joblib

import cv2

import numpy as np

def window_leveling(x, c=50, w=400, ymin=0, ymax=255):

    output = x.copy()

    wmin = (c - 0.5) - ((w - 1) / 2)

    wmax = (c - 0.5) + ((w - 1) / 2)

    output = ((wmin &lt; x) &amp; (x &lt;= wmax)) * ((x - (c - 0.5)) / (w - 1) + 0.5) * (ymax - ymin) + ymin

    output[x &lt;= wmin] = ymin

    output[x &gt; wmax] = ymax

    return output.astype(np.uint8)

def main():

    path = 'cross_validation path'

    out_path = 'saved patches'

    emr_feature_number = [1, 6, 9, 11, 14, 15, 16]

    ct_slice_num = 80

    default_value = 0

    for i in range(5):

        ct_data = joblib.load(f'{path}/cross_validation_ct_threshold_{i+1}.pkl')

        voxel_list = joblib.load(f'{path}/cross_validation_voxel_list_postprocess_{i+1}.pkl')

        emr_data = joblib.load(f'{path}/emr_data_{i+1}.pkl')

       

        sub_path = f'{out_path}/fold_{i+1}/'

        os.makedirs(f'{sub_path}')

       

        for voxel_info in voxel_list:

            image_index, z, y, x, answer = voxel_info

            emr = np.array(emr_data[image_index])[emr_feature_number]

            ct = window_leveling(ct_data[image_index])

            start_index = (ct_slice_num - ct.shape[0]) //2

            np_extended = np.full((ct_slice_num, 512, 512), default_value, dtype=np.uint8)

            np_extended[start_index:start_index + ct.shape[0], :, :] = ct

            np_padding = np.pad(np_extended, ((10, 10), (65, 65), (65, 65)), 'constant', constant_values=default_value)

           

            ct_z = z + 10

            ct_y = y + 65

            ct_x = x + 65

           

            axial =  np_padding[ct_z, ct_y - 32:ct_y + 33, ct_x - 32:ct_x + 33]

            sagittal = np_padding[ct_z - 10:ct_z + 11, ct_y - 32:ct_y + 33, ct_x]

            coronal = np_padding[ct_z - 10:ct_z + 11, ct_y, ct_x - 32:ct_x + 33]

           

            data = [axial, sagittal, coronal, emr, int(answer)]

            save_path = f'{sub_path}/fold_{i+1}_{image_index:02d}_{answer}_{z:02d}_{y:03d}_{x:03d}.pkl'

            joblib.dump(data, save_path)

if __name__ == '__main__':

    main()
</code></pre>

username: Matias_Vasquez
cooked: <h2><a name="h-1-is-it-possible-to-create-images-from-it-1" class="anchor" href="#h-1-is-it-possible-to-create-images-from-it-1"></a>1. Is it possible to create images from it?</h2>
<p>Yes, you can use the library <code>matplotlib.pyplot</code> to plot numpy arrays. If you have 2d images you can use something like <code>imshow</code>.</p>
<p><a href="https://matplotlib.org/3.5.0/tutorials/introductory/images.html#sphx-glr-tutorials-introductory-images-py" class="onebox" target="_blank" rel="noopener nofollow ugc">https://matplotlib.org/3.5.0/tutorials/introductory/images.html#sphx-glr-tutorials-introductory-images-py</a></p>
<p>If you have 3d data, there are some ways to visualize it, but if the data is dense, it will be difficult to understand what is going on.</p>
<p><a href="https://matplotlib.org/2.0.2/mpl_toolkits/mplot3d/tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://matplotlib.org/2.0.2/mpl_toolkits/mplot3d/tutorial.html</a></p>
<hr>
<h2><a name="h-2-did-the-author-extract-3d-patches-from-it-2" class="anchor" href="#h-2-did-the-author-extract-3d-patches-from-it-2"></a>2. Did the author extract 3d patches from it?</h2>
<p>I believe the author is extrading three 2d patches. If you see how he extracts the slices there is always one dimension that does not “move”. For the <code>axial</code> dim <code>ct_z</code> remains constant, for <code>sagittal</code> is <code>ct_x</code> and for <code>coronal</code> is <code>ct_y</code>.</p>
<p>What this means is that for a given <code>voxel</code> you have three 2d images, one for each plane with the desired <code>voxel</code> in the middle.</p>
<aside class="quote no-group" data-username="farah_jabeen" data-post="1" data-topic="151142">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/farah_jabeen/48/46119_2.png" class="avatar"> farah_jabeen:</div>
<blockquote>
<pre><code class="lang-auto">axial =  np_padding[ct_z, ct_y - 32:ct_y + 33, ct_x - 32:ct_x + 33]
sagittal = np_padding[ct_z - 10:ct_z + 11, ct_y - 32:ct_y + 33, ct_x]
coronal = np_padding[ct_z - 10:ct_z + 11, ct_y, ct_x - 32:ct_x + 33]
</code></pre>
</blockquote>
</aside>
<hr>
<h2><a name="h-3-will-it-be-a-3d-image-3" class="anchor" href="#h-3-will-it-be-a-3d-image-3"></a>3. Will it be a 3D image?</h2>
<p>As we saw in the last question, it actually return three 2D images. If you wanted to get a 3D image you could do something like this.</p>
<pre data-code-wrap="python"><code class="lang-python">scan_3d = np_padding[ct_z - 10:ct_z + 11, ct_y - 32:ct_y + 33, ct_x - 32:ct_x + 33]
</code></pre>
<p>However, as we saw in the first answer, it might be difficult to plot this 3D data in an understandable way.</p>
<p>Maybe one of these tutorials might helps you plotting the 3d data.</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://www.datacamp.com/tutorial/matplotlib-3d-volumetric-data">
  <header class="source">
      <img src="https://www.datacamp.com/marketing-backgrounds/favicons/favicon-196x196.png" class="site-icon" width="" height="">

      <a href="https://www.datacamp.com/tutorial/matplotlib-3d-volumetric-data" target="_blank" rel="noopener nofollow ugc">datacamp.com</a>
  </header>

  <article class="onebox-body">
    

<h3><a href="https://www.datacamp.com/tutorial/matplotlib-3d-volumetric-data" target="_blank" rel="noopener nofollow ugc">Python Matplotlib 3D Visualization with Volumetric Data</a></h3>

  <p>Learn how to use Matplotlib's event handler API to display the slices of an MRI dataset. Follow our tutorial with code examples today!</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://goodboychan.github.io/python/datacamp/deep_learning/vision/2020/08/13/01-Exploration-in-Biomedical-Image-Analysis.html">
  <header class="source">
      <img src="https://goodboychan.github.io/images/favicon.ico" class="site-icon" width="25" height="35">

      <a href="https://goodboychan.github.io/python/datacamp/deep_learning/vision/2020/08/13/01-Exploration-in-Biomedical-Image-Analysis.html" target="_blank" rel="noopener nofollow ugc" title="12:00AM - 13 August 2020">Chan`s Jupyter – 13 Aug 20</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:600/337;"><img src="https://goodboychan.github.io/images/Ch1_L3_Axial16x9.gif" class="thumbnail" width="600" height="337"></div>

<h3><a href="https://goodboychan.github.io/python/datacamp/deep_learning/vision/2020/08/13/01-Exploration-in-Biomedical-Image-Analysis.html" target="_blank" rel="noopener nofollow ugc">Exploration in Biomedical Image Analysis</a></h3>

  <p>Prepare to conquer the Nth dimension! To begin the course, you’ll learn how to load, build and navigate N-dimensional images using a CT image of the human chest. You’ll also leverage the useful ImageIO package and hone your NumPy and matplotlib...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<hr>
<h2><a name="h-4-size-of-extracted-patches-are-6464-4" class="anchor" href="#h-4-size-of-extracted-patches-are-6464-4"></a>4. Size of extracted patches are 64*64?</h2>
<p>Sizes:</p>
<ul>
<li>Axial = Y: 64, X: 64</li>
<li>Sagittal = Z: 20, Y: 64</li>
<li>Coronal = Z: 20, X: 64</li>
</ul>
<p>Hope this helps <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=12" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>

username: farah_jabeen
cooked: <p>thank you, you explain everything very well <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: farah_jabeen
cooked: <p>Sorry to bother you again but as you can see these are 2D images of axial/sagittal and coronal planes.<br>
I want to implement 3D CNN model for classification and further I want to implement localization techniques as well for automatic calcium scoring.</p>
<p>This is an example of 2D CNN model implement on extract patches</p>
<aside class="quote quote-modified" data-post="1" data-topic="142668">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/farah_jabeen/48/46119_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/runtimeerror-given-input-size-512x1x3-calculated-output-size-512x0x1-output-size-is-too-small/142668">RuntimeError: Given input size: (512x1x3). Calculated output size: (512x0x1). Output size is too small</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    I am working on Cardiac Ct images by implementing the 2DCNN model. I am trying to train vgg model. But every time I got this issue. I know this is related to layers as I am new in PyTorch I am not able to understand. If someone can guide me related to it 


Here is the error and code. My torch.Size([2, 1, 65, 65]) I am using the dense-net model and it has the same torch.Size([2, 1, 65, 65]) and it’s working fine . 


VGG_MODEL 


Traceback (most recent call last):
  File "ct_pretrained.py", lin…
  </blockquote>
</aside>

<p>but I am little confuse and following is my question if you can guide me</p>
<p>How can I change this model into 3D CNN. As this is implemented on all X/y/z planes as I follow some articles but there is no example of all planes they just implement it on Axial images?</p>
<p>Thank you in advance</p>

username: Matias_Vasquez
cooked: <p>If you want to analyze your data in 3D I would really recommend this book written by <a class="mention" href="/u/tom">@tom</a> et. al.</p>
<p>In the first part they teach the most important aspects of pytorch.</p>
<p>The second part is completely dedicated to lung cancer detection in 3D.</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://www.manning.com/books/deep-learning-with-pytorch#toc">
  <header class="source">

      <a href="https://www.manning.com/books/deep-learning-with-pytorch#toc" target="_blank" rel="noopener nofollow ugc">Manning Publications</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:360/451;"><img src="https://images.manning.com/360/480/resize/book/3/8e5d003-09e3-430e-a5a3-f42ee1cafb5f/Stevens-DLPy-HI.png" class="thumbnail" width="360" height="451"></div>

<h3><a href="https://www.manning.com/books/deep-learning-with-pytorch#toc" target="_blank" rel="noopener nofollow ugc">Deep Learning with PyTorch</a></h3>

  <p>Create neural networks and deep learning systems with PyTorch. Discover best practices for the entire DL pipeline, including the PyTorch Tensor API and loading data in Python.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Since this is a big topic, I really thing this book will help you more to understand what is happening.</p>
<p>You can read it for free in the link that I gave you.</p>