Need serious Help on the bases of below explanation Please read once

Aliktk: <p>Hello respected all <code>forum community</code> I am glad to ask question here in such a responsive environment <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"><br>
I am working on <strong>CNN based detection and classification of Alzheimer disease using MRI and Clinical data</strong><br>
For that I am using <code>ADNI</code> dataset where all the nifty MRI scans are not same shape. But the size of maximum number of files are (256,256,166).</p>
<p>I collected data from ADNI website and I cleaned it and splitted into testing and training folder manually with their respective classes for training my modal in future.<br>
The data size is almost <strong>17 GB</strong></p>
<p>Now my task is to load and visualize all those <code>.nii</code> files from directory to the memory and convert them into <strong>tensor torch</strong>  or <strong>numpy array</strong>. Both will be applicable in my case<br>
I am following data loading topics and already asked two question for data loading to my memory.</p>
<p>I tried a lot my all tries find for exploring my problem here in Github <a href="https://github.com/Aliktk/Code-for-AD-Dataloading-" rel="noopener nofollow ugc">My Github Files</a>. It contains what I did in last two months but I still didn’t get the right direction.<br>
I think here in this <code>forum</code> someone worked on 3D datasets like <code>ADNI</code>, <code>OSIS</code> etc. need help for solving dataloading problem and many question about dataloading.</p>
<p><strong>Question are:</strong></p>
<p>1-</p>
<blockquote>
<p>Loading 3D or 4D data in pytorch in not a big deal as they have strong libraries for computing this data. Beside dataloading tutorials anyone have such kinds of methods or code which load 3D data without any preprocessing and augmentation or subsampling??</p>
</blockquote>
<p>1-</p>
<blockquote>
<p>Is it important to convert each MRI file into patches for feed into my memory and further processing?</p>
</blockquote>
<p>2-</p>
<blockquote>
<p>What is the best to subsample my MRI files?</p>
</blockquote>
<p>3-</p>
<blockquote>
<p>Loading MRI files need preprocessing?</p>
</blockquote>
<p>4-</p>
<blockquote>
<p>Does slicing make my work easy?</p>
</blockquote>
<p>My data dimension are 256, 256, 166 and it may varying for some images and the these files are in <code>nii</code> file format.</p>
<p><strong>Note:</strong><br>
<a class="mention" href="/u/nabsabs">@nabsabs</a> share with that for loading such data I simply do <strong>typically do is create a list of direct paths to the patient directories and then in the get_item method, I will index this list and directly read and convert the nifti to a torch Tensor</strong>.</p>
<p>but it didn’t work for me or i don’t know the code or something else but I failed to load the data in such way.<img src="https://discuss.pytorch.org/images/emoji/apple/disappointed.png?v=12" title=":disappointed:" class="emoji" alt=":disappointed:" loading="lazy" width="20" height="20">. <strong>So I am requesting with much respect that please help me someone in step by step way please</strong></p>
<p>I have many question but the problem is that here in my institute no one sharing and explaining these things. everyone says that you have to solve by your own knowledge and google any problem. But I couldn’t find way to figure out my problem</p>
<p><strong>Experts and my field related please help me</strong> <img src="https://discuss.pytorch.org/images/emoji/apple/pray.png?v=12" title=":pray:" class="emoji" alt=":pray:" loading="lazy" width="20" height="20"></p>

nabsabs: <p>I remember how hard it was when I started out in ML/PyTorch/Keras so here is some starter code. That being said, there is a wealth of information, both on this forum and online - I suggest you break up your questions into smaller parts to solve first by yourself, and then post your errors on the forum. It is generally not good practice to post your entire problem for the community to solve without efforts on your part (unless its an open research question).</p>
<p>Any pytorch dataset can be custom made following this boiler plate code below. You need to create:</p>
<ol>
<li>An <code>__init__</code> method that will house all the variables you need to call in future methods (like maybe a list of patients as below).</li>
<li>A <code>__len__</code> method that will output the total size of your dataset, in this case its the number of patients you have in the folder</li>
<li>And finally, the heart of the dataset, the <code>__getitem__</code> method which the dataloader will call on everytime with an index which is a number from 0 to size_of_dataset (ie <code>len(self.patients)</code>).</li>
</ol>
<ul>
<li>This method takes the idx-th index from the list of <code>self.patients</code>
</li>
<li>
<code>path_to_pat</code> is a direct path to the patient you want to load in this iteration</li>
<li>Using <code>nib</code>, you can load the patient’s full volume into the <code>mri</code> variable.</li>
<li>At this point, everything is numpy and you can do normalization as you please and at the end, return the <code>mri</code> volume as one datasample.</li>
<li>Now, if your batch size is greater than 1, you have to make sure all your patient volumes are the same size in all dimensions otherwise you will not be able to batch them. (You can do this by chunking or resampling, your design choice)</li>
</ul>
<pre><code class="lang-auto">import os
import nibabel as nib
from torch.utils.data import Dataset, DataLoader

class ADNI(Dataset):

    def __init__(self, path_to_adni_files):      
        self.data_path = path_to_adni_files
        self.patients = os.listdir(self.data_path)
            
    def __len__(self):
        return len(self.patients)
            
    def __getitem__(self, idx):
        path_to_pat = os.path.join(self.data_path, self.patients[idx])
        mri = nib.load(path_to_pat).get_data()
        mri = (mri - mri.min())/(mri.max() - mri.min())
        mri = mri - mri.mean()       
        return mri

# example use
path_to_adni_files = "define_this_path"
dataset = ADNI(path_to_adni_files)
dataloader =  DataLoader(dataset, batch_size=1, shuffle=True)
one_batch_of_pats = next(iter(dataloader))
print(one_batch_of_pats.shape)     

</code></pre>

Nickolas_Luo: <p>Thanks a lot.I’m also working as a beginner to deal with some MRI files through deep learning pytorch.particularly,I am trying to bulid a 3D Convolutional neural network to extract features and predict the gender labels.</p>

nabsabs: <p>Good to hear! In the <code>__getitem__</code> method you should also load the label, as in your case, the gender label and return it as the output. You can load it to similar to how you load the patient data.</p>

Nickolas_Luo: <p>Thank you very much bro ! For me, my goal is to build a gender classification model from  some existing MRI files to predict their gender labels. It’s just the first step to input the whole MRI files and their gender labels into pytorch as tensor. I don’t know how to build my classification model next, especially, I want to build a 3D convolution model:hot_face:</p>