Creating 3D VOLUME from respective 2D slices

aleemsidra: <p>I am working with 3D brain MRI scans (nifti format). I am using 2D slices from 3D volume to train a 2D network. Now, I want to do the evaluation on 3D volume by stacking  the slices of respective subject together and calculate surface dice co-efficeint. How can I stack all the slices of respective subject together to get the 3D volume back?</p>

ege_b: <p>Have you tried <code>torch.concatenate(&lt;list_of_tensors&gt;, axis=0)</code>? And then you can permute the 3D tensor according to how you sliced it.</p>

aleemsidra: <p><a class="mention" href="/u/ege_b">@ege_b</a> , my data consists of 3D brain MRI scans. I am treating the 2D slices independantly and training the 2D model on them. Below is the part of  dataset class showing how actually I am reading the images:</p>
<pre><code class="lang-auto">for i, f in enumerate(files):

            nib_file = nib.load(os.path.join(images_path, f))
            lbl = nib.load(os.path.join(data_path, 'Silver-standard', self.folder, f[:-7] + '_ss.nii.gz')).get_fdata(
                'unchanged', dtype=np.float32)

            if self.scale:
                transformed = scaler.fit_transform(np.reshape(img, (-1, 1)))
                img = np.reshape(transformed, img.shape)

            if not self.sagittal:
                img = np.moveaxis(img, -1, 0)
          
            if self.rotate:
                img = np.rot90(img, axes=(1, 2))

            if img.shape[1] != img.shape[2]:
  
                img = self.pad_image(img)

            images.append(img)

            if not self.sagittal:
                lbl = np.moveaxis(lbl, -1, 0)
            if self.rotate:
                lbl = np.rot90(lbl, axes=(1, 2))
            if lbl.shape[1] != lbl.shape[2]:
                lbl = self.pad_image(lbl)
            labels.append(lbl)

            spacing = [nib_file.header.get_zooms()] * img.shape[0]
            self.voxel_dim.append(np.array(spacing))  


        images, labels = self.unify_sizes(images, labels)
        self.data = np.expand_dims(np.vstack(images), axis=1)
        self.label = np.expand_dims(np.vstack(labels), axis=1)
        self.voxel_dim = np.vstack(self.voxel_dim)

def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        data = self.data[idx]
        labels = self.label[idx]
        voxel_dim = self.voxel_dim[idx]
        # asd
        return data, labels, voxel_dim


</code></pre>
<p>Now, when I pass the dataset into dataloader, the images are independant 2D slices, the information that which 2D image belonged to which slice is not there anymore.  And are randomly passed in the loader depending on the batch size as follows:</p>
<pre><code class="lang-auto">train_loader = DataLoader(train_data, batch_size=config.batch_size,
                              shuffle=True, num_workers=10, drop_last=True)
</code></pre>
<p>Now for evaluation, I want to stack all the 2D images to get back the respective 3D volume.  I am not sure of how they were permuted before passing to <code>DataLoader</code> . I donot have the slice information too. Can you please help me with this.</p>

ege_b: <p>I mean it seems that you are not sure how to get 2D images in the correct order again, for this you can just create a new data loader with <code>eval_loader = DataLoader(t&lt;your_other_args&gt;, shuffle=False, drop_last=False)</code>, which forces the data loader to yield the 2D images in the same order as you saved them to <code>self.data</code>.  An other option would be, if you want to reconstruct these during training, which is obviously not possible unless you have every single 2D slice of the 3D image, then you can also force <code>__getitem__</code> to return the <code>idx</code> so you do know which order of the slice you get. And then you should apply the inverse of every transformation you apply to your 2D slices, put them in the right order depending on their indices, and concatenate them to a torch tensor using <code>torch.concatenate(&lt;list_of_vectors&gt;, axis=0&gt;)</code>. But the above code snippet does not have any information to how you actually get 2D slices from your 3D volume.</p>

aleemsidra: <p><a class="mention" href="/u/ege_b">@ege_b</a> , yes exactly I donot know how to put back the 2D slices together to get 3D volume back. With your suggestion:</p>
<p><code>eval_loader = DataLoader(t&lt;your_other_args&gt;, shuffle=False, drop_last=False)</code></p>
<p>If I do this, the size of one 3D is <code>200,256,256</code>. Now as I have stored them as slices is <code>self.data</code>. The above line of code is not goint to help me to take all the slices of of an image. Since the batch_size is different than the number of slices in the 3D volume.</p>
<p>Moreover, I want to do the same procedure (stacking 2D to get 3D volume) for training set as well. As the same evaluation metric is used for both of train and val set.</p>
<p>I modified the validation code as below. In which I am actually laoding the 3D volume locally, doing prediction on each slice and then concatenating them together to the original 3D volume.</p>
<pre><code class="lang-auto"># Train the model on 2D slices
for epoch in range(num_epochs):
    for inputs, targets in slice_dataset:
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = loss_fn(outputs, targets)
        loss.backward()
        optimizer.step()

# Evaluate on full 3D volume
model.eval()
with torch.no_grad():
    # Get 2D slices from the 3D volume
    slices = []
    for z in range(volume.shape[0]):
        slice = volume[z, :, :]
        slices.append(slice)

    # Stack the 2D slices to reconstruct the 3D volume
    stacked = np.stack(slices, axis=0)

    # Run inference on the stacked 2D slices
    inputs = torch.from_numpy(stacked).unsqueeze(1)
    outputs = model(inputs)

    # Convert the output back to a 3D volume
    predicted_volume = np.squeeze(outputs.numpy(), axis=1)
</code></pre>
<p>However, for training part , I am not sure how to do it. Since I want to pass the slices randomly per batch, which obviously will not have all the slices from one 3D volume.</p>

ege_b: <p>To achieve this, you should use a DataLoader with <code>drop_last=False</code> and should return the index of the slice with the data itself in <code>__getitem__</code>, i.e. <code>return data, idx, labels, voxel_dim</code> or however you want and unpack accordingly. On every iteration of the loop unpack with <code>for &lt;batch_nr&gt;,  (&lt;data&gt;, &lt;slice_idx&gt;, &lt;labels&gt;, &lt;voxel_dim&gt;) in enumerate(train_loader):</code> and store both <code>data</code> and <code>&lt;slice_idx&gt;</code> (ofcourse name this however you want) in a list. <strong>AFTER THE LOOP</strong> , flatten the list of <code>&lt;slice_idx&gt;</code> (I’m not sure of the format that will be returned here, you might want to check that with a debugger or a print statement) and achieve a list of slice indices <code>[143, 156, 2, 42, ...]</code>. Then concatenate your list of batches with <code>tens = torch.concatenate(&lt;list_of_batches&gt;, axis=0)</code>. Sort your indices and get their real orders with: <code>&lt;order_np_array&gt; = np.array(&lt;list_of_indices&gt;).argsort()</code>. Lastly, reorder them using <code>&lt;3d_image&gt; = tens[&lt;order_np_array&gt;]</code>. This means you can <strong>ONLY</strong> reconstruct your 3D image <strong>AFTER</strong> every epoch.</p>

aleemsidra: <p><a class="mention" href="/u/ege_b">@ege_b</a> I understood your suggestion in general. But from implementation perspective.<br>
My list of slice indices would not be integers<code>[143, 156, 2, 42, ...]</code> The slice id is not a unique identifier. For instance, one image has 200 slices and the second image has 100 slices. Now  in <code>__getitem__</code> ,I am storing the slice id as <code>image_name + idx + slice_id</code> (e.g. CC0304_ge_3_57_F.nii.gz_id_397). If I sort the slice id based on <code>image_name</code> and slice_id. How would I retirve the corresponding<code> prediction, ground truth, voxel_dim</code>. Since the list of slices was not unique for every image.</p>
<p>Also I donot understand this line:</p>
<p>Then concatenate your list of batches with <code>tens = torch.concatenate(&lt;list_of_batches&gt;, axis=0)</code></p>
<p>Would not this concatenate all the tensors in one single tensor. But this is not the objective here.</p>

ege_b: <aside class="quote no-group" data-username="aleemsidra" data-post="7" data-topic="175422">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aleemsidra/48/38723_2.png" class="avatar"> aleemsidra:</div>
<blockquote>
<p>Would not this concatenate all the tensors in one single tensor. But this is not the objective here.</p>
</blockquote>
</aside>
<p>I’m sorry, I did not quite understand what the objective is then. But when you are slicing the 3D images, you should have some unique identifier of the position of that 2D slice. The idea is to pass this through in <code>__getitem__</code> and use this to reconstruct your image.</p>
<p>I thought you were reconstructing the 3D volume by stacking the 2D slices in the correct order, as if the 3D volume was a cube and 2D slices were the squres corresponding to every height level so to say. But without knowledge of what the actual 3D image looks like, I don’t think I can help you no more <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> If you have any questions for any specific operation though, I might be of further help. Good luck!</p>

Kavya: <p>Could You please share us the dataset which you are using.We are also building project on a similar idea.please, we need the dataset .Kindly,share us the dataset.Thank you.<br>
Email ID: <a href="mailto:dugyanikavya123@gmail.com">dugyanikavya123@gmail.com</a></p>