Torch Dataloader gives torch.Tensor as ouput

din0328: <p>I am coding Dataloader for my own data. I return output as numpy but dataloader gives me torch.Tensor as the output. Don’t understand why.</p>
<pre><code class="lang-auto">from torch.utils import data
import torch
import nibabel as nib

class getdata(data.Dataset):
    '''
    Initializes a dataset for the network
    Assumes that the data_dir has files named MRimages and CTimages that contain all the images 
    for all the patients in .hdr format.
    '''
    def __init__(self,data_dir,transform):
        'Initialization'
        self.data_dir = data_dir
        self.transform = transform
        self.list_IDs = np.arange(nib.load(self.data_dir+ '/MRimages.img').shape[2]) #list of all patients.
        
    def __len__(self):
        'Total no. of samples. Make sure that number of MR and CT samples are same.'
        num = len(self.list_IDs)  #total number of slices.
        return num
        
    def __getitem__(self,index): #index is patient ID
        'Generate one sample of data'
        ID = self.list_IDs[index]
        MR = np.asarray(nib.load(self.data_dir+ '/MRimages.img').get_data()[:,:,ID])
        CT = np.asarray(nib.load(self.data_dir+ '/CTimages.img').get_data()[:,:,ID])

        sample = {'MR': MR,'CT': CT}
        if self.transform:
            sample = self.transform(sample)

        return sample
</code></pre>
<p>Calling function:</p>
<pre><code class="lang-auto">data_dir = '...'#my data directory
a = np.arange(nib.load(data_dir+ '/MRimages.img').shape[2])
params = {'batch_size': 64,
          'shuffle': True,
          'num_workers': 2}
train_set = getdata(data_dir,transform=None)
train_gen = data.DataLoader(train_set,**params)
for s in train_gen:
    print(type(s['MR']))
</code></pre>
<p>This gives me &lt;class ‘torch.Tensor’&gt; for every batch.<br>
I want to make the self.tranform as class that works on numpy matrix and not torch matrix.<br>
Any suggestions?</p>

ptrblck: <p>The reason for your <code>DataLoader</code> returning <code>torch.tensors</code> even though are are returning numpy arrays is most likely due to the usage of the <a href="https://github.com/pytorch/pytorch/blob/35c8f93fd238d42aaea8fd6e730c3da9e18257cc/torch/utils/data/dataloader.py#L218">default_collate</a> method. You can see in the line of code I’m referring to how numpy arrays are wrapped in <code>torch.tensors</code>.<br>
If you check the type of <code>train_set[0]</code> you should get a numpy array, which means that the transform in <code>__getitem__</code> is actually working on numpy arrays. The <code>DataLoader</code> just makes your life a bit easier as you probably want to use <code>torch.tensors</code> in your training loop.</p>

din0328: <p>That makes sense. So my numpy arrays are contained in a torch batch, so I can do transform on numpy.<br>
Another question. So I am assuming that since my batch size is 64, when <strong>getitem</strong> is called, the list ID should have 64 random indices that are used to get the images.<br>
But when I do <code>MR.transpose((2,1,0))</code>, I get <code>ValueError: axes don't match array</code> error<br>
And then <code>MR.transpose((1,0))</code> works fine. So it seems that it’s getting ID one by one.<br>
I want to do my <code>self.transpose</code> on 3D arrays and not on 2D arrays.<br>
Is there something I am doing wrong? Also my pytorch version is <code>0.4.1.post2</code></p>
<p>Thanks</p>

ptrblck: <p>The <code>__getitem__</code> method uses an index to get a single samples not a batch, i.e. the batch dimension of your data is missing in <code>__getitem__</code>.<br>
Usually this makes developing of a custom <code>Dataset</code> really easy, as you just have to think about how to get a single samples of data. The <code>DataLoader</code> yields a complete batch of samples and provides some additional functionalities like shuffling the dataset or using multiple workers.</p>
<p>It seems in your use case you would like to load a whole bunch of slices of your MRI images.<br>
Could you print the shapes of <code>MR</code> and <code>CT</code> in <code>__getitem__</code> since I’m currently not sure how your indexing works.</p>

din0328: <p>Here it is:<br>
Sample <em>getitem</em>_ with some transpose:</p>
<pre><code class="lang-auto">    def __getitem__(self,index): #index is patient ID
        'Generate one sample of data'
        ID = self.list_IDs[index]
        MR = np.asarray(nib.load(self.data_dir+ '/MRimages.img').get_data()[:,:,ID])
        CT = np.asarray(nib.load(self.data_dir+ '/CTimages.img').get_data()[:,:,ID])
        
        hMR, wMR = MR.shape[:2]
        new_hMR, new_wMR = 200,200

        top = np.random.randint(0, hMR - new_hMR)
        left = np.random.randint(0, wMR - new_wMR)

        MR = MR[top: top + new_hMR,
                      left: left + new_wMR]

        CT = CT[top: top + new_hMR,
                      left: left + new_wMR]
        
        flip = np.random.randint(2)
        if flip:
            MR.transpose((1,0))
            CT.transpose((1,0))

        sample = {'MR': MR,'CT': CT}
        #if self.transform:
            #sample = self.transform(sample)

        return sample
</code></pre>
<p>Give me: <code>torch.Size([64, 200, 200])</code></p>

ptrblck: <p>I assume the shape you’ve printed is the shape of the batch from the <code>DataLoader</code>.<br>
If that’s the case your <code>MR</code> and <code>CT</code> data have the shape <code>[200, 200]</code> in <code>__getitem__</code>.</p>
<p>Based on your description it seems you would like to apply the same transformation on the whole batch instead of each single image.</p>
<p>If that’s the case one approach would be to create an own sampler and provide a list of random indices to your <code>Dataset</code>.<br>
In the <code>__getitem__</code> you would get a list of indices of the length <code>batch_size</code>, could load all images one by one and apply the same transformation on them.<br>
In the training loop you would get an additional batch dimension and can just squeeze it.<br>
Here is a small example:</p>
<pre><code class="lang-python">class MyDataset(Dataset):
    def __init__(self, data):
        self.data = data
        
    def __getitem__(self, index):
        x = torch.stack([self.data[i] for i in index])
        return x
    
    def __len__(self):
        return len(self.data)


class RandomBatchSampler(torch.utils.data.sampler.Sampler):
    def __init__(self, data_source, batch_size):
        self.data_source = data_source
        self.batch_size = batch_size

    def __iter__(self):
        rand_idx = torch.randperm(len(self.data_source)).tolist()
        data_iter = iter([rand_idx[i:i+self.batch_size] for i in range(0, len(rand_idx), self.batch_size)])
        return data_iter

    def __len__(self):
        return len(self.data_source)//self.batch_size


data = torch.randn(100, 3, 24, 24)
dataset = MyDataset(data)

batch_size = 64
sampler = RandomBatchSampler(data, batch_size=batch_size)

loader = DataLoader(
    dataset,
    batch_size=1,
    num_workers=2,
    sampler=sampler
)

for x in loader:
    x.squeeze_(0)
    print(x.shape)
</code></pre>

din0328: <p>Makes sense! I will implement it as you described!<br>
Thank you!</p>

MikeTensor: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Is there a way to switch the default_collate fn off, since I dont needit, and it makes my live much more complicated …</p>
<p>My return from the CustomDataset would be a pandas.DF (which doesn’t work with the collate_fn).<br>
If I use the list as return, the collate-fn makes very ugly-unreadable tensors out of it.</p>

MikeTensor: <p><strong>beta[1]:</strong> Is the usual return of the CustomDataset (type: list)<br>
<strong>df_train:</strong> Is the pd.Dataframe of beta[1]<br>
<strong>pd.DataFrame(df_train_dl):</strong> Is the output of the DataLoader as pd.DataFrame</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/6/f67547c01ac35efd5929bca967b2cb53c82dbbb8.png" data-download-href="https://discuss.pytorch.org/uploads/default/f67547c01ac35efd5929bca967b2cb53c82dbbb8" title="Bildschirmfoto vom 2023-02-28 06-51-01"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/6/f67547c01ac35efd5929bca967b2cb53c82dbbb8_2_690x479.png" alt="Bildschirmfoto vom 2023-02-28 06-51-01" data-base62-sha1="zagJeqMzaQoO67cKrTjBVQ8bc0w" width="690" height="479" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/6/f67547c01ac35efd5929bca967b2cb53c82dbbb8_2_690x479.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/6/f67547c01ac35efd5929bca967b2cb53c82dbbb8_2_1035x718.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/6/f67547c01ac35efd5929bca967b2cb53c82dbbb8_2_1380x958.png 2x" data-dominant-color="313233"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Bildschirmfoto vom 2023-02-28 06-51-01</span><span class="informations">1558×1082 118 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Would it be possible to provide a custom <code>collate_fn</code> and return the desired outputs instead?</p>

MikeTensor: <p>Wow – you <a class="mention" href="/u/ptrblck">@ptrblck</a> where faster than I could upload my image with the overview of the problem. Thanks.</p>
<p>Hmm. About the custom <code>collate_fn</code> this seems pretty complicated, and I am not sure if I cause even bigger problems by trying to use this …</p>
<p>Maybe this is naiv, but wouldn’t it be easier to convert the image and target (list) (my returns of the DL) manually with one or two lines of code?</p>

ptrblck: <p>I think the main problem is:</p>
<aside class="quote no-group" data-username="MikeTensor" data-post="8" data-topic="31084">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/5daacb/48.png" class="avatar"> MikeTensor:</div>
<blockquote>
<p>My return from the CustomDataset would be a pandas.DF (which doesn’t work with the collate_fn).<br>
If I use the list as return, the collate-fn makes very ugly-unreadable tensors out of it.</p>
</blockquote>
</aside>
<p>as stacked tensors are expected.<br>
You should be able to directly return the <code>pd.DataFrame</code>s, but then your model would most likely not accept it (or you would transform the input to proper tensors in your <code>forward</code> which also sounds quite late).</p>
<aside class="quote no-group" data-username="MikeTensor" data-post="11" data-topic="31084">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/5daacb/48.png" class="avatar"> MikeTensor:</div>
<blockquote>
<p>Maybe this is naiv, but wouldn’t it be easier to convert the image and target (list) (my returns of the DL) manually with one or two lines of code?</p>
</blockquote>
</aside>
<p>Yes, I would recommend transforming the data into a tensor inside the <code>__getitem__</code> so that the default collate function could then create batches from these samples.<br>
Would this work or does it not fit your use case?</p>

MikeTensor: <p>Thanks again!</p>
<p>I don’t need any batches, so just returning the following would be good enough:<br>
<strong>1. feature:</strong> image – for CNN feature extractor –– could be easily converted to Tensor in the <code>__getitem__</code>)<br>
<em><strong>2. feature:</strong> BoundingBoxes – for ROI pooling</em> –– (not needed as Tensor)<br>
<em><strong>3. feature:</strong> Keypoints (maye used later)</em>  –– (not needed as Tensor)<br>
<em><strong>4. feature:</strong> type (object classes) – not sure; if I really use it, since I want to use a classifier</em>  –– (could be easily converted to Tensor in the <code>__getitem__</code>)<br>
<em><strong>y: label:</strong> distance</em> –– (could be easily converted to Tensor in the <code>__getitem__</code>)</p>
<p>Items 2–y usually would be in the pd.DF. But I wouldn’t have issues with return them one-by-one.</p>
<p>The issue is, that right now everything already works manually (every transform, resize, …).<br>
From fetching the Data, to the CNN-feature-extractor, to the roi_pooling and my last two “models” aka “custom_classifier” and “distance_regressor” (which is not done).</p>
<p>Just wanted to automate it now with the DataLoader and stuck with the <code>collate_fn</code> which is causing me problems now …</p>
<p>So my questions would be:</p>
<ol>
<li>
<p>Do I really need the <code>collate_fn</code>?<br>
What kind of real benefit does the <code>collate_fn</code> have (since my model already works manually great)?</p>
</li>
<li>
<p>What kind of disadvanteges do I have by using:<br>
<code>collate_fn= None, batch_size=None</code><br>
It seems like the DataLoader still returns my image, and my target (pd.Dataframe), but doesn’t apply the collate_fn (as described by pytorch-Doc.).<br>
But is this kind of “equal” to “batch_size=1”, just that I don’t get the extra batch_dim in my image, data, …?</p>
</li>
</ol>
<p>So let’s say I would use 2): Do you see any problems by doing so?</p>

MikeTensor: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Since I only need the image and maybe the label (distance) as a tensor …<br>
I would both convert to type.tensor within the <code>__getitem__</code>.</p>
<p>Only <em>image</em> is passed to the model (feature_extractor; freezed vgg.features).<br>
My <code>roi_align</code> and other functions just work fine with non-tensor-type.</p>
<p>So can I just switch the <code>collate_fn</code> off by setting <code>collate_fn= None, batch_size=None</code> (see comment above)?</p>

ptrblck: <ol>
<li>
<p>You don’t necessarily need to use a <code>collate_fn</code>, but then automatic batching would be disabled. The <code>DataLoader</code> could still be useful, e.g. if you want to shuffle the dataset, but you could also directly iterate the <code>Dataset</code> alternatively.</p>
</li>
<li>
<p>Yes, this approach would be similar to just specifying a batch size of 1, but note that you might need to further process the data (in case its not in tensors already).</p>
</li>
</ol>
<aside class="quote no-group" data-username="MikeTensor" data-post="14" data-topic="31084">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/5daacb/48.png" class="avatar"> MikeTensor:</div>
<blockquote>
<p>So can I just switch the <code>collate_fn</code> off by setting <code>collate_fn= None, batch_size=None</code> (see comment above)?</p>
</blockquote>
</aside>
<p>Yes, this should work but are you sure that a batch size of 1 is really what you want and that you won’t increase it during training?</p>

MikeTensor: <p>Thanks!<br>
<span class="mention">@1</span>) Yes; using the DL for shuffling is still great. That’s why I want to use the DL.<br>
<span class="mention">@2</span>) Great – good to know.<br>
Batch size = 1 should be good, and enough, since every picture has a couple of <code>roi</code>s (up to 9+ per image, which are passing <code>my_classifier</code> and <code>distance_regressor</code> one after another. So these rois are already kind of my “batch” with which my two models are trained.</p>
<p>(The paper I am trying to reproduce also just used batch size = 1; so hopefully they were not sooo wrong <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"> )</p>