Tutorial WRITING CUSTOM DATASETS, DATALOADERS AND TRANSFORMS tutorial

Anna_Inberg: <p>Good afternoon!<br>
I have questions about the following tutorial:<br>
<a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/data_loading_tutorial.html </a></p>
<p>I have a similar dataset (images + landmarks). I’ve built the custom dataloader following the tutorial and checked the types of dataloader components (torch.float64 for both images and landmarks).</p>
<p>Then I applied the dataloader to the classification model with this training class:</p>
<pre><code class="lang-python">class Trainer():
  
    def __init__(self,criterion = None,optimizer = None,schedular = None):
      
        self.criterion = criterion
        self.optimizer = optimizer
        self.schedular = schedular
  
    def train_batch_loop(self,model,train_dataloader):    
        train_loss = 0.0
        train_acc = 0.0     
        for images,landmarks, labels in train_dataloader: 
                images = images.to(device)
                landmarks = landmarks.to(device)
                labels = labels.to(device)        
                 self.optimizer.zero_grad()
</code></pre>
<p>I won’t be elaborating further because the training crushes at <strong>images = images.to(device)</strong> with the following error: <strong>AttributeError: ‘str’ object has no attribute 'to’</strong></p>
<p>I don’t understand where this string is coming from if all the dataloader components are torch.float64.<br>
I went back to check the initial data: in the tutorial, the landmarks are summarized in a pandas dataframe with landmark values as int64 and image name as “object”.<br>
In my summary dataframe image name is an “object” as well and landmarks are numpy.float64. Again, no strings anywhere…</p>
<p>I will share the code for creating a summary datatable in the first comment.<br>
Appreciate any ideas!</p>

Anna_Inberg: <pre><code class="lang-python">def load_landmarks(path, header):
    filename = path.split("\\")[-1].replace("txt", "png")
    instance = np.swapaxes(pd.read_csv(path, delimiter ="\t", index_col=False).values, 0,1)
    data_list = list(zip(instance[1], instance[2]))
    flattened = [item for sublist in data_list for item in sublist]
    flattened.insert(0, filename)
    frame = pd.DataFrame(np.array(flattened).reshape(-1,len(flattened)), columns = header)
    return frame
</code></pre>
<pre><code class="lang-python">def data_summary(path, num_landmarks):
    header = ['image_name']
    for i in range(num_landmarks):
        header += ['landmark_{}_x'.format(i), 'landmark_{}_y'.format(i)]
    data_table = pd.DataFrame(columns = header)
    labels_lst = []
    for label in os.listdir(path):
        for roots, dirs, filenames in os.walk(os.path.join(path, label)):
            for file in filenames:
                filename = os.path.splitext(file)
                if filename[1] == ".png":
                    label = label
                    cat_number = file.split("_")[1]
                    labels_lst.append(tuple((int(label), cat_number)))
                    shutil.copy(os.path.join(roots, file), images_path)
                if filename[1] == ".txt":
                    temp_path = os.path.abspath(os.path.join(path, label, file))
                    landmark = load_landmarks(temp_path, header)
                    data_table = data_table.append(landmark)
    labels = [i[0] for i in labels_lst]
    cat_numbers = [i[1] for i in labels_lst]
    
    data_table.insert(0, 'cat_number', cat_numbers)
    data_table.insert(1, 'label', labels)
    data_table['cat_number'] = data_table['cat_number'].astype('int64')
    return data_table
</code></pre>
<pre><code class="lang-python">class FaceLandmarksDataset(Dataset):
    def __init__(self, data_frame, root_dir, transform=None):
        self.data_frame = data_frame
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.data_frame)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        img_name = os.path.join(self.root_dir, self.data_frame.iloc[idx, 2])
        image = io.imread(img_name)
        landmarks = self.data_frame.iloc[idx, 3:]
        landmarks = np.array([landmarks])
        landmarks = landmarks.astype('float').reshape(-1, 2)
        labels = self.data_frame.iloc[idx, 1].reshape(1)
        sample = {'image': image, 'landmarks': landmarks, 'labels': labels}

        if self.transform:
            sample = self.transform(sample)

        return sample
</code></pre>

ptrblck: <p>Print the <code>dtypes</code> of the <code>sample</code> in the <code>__getitem__</code> before and after applying <code>self.transform</code> on them and check if some of them are strings.</p>

Anna_Inberg: <p>Thanks for your response.<br>
I’ve checked that - maybe I did something wrong?</p>
<p>Before transformation:</p>
<pre><code class="lang-auto">face_dataset = FaceLandmarksDataset(full_data_table, images_path)
for i in range(len(face_dataset)):
    sample = face_dataset[i]
    print(i, sample['image'].shape, sample['landmarks'].shape, sample['labels'].shape)
    print(i, sample['image'].dtype, sample['landmarks'].dtype, sample['labels'].dtype)

    # image - (1000, 1000, 4), uint8 
    # landmarks  - (48, 2), float64
    # label -  (1,), int64
</code></pre>
<p>After transformation:</p>
<pre><code class="lang-auto">scale = Rescale(224)
transformed_dataset = FaceLandmarksDataset(full_data_table, images_path,
                                           transform=transforms.Compose([scale, ToTensor()]))

for i in range(len(transformed_dataset)):
    sample = transformed_dataset[i]

    print(i, sample['image'].size(), sample['landmarks'].size(), sample['labels'].size())
    print(i, sample['image'].dtype, sample['landmarks'].dtype, sample['labels'].dtype)

    # image - [4, 224, 224]), torch.float64 
    # landmarks  - torch.Size([48, 2]), torch.float64
    # label -  torch.Size([1]), torch.int64
</code></pre>

ptrblck: <p>The <code>dtypes</code> look correct. Are you seeing the error in a specific iteration or directly in the first one?<br>
In the former case, you could keep the print statements, start the training, and check if the <code>dtype</code> changes in one iteration. I would recommend not to shuffle the dataset so that you could check which index was used to create the data.</p>

Anna_Inberg: <p>The error shows up directly in the first iteration and I don’t shuffle the data at Dataloader stage. the dataset, fed into dataloader already comes shuffled.</p>

ptrblck: <p>OK, that’s interesting as it seems that iterating <code>transformed_dataset</code> creates the expected tensors while iterating the <code>DataLoader</code> returns a string and causes the error.<br>
Could you use a batch size of 1 and iterate the <code>DataLoader</code> again with the same print statements?</p>

Anna_Inberg: <p>Tried - same results. Still the strings are not found.</p>
<p>Is it possible that somewhere during data processing dataloader gets an image name instead of an actual image? Where can it be (maybe data_summary function?)</p>

ptrblck: <p>If the strings are not found anymore, <code>images = images.to(device)</code> wouldn’t be failing with <code>AttributeError: ‘str’ object has no attribute 'to’</code>, would it?</p>
<p>In case it’s still failing you, it seems you are hitting these issues now:</p>
<ul>
<li>the data has definitely the correct <code>dtype</code> in the <code>__getitem__</code> before the <code>return</code> statement</li>
<li>inside the <code>DataLoader</code> loop the <code>images</code> are strings and the <code>to</code> call is failing</li>
</ul>
<p>If so, then the place to look next into would be the <code>collate_fn</code> and I assume you might be using a custom one. By default this method stacks the samples returned by <code>Dataset.__getitem__</code> into a batch and does not manipulate the data.<br>
However, I don’t know if a custom <code>collate_fn</code> is used which might not be interacting nicely with the returned <code>dict</code>.</p>
<p>Another idea: are you indexing the <code>dict</code> with the right keys or are you iterating the returned <code>dict</code> and are trying to use the <code>keys</code> as the data?<br>
Remove the <code>dict</code> usage for the sake of debugging and just return the samples:</p>
<pre><code class="lang-python">        sample = {'image': image, 'landmarks': landmarks, 'labels': labels}

        if self.transform:
            sample = self.transform(sample)

        return sample['image'], sample['landmarks'], sample['labels']
</code></pre>

Anna_Inberg: <p>Thanks for your response. I tried the dict manipulation you suggested, dtypes are still torch floats.</p>
<p>I’ve just found the string. The for-loop in Trainer class “for images,landmarks, labels in train_dataloader: …” is iterating incorrectly over the dataloder.<br>
Dataoader dtypes are fine, but this for-loop returns strings instead of dataloader items.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/0/90dc3f149e72da54057aa1e3176a56ba11181b75.png" alt="image" data-base62-sha1="kFuD9tJuoQcUtBbZNvKb6lL0Nut" width="624" height="95"></p>
<p>the printout shows “str”</p>
<p>Now I need to figure out the way to fix it <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>