How to represent class_to_idx map for custom dataset in Pytorch

username: Shiva_Kumar
cooked: <p>I am attempting transfer learning with a CNN (vgg19) on the Oxford102 category dataset consisting of 8189 samples of flowers labeled from 1 through 102. Instead of loading the data with ImageFolder, which requires a tedious process of structuring my data into train, valid and test folders with each class being a sub-folder holding my images, I decided to load it in using the Custom Dataset class following  <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html" rel="nofollow noopener">This tutorial</a></p>
<p>My directory paths set up for the image, the labels (from 1 through 102), and class label map which<br>
maps the label to flower name</p>
<pre><code>    data_dir_path = 'data/images/'
    labels_path = 'data/imagelabels.mat'
    class_label_path = 'data/class_label_map'y 
</code></pre>
<p>my data is not in the ImageFolder, and as far as I know If writing a  custom dataset, you don’t need your data to be in that format.</p>
<p>data/images/image_00001.jpg<br>
data/images/image_00002.jpg<br>
data/images/image_00003.jpg<br>
data/images/image_00004.jpg<br>
.<br>
.<br>
.<br>
data/images/image_08189.jpg</p>
<p>my labels list which is in data/imagelabels.mat is as follows,</p>
<p>[77 77 77 … 62 62 62]</p>
<p>The transforms I am using for train, test and valid are</p>
<pre><code>    data_transforms = {
'train': transforms.Compose([
    transforms.RandomRotation(45),
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406],
                         [0.229, 0.224, 0.225])
]),
'valid': transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406],
                         [0.229, 0.224, 0.225])
]),
'test': transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406],
                         [0.229, 0.224, 0.225])
]),
</code></pre>
<p>}</p>
<p>Here is my code to load in data, it works as I can, index into my images and labels.</p>
<pre><code>    class MyDataset(Dataset):

def __init__(self, image_labels, data_dir, transform=None):

    """

    :param image_labels_path: path to our labels
    :param root_dir: the directory which houses our images
    :param transform: apply any transform on our sample
    """

    self.image_labels = image_labels
    self.root_dir = data_dir
    self.transform = transform

def __len__(self):

    label_dict = scipy.io.loadmat(self.image_labels)
    return len(label_dict['labels'][0]) # [0] cause its a list of lists

def __getitem__(self, idx):

    image_path_list = [os.path.join(self.root_dir, filename) for filename in os.listdir(self.root_dir)]

    image = Image.open(image_path_list[idx])

    label_dict = scipy.io.loadmat(self.image_labels)
    label_list = label_dict['labels'][0]

    # labels for pytorch should start from zero
    label_list[:] = [i - 1 for i in label_list]
    label = label_list[idx]  

    if self.transform:
        image = self.transform(image)

    return image, label
</code></pre>
<h1>Here I am creating my trainset, validset and testset</h1>
<pre><code>    image_datasets = {x: MyDataset(image_labels=labels_path, data_dir=data_dir_path, transform=data_transforms[x]) for x in
              ['train', 'valid', 'test']}
</code></pre>
<p>I subtracted 1 from my label list since Pytorch expects labels to start from 0. Thus 0 through 101 for 102 labels.</p>
<p>Correct me if I am wrong cause I get a “current target &gt;=0 and current target &lt;= n_classes failed” error if I don’t subtract one. Since this fails at 102, I thus surmised that class representation starts from 0.</p>
<p>I have a class_label_map is a dict which maps class labels to flowers names</p>
<p>{ 1": “pink primrose”,<br>
“2”: “hard-leaved pocket orchid”,<br>
“3”: “canterbury bells”,<br>
“4”: “sweet pea”,<br>
“5”: “english marigold”,<br>
“6”: "tiger lily }</p>
<p>My big problem is to get a class_to_idx mapping, how do I do this, my flower names do not match the images if I visualize them, I get totally different flower names for my flowers.</p>
<p>I first created a mapping by having a dict with key my original label before I subtracted 1, and value the one after. Example:</p>
<pre><code>        class_to_idx = {77:76, 73:72, 1:0, 65:64......102:101...65:54}
</code></pre>
<p>This is beyond doubt wrong as I was getting totally wrong label for my images.</p>
<p>The first label of my image in my data_dir_path = ‘data/images’ is 77, upon subtracting one I get 76. Would this mean the index for all labels 76 is 0, and if the next class is 72 , would that mean the index for all classes 72 is 1? So…</p>
<pre><code>        class_to_idx = {76:0, 72:1, 0:2, 65:3....and so on}
</code></pre>
<p>But this again works if the class number does not repeat after a certain classes, i.e. if 72 were to repeat<br>
after class 65, then 72 is no more the 2nd class. The code for how ImageFolder find class_to_index is…</p>
<pre><code>     classes = [d for d in os.listdir(dir) if os.path.isdir(os.path.join(dir, d))]
     classes.sort()
     class_to_idx = {classes[i]: i for i in range(len(classes))}
</code></pre>
<p>I don’t properly understand this. Performing it on my label_list give me the largest index of the a certain<br>
class  {1: 39, 2: 99, 3: 139, 4: 195, 5: 260, 6: 305, 7: 345, 8: 430, 9: 476}.</p>
<p>The ImageFolder seems to have a class_to_idx attribute which if used on my Dataset throws an error,</p>
<pre><code>    image_datasets['train'].class_to_idx
</code></pre>
<p>AttributeError: ‘MyDataset’ object has no attribute ‘class_to_idx’</p>
<p>This is obviously the case because my Dataset class does not contain any such attribute. If I am reasoing is correct, Pytorch gives out a tuple of probabilities and corresponding index when we do probs.max(dim=1). This index is the value in our class_to_index map. The key is the also the key in our<br>
label_map with, the value of which helps us trace the flower name of the our prediction.</p>
<p>So How do I map my classes to my index in this scenario? It isn’t structured like ImageFolder.<br>
This is super important as I need to checkpoint my model and load it back again to fire out predictions.It may sound really silly, but I really don’t know what to do here, please please please help?</p>

username: ptrblck
cooked: <p>Yes, you are correct in your assumption that Python uses 0-based indices.<br>
Your labels should indeed be in the range <code>[0, nb_classes-1]</code>.</p>
<p>Since your map starts at the index 1, you should just add 1 to your label:</p>
<pre><code class="lang-python">idx_to_class = {
    1: "pink primrose",
    2: "hard-leaved pocket orchid",
    3: "canterbury bells",
    4: "sweet pea",
    5: "english marigold",
    6: "tiger lily"
}

label = 0
sample_class = idx_to_class[label+1]
</code></pre>
<p>No need to create another mapping.</p>
<p>Also as a small side note: it seems you are using the whole data for your training, validation, and test datasets, since you are just changing the transformations.</p>

username: Shiva_Kumar
cooked: <p>Thanks a lot, the thing I am trying to grok here is that Pytorch doesn’t actually return the label of the image but the index of the highest probability in the output tensor right? Does it internally sort classes from 0 through n-1 classes such that index correspond to the class.</p>
<p>So though my label_list doesn’t  start with class 0 but class 77 , and suppose my model returns index 48 as the highest probability when I do probs.max(dim=1)[1] by passing in some random image, does the index 48 mean class 48, which I can then look in my class_dict . Hope my question is clear and thank you once again.</p>
<p>Regarding the whole data for train, valid and test, I have a function that does the splitting into each.</p>
<pre><code>    def get_train_valid_split(dataset, shuffle=True, first_split=0.8, second_split=0.9, random_seed=1):

"""

:param dataset: dataset
:param shuffle: shuffles the indices so random elements from our data find themselves in our splits.
:param first_split: this is for train
:param second_split: for validation
:param random_seed: for test
:return: train, validation and test loaders
"""
dataset_size = len(dataset)  # get any, we just need the length, say its n
indices = list(range(dataset_size))  # 0 though n-1.
split1 = int(np.floor(first_split * dataset_size))
split2 = int(np.floor(second_split * dataset_size))

if shuffle:
    np.random.seed(random_seed)
    np.random.shuffle(indices)

train_idx = indices[:split1]
val_idx = indices[split1:split2]
test_idx = indices[split2:]

samplers = {'train': SubsetRandomSampler(train_idx), 'valid': SubsetRandomSampler(val_idx),
            'test': SubsetRandomSampler(test_idx)}

dataloader = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=64, sampler=samplers[x]) for x in
               ['train', 'valid', 'test']}

return dataloader
</code></pre>
<p>data_loaders = get_train_valid_split(image_datasets[‘train’])  # any key works, need to get length in order to split.</p>
<pre><code>    print(data_loaders)
   {'train': &lt;torch.utils.data.dataloader.DataLoader object at 0x7f7966ceb860&gt;, 'valid': &lt;torch.utils.data.dataloader.DataLoader object at 0x7f7966ceb978&gt;, 'test': &lt;torch.utils.data.dataloader.DataLoader object at 0x7f7966ceb9e8&gt;}

    dataset_sizes = {x: len(data_loaders[x]) for x in ['train', 'valid', 'test']}
    print(dataset_sizes)
   {'train': 103, 'valid': 13, 'test': 13} # batch_size = 64.</code></pre>

username: ptrblck
cooked: <p><code>probs.max(dim=1)[1]</code> will give you the predicted class index. This class index does not inherit any class information, but will be learned by providing the right labels.<br>
The ground truth label is provided by the <code>Dataset</code>.<br>
Could you explain a bit more about <code>imagelabels.mat</code>?<br>
Is it just a MATLAB matrix containing indices?<br>
If so, do you know how the mapping was created?<br>
Currently it looks like you are trying to map the sorted file paths to the values in <code>imagelabels.mat</code>, which might be wrong. E.g. does <code>image_00001.jpg</code> belong to class77 (MATLAB index)?</p>
<p>Thanks for the info regarding the datasets. I just thought this could be another issue after fixing this one here <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"></p>

username: Shiva_Kumar
cooked: <p>The imagelabels.mat is  an array of labels.</p>
<pre><code>    label_list = [77 77 77 ... 62 62 62]

    &lt;class 'numpy.ndarray'&gt;
</code></pre>
<p>The sorted version of my file path is as follows</p>
<pre><code>    image_path_list = sorted([os.path.join('data/images/', filename) for filename in os.listdir('data/images/')])
</code></pre>
<p>This gives<br>
[‘data/images/image_00001.jpg’, ‘data/images/image_00002.jpg’, ‘data/images/image_00003.jpg’, ‘data/images/image_00004.jpg’…and so on]</p>
<p>The corresponding class indeed is 77. And it maps perfectly to Passion flower. A few other randomly tested map too.<br>
Me being the very embodiment of stupidity did not sort my labels. And the order they were in since I not sorted them was totally arbitrary as os.listdir() doesn’t read files in the order they appear in the directory.</p>
<p>So, I ended up subtracting one from label_list with 77 becoming 76 and 73 72 and so on. Create a<br>
new class_to_label map that starts from 0.</p>
<pre><code>    {
"0": "pink primrose",
"1": "hard-leaved pocket orchid",
"2": "canterbury bells",
"3": "sweet pea",
"4": "english marigold",
"5": "tiger lily",
"6": "moon orchid",
"7": "bird of paradise",
"8": "monkshood",
"9": "globe thistle",
"10": "snapdragon",
"11": "colt's foot",
"12": "king protea",
 and so on...
 "76": "passion flower"
</code></pre>
<p>}</p>
<p>I now think I am good to go. Thanks a ton.</p>

username: ptrblck
cooked: <p>Awesome it’s working!<br>
So the issue was that your file paths weren’t sorted?</p>

username: Shiva_Kumar
cooked: <p>Yes that was the issue. I ended up printing out a few lines to check. Should have done that before. Thanks.</p>

username: Shiva_Kumar
cooked: <p>But going forward, are you aware of any best practice with this class_to_idx thing for Pytorch. This was the one that tripped me up the most. Thank you very much for your time and patience.</p>

username: ManicJon
cooked: <p>Hi,<br>
I have an issue using CIFER and rotation. I have a secondary information table I load which I wish to cross reference with the image I get from torch.index_select. How on earth do I get the Tensor index (as in it’s position in the dataset from say 0 - 50000)??</p>

username: ptrblck
cooked: <p>You can write a custom <code>Dataset</code> (e.g. by deriving from the <code>CIFAR</code> dataset) as described e.g. <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">here</a> and return the <code>index</code> in the <code>__getitem__</code> method with the data and target tensors.</p>

username: ManicJon
cooked: <p>Ok, this may sound stupid but how do I use the <strong>get</strong> item (easy to change as I have modified the CIFAR loader to work with my data) just not sure the code to go from index_select to sub data<br>
(Thanks for the reply)</p>

username: ptrblck
cooked: <p>Something like this would work:</p>
<pre><code class="lang-python">class MyCIFAR10(torch.utils.data.Dataset):
    def __init__(self, transform=None):
        self.dataset = datasets.CIFAR10(root='~/python/data', download=False, transform=transform)
        
    def __len__(self):
        return len(self.dataset)
    
    def __getitem__(self, index):
        x, y = self.dataset[index]
        return x, y, index
    
dataset = MyCIFAR10(transform=transforms.ToTensor())
loader = torch.utils.data.DataLoader(dataset, batch_size=2)
x, y, index = next(iter(loader))
print(index)
# &gt; tensor([0, 1])
</code></pre>