IndexError: Target 24000 is out of bounds

VickneshBalabaskaran: <p>Tried to train a model for Image segmentation.  Used a pretrained model by torchvision FCN-Resnet101 and changed the final two layers to have a model for output number of classes = 30.</p>
<p>I chose the Network as:</p>
<blockquote>
<p>def Net():<br>
print(“Loading in pretrained network …”)<br>
model = models.segmentation.fcn_resnet101(pretrained=True).eval().to(device)<br>
for param in model.parameters():<br>
param.requires_grad = False</p>
<pre><code>model.classifier = nn.Sequential(
    nn.Conv2d(2048, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False),
    nn.BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),
    nn.ReLU(),
    nn.Dropout(p=0.1, inplace=False),
    nn.Conv2d(512, 30, kernel_size=(1, 1), stride=(1, 1))
    )
model.aux_classifier = nn.Sequential(
    nn.Conv2d(1024, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False),
    nn.BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),
    nn.ReLU(),
    nn.Dropout(p=0.1, inplace=False),
    nn.Conv2d(256, 30, kernel_size=(1, 1), stride=(1, 1))
    )
print(model)
return model
</code></pre>
</blockquote>
<p>and the training was done in such a manner:</p>
<blockquote>
<p>def train_model(model, train_loader, val_loader, epochs=1):<br>
learning_rate = 1e-4<br>
optimizer = optim.Adam(model.aux_classifier.parameters(), lr=learning_rate)</p>
<pre><code>model = model.to(device)  # move the model parameters to CPU/GPU
for e in range(epochs):
    print("Epoch: ", e)
    for t, (x, y) in enumerate(train_loader):
        model.train()
        x = x.to(device)
        y = y.type(torch.LongTensor)
        y = y.to(device)
        y = y.squeeze(1)

        scores = model(x)['out']
        loss = F.cross_entropy(scores, y)
        torch.cuda.empty_cache()
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if t % 100 == 0:
            print('Iteration %d, loss = %.4f' % (t, loss.item()))
            check_val_accuracy(val_loader, model)
            print()
</code></pre>
</blockquote>
<p>I got this error:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “main.py”, line 341, in module<br>
main()<br>
File “main.py”, line 334, in main<br>
train_model(model, train_loader, val_loader, epochs=3)<br>
File “main.py”, line 125, in train_model<br>
loss = F.cross_entropy(scores, y)<br>
File “/media/vicknesh/Studies/CSCI508/Project/venv/lib/python3.6/site-packages/torch/nn/functional.py”, line 2021, in cross_entropy<br>
return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)<br>
File “/media/vicknesh/Studies/CSCI508/Project/venv/lib/python3.6/site-packages/torch/nn/functional.py”, line 1840, in nll_loss<br>
ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)<br>
IndexError: Target 24000 is out of bounds.</p>
</blockquote>
<p>Even though my output number of classes are 30 my targets are going out of bounds and that too 24000. I even tried CrossEntropyLoss(). What mistake can you see?</p>

ptrblck: <p>How did you define your target and could you check the min and max values for it?<br>
For a segmentation use case the target should have the shape <code>[batch_size, height, width]</code> and contain the class indices in the range <code>[0, nb_classes-1]</code>.</p>

VickneshBalabaskaran: <p>Thanks, looks like my class indices aren’t in the range as specfied. I printed the minimum and maximum values of my label for few iterations and this is what I got:</p>
<blockquote>
<p>tensor(1) tensor(33008)<br>
tensor(0) tensor(33004)<br>
tensor(0) tensor(33001)<br>
tensor(0) tensor(33003)<br>
tensor(1) tensor(33003)<br>
tensor(0) tensor(33001)<br>
tensor(1) tensor(33000)<br>
tensor(1) tensor(33000)<br>
tensor(0) tensor(33003)<br>
tensor(0) tensor(33000)<br>
tensor(0) tensor(33007)<br>
tensor(0) tensor(33005)<br>
tensor(0) tensor(33004)<br>
tensor(0) tensor(33006)<br>
tensor(0) tensor(33000)<br>
tensor(0) tensor(33001)<br>
tensor(0) tensor(33000)<br>
tensor(0) tensor(33005)<br>
tensor(1) tensor(33001)<br>
tensor(0) tensor(33002)<br>
tensor(0) tensor(33001)<br>
tensor(0) tensor(33002)<br>
tensor(0) tensor(33000)<br>
tensor(0) tensor(33001)<br>
tensor(1) tensor(33000)<br>
tensor(1) tensor(33000)<br>
tensor(0) tensor(33000)<br>
tensor(0) tensor(33003)<br>
tensor(0) tensor(33006)<br>
tensor(0) tensor(33002)<br>
tensor(1) tensor(33002)<br>
tensor(1) tensor(33005)<br>
tensor(0) tensor(33002)<br>
tensor(1) tensor(33000)<br>
tensor(1) tensor(33000)<br>
tensor(0) tensor(33003)<br>
tensor(1) tensor(33006)<br>
tensor(1) tensor(33000)<br>
tensor(1) tensor(27000)<br>
tensor(1) tensor(33003)<br>
tensor(1) tensor(33001)<br>
tensor(0) tensor(33002)<br>
tensor(0) tensor(33000)<br>
tensor(1) tensor(33002)<br>
tensor(0) tensor(33000)<br>
tensor(1) tensor(33001)<br>
tensor(0) tensor(33002)<br>
tensor(1) tensor(33001)<br>
tensor(0) tensor(33005)<br>
tensor(0) tensor(33004)</p>
</blockquote>
<p>How do I tackle this now?</p>

ptrblck: <p>It depends how you’ve created these targets.<br>
Did you load these values from a file or generated them somehow?</p>

VickneshBalabaskaran: <p>I used the torchvision.datasets.Cityscapes to load the model.</p>
<blockquote>
<p>model = Net()<br>
model = model.to(device)<br>
train_transform = transforms.Compose([<br>
<span class="hashtag-raw">#transforms</span>.RandomResizedCrop(size=224, scale=(0.9,1.0), ratio=(0.9,1.1)),<br>
<span class="hashtag-raw">#transforms</span>.Resize(256),<br>
transforms.RandomHorizontalFlip(),<br>
transforms.ToTensor(),<br>
transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])<br>
])<br>
test_transform = transforms.Compose([<br>
<span class="hashtag-raw">#transforms</span>.Resize(256),<br>
<span class="hashtag-raw">#transforms</span>.CenterCrop(224),<br>
transforms.ToTensor(),<br>
transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])<br>
])<br>
train_dataset = torchvision.datasets.Cityscapes(root = “./Cityscapes”, split=“train”, transform=train_transform)<br>
num_train = len(train_dataset)<br>
val_dataset = torchvision.datasets.Cityscapes(root = “./Cityscapes”, split=“val”, transform=train_transform)<br>
num_val = len(val_dataset)<br>
print(“Number of training images: %d” % len(train_dataset))<br>
print(“Number of validation images: %d” % len(val_dataset))</p>
<pre><code>    batch_size = 3

    train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True, num_workers=1)
    val_loader = torch.utils.data.DataLoader(val_dataset, batch_size=batch_size, shuffle=True, num_workers=1)
    test_dataset = torchvision.datasets.Cityscapes(root = "./Cityscapes", split="test", transform=test_transform)
    test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=True, num_workers=1)
    print("Number of test images: %d" % len(test_dataset))
    torch.save(train_loader,'train_loader')
    torch.save(val_loader,'val_loader')
    torch.save(test_loader,'test_loader')
    train_model(model, test_loader, val_loader, epochs=3)
</code></pre>
</blockquote>

ptrblck: <p>How does your folder structure in <code>./Cityscapes</code> look?<br>
I’m not deeply familiar with this dataset, but it seems the default <code>target_type</code> is set as <code>'instance'</code>.<br>
Do you know, how these targets are defined internally?</p>

VickneshBalabaskaran: <p>Yes, target_type is set to instance. What should I be using. I don’t know the background of these.<br>
The cityscapes class is as below:</p>
<blockquote>
<p><code>torchvision.datasets.</code> <code>Cityscapes</code> ( <em>root</em> ,  <em>split=‘train’</em> ,  <em>mode=‘fine’</em> ,  <em>target_type=‘instance’</em> ,  <em>transform=None</em> ,  <em>target_transform=None</em> ,  <em>transforms=None</em> )<a href="https://pytorch.org/docs/stable/_modules/torchvision/datasets/cityscapes.html#Cityscapes" rel="noopener nofollow ugc">[SOURCE]</a></p>
<p><a href="http://www.cityscapes-dataset.com/" rel="noopener nofollow ugc">Cityscapes</a> Dataset.</p>
<p>Parameters</p>
<ul>
<li><strong>root</strong>  ( <em>string</em> ) – Root directory of dataset where directory  <code>leftImg8bit</code>  and  <code>gtFine</code>  or  <code>gtCoarse</code>  are located.</li>
<li><strong>split</strong>  ( <em>string</em> <em>,</em>  <em>optional</em> ) – The image split to use,  <code>train</code> ,  <code>test</code>  or  <code>val</code>  if mode=”fine” otherwise  <code>train</code> ,  <code>train_extra</code>  or  <code>val</code></li>
<li><strong>mode</strong>  ( <em>string</em> <em>,</em>  <em>optional</em> ) – The quality mode to use,  <code>fine</code>  or  <code>coarse</code></li>
<li><strong>target_type</strong>  ( <em>string</em>  <em>or</em>  <a href="https://docs.python.org/3/library/stdtypes.html#list" rel="noopener nofollow ugc"> <em>list</em> </a> <em>,</em>  <em>optional</em> ) – Type of target to use,  <code>instance</code> ,  <code>semantic</code> ,  <code>polygon</code>  or  <code>color</code> . Can also be a list to output a tuple with all specified target types.</li>
<li><strong>transform</strong>  ( <em>callable</em> <em>,</em>  <em>optional</em> ) – A function/transform that takes in a PIL image and returns a transformed version. E.g,  <code>transforms.RandomCrop</code></li>
<li><strong>target_transform</strong>  ( <em>callable</em> <em>,</em>  <em>optional</em> ) – A function/transform that takes in the target and transforms it.</li>
<li><strong>transforms</strong>  ( <em>callable</em> <em>,</em>  <em>optional</em> ) – A function/transform that takes input sample and its target as entry and returns a transformed version.</li>
</ul>
</blockquote>
<p>Edit: I just saw that we have a target_type called semantic. I’ll try using it.</p>