Error: Target 4 is out of bounds. Using cross entropy loss

Sueanny_Moreno: <p>I am newbie in pytorch, I am trying to create a convolutional neural network with my own data but I have a problem running the cross entropy loss function. I am not sure but maybe it is due to the structure of my data. Data train contains 16X16 arrays that simulate 9 channel images, hence the train_x has structure of (4608,9,16,16), the labels of this data of (4608). The validation data has dimension of (521,9,16,16) and labels of (512).</p>
<h1>
<a name="importing-the-libraries-1" class="anchor" href="#importing-the-libraries-1"></a>importing the libraries</h1>
<p>import pandas as pd<br>
import numpy as np</p>
<h1>
<a name="for-reading-and-displying-images-2" class="anchor" href="#for-reading-and-displying-images-2"></a>for reading and displying images</h1>
<p>from <a href="http://skimage.io" rel="noopener nofollow ugc">skimage.io</a> import imread<br>
import matplotlib.pyplot as plt<br>
%matplotlib inline</p>
<h1>
<a name="for-creating-validation-set-3" class="anchor" href="#for-creating-validation-set-3"></a>for creating validation set</h1>
<p>from sklearn.model_selection import train_test_split</p>
<h1>
<a name="for-evaluating-the-model-4" class="anchor" href="#for-evaluating-the-model-4"></a>for evaluating the model</h1>
<p>from sklearn.metrics import accuracy_score<br>
from tqdm import tqdm</p>
<h1>
<a name="pytorch-libraries-and-modules-5" class="anchor" href="#pytorch-libraries-and-modules-5"></a>PyTorch libraries and modules</h1>
<p>import torch<br>
from torch.autograd import Variable<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
import torch.optim as optim</p>
<h1>
<a name="check-if-cuda-is-available-6" class="anchor" href="#check-if-cuda-is-available-6"></a>check if CUDA is available</h1>
<p>train_on_gpu = torch.cuda.is_available()<br>
if not train_on_gpu:<br>
print(‘CUDA is not available.  Training on CPU …’)<br>
else:<br>
print(‘CUDA is available!  Training on GPU …’)<br>
train_data=images_training</p>
<p>test_data=images_testing<br>
train_x = np.array(train_data)<br>
test_x = np.array(test_data)</p>
<p><span class="hashtag">#agregando</span> las etiquetas, es decir, identificando a qué clase pertenece cada imagen<br>
train_y = np.array(label_train)<br>
test_y = np.array(label_test)<br>
<span class="hashtag">#creando</span> un set de validación, estos datos los tomamos de los datos de entrenamiento</p>
<p>train_x, val_x, train_y, val_y = train_test_split(train_x, train_y, test_size = 0.1)</p>
<h1>
<a name="convirtiendo-las-imagenes-de-entrenamiento-al-formato-torch-para-su-manejo-dentro-de-la-red-7" class="anchor" href="#convirtiendo-las-imagenes-de-entrenamiento-al-formato-torch-para-su-manejo-dentro-de-la-red-7"></a>convirtiendo las imagenes de ENTRENAMIENTO al formato torch para su manejo dentro de la red</h1>
<p><span class="hashtag">#train_x</span> tiene la forma (4608,9,16,16)<br>
train_x  = torch.from_numpy(train_x)</p>
<h1>
<a name="convirtiendo-las-etiquetas-de-cada-imagen-en-formato-torch-8" class="anchor" href="#convirtiendo-las-etiquetas-de-cada-imagen-en-formato-torch-8"></a>convirtiendo las etiquetas de cada imagen en formato torch</h1>
<p>train_y = train_y.astype(int);<br>
train_y = torch.from_numpy(train_y)</p>
<h1>
<a name="estructura-de-los-datos-de-entrenamiento-9" class="anchor" href="#estructura-de-los-datos-de-entrenamiento-9"></a>estructura de los datos de entrenamiento</h1>
<p>train_x.shape, train_y.shape</p>
<h1>
<a name="convirtiendo-las-imagenes-de-validacin-al-formato-torch-para-su-manejo-dentro-de-la-red-10" class="anchor" href="#convirtiendo-las-imagenes-de-validacin-al-formato-torch-para-su-manejo-dentro-de-la-red-10"></a>convirtiendo las imagenes de VALIDACIÓN al formato torch para su manejo dentro de la red</h1>
<p><span class="hashtag">#train_x</span> tiene la forma (521,9,16,16)<br>
val_x  = torch.from_numpy(val_x)</p>
<h1>
<a name="converting-the-target-into-torch-format-11" class="anchor" href="#converting-the-target-into-torch-format-11"></a>converting the target into torch format</h1>
<p>val_y = val_y.astype(int);<br>
val_y = torch.from_numpy(val_y)</p>
<p>class Net(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Net, self).<strong>init</strong>()<br>
# Defining a 2D convolution layer</p>
<pre><code>    #convirtiendo la imagen de profundidad 9 a 16 con un kernel de 3x3
    self.conv1=nn.Conv2d(9, 16, kernel_size=3, stride=1, padding=1)

    #convirtiendo la imagen de profundidad 16 a 32 con un kernel de 3x3
    self.conv2=nn.Conv2d(16, 32, kernel_size=3, stride=1, padding=1)

    #convirtiendo la imagen de profundidad 32 a 64 con un kernel de 3x3
    #self.conv3=nn.Conv2d(36, 72, kernel_size=3, stride=1, padding=1)

    #Capa macpooling
    self.pool=nn.MaxPool2d(2,2)

    #la dimensión de la imagen era de 16*16 después de la primera pool tengo 8*8 después de la segunda 4*4 despuès de la tercera 2*2

    self.fc1=nn.Linear(72*4*4, 256)
    self.fc2=nn.Linear(256,4)
    self.dropout=nn.Dropout(0.25)
    

# Defining the forward pass    

def forward(self, x):
    #agregando la secuencia de las capas de convolución y maxpooling
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    #x = self.pool(F.relu(self.conv3(x)))
    #flatten, convitiéndolo a forma de vector
    x=x.view(-1,72*4*4)

    #agregando la capa de dropout para prevenir sobreajuste
    x=self.dropout(x)
    #pasando la primera capa oculta con la activación relu
    x=F.relu(self.fc1(x))
    #agregando la capa de dropout para prevenir sobreajuste
    x=self.dropout(x)
   #agregando la segunda capa oculta del fully connectec para obtener la clasificación  
    x=self.fc2(x)
    return x
</code></pre>
<p><span class="hashtag">#Creando</span> la CNN<br>
model= Net()<br>
print (model)<br>
<span class="hashtag">#moviéndolo</span> a una GPU en caso de que tenga CUDA disponible<br>
if train_on_gpu:<br>
model.cuda()<br>
criterion = criterion.cuda()<br>
<span class="hashtag">#Definiendo</span> que usaremos la función de perdida</p>
<p>criterion=nn.CrossEntropyLoss()<br>
<span class="hashtag">#especificando</span> un optimizador con un gradiente descendiente estocástico y una<br>
<span class="hashtag">#tasa</span> de aprendizaje de 0.01</p>
<p>optimizer=optim.SGD(model.parameters(),lr=0.01)<br>
def train(epoch):</p>
<pre><code>###################

# train the model #

###################

model.train()
tr_loss = 0
# getting the training set
x_train, y_train = Variable(train_x), Variable(train_y)
# getting the validation set
x_val, y_val = Variable(val_x), Variable(val_y)
# converting the data into GPU format
if torch.cuda.is_available():
    x_train = x_train.cuda()
    y_train = y_train.cuda()
    x_val = x_val.cuda()
    y_val = y_val.cuda()


# clear the gradients of all optimized variables
optimizer.zero_grad()
# forward pass: compute predicted outputs by passing inputs to the model
output_train = model(x_train.float().squeeze(1))
output_val = model(x_val.float().squeeze(1))
print('train ',x_train.shape, 'label ', y_train.shape, 'prediccion ', output_train )
print(output_train)

# computing the training and validation loss
loss_train = criterion(output_train, y_train)
print(loss_train)
loss_val = criterion(output_val, y_val)
print(loss_val)
train_losses.append(loss_train)
val_losses.append(loss_val)
#En esta sección se actualizan los pesos praa todos los modelos
# backward pass: compute gradient of the loss with respect to model parameters

loss_train.backward()
# perform a single optimization step (parameter update)
optimizer.step()
# update training loss
tr_loss = loss_train.item()
print('Epoch: {} \tTraining Loss: {:.5f} \tValidation Loss: {:.5f}'.format(
    epoch, loss_train, loss_val))
</code></pre>
<h1>
<a name="defining-the-number-of-epochs-12" class="anchor" href="#defining-the-number-of-epochs-12"></a>defining the number of epochs</h1>
<p>n_epochs = 10</p>
<h1>
<a name="empty-list-to-store-training-losses-13" class="anchor" href="#empty-list-to-store-training-losses-13"></a>empty list to store training losses</h1>
<p>train_losses = []</p>
<h1>
<a name="empty-list-to-store-validation-losses-14" class="anchor" href="#empty-list-to-store-validation-losses-14"></a>empty list to store validation losses</h1>
<p>val_losses = []</p>
<h1>
<a name="training-the-model-15" class="anchor" href="#training-the-model-15"></a>training the model</h1>
<p>for epoch in range(n_epochs):<br>
train(epoch)</p>
<p>This is the mistake<br>
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)<br>
2822     if size_average is not None or reduce is not None:<br>
2823         reduction = _Reduction.legacy_get_string(size_average, reduce)<br>
 → 2824     return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)<br>
2825<br>
2826</p>
<p>IndexError: Target 4 is out of bounds.</p>

ptrblck: <p>Your model yields an output in the shape <code>[batch_size, 4]</code> (defined by the last linear layer) which contains logits for 4 classes. The target should thus contain class indices in the range <code>[0, 3]</code> and an index of <code>4</code> will be out of bounds. I’m not familiar with your use case, but in case your target contains 5 unique class indices, the model output should also have the shape <code>[batch_size, nb_classes=5]</code>.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Sueanny_Moreno: <p>That was the problem, thank you very much for your help, I appreciate it.</p>