Getting the "TypeError: 'int' object is not subscriptable" error while constructing a basic LSTM network?

talhak: <p>Hey everyone,</p>
<p>I’m getting the error below while constructing a basic <code>LSTM</code> network for research purposes (which is based on the source code available in <a href="https://github.com/bentrevett/pytorch-sentiment-analysis/blob/master/1%20-%20Simple%20Sentiment%20Analysis.ipynb" rel="nofollow noopener">this GitHub repository</a>). Could you please help me to find out the issue?</p>
<p>p.s. I’m utilizing the <code>Google Colab</code> platform.</p>
<pre><code class="lang-auto">TypeError Traceback (most recent call last)

[&lt;ipython-input-10-8883ef0b1eb5&gt;](https://iyhna5ktvk-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20200708-085602-RC00_320186311#) in &lt;module&gt;() 195 196 if __name__ == '__main__': --&gt; 197 main()

6 frames

[/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py](https://iyhna5ktvk-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20200708-085602-RC00_320186311#) in check_forward_args(self, input, hidden, batch_sizes) 520 expected_hidden_size = self.get_expected_hidden_size(input, batch_sizes) 521 --&gt; 522 self.check_hidden_size(hidden[0], expected_hidden_size, 523 'Expected hidden[0] size {}, got {}') 524 self.check_hidden_size(hidden[1], expected_hidden_size,

TypeError: 'int' object is not subscriptable
</code></pre>
<pre><code class="lang-auto">import os
import random
import time

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import Dataset
from torchtext import data
from torchtext.datasets import IMDB


def epoch_time(start_time, end_time):
    elapsed_time = end_time - start_time
    elapsed_mins = int(elapsed_time / 60)
    elapsed_secs = int(elapsed_time - (elapsed_mins * 60))
    return elapsed_mins, elapsed_secs


class LSTM_IMDB(nn.Module):
    def __init__(self, input_dim, embedding_dim, hidden_dim, output_dim):
        super().__init__()
        self.hidden_dim = hidden_dim
        self.embedding = nn.Embedding(input_dim, embedding_dim)
        self.lstm = nn.LSTM(embedding_dim, hidden_dim)
        self.fc = nn.Linear(hidden_dim, output_dim)

    def forward(self, text):
        embedded = self.embedding(text)

        lstm_out, hidden=self.lstm(embedded, self.hidden_dim)

        #stack up the lstm output
        lstm_out=lstm_out.contiguous().view(-1, hidden)

        return self.fc(lstm_out)


def binary_accuracy(preds, y):
    """
    Returns accuracy per batch, i.e. if you get 8/10 right, this returns 0.8, NOT 8
    """

    # round predictions to the closest integer
    rounded_preds = torch.round(torch.sigmoid(preds))
    correct = (rounded_preds == y).float()  # convert into float for division
    acc = correct.sum() / len(correct)
    return acc


def train(model, iterator, optimizer, criterion):
    epoch_loss = 0
    epoch_acc = 0

    model.train()

    for batch in iterator:
        optimizer.zero_grad()

        predictions = model(batch.text).squeeze(1)

        loss = criterion(predictions, batch.label)

        acc = binary_accuracy(predictions, batch.label)

        loss.backward()

        optimizer.step()

        epoch_loss += loss.item()
        epoch_acc += acc.item()

    return epoch_loss / len(iterator), epoch_acc / len(iterator)


def evaluate(model, iterator, criterion):
    epoch_loss = 0
    epoch_acc = 0

    model.eval()

    with torch.no_grad():
        for batch in iterator:
            predictions = model(batch.text).squeeze(1)

            loss = criterion(predictions, batch.label)

            acc = binary_accuracy(predictions, batch.label)

            epoch_loss += loss.item()
            epoch_acc += acc.item()

    return epoch_loss / len(iterator), epoch_acc / len(iterator)


def main():
    SEED = 1234
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    torch.manual_seed(SEED)
    torch.backends.cudnn.deterministic = True

    TEXT = data.Field(tokenize='spacy')
    LABEL = data.LabelField(dtype=torch.float)

    train_data, test_data = IMDB.splits(TEXT, LABEL)
    train_data, valid_data = train_data.split(split_ratio=0.5, random_state=random.seed(SEED))

    print(f'Number of training examples: {len(train_data)}')
    print(f'Number of validation examples: {len(valid_data)}')
    print(f'Number of testing examples: {len(test_data)}')

    MAX_VOCAB_SIZE = 25_000
    TEXT.build_vocab(train_data, max_size=MAX_VOCAB_SIZE)
    LABEL.build_vocab(train_data)

    BATCH_SIZE = 64
    train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits(
        (train_data, valid_data, test_data),
        batch_size=BATCH_SIZE,
        device=device)

    INPUT_DIM = len(TEXT.vocab)
    EMBEDDING_DIM = 100
    HIDDEN_DIM = 256
    OUTPUT_DIM = 1

    model = LSTM_IMDB(INPUT_DIM, EMBEDDING_DIM, HIDDEN_DIM, OUTPUT_DIM)
    model.to(device)

    lr = 1e-2
    criterion = nn.BCELoss()
    criterion = criterion.to(device)
    optimizer = optim.SGD(model.parameters(), lr=lr, momentum=0.9)

    N_EPOCHS = 5

    best_valid_loss = float('inf')

    for epoch in range(N_EPOCHS):

        start_time = time.time()

        train_loss, train_acc = train(model, train_iterator, optimizer, criterion)
        valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)

        end_time = time.time()

        epoch_mins, epoch_secs = epoch_time(start_time, end_time)

        if valid_loss &lt; best_valid_loss:
            best_valid_loss = valid_loss
            torch.save(model.state_dict(), 'tut1-model.pt')

        print(f'Epoch: {epoch + 1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')
        print(f'\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc * 100:.2f}%')
        print(f'\t Val. Loss: {valid_loss:.3f} |  Val. Acc: {valid_acc * 100:.2f}%')


if __name__ == '__main__':
    main()
</code></pre>

ptrblck: <p>Could you check, if you are accidentally passing an <code>int</code> to the model instead of a tensor?<br>
The error is raised, if you are trying to index an <code>int</code> via:</p>
<pre><code class="lang-python">1[0]
&gt; TypeError: 'int' object is not subscriptable
</code></pre>
<p>which seems to be called somewhere inside the <code>forward</code> method.</p>

talhak: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a>,<br>
Thanks for your interest. I don’t purposely do such thing as I use the <code>iterator</code> mechanism on a pre-defined, standard dataset, namely, <code>IMDB</code>. The error might be due to the confusion between the <code>hidden</code> and <code>self.hidden_dim</code> params located in the <code>forward</code> function. When you have some free time, could you please check them?</p>

ptrblck: <p>The error is raised in:</p>
<pre><code class="lang-python">lstm_out, hidden=self.lstm(embedded, self.hidden_dim)
</code></pre>
<p>since you are passing in the <code>self.hidden_dim</code>, which is an integer instead of the expected <code>hidden</code> tensor.<br>
Have a look at the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.LSTM">docs</a> for more information regarding its shape etc. or just remove it, if you want to use the default zero-initialized <code>hidden</code>.</p>
<p>Once this is fixed, you would run into another issue, as you are trying to use the returned <code>hidden</code> tensor to reshape the <code>lstm_out</code> in:</p>
<pre><code class="lang-python">lstm_out=lstm_out.contiguous().view(-1, hidden)
</code></pre>
<p>so you might want to use <code>self.hidden_dim</code> for it.<br>
That being said, note that this <code>view</code> operation would collapse the temporal and batch dimension.</p>

talhak: <p>Thank you very much for your explanation, dear <a class="mention" href="/u/ptrblck">@ptrblck</a>. In order to not collapse the temporal and batch dimension as a result of the <code>view</code> operation, now I don’t apply it. But, still I’m getting the <code>ValueError: Target and input must have the same number of elements. target nelement (64) != input nelement (76800)</code> error.</p>
<p>p.s. Here is the corrected version of the <code>forward</code> function:</p>
<pre><code class="lang-auto">def forward(self, text):
    embedded = self.embedding(text)

    lstm_out, hidden = self.lstm(embedded)

    return self.fc(lstm_out)
</code></pre>

ptrblck: <p>I guess 64 would be the batch size, while 76800 would probably be the temporal size (or the flattened batch dim and temp. dim)?</p>
<p>Anyway, I think you might want to consider using the last time step only or reducing the temporal dimension somehow (e.g. with a <code>mean</code>) to get an output of <code>[batch_size, out_features]</code>.</p>
<p>I’m not familiar with your use case, so please feel free to add more information in case my suggestions wouldn’t work.</p>

talhak: <p>Exactly, <code>64</code> is the batch size, and <code>76,800</code> is the temporal size. But could not get what you mean by using the last time step only? Actually, I’ve added the whole source code in the OP. Could you please guide me on it?</p>