Torch.cat issues in creating RNN

username: Danielr13
cooked: <p>Hello, I am creating a RNN for binary classification. The goal is to look at binary arrays of length 60 in which arrays containing  2 0r more consecutive 1s are not a part of the grammar (target = 0) and those that do not are a part of the grammar (target = 1). I am attempting to modify this model for my data set: <a href="https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial.html" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial.html</a>.</p>
<p>here is my model:</p>
<pre><code class="lang-auto">class RNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(RNN, self).__init__()

        self.hidden_size = hidden_size

        self.i2h = nn.Linear(input_size + hidden_size, hidden_size)
        self.i2o = nn.Linear(input_size + hidden_size, output_size)
        self.softmax = nn.LogSoftmax(dim=1)

    def forward(self, input, hidden):
        print(input)
        print(input.size())
        print(hidden.size())
        input = torch.unsqueeze(input, 0)
        print(input.size())
        combined = torch.cat((input, hidden), 1)
        hidden = self.i2h(combined)
        output = self.i2o(combined)
        output = self.softmax(output)
        return output, hidden

    def initHidden(self):
        return torch.zeros(1, self.hidden_size)

n_elements = 60
n_hidden = 128
n_categories = 2
rnn = RNN(n_elements, n_hidden, n_categories)
</code></pre>
<p>train function:</p>
<pre><code class="lang-auto">criterion = nn.NLLLoss()

learning_rate = 0.005 

def train(category_tensor, line_tensor):
    hidden = rnn.initHidden()

    rnn.zero_grad()

    for i in range(line_tensor.size()[0]):
        output, hidden = rnn(line_tensor[i], hidden)

    loss = criterion(output, category_tensor)
    loss.backward()

    # Add parameters' gradients to their values, multiplied by learning rate
    for p in rnn.parameters():
        p.data.add_(p.grad.data, alpha=-learning_rate)

    return output, loss.item()
</code></pre>
<p>training loop:</p>
<pre><code class="lang-auto">import time
import math

n_iters = 1000
print_every = 5000
plot_every = 1000



# Keep track of losses for plotting
current_loss = 0
all_losses = []

def timeSince(since):
    now = time.time()
    s = now - since
    m = math.floor(s / 60)
    s -= m * 60
    return '%dm %ds' % (m, s)

start = time.time()

for iter in range(1, n_iters + 1):
    line_tensor = x_train[iter - 1, :]
    #print(line_tensor)
    category_tensor = y_train[iter - 1] 
    #print(category_tensor)
    output, loss = train(category_tensor, line_tensor)
    current_loss += loss

    # Print iter number, loss, name and guess
    if iter % print_every == 0:
        #guess, guess_i = categoryFromOutput(output)
        guess = output.data.max(1, keepdim = True)[1]
        correct = '✓' if guess == category else '✗ (%s)' % category
        print('%d %d%% (%s) %.4f %s / %s %s' % (iter, iter / n_iters * 100, timeSince(start), loss, line, guess, correct))

    # Add current loss avg to list of losses
    if iter % plot_every == 0:
        all_losses.append(current_loss / plot_every)
        current_loss = 0
</code></pre>
<p>Error:</p>
<pre><code class="lang-auto">IndexError                                Traceback (most recent call last)
&lt;ipython-input-17-5369f07dfbd7&gt; in &lt;module&gt;
     26     category_tensor = y_train[iter - 1]
     27     #print(category_tensor)
---&gt; 28     output, loss = train(category_tensor, line_tensor)
     29     current_loss += loss
     30 

&lt;ipython-input-16-be1621d99dcf&gt; in train(category_tensor, line_tensor)
      9 
     10     for i in range(line_tensor.size()[0]):
---&gt; 11         output, hidden = rnn(line_tensor[i], hidden)
     12 
     13     loss = criterion(output, category_tensor)

~\anaconda3\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--&gt; 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

&lt;ipython-input-13-d9c935b3b378&gt; in forward(self, input, hidden)
     15         input = torch.unsqueeze(input, 0)
     16         print(input.size())
---&gt; 17         combined = torch.cat((input, hidden), 1)
     18         hidden = self.i2h(combined)
     19         output = self.i2o(combined)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
</code></pre>
<p>I have tried messing with the dimensions of the inputs to self.forward() and it prints the following:</p>
<pre><code class="lang-auto">tensor(0, dtype=torch.int32)
torch.Size([])
torch.Size([1, 128])
torch.Size([1])
</code></pre>
<p>I am not sure how to fix this issue, can anyone help?</p>

username: ptrblck
cooked: <p>Is the shape of <code>input</code> variable and return the posted values or is the output referring to multiple tensors?</p>