Seq2seq: For unbatched 2-D input, hx and cx should also be 2-D but got (3-D, 3-D) tensors

Ninlawat_Phuangchoke: <p>I debug program each step that the first is encoder layer I don’t got any error as below</p>
<pre><code class="lang-auto">input_dim = len(en_vocab) 
output_dim = 11
encoder_embedding_dim = 512
decoder_embedding_dim = 512
hidden_dim = 1024
n_layers = 2
encoder_dropout = 0.5
decoder_dropout = 0.5
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

encoder = Encoder(
    input_dim,
    encoder_embedding_dim,
    hidden_dim,
    n_layers,
    encoder_dropout,
)
hidden, cell = encoder(batch["numericalized_input"])
hidden.shape, cell.shape
</code></pre>
<p>Output:</p>
<blockquote>
<p>src shape: torch.Size([26, 3])<br>
embedded shape: torch.Size([26, 3, 512])<br>
hidden shape: torch.Size([2, 26, 1024])<br>
(torch.Size([2, 26, 1024]), torch.Size([2, 26, 1024]))</p>
</blockquote>
<p>My decoder layer</p>
<pre><code class="lang-auto">class Decoder(nn.Module):
    def __init__(self, output_dim,hidden_dim, n_layers, dropout):
        super().__init__()
        self.output_dim = output_dim
        self.hidden_dim = hidden_dim
        self.n_layers = n_layers
        self.rnn = nn.LSTM(output_dim, hidden_dim, n_layers, dropout=dropout,batch_first=True)
        self.fc_out = nn.Linear(hidden_dim, output_dim)
        self.dropout = nn.Dropout(dropout)

    def forward(self, input, hidden,cell):
        input = input.unsqueeze(0) 
        input = input.float()
        print('input :',input)
        output,(hidden,cell) = self.rnn(input,(hidden,cell))
        prediction = self.fc_out(output.squeeze(0))
        return prediction,hidden, output
</code></pre>
<p>When I run this program</p>
<pre><code class="lang-auto">decoder = Decoder(
    output_dim,
    hidden_dim,
    n_layers,
    decoder_dropout,
)
prediction, hidden, output = decoder(batch["target"][0], hidden,cell)
prediction.shape, hidden.shape
</code></pre>
<p>I’m getting the following error</p>
<blockquote>
<p>RuntimeError: For unbatched 2-D input, hx and cx should also be 2-D but got (3-D, 3-D) tensors</p>
</blockquote>
<p>Anyone with the solution. Kindly reply. Thanks in advance.</p>

vdw: <p>What are you trying to do in the first place, i.e., what kind of task are you trying to learn.</p>
<p>I’m just asking since it seems odd that <code>batch["target"][0]</code> is the  <code>input</code> parameter of the forward() method of the decoder.</p>

Ninlawat_Phuangchoke: <p>I try to do seq2seq model with text as input and numerical data that is output. I prepare data text to word index to pass through encoder and numerical data for decoder</p>
<pre><code class="lang-auto">batch_size = 3
train_data_loader = get_data_loader(train_data, batch_size, pad_index, shuffle=True)
</code></pre>
<p>I just run to <code>batch["numericalized_input"], batch["target"] </code> to debug the program</p>
<pre><code class="lang-auto">for i, batch in enumerate(train_data_loader):
    print('i', i)
    src = batch["numericalized_input"]
    trg = batch["target"]
    print('src:',src)
    print('trg:',trg)
</code></pre>
<p>Output</p>
<blockquote>
<p>i 0<br>
src: tensor([[ 2,  2,  2],<br>
[ 4,  4,  4],<br>
[ 0,  0,  0],<br>
[ 5,  5,  5],<br>
[ 4,  4,  4],<br>
[ 0,  0,  0],<br>
[ 0,  0,  6],<br>
[18,  3,  0],<br>
[ 4,  1,  0],<br>
[23,  1,  4],<br>
[ 6,  1, 11],<br>
[ 5,  1,  0],<br>
[ 0,  1,  7],<br>
[ 9,  1,  0],<br>
[ 0,  1,  7],<br>
[ 0,  1,  6],<br>
[21,  1, 13],<br>
[ 7,  1,  4],<br>
[20,  1, 16],<br>
[ 8,  1,  8],<br>
[ 4,  1, 17],<br>
[11,  1,  8],<br>
[ 3,  1,  4],<br>
[ 1,  1, 15],<br>
[ 1,  1, 18],<br>
[ 1,  1,  4],<br>
[ 1,  1, 14],<br>
[ 1,  1,  3]])<br>
trg: tensor([[ 2,  2,  2],<br>
[ 9, 15,  7],<br>
[ 6,  5,  8],<br>
[ 7,  7,  9],<br>
[22,  6, 22],<br>
[ 5,  9,  5],<br>
[15,  8, 15],<br>
[15,  9, 15],<br>
[22,  8, 22],<br>
[33,  9, 33],<br>
[ 3,  3,  3]])<br>
i 1<br>
src: tensor([[ 2,  2,  2],<br>
[ 9, 19,  6],<br>
[ 0,  5,  5],<br>
[ 4, 12,  0],<br>
[11,  0,  9],<br>
[24,  3,  4],<br>
[ 0,  1,  0],<br>
[ 0,  1,  7],<br>
[22,  1,  6],<br>
[ 0,  1,  5],<br>
[10,  1,  0],<br>
[ 0,  1,  0],<br>
[ 4,  1,  0],<br>
[16,  1,  4],<br>
[ 8,  1, 15],<br>
[17,  1, 21],<br>
[ 0,  1,  7],<br>
[ 6,  1, 20],<br>
[ 0,  1,  0],<br>
[ 0,  1, 24],<br>
[10,  1,  0],<br>
[ 0,  1,  7],<br>
[ 4,  1,  0],<br>
[23,  1, 12],<br>
[ 3,  1,  0],<br>
[ 1,  1,  3]])<br>
trg: tensor([[ 2,  2,  2],<br>
[ 8, 22,  9],<br>
[ 9,  5,  5],<br>
[ 4,  8, 33],<br>
[22, 22,  7],<br>
[ 5,  5,  8],<br>
[15, 15,  9],<br>
[15, 15,  7],<br>
[22, 22,  8],<br>
[33, 33,  9],<br>
[ 3,  3,  3]])</p>
</blockquote>