Correct way to feed data to RNN in PyTorch

username: learner47
cooked: <p>I have a data sequence <code>a</code> which is of shape <code>[seq_len, 2]</code>, <code>seq_len</code> is the length of the sequence. There is time correlation among elements of <code>a[:, 0]</code> and <code>a[:, 1]</code>, but <code>a[:, 0]</code> and <code>a[:, 1]</code> are independent of each other. For training I prepare data of shape <code>[batch_size, seq_len, 2]</code>. The initialization of BRNN that I use is</p>
<pre><code class="lang-auto">birnn_layer = nn.RNN(input_size=2, hidden_size=100, batch_first=True, bidirectional=True)
</code></pre>
<p>From the <a href="https://pytorch.org/docs/stable/generated/torch.nn.RNN.html#torch.nn.RNN" rel="noopener nofollow ugc">docs</a>,</p>
<blockquote>
<p>input_size – The number of expected features in the input x</p>
</blockquote>
<blockquote>
<p>hidden_size – The number of features in the hidden state h</p>
</blockquote>
<p>What does “number of expected features” mean? Since there is correlation along the <code>seq_len</code> axis should my <code>input_size</code> be set as <code>seq_len</code> and the input be permuted? Thanks.</p>

username: suraj.pt
cooked: <p>Hi,</p>
<p><code>input_size</code> or “no. of expected features” denotes the dimensionality of each observation; in this case, 2.</p>
<p>Also, your input to <code>nn.RNN</code>  should be in the shape of <code>[seq_len, batch_size, input_size]</code>. At every timestep, the RNN receives a <code>[t, :, :]</code> matrix that contains all the observations at timestep <em>t</em> from all the batches.</p>

username: learner47
cooked: <p>Hi,</p>
<p>I have mentioned <code>batch_size=True</code> as one of the parameters, so the <code>batch_size</code> dimension should come first right?</p>

username: suraj.pt
cooked: <p>Oh, I didn’t see the arg - yes that’s correct.</p>

username: learner47
cooked: <aside class="quote no-group" data-username="suraj.pt" data-post="2" data-topic="99204">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/suraj.pt/48/53186_2.png" class="avatar"> suraj.pt:</div>
<blockquote>
<p>the RNN receives a <code>[t, :, :]</code> matrix that contains all the observations at timestep <em>t</em> from all the batches.</p>
</blockquote>
</aside>
<p>Can you please elaborate on this?</p>

username: suraj.pt
cooked: <p>Let’s say your  <code>batch_size=5</code> and <code>seq_len=3</code>. So each batch looks like</p>
<pre><code class="lang-auto">batch = [
        [x1_1, x1_2, x1_3],
        [x2_1, x2_2, x2_3],
        ...
        [x5_1, x5_2, x5_3]
       ] # shape (batch_size, seq_len, input_size) 

where x{seq_id}_{timestep}
</code></pre>
<p>For a given timestep <em>t</em>, the RNN reads the <em>t</em>-th observations from all the sequences in the batch. So at timestep <code>t</code>, it will read and process <code>input_t</code></p>
<pre><code class="lang-auto">input_1 = [x1_1, x2_1..., x5_1] # shape (batch_size, input_size)
input_2 = [x1_2, x2_2..., x5_2]
input_3 = [x1_3, x2_3..., x5_3]
</code></pre>
<p>However, you don’t actually need to split your data into time-stepped inputs like this, the RNN just needs</p>
<pre><code class="lang-auto">rnn_input = [input_1, input_2, input_3] # (seq_len, batch_size, input_size)
</code></pre>
<p>from you.</p>
<p>Hope this helps!</p>

username: learner47
cooked: <p>I have one last question. In the above example what would be the shape of <code>data</code>? Thank you so much for your effort</p>

username: suraj.pt
cooked: <p>edited my answer, hopefully this is clearer</p>

username: learner47
cooked: <p>Perfect thanks. This helped.</p>