Could someone explain batch_first=True in LSTM

n00b: <p>I can’t figure out how it works. I try to change bs,time step, input size with batch_first = True or no batch_first. It return the dimension that I feed to model. so I have to change it manually ??</p>

jpeg729: <p>If your input data is of shape <code>(seq_len, batch_size, features)</code> then you don’t need <code>batch_first=True</code> and your LSTM will give output of shape <code>(seq_len, batch_size, hidden_size)</code>.</p>
<p>If your input data is of shape <code>(batch_size, seq_len, features)</code> then you need <code>batch_first=True</code> and your LSTM will give output of shape <code>(batch_size, seq_len, hidden_size)</code>.</p>
<p>Is that not what you expect?</p>

n00b: <p>but when my input’s shape is <code>(batch_size, seq_len, features)</code> without <code>batch_first=True</code> the output still as same as the input that why i had a little doubt</p>

n00b: <pre><code class="lang-auto">no batch_first=True
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/9/9d4cf8b7510deb0f7c3c24cc87e9678e2dc24177.PNG" data-download-href="https://discuss.pytorch.org/uploads/default/9d4cf8b7510deb0f7c3c24cc87e9678e2dc24177" title="no%20batch_first"><img src="https://discuss.pytorch.org/uploads/default/original/2X/9/9d4cf8b7510deb0f7c3c24cc87e9678e2dc24177.PNG" alt="no%20batch_first" data-base62-sha1="mrxSagP9gocv5CYjVsRLDAEUv1Z" width="690" height="462" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">no%20batch_first</span><span class="informations">718×481 17 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<a href="https://i.imgur.com/51SRQni.png" rel="noopener nofollow ugc">https://i.imgur.com/51SRQni.png</a></p>
<pre><code class="lang-auto">had batch_first=True
</code></pre>
<p><img src="https://discuss.pytorch.org/uploads/default/original/2X/a/a274028a2179e6599cc9ca36847b672d4eacd4b9.PNG" alt="had%20batch_first" data-base62-sha1="nb7TOPA5WAbHjh83TpEvxywMJFn" width="670" height="487"><br>
<a href="https://i.imgur.com/PI0Excd.png" rel="noopener nofollow ugc">https://i.imgur.com/PI0Excd.png</a></p>

jpeg729: <p>If you feed it input of shape <code>(10, 20, in_features)</code> then the output will always be of shape <code>(10, 20, hidden_size)</code></p>
<p>Without <code>batch_first=True</code> it will use the first dimension as the sequence dimension.<br>
With <code>batch_first=True</code> it will use the second dimension as the sequence dimension.</p>

n00b: <p>It does not work the result should become 3x2</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/e3f4f19ec52d29340086a88e17168b83826e3d3a.PNG" data-download-href="https://discuss.pytorch.org/uploads/default/e3f4f19ec52d29340086a88e17168b83826e3d3a" title="not%20work%20batch_first"><img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e3f4f19ec52d29340086a88e17168b83826e3d3a.PNG" alt="not%20work%20batch_first" data-base62-sha1="wwBaSIwP86F3OjRTr1Foa9gqMEW" width="543" height="500" data-dominant-color="F4F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">not%20work%20batch_first</span><span class="informations">585×538 16.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I really appreciate that your reply to all my topic thanks alot</p>

jpeg729: <p>When you do</p>
<pre><code class="lang-auto">print(out[-1])
</code></pre>
<p>you are taking the last element of the <em>batch</em> dimension.</p>
<p>You probably wanted to do</p>
<pre><code class="lang-auto">print(out[:, -1])</code></pre>

vdw: <p>Note that both version are only meaningful when <code>bidirectional=False</code>.</p>

edwardpwtsoi: <p>Hi <a class="mention" href="/u/vdw">@vdw</a> Chris, would you mind to explain a bit more why this doesn’t matter if bidirectional is True. I recently found out that my model was trained with batch first sequence but the bilstm is set to batch_first = False. And surprisingly, the model still converge. I wish I could to correct it without retrain the model.</p>

vdw: <p>The  line in the <code>forward()</code> method is</p>
<pre><code>out, _ = self.lstm(x)
</code></pre>
<p>So</p>
<pre><code class="lang-auto">out[-1]    # If batch_first=True OR
out[:, -1] # If batch_dirst=False
</code></pre>
<p>will give you the hidden state after the <strong>LAST hidden state with respect to the forward pass</strong> but the <strong>FIRST hidden state with respect to the backward pass</strong>; see <a href="https://discuss.pytorch.org/t/rnn-output-vs-hidden-state-dont-match-up-my-misunderstanding/43280">this old post of mine</a>. What you want is also the last hidden state with respect to the backward pass. This one cna be found in <code>out[0]</code> or <code>out[:, 0]</code> respectively.</p>
<p>Your network will still learn but essentially making only meaningful use of the forward pass.</p>
<p>The cleanest way is the following</p>
<pre><code>out, (h, c) = self.lstm(x)
# Split num_layers and num_directions (1 or 2)
h = h.view(self.num_layers, self.num_directions, batch_size, self.rnn_hidden_dim)
# Last hidden state w.r.t. to then number of layers, NOT the time steps
h = h[-1]   
# Get last hidden states of backward and forward passes
h_forward, h_backward = h[0], h[1]
</code></pre>
<p>Then you can add or concatenate <code>h_foward</code> and <code>h_backward</code> for further layers.</p>