Different Between LSTM and LSTMCell Function

herleeyandi: <p>Hello I am still confuse what is the different between function of LSTM and LSTMCell. I have read the documentation however I can not visualize it in my mind the different between 2 of them. Suppose I want to creating this network in the picture.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/5/5c5b08073bfae7f001f75674a4d27acd2671c75f.png" width="408" height="477"><br>
Suppose green cell is the LSTM cell and I want to make it with <code>depth=3</code>, <code>seq_len=7</code>, <code>input_size=3</code>. Red cell is input and blue cell is output. What function should I use for this and how to use it?, Suppose I just want the output of the last sequence can I just using <code>out[:, -1, :]</code> ?<br>
-Thank you-</p>

moskomule: <p>Hi. As far as I understand your figure, the red box below is the <code>Cell</code>.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/2X/f/fba1890185c48183e1cf8eb91086fc0f1b6df322.png" width="408" height="477"></p>
<p>That is the reason Cell return</p>
<blockquote></blockquote>
<pre><code>    h_1 (batch, hidden_size): tensor containing the next hidden state for each element in the batch
    c_1 (batch, hidden_size): tensor containing the next cell state for each element in the batch
</code></pre>

herleeyandi: <p>Yes as I mention before suppose that <code>green is LSTM</code>, <code>red is input</code>, and <code>blue is output</code>. How can you use function of <code>LSTM</code> and <code>LSTMCell</code> which available in Pytorch and what is their difference if we have visualization like in the figure.</p>

moskomule: <p><code>LSTMCell</code> is more flexible and you need less code with <code>LSTM</code> .</p>
<p>So with <code>LSTMCell</code>,</p>
<pre><code class="lang-auto">def forward(self, x):
        h = self.get_hidden() 
        for input in x:  
            h = self.rnn(input, h) # self.rnn = self.LSTMCell(input_size, hidden_size)
</code></pre>
<p>while with <code>LSTM</code> it is</p>
<pre><code class="lang-auto">def forward(self, x):
        h_0 = self.get_hidden()
        output, h = self.rnn(x, h_0) # self.rnn = self.LSTM(input_size, hidden_size)
</code></pre>
<p><code>output</code> is the blue rectangles in your fig.</p>

analvikingur: <p>Be aware of using various RNNCells instead of RNN/GRU/LSTM models itself, since Cells don’t have cuDNN optimisation and thus don’t have any fused operations, etc.</p>

Intel_Novel: <p><a href="https://discuss.pytorch.org/u/analvikingur">analvikingur</a>: Are you trying to say that LSTMCell will not work well on GPU, where the LSMT class will work fine with GPU?</p>

moskomule: <p>RNN/LSTM/GRU is boosted if cudnn is available.</p>
<p>Maybe, torch.jit will be ease the difference someday.</p>

moskomule: <p>Just FYI <a href="https://github.com/pytorch/benchmark/tree/master/rnns" rel="nofollow noopener">https://github.com/pytorch/benchmark/tree/master/rnns</a> includes JIT boosted RNNs</p>

ThisuriCham: <p><a class="mention" href="/u/moskomule">@moskomule</a> is that mean LSTMCell and LSTM do the same task? no difference in performanc. right? do the same task? only difference is in code?</p>

ThisuriCham: <p>I think the expected answer is in this. [<a href="https://discuss.pytorch.org/t/lstm-and-lstmcell/7488" class="inline-onebox">LSTM and LSTMCell ？</a>]</p>

Carlos_Jose_Gonzalez: <p>I made this example and hope it helps <a href="https://github.com/CarlosJose126/RNNs/blob/main/LSTMCellvsLSTM.ipynb" class="inline-onebox" rel="noopener nofollow ugc">RNNs/LSTMCellvsLSTM.ipynb at main · CarlosJose126/RNNs · GitHub</a></p>