Confusing with the implementation of Many to Many LSTM

username: Peterwisu
cooked: <p>Hi, I am very new to RNN and LSTM networks. I have tried implementing some of the One to Many models with RNN and LSTM and trying to implement a Many to Many  LSTM network like in this picture<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/6/06f80713eaa5ba5c577b45568b361978afbd2978.png" alt="Screenshot from 2022-12-10 04-52-31" data-base62-sha1="ZEgdCh9PRj5hCC5HrFsRZJboco" width="252" height="414"></p>
<p>but most tutorials are many to one or if it is Many to Many it gonna a tutorial for NLP which is very different from what I am expecting. Can anyone give me an example of this?</p>
<p>And also I found this code from the internet about using LSTM in time series tasks.</p>
<pre><code class="lang-auto">class LSTM(nn.Module):
    def __init__(self, hidden_layers=64):
        super(LSTM, self).__init__()
        self.hidden_layers = hidden_layers
        # lstm1, lstm2, linear are all layers in the network
        self.lstm1 = nn.LSTMCell(1, self.hidden_layers)
        self.lstm2 = nn.LSTMCell(self.hidden_layers, self.hidden_layers)
        self.linear = nn.Linear(self.hidden_layers, 1)
        
    def forward(self, y, future_preds=0):
        outputs, num_samples = [], y.size(0)
        h_t = torch.zeros(n_samples, self.hidden_layers, dtype=torch.float32)
        c_t = torch.zeros(n_samples, self.hidden_layers, dtype=torch.float32)
        h_t2 = torch.zeros(n_samples, self.hidden_layers, dtype=torch.float32)
        c_t2 = torch.zeros(n_samples, self.hidden_layers, dtype=torch.float32)
        
        for time_step in y.split(1, dim=1):
            # N, 1
            h_t, c_t = self.lstm1(input_t, (h_t, c_t)) # initial hidden and cell states
            h_t2, c_t2 = self.lstm2(h_t, (h_t2, c_t2)) # new hidden and cell states
            output = self.linear(h_t2) # output from the last FC layer
            outputs.append(output)
            
        for i in range(future_preds):
            # this only generates future predictions if we pass in future_preds&gt;0
            # mirrors the code above, using last output/prediction as input
            h_t, c_t = self.lstm1(output, (h_t, c_t))
            h_t2, c_t2 = self.lstm2(h_t, (h_t2, c_t2))
            output = self.linear(h_t2)
            outputs.append(output)
        # transform list to tensor    
        outputs = torch.cat(outputs, dim=1)
        return outputs
</code></pre>
<p>Does this code do the Many to Many tasks like in the first pictures and why in the code there are 2 LSTM layers (lstm1 and lstm2) Thank you.</p>

username: vdw
cooked: <p>There are two types of many-to-many models:</p>
<ul>
<li>
<p>Sequence labeling (e.g., POS tagging NER): Here you predict some tag or label for each item in your sequence</p>
</li>
<li>
<p>Sequence translation (e.g., machine translation): Here you encode your input sequence and use this internal representation to condition a decoder that generates the target sequence. This is what the code block you included is doing</p>
</li>
</ul>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/3/f3cee3a3e8b310dd1c2b5d46c121dd41031b2468.png" data-download-href="https://discuss.pytorch.org/uploads/default/f3cee3a3e8b310dd1c2b5d46c121dd41031b2468" title="Screenshot from 2022-12-10 14-25-59"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/3/f3cee3a3e8b310dd1c2b5d46c121dd41031b2468_2_690x389.png" alt="Screenshot from 2022-12-10 14-25-59" data-base62-sha1="yMPhzuTeIZPIpFKxNwKWmW0En4k" width="690" height="389" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/3/f3cee3a3e8b310dd1c2b5d46c121dd41031b2468_2_690x389.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/3/f3cee3a3e8b310dd1c2b5d46c121dd41031b2468_2_1035x583.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/3/f3cee3a3e8b310dd1c2b5d46c121dd41031b2468.png 2x" data-dominant-color="E7ECF3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-12-10 14-25-59</span><span class="informations">1064×601 45.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/7/b78ecb5141be1ad00a602c4d4ba3070c9dbef6d6.png" data-download-href="https://discuss.pytorch.org/uploads/default/b78ecb5141be1ad00a602c4d4ba3070c9dbef6d6" title="Screenshot from 2022-12-10 14-29-50"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/7/b78ecb5141be1ad00a602c4d4ba3070c9dbef6d6_2_690x389.png" alt="Screenshot from 2022-12-10 14-29-50" data-base62-sha1="qbPjmnrRPA8M8lD9vegcX9KcVjU" width="690" height="389" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/7/b78ecb5141be1ad00a602c4d4ba3070c9dbef6d6_2_690x389.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/7/b78ecb5141be1ad00a602c4d4ba3070c9dbef6d6_2_1035x583.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/7/b78ecb5141be1ad00a602c4d4ba3070c9dbef6d6.png 2x" data-dominant-color="E9F0F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-12-10 14-29-50</span><span class="informations">1064×601 39.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: GurjeetSinghSangra
cooked: <p>If your problem is not very high-dimensional, you can frame the one-to-many as many-to-many by repeating the input tensor by the sequence length.</p>
<p>For example using <code>torch.nn.LSTM</code></p>
<pre><code class="lang-auto">lstm = torch.nn.LSTM(
            input_size=self.x_dim,
            hidden_size=self.out_dim,
            num_layers=lstm_layers,
            batch_first=True,
        )
out = x.unsqueeze(1).repeat(1, seq_len, 1) # (n_batches, L, H_in)
out, _ = self.lstm(out)
</code></pre>