akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Is there a toy task that requires forward output [batch, time_steps, output_dim] in the model using lstm?

Keiku: <p>It’s a task that has a label for each time_steps for a batch and I have to calculate the loss for that label. I previously asked about the case that the action recognition task is labeled per frame instead of per video, but I didn’t get any answers. If anyone knows a toy task related to the above, please let me know.</p>
<aside class="quote quote-modified" data-post="1" data-topic="116138">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/keiku/48/36340_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-can-i-train-action-recognition-for-frame-by-frame-of-video-using-pytorch/116138">How can I train action recognition for frame by frame of video using pytorch?</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    I am referring to <a href="https://github.com/eriklindernoren/Action-Recognition" rel="noopener nofollow ugc">eriklindernoren/Action-Recognition</a> now. The <a href="https://www.crcv.ucf.edu/data/UCF101.php" rel="noopener nofollow ugc">UCF101</a> dataset has “one action” for each video. Therefore, there is no problem with the following model output. 
        # output_layers of models.py
        self.output_layers = nn.Sequential(
            nn.Linear(2 * hidden_dim if bidirectional else hidden_dim, hidden_dim),
            nn.BatchNorm1d(hidden_dim, momentum=0.01),
            nn.ReLU(),
            nn.Linear(hidden_dim, num_classes),
            nn.Softmax(dim=-1),
  …
  </blockquote>
</aside>


omarfoq: <p>Hi,</p>
<p>What do you exactly mean by a toy task? a simple toy example where output is of shape <code>[batch, time_steps, output_dim]</code> is language modeling. Check this <a href="https://towardsdatascience.com/writing-like-shakespeare-with-machine-learning-in-pytorch-d77f851d910c" rel="noopener nofollow ugc">link</a> for a simple example.</p>
Why Loss is not decreasing and remains constant in Relu () but linear () working fine

bc060400164: <p>train_window = 24</p>
<p>def create_sequences(input_data, length_of_sequence):<br>
sequence = []<br>
L = len(input_data)<br>
for i in range(L-length_of_sequence):<br>
train_seq = input_data[i:i+length_of_sequence]<br>
train_label = input_data[i+length_of_sequence:i+length_of_sequence+1]<br>
sequence.append((train_seq, train_label))<br>
return sequence</p>
<p>class LSTM(nn.Module):<br>
def <strong>init</strong>(self, input_size=1, hidden_layer_size=40, output_size=1):<br>
super().<strong>init</strong>()</p>
<pre><code>self.hidden_layer_size = hidden_layer_size
self.lstm = nn.LSTM(input_size, hidden_layer_size)
self.relu = nn.ReLU()     

self.hidden_cell = (torch.zeros(1, 1, self.hidden_layer_size),    
                    torch.zeros(1, 1, self.hidden_layer_size))
</code></pre>
<p>def forward(self, input_seq):<br>
lstm_out, self.hidden_cell = self.lstm(input_seq.view(len(input_seq), 1, -1), self.hidden_cell)<br>
predictions = self.relu(lstm_out.view(len(input_seq), -1))<br>
return predictions[-1]</p>
<p>model = LSTM()<br>
loss_function = nn.MSELoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr = 0.01)</p>
<p>epochs = 500</p>
<p>for i in range(epochs):<br>
for seq, labels in train_inout_seq:<br>
optimizer.zero_grad()<br>
model.hidden_cell = (torch.zeros(1, 1, model.hidden_layer_size),<br>
torch.zeros(1, 1, model.hidden_layer_size))<br>
y_pred = model(seq)</p>
<pre><code>single_loss = loss_function(y_pred, labels)
single_loss.backward()
optimizer.step()
</code></pre>
<p>if i%25 == 1:<br>
print(f’epoch: {i:3} loss: {single_loss.item():10.8f}’)</p>
<p>print(f’epoch: {i:3} loss: {single_loss.item():10.10f}’)</p>
<p>and the result is coming loss is not decreasing any guidline?:</p>
<p>epoch:   1 loss: 0.72129285<br>
epoch:  26 loss: 0.72129285</p>
Libtorch,torch::sigmoid() function return value can changle channel of tensor

yzqxmuex: <p>libtorch:I use torch::sigmoid() function with [1,1,h,w] shape parameter,it return [1,3,h,w]?why??</p>

ptrblck: <p>This shouldn’t be the case.<br>
Could you post a code snippet to reproduce this issue, please?</p>