akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Time series data preparation for LSTM

Shafaat_Sheikh: <p>Hi, I have time series data in form of 200 mat files.  I need to perform binary classification of these mat files. The 200 labels for these mat files are stored in a csv file.</p>
<p>Can somebody please guide me how to prepare dataset  and dataloader in this case .<br>
Also I need to do stratified sampling of test, train and validation set due to imbalance data.</p>
<p>Regards<br>
shafaat</p>
Using scheduler lead to out of memory

clearlovewl: <p>my gpu memory is increasing in circulation of define scheduler</p>
<pre><code class="lang-auto">from torch.optim import lr_scheduler
import torch.nn as nn
import torch
class network(torch.nn.Module):
    def __init__(self):
        nn.Module.__init__(self)
        self.layer=nn.Sequential(
            nn.Linear(4096, 2048),
            nn.ReLU(),
            nn.Linear(2048, 1024),
            nn.ReLU(),
            nn.Linear(1024, 512),
            nn.ReLU(),
        )
    def forward(self, ftr):
        pass
device=torch.device('cuda:0')


for i in range(100**100):

    net = network().to(device)
    optimizer = torch.optim.SGD(net.parameters(), lr=0.01, momentum=0.9,
                                weight_decay=0.9)
    scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.9)

    for epoch in range(2):
        optimizer.step()
        scheduler.step()


</code></pre>

clearlovewl: <p>The code which i use actually is more complex. This simple one is still have the problem. The first time when i use lr_scheduler function ,my gpu memory is low. While As the iteration goes on, this code need more gpu memory. I don’t know why it would be like this, but when a del the lr_scheduler, the increasing of gpu memory disappear. Is there somebody watch this quesion? I really need help.</p>

ptrblck: <p>Thanks for the executable code snippet!<br>
I failed to reproduce the issue with <code>1.4.0.dev20191109</code>, as my memory usage stays constant.<br>
Which PyTorch version are you using?</p>

clearlovewl: <p>i use pytorch 1.2.0 ,which was download from the <a href="https://pytorch.org/" rel="nofollow noopener">https://pytorch.org/</a></p>

clearlovewl: <p>This code may be more clear</p>
<pre><code class="lang-auto">from torch.optim import lr_scheduler
import torch.nn as nn
import torch
class network(torch.nn.Module):
    def __init__(self):
        nn.Module.__init__(self)
        self.layer=nn.Sequential(
            nn.Linear(4096, 2048),
            nn.ReLU(),
            nn.Linear(2048, 1024),
            nn.ReLU(),
            nn.Linear(1024, 512),
            nn.ReLU(),
        )
    def forward(self, ftr):
        pass
device=torch.device('cuda:0')


for i in range(100**100):
    print(i)
    net = network().to(device)
    optimizer = torch.optim.SGD(net.parameters(), lr=0.01, momentum=0.9,
                                weight_decay=0.9)
    scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.9)

    for epoch in range(1000*100):

        optimizer.step()
        scheduler.step()


</code></pre>

ptrblck: <p>Could you update to the latest stable release (<code>1.3.1</code>) and rerun the code again?</p>

clearlovewl: <p>yes, it is solved. Thank you  very much.</p>
(==) operation gives wrong result for same metrics when using torch.bmm

Fawaz_Sammani: <p>Hi,<br>
I need to detect equality in my tensors. I am using two equivalent ways to multiply 4D metrics, which is element-wise then sum, and torch.bmm. The results are the same, but when checking with the ("==<br>
") operator, they dont seem to be.</p>
<pre><code class="lang-auto">import torch
a = torch.rand(3,2,100)
b = torch.randn(3,4,2)
a_expanded = a.unsqueeze(1).expand(-1, b.shape[1], -1, -1)
mul1 = (b.unsqueeze(3) * a_expanded).sum(dim = 2)
mul2 = torch.bmm(b, a)
</code></pre>
<p>Now, if we do: <code>mul1==mul2</code><br>
The result is:</p>
<pre><code class="lang-auto">tensor([[[1, 1, 0,  ..., 1, 1, 1],
         [0, 0, 1,  ..., 1, 1, 1],
         [0, 1, 1,  ..., 1, 1, 1],
         [1, 1, 1,  ..., 1, 1, 1]],

        [[0, 1, 0,  ..., 1, 1, 1],
         [1, 0, 1,  ..., 1, 1, 1],
         [0, 0, 1,  ..., 1, 1, 1],
         [1, 0, 0,  ..., 1, 1, 1]],

        [[1, 1, 0,  ..., 1, 1, 1],
         [1, 0, 1,  ..., 1, 1, 1],
         [1, 1, 1,  ..., 1, 1, 1],
         [1, 1, 1,  ..., 1, 1, 1]]], dtype=torch.uint8)
</code></pre>
<p>Examining the elements where it gave 0:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/1/410c0c83729cd7c8c0be55cad9092877575c478f.png" data-download-href="https://discuss.pytorch.org/uploads/default/410c0c83729cd7c8c0be55cad9092877575c478f" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/1/410c0c83729cd7c8c0be55cad9092877575c478f.png" alt="Capture" data-base62-sha1="9hqQI7oidlBbUwCKuUpXg79xT2f" width="502" height="500" data-dominant-color="F0F0EE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">728×724 25.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>They are same. What is wrong?</p>

ptrblck: <p>Due to floating point precision you might get small errors (usually at ~1e-6), so I would suggest to compare the results with <code>torch.allclose</code> (and specifying the tolerance manually, if necessary).<br>
A direct comparison for floating point numbers is often not the best idea.</p>