akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
UCF101 training pauses periodically - bottleneck

username: Dhruvan_Ganesh
cooked: <pre><code class="lang-auto">num_cpu=12
train_dataset = torchvision.datasets.UCF101(
                UCF101_ROOT_PATH,
                UCF101_ANNO_PATH,
                frames_per_clip=12,
                step_between_clips=100,
                num_workers=num_cpu,
                train=True,
                transform=train_transforms,
                fold=self.fold,
            )

dataloader = DataLoader(
            self.train_dataset,
            batch_size=self.batch_size,
            num_workers=num_cpu,
            collate_fn=custom_collate,
            shuffle=True,
            pin_memory=True,
        )
</code></pre>
<p>Training freezes every 50 epochs, one CPU core is active 100% (but I have 12 cores), and GPU-util is 0% on nvidia-smi.<br>
After some time training resumes. This behavior repeats every 50 epochs.</p>

username: Dexter
cooked: <p>Did you explicitly moved the data to the cuda?</p>

username: Dhruvan_Ganesh
cooked: <p>I use Pytorch Lightning and it takes care of moving data to the CUDA device</p>

username: Dexter
cooked: <p>Well, I don’t have any experience in using the lightning version of PyTorch but seems like a problem is occurring while transferring your data to your GPU.  Is your network architecture too big? if so try a smaller one and see if the same problem occurs or not.</p>

username: Dhruvan_Ganesh
cooked: <p>It was as you said there was a lot of data moved between devices. The culprit was pytorch lightning’s LightningDataModule.</p>

username: dangne
cooked: <p>I am having the same problem. Any solution to this?</p>
Is Pytorch able to grab data from GPU calculated from Cuda C directly

username: Emerald01
cooked: <p>Hi all,</p>
<p>I have a data processor that is written by Cuda C, and I used PyCuda as the API to call this. So essentially, the kernel creates the data at GPU</p>
<pre><code class="lang-auto">  __global__ void create_data(float* data)
  { 
   // process the data using Cuda kernel
 }
</code></pre>
<p>the data will be processed within the Cuda kernel function, and finally update this <code>float* data</code> which is sitting, I guess, in the global memory at GPU. Now, the straightforward way is I can copy this data back to CPU host, this is easy by Cuda. Then I call Pytorch to send data in CPU again back to GPU using like data.to(“cuda”), this is also easy.</p>
<p>But obviously this is redundant if Pytorch has some API that directly access to the <code>float* data</code> in GPU memory, so the program does not need to do this unnecessary GPU-&gt;CPU-&gt;GPU data transfer</p>

username: tom
cooked: <p>If you have a CUDA tensor <code>t</code> in PyTorch <code>t.data_ptr&lt;float&gt;()</code> will give you the <code>float*</code> to the tensor’s memory. You would have to deal with sizes, strides etc. yourself.</p>
<p>There also is <code>from_blob</code> to make a tensor pointing to a given memory blob, but <em>you have to guarantee that the pointer will stay valid for the tensor’s lifetime</em> (until the deleter callback that you can pass is called by PyTorch to signal that it’s done using it). TensorOptions allow you to specify that it is a GPU tensor and the dtype etc.</p>
<p>Best regards</p>
<p>Thomas</p>
Setting weight to the NLLLoss for imbalanced data

username: tyron
cooked: <p>I have 2 imbalanced classes with the size 3089 and 6179. How I can to weight the loss function in order to penalize errors on the smaller class. I am using nn.NLLLoss()</p>

username: Aman_Singh
cooked: <p><code>WeightedRandomSampler</code> might be helpful to you.<br>
Here’s a discussion about it.</p>
<aside class="quote quote-modified" data-post="10" data-topic="2703">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/balanced-sampling-between-classes-with-torchvision-dataloader/2703/10">Balanced Sampling between classes with torchvision DataLoader</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    Ok, I found some other issues. I changed your code a bit. This should work: 
numDataPoints = 1000
data_dim = 5
bs = 100

data = torch.FloatTensor(numDataPoints, data_dim)
target = np.hstack((np.zeros(int(numDataPoints * 0.9), dtype=np.int32),
                    np.ones(int(numDataPoints * 0.1), dtype=np.int32)))

dataset_x = data.numpy()
dataset_y = target
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(dataset_x,
                        …
  </blockquote>
</aside>


username: KFrank
cooked: <p>Hi Liu!</p>
<aside class="quote no-group" data-username="tyron" data-post="1" data-topic="116147" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/b38774/48.png" class="avatar"> tyron:</div>
<blockquote>
<p>How I can to weight the loss function in order to penalize errors on the smaller class. I am using nn.NLLLoss()</p>
</blockquote>
</aside>
<p>You can use the <code>weight</code> argument that is passed to <code>NLLLoss</code>’s<br>
constructor.</p>
<p>As an aside, if you do, in fact, have only two classes, you could<br>
treat your problem as a binary classification problem (rather than<br>
a general multi-class problem that happens to have two classes),<br>
use <code>BCEWithLogitsLoss</code>, and use its <code>pos_weight</code> argument.</p>
<p>(Aman’s <code>WeightedRandomSampler</code> suggestion is also reasonable.<br>
It and adding weights to the loss function are two somewhat different<br>
approaches for compensating for imbalanced data.)</p>
<p>Best.</p>
<p>K. Frank</p>
cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

username: spadel
cooked: <p>I receive the following error:</p>
<pre><code class="lang-auto">    filtered_y_y = F.conv2d(y_c, sobel_y_filter, stride=1, padding=1)
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED
</code></pre>
<p>The problem occurs, when I apply a sobel filter to a 2d feature map and run it on cuda. The code looks as follows:</p>
<pre><code class="lang-auto">def AbsDetJacobian(batch_meshgrid, device):
    """
        :param batch_meshgrid: takes meshgrid tensor of dim [bn, 2, h, w] (conceptually meshgrid represents a two dimensional function f = [fx, fy] on [bn, h, w] )
        :return: returns Abs det of  Jacobian of f of dim [bn, 1, h, w]
        """
    y_c = batch_meshgrid[:, 0, :, :].unsqueeze(1)
    x_c = batch_meshgrid[:, 1, :, :].unsqueeze(1)
    sobel_x_filter = 1 / 4 * torch.tensor([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]).reshape(1, 1, 3, 3).to(device)
    sobel_y_filter = sobel_x_filter.permute(0, 1, 3, 2).to(device)


    filtered_y_y = F.conv2d(y_c, sobel_y_filter, stride=1, padding=1)
    filtered_y_x = F.conv2d(y_c, sobel_x_filter, stride=1, padding=1)
    filtered_x_y = F.conv2d(x_c, sobel_y_filter, stride=1, padding=1)
    filtered_x_x = F.conv2d(x_c, sobel_x_filter, stride=1, padding=1)

    Det = torch.abs(filtered_y_y * filtered_x_x - filtered_y_x * filtered_x_y)

    return Det
</code></pre>
<p>Does anyone have an idea what’s going on here?</p>

username: ptrblck
cooked: <p>I cannot reproduce this issue using this code snippet:</p>
<pre><code class="lang-python">device = 'cuda'
batch_meshgrid = torch.empty(4, 2, 24, 24).uniform_(-1, 1).to(device)

y_c = batch_meshgrid[:, 0, :, :].unsqueeze(1)
x_c = batch_meshgrid[:, 1, :, :].unsqueeze(1)
sobel_x_filter = 1 / 4 * torch.tensor([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]).reshape(1, 1, 3, 3).to(device)
sobel_y_filter = sobel_x_filter.permute(0, 1, 3, 2).to(device)


filtered_y_y = F.conv2d(y_c, sobel_y_filter, stride=1, padding=1)
filtered_y_x = F.conv2d(y_c, sobel_x_filter, stride=1, padding=1)
filtered_x_y = F.conv2d(x_c, sobel_y_filter, stride=1, padding=1)
filtered_x_x = F.conv2d(x_c, sobel_x_filter, stride=1, padding=1)

Det = torch.abs(filtered_y_y * filtered_x_x - filtered_y_x * filtered_x_y)
print(Det.shape)
</code></pre>
<p>Could you post the shapes you’ve used to run into this issue?</p>

username: Vincent_Li
cooked: <p>Have you solved the issue? If so, would you mind posting the solution? Thank you</p>
In forward process,i need loops multiple times. How can Backward update the only once loop

username: Haoxl
cooked: <pre><code>class Fun(torch.autograd.Function):

    @staticmethod
    def forward(ctx, *args, **kwargs):


    @staticmethod
    def backward(ctx, *grad_outputs):

class NN(nn.Module):
    def __init__(self):
        super(NN, self).__init__()
        self.fc1 = nn.Linear(784, 800)
        self.fc2 = nn.Linear(800, 10)

    def forward(self, inpu, L=5):
        for step in rang(L):
            x = input.view(batch,-1)
            x = self.fc1(x)
            x = self.fc2(x)
        return x

nn = NN()
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(nn.parameters(), lr=learning_rate,)

for epoch in range(e):
    for i, (x, y) in enumerate(train_loader):
        outputs = nn(x)
        y_ = torch.zeros(batch_size, 10).scatter_(1, y.view(-1, 1), 1)# ont-hot encoding
        loss = criterion(outputs.cpu(), y)
        loss.backward()
</code></pre>
<p>As shown above, I need to loop multiple times in the forward stage, but I want to update parameters only once in the Backward process, not all forward loops need to be updated.<br>
What should I do, please.Thank you！</p>

username: AbdulsalamBande
cooked: <p>Hello , You can just remove the second loop. Now, the Neural Network will take all the data instead of a batched data. <code>batch_size </code> will now be the size of the whole dataset. Hope it helps</p>
Choosing a seed for Generator (what is a good seed value?)

username: loganfrank
cooked: <p>In the <a href="https://pytorch.org/docs/stable/generated/torch.Generator.html" rel="noopener nofollow ugc">Generator class</a> under the <code>manual_seed</code> function, it says “It is recommended to set a large seed, i.e. a number that has a good balance of 0 and 1 bits. Avoid having many 0 bits in the seed.”</p>
<p>Does anyone have an answer or intuition as to why this is the case? Why does the seed need to be a large number, have a good balance of 0 and 1 bits, and not have many 0 bits?</p>

username: Nikronic
cooked: <p>Hi,</p>
<p>I had never thought about your question and I usually run my programs with single digit natural numbers since I wanted to experiments be repeatable.<br>
But after your question I read the following article and got the idea why you need <em>high entropy</em> numbers to reduce <em>predictability</em> of sequence.</p>
<p><a href="http://www0.cs.ucl.ac.uk/staff/d.jones/GoodPracticeRNG.pdf" rel="noopener nofollow ugc">GoodPracticeRNG.pdf (ucl.ac.uk)</a></p>
<p>Following sections seems can represent the topic:</p>
<ol>
<li>Rule 3: Properly seed your generator</li>
<li>Unbiased Shuffling</li>
<li>Warm up your generator first</li>
</ol>
<p>Bests</p>
Building from source with very recent versions of Intel tools

username: Felix_Crazzolara
cooked: <p>Hello everyone<br>
I want to build PyTorch from source with TBB and MKL support. For other projects of mine, I have very recent versions of Intel MKL/TBB/MPI and so on. They are located in /mnt/sdb/intel/oneapi/. In order for CMake to find MKL I had to slightly adjust pytorch/cmake/Modules/FindMKL.cmake, but in the end everything compiled. However, I unpleasently found that PyTorch ships its own version of TBB, but doesn’t ship MKL. I guess, I’m missing something here, but this doesn’t make a lot of sense to me. In my case, MKL needs libtbb.so.12 (as determined with readelf -d libmkl_tbb_thread.so.1), which is readily available in /mnt/sdb/intel/oneapi/. However, the TBB version shipped with PyTorch seems to be outdated and libtbb.so (shipped with PyTorch) and libtbb.so.12 do not provide the same symbols. I am wondering now, what version of MKL are you supposed to use when building from source? I should note at this point, that I’m using Ubuntu 18.04. Also, I fear there is no way, that I could use the most recent version of TBB for PyTorch, is that correct?<br>
Cheers,<br>
Felix Crazzolara</p>
Running PyTorch on TPU [Theoretical]

username: nzinc
cooked: <p>I have a purely theoretical question.<br>
Recently read that it is possible to run pytorch code on TPU. In this regard, the question is whether the forces of pytorch and TF have become equal. That is, pytroch itself is much more convenient than TF, but now it can be as effective as TF on TPU clusters, and now, with any identical hardware or service (even Google cloud), they are equal in power. I’m right?</p>
Brodcasting 2D index select in pytorch?

username: dvirginz
cooked: <p>I have a tensor of shape <code>P.shape=[N,k]</code> and an index tensor of shape <code>ind.shape=[L,N]</code>, where <code>ind[i,j]</code>  is a column in <code>P[j]</code> (<code>ind[i,j] &lt; k</code> always).<br>
I wish to create a new tensor of dims <code>[L,n]</code>, where the functionality can be generated using a for loop in the following manner:</p>
<pre><code class="lang-auto">    new= []
    num_points = P.shape[-1]
    for experiment in range(ind.shape[0]):
        new.append(P[torch.arange(num_points),ind[exp]])
    new= torch.stack(new)
</code></pre>
<p>But as <code>L</code> is really big, the code is extremely slow.<br><br>
Using <code>repeat</code> I managed to replicate the functionality</p>
<pre><code class="lang-auto">new = P.unsqueeze(1).repeat(1,L,1,1).reshape(-1,*P.shape[1:])
new = new.gather(2,ind.unsqueeze(2)).squeeze(2)
</code></pre>
<p>But as <code>L</code> is really big I have an OOM exception on the <code>.repeat(1,L,1,1)</code> line.<br>
I was wondering if I can accomplish something similar using broadcasting?</p>
How can I train action recognition for frame by frame of video using pytorch?

username: Keiku
cooked: <p>I am referring to <a href="https://github.com/eriklindernoren/Action-Recognition" rel="noopener nofollow ugc">eriklindernoren/Action-Recognition</a> now. The <a href="https://www.crcv.ucf.edu/data/UCF101.php" rel="noopener nofollow ugc">UCF101</a> dataset has “one action” for each video. Therefore, there is no problem with the following model output.</p>
<pre><code class="lang-python">        # output_layers of models.py
        self.output_layers = nn.Sequential(
            nn.Linear(2 * hidden_dim if bidirectional else hidden_dim, hidden_dim),
            nn.BatchNorm1d(hidden_dim, momentum=0.01),
            nn.ReLU(),
            nn.Linear(hidden_dim, num_classes),
            nn.Softmax(dim=-1),
        )
</code></pre>
<p>Now, I have data with multiple actions for one video. How can I train a dataset that has multiple action labels for frame by frame of this one video? I know that I have to extend one dimension from the above model. Please let me know if you know any existing implementations or repositories that might be helpful to me.</p>
Unexpected sparse encodings

username: horuswoo
cooked: <p>Hi! I trained an encoderdecoder with a simple CNN architecture, with the encoded / latent variable being 128x1x1 size. What I found surprising was, after the model converged to a low loss and can produce mostly meaningful reconstructions, when I feed any example (it has seen in training), the latent variable is very sparse. The 128x1x1 latent variable has about 96 zeros.<br>
I trained it using MSE loss with no regularization.<br>
Why would this be the case? Is there potentially any bug or is this natural?</p>
WeightedRandomSampler Not Random

username: Dadatata-JZ
cooked: <p>Hi there,</p>
<p>I have an imbalanced dataset, and I am trying to make each batch contains every class, regardless of the numbers as long as one exists. I found WeightedRandomSampler may be working. However, after I assign weights, the first batch always only draws samples from class0.</p>
<p>There are six classes in total.</p>
<pre><code class="lang-auto">class_sample_counts = [568330.0, 430.0, 349.0, 209.0, 14590.0, 9712.0]
class_weights = 1./torch.tensor(class_sample_counts)

sampler = WeightedRandomSampler(weights=class_weights,num_samples= HDF5Dataset(path=hdf5_path,set_name="train").__len__(),replacement=True)

train_loader = DataLoader(HDF5Dataset(path=hdf5_path,set_name="train"), batch_size=config['batch_size'],drop_last=False,sampler=sampler)
val_loader = DataLoader(HDF5Dataset(path=hdf5_path,set_name="val"), batch_size=config['batch_size'],shuffle=True,drop_last=False)
test_loader = DataLoader(HDF5Dataset(path=hdf5_path,set_name="test"), batch_size=128,shuffle=True,drop_last=False)

loaders = {"train":train_loader,"val":val_loader,"test":test_loader}

a = iter(train_loader)
print((next(a)[1]))
</code></pre>

username: Dadatata-JZ
cooked: <p>Here is the output.</p>
<pre><code class="lang-auto">tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0.])

</code></pre>

username: ptrblck
cooked: <p>The <code>weights</code> should be passed as sample weights (a weight is assigned to each sample), not class weights.<br>
<a href="https://discuss.pytorch.org/t/how-to-handle-imbalanced-classes/11264/2">Here</a> is an example on how to use the <code>WeightedRandomSampler</code>.</p>