akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
IndexError: index 50 is out of bounds for dimension 0 with size 50

Berny: <p>I’ m doing a filter function:</p>
<pre><code>def smoothfilter(extremeBands, CHSC, X):
    filteredSmooth = torch.zeros(np.size(CHSC, 1) * np.size(CHSC, 2), X.shape[3], 
    dtype=torch.float64)
    for i in range(np.size(CHSC, 1) * np.size(CHSC, 2)-1):
        temp = extremeBands[i, :]
        for j in range(X.shape[3]):
            if j == 1:
                temp[j] = torch.mean(extremeBands[i, j - 1:j + 2])
            if j == (X.shape[3] - 2):
                temp[j] = torch.mean(extremeBands[i, j - 1:])
            if j &gt; 1 and j != (X.shape[3] - 3):
                temp[j] = torch.mean(extremeBands[i, j - 2:j + 3])
            if j == (X.shape[3] - 3):
                temp[j] = torch.mean(extremeBands[i, j - 2:])
        filteredSmooth = temp
    return filteredSmooth
</code></pre>
<p>I get the IndexError in the line:  temp[j] = torch.mean(extremeBands[i, j - 2:j + 3]). Does someone know how to fix it?</p>
PyTorch ROCm does not recognize AMD MI250X GPUs

Eugene_Walker: <p>I’ve installed PyTorch 1.11.0 with ROCm support using pip per official instructions.</p>
<p>And I’m on a system with 4x AMD MI250X GPUs (gfx90a), and torch.cuda.is_available() returns False. Is there a fundamental incompatibility at this point with these particular AMD GPUs?</p>
<p>The same installation procedure using the same version of Python and same ROCm version works fine on another system with MI100. It just doesn’t work on system with MI250X</p>
<p>Python version is 3.6.13</p>

davide445: <p><a class="mention" href="/u/eugene_walker">@Eugene_Walker</a> did you finally solved the problem?<br>
Interested using PyTorch on MI250 for big dataset training, leveraging the aggregated 128GB memory, wanted to ask if is will be truely able to leverage that amount or in fact will be limited to 64GB on each die.</p>
Excessive use of Cuda memory calling backward() on new tensor

fabiozappo: <p>Hi everyone, this is my first post here so don’t be too mean please.<br>
I am struggling with a weird problem that consumes the double cuda memory of what I expected.</p>
<p>Here is a quick snippet to replicate the problem:</p>
<pre><code class="lang-auto">from torchvision.models import resnet50
import torch


# This loss consumes a lot of memory 
class ZeroLoss(torch.nn.Module):
    def forward(self, embeddings):
        return torch.tensor([0.], requires_grad=True)


# This doesn't
class MeanLoss(torch.nn.Module):
    def forward(self, embeddings):
        return embeddings.mean() * 0.0


use_zeroloss = True

device = "cuda:0" if torch.cuda.is_available() else "cpu"

model = resnet50().to(device)
x = torch.randn(128, 3, 224, 224).to(device)

optimizer = torch.optim.SGD(model.parameters(), lr=1e-5, momentum=0)

# MeanLoss use 15GB of Vram ZeroLoss use 24GB of Vram
criterion = ZeroLoss() if use_zeroloss else MeanLoss()

while True:
    embeddings = model(x)

    loss = criterion(embeddings)
    loss.backward()

    optimizer.step()
    optimizer.zero_grad()
</code></pre>
<p>This is the minimum amount of code to replicate my problem.</p>
<p>Cuda memory used in above script with <code>use_zeroloss = False</code> (up) and <code>use_zeroloss = True</code> (down)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/6/36e4a3b1cd6ac4e76c7955f824d8e662c8521e87.png" data-download-href="https://discuss.pytorch.org/uploads/default/36e4a3b1cd6ac4e76c7955f824d8e662c8521e87" title="untitled (3)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/6/36e4a3b1cd6ac4e76c7955f824d8e662c8521e87.png" alt="untitled (3)" data-base62-sha1="7PBDKuG0PPXcmnJsiu32wkpCaHl" width="690" height="199" data-dominant-color="474747"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">untitled (3)</span><span class="informations">715×207 15.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I expected ZeroLoss to save more memory than MeanLoss, but it’s actually the opposite and the gap is huge. Why?!?!?</p>

ptrblck: <p>Your <code>ZeroLoss</code> creates a new tensor which is not attached to the computation graph. Calling <code>backward</code> will not compute any gradients for the parameters of the model and will thus also not clear the computation graph and will not delete the intermediate forward activations needed for the gradient computation, which could explain the memory usage. Try to <code>del embeddings</code> and the memory usage should go down.</p>
Returning final output in the One vs. One classification

Gianmarco_Luchetti_S: <p>Hello everyone,<br>
I am tackling a multiclass classification problem related to the activity of some drugs using Pytorch Geometric neural network, I have three activity classes (0, 1 and 2), to tackle the problem I have adopted the one vs. one approach, thus creating three binary classifiers: 0 vs. 1, 1 vs. 2 and 2 vs. 0. After creating the three classifiers I have trained and optimized them, now my question is, how can I concatenate the outputs of the three classifiers to get the prediction? I tried to do a simple average among the three predictions getting poor results, although the three classifiers are well trained, and individually achieve high performance</p>
<p>Thank you all for your help!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/1/71bf6209a38defb93c17c632b23cf42c71383ac3.png" data-download-href="https://discuss.pytorch.org/uploads/default/71bf6209a38defb93c17c632b23cf42c71383ac3" title="download"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/71bf6209a38defb93c17c632b23cf42c71383ac3_2_495x500.png" alt="download" data-base62-sha1="gefZoyOWfIc5N9AgeufwoDaCg3V" width="495" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/71bf6209a38defb93c17c632b23cf42c71383ac3_2_495x500.png, https://discuss.pytorch.org/uploads/default/original/3X/7/1/71bf6209a38defb93c17c632b23cf42c71383ac3.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/1/71bf6209a38defb93c17c632b23cf42c71383ac3.png 2x" data-dominant-color="E8E8EF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">download</span><span class="informations">652×658 63.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

srishti-git1110: <p>Hi Gianmarco,</p>
<p>What does your task look like?<br>
Is it a multi-class (single-label) problem where one and only one class out of 0, 1, 2 is the right answer?</p>
<p>Or is it a multi-class multi-label problem where a data point could belong to one or more than one classes?</p>
<p>In any case, you may not need to train 3 separate classifiers. Rather, you could try modifying your network such that the output layer has three nodes and use a suitable loss function - <code>CrossEntropyLoss</code> in the former case, and <code>BCEWithLogitsLoss</code> in the latter.</p>

Gianmarco_Luchetti_S: <p>Hi <a class="mention" href="/u/srishti-git1110">@srishti-git1110</a>, thank you for your response.</p>
<p>It is a multi-class problem, my dataset is composed of drugs and the label (0, 1, 2) represents a class of activity against cancer. I have already tried to train a single classifier using <code>CrossEntropyLoss</code> as a loss function, but I didn’t get very good results. The main problem is that in my dataset I have compounds that are very similar structurally but often have different labels (I attach an image in the post)</p>
Computing Hue from RGB color space

anjali-chadha: <p>Can we compute hue from an RGB tensor in a differentiable way?</p>
<p>My model is n RGB color space and I want to write a loss function to minimise the difference in hue component in the input image and the image generated by the model.</p>
<pre><code class="lang-auto">Hue Loss = |Hue Input - Hue output|
</code></pre>
<p>Thank you</p>

ptrblck: <p>You can find the transformation from RGB to HSV in <a href="http://www.niwa.nu/2013/05/math-behind-colorspace-conversions-rgb-hsl/" rel="nofollow noopener">this post</a>. It looks like it should be differentiable, but let me know, if you run into any trouble implementing it.</p>

xiang-zhe: <p>Hi, <a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/anjali-chadha">@anjali-chadha</a> ,I’m confused that why you emphasized that</p>
<blockquote>
<p>It looks like it should be differentiable</p>
</blockquote>
<p>i also saw that some others emphasized that <strong>differentiable</strong><br>
as far as i know,  rgb image tensor range [0,1], and hsv  image tensor also range [0,1], if rgb is always differnetiable, why hsv not.<br>
Thank you!</p>

ptrblck: <aside class="quote no-group" data-username="xiang-zhe" data-post="3" data-topic="37414">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xiang-zhe/48/44315_2.png" class="avatar"> xiang-zhe:</div>
<blockquote>
<p>if rgb is always differnetiable, why hsv not.</p>
</blockquote>
</aside>
<p>I’m not sure I understand the question completely, but I was pointing out that the transformation should be differentiable (I haven’t written and confirmed it), not the tensors in isolation.</p>

xiang-zhe: <p>sorry for my bad expression, what my question is that:<br>
what is the differences between rgb color space image and hsv color space image when calculating differentiation<br>
or could you give me a hsv color space image example/case that it is non-differentiable.<br>
thank you!</p>

ptrblck: <p>I’m not claiming any operations with input images in a specific color space might or might not be differentiable. In my post I was pointing out that the RGB to HSV transformation (only this operation) seems to be differentiable.</p>

xiang-zhe: <p>I got it, thank you very much, for the detailed interpretation</p>
Apply function in Pytorch

Prakhar_Pradhan1: <p>Hi Everyone,</p>
<p>I have one function, which I want to apply to all the kernels of a neural net, example kernels size: [out_ch, in_ch, H, W], I want to apply a function to all the out_ch (dim=0 of kernels) without using any for loop, how can I do this ?</p>
<p>Thanks in advance</p>