akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Modify the border/stroke of an image

username: Farzam
cooked: <p>Hi everybody, I need to modify the border of an image, like reducing the stroke from this image. Would you mind helping me with that? A sample has been uploaded.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/2/1205ef20fb9d5c5efbf5030cd0d48afece910fd3.png" data-download-href="https://discuss.pytorch.org/uploads/default/1205ef20fb9d5c5efbf5030cd0d48afece910fd3" title="6to6"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/1205ef20fb9d5c5efbf5030cd0d48afece910fd3_2_690x300.png" alt="6to6" data-base62-sha1="2zrj6LCYhxRFnPx80JEBzAwMVvJ" width="690" height="300" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/1205ef20fb9d5c5efbf5030cd0d48afece910fd3_2_690x300.png, https://discuss.pytorch.org/uploads/default/original/3X/1/2/1205ef20fb9d5c5efbf5030cd0d48afece910fd3.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/2/1205ef20fb9d5c5efbf5030cd0d48afece910fd3.png 2x" data-dominant-color="E5DDDC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">6to6</span><span class="informations">808×352 15.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Sample:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/b/ab694750c677b12f005a029c38c93d4488e6f1e2.png" alt="6" data-base62-sha1="osncKVsTmp0CJAz4BGmmZmoEsnw" width="30" height="42"></p>
<p>Ps: I used <code>torchvision.transforms.Pad</code> class , but that wasn’t what I wanted.</p>

username: johnnv1
cooked: <p>Hey, I believe what you want/needs here is the morphological operation erosion (<a href="https://en.wikipedia.org/wiki/Mathematical_morphology" class="inline-onebox" rel="noopener nofollow ugc">Mathematical morphology - Wikipedia</a>)</p>
<p>You can achieve that with pytorch using <a href="https://github.com/kornia/kornia" rel="noopener nofollow ugc">kornia</a> – Kornia docs erosion morph (<a href="https://kornia.readthedocs.io/en/stable/morphology.html#kornia.morphology.erosion" class="inline-onebox" rel="noopener nofollow ugc">kornia.morphology - Kornia</a>)</p>
<p>Kornia tutorials morph (<a href="https://kornia.github.io/tutorials/nbs/morphology_101.html#erosion" class="inline-onebox" rel="noopener nofollow ugc">Kornia - Introduction to Morphological Operators</a>)</p>

username: Farzam
cooked: <p>Thank you for your comprehensive reply, I’ll take a look at all of them!</p>
Why pytorch call cuda math api in two different way, like ::aten or std::aten

username: Shan19900305
cooked: <p>why pytorch call cuda match api in two different way, like ::aten or std::aten.<br>
In op function, the ::aten is called. <a href="https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/UnaryGeometricAtanKernel.cu#L50" class="inline-onebox" rel="noopener nofollow ugc">pytorch/aten/src/ATen/native/cuda/UnaryGeometricAtanKernel.cu at main · pytorch/pytorch · GitHub</a><br>
and in foreach op function, std::aten is called.<br>
<a href="https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/ForeachUnaryOp.cu#L209" class="inline-onebox" rel="noopener nofollow ugc">pytorch/aten/src/ATen/native/cuda/ForeachUnaryOp.cu at main · pytorch/pytorch · GitHub</a></p>
<p><a class="mention" href="/u/ezyang">@ezyang</a> <a class="mention" href="/u/alband">@albanD</a></p>

username: Shan19900305
cooked: <p><a class="mention" href="/u/crcrpar">@crcrpar</a> Hi, do you know anything about this. Thx</p>
How to disable PyTorch elastic restarts for non-retriable errors

username: bgedik
cooked: <p>I am using PyTorch distributed elastic for multi-gpu, multi-node runs. There are a fixed set of retries configured for my training jobs. But I want to have the ability to not retry and fail early, if the failure is such that it is non-retriable (for instance node level validations failed or there is a known kind of exception for which I don’t want to retry). I couldn’t find how elastic handles this. It seems to always restart user errors (aka exceptions from the trainer subprocess).</p>
Will the Conv3D operation be supported on MPS through PyTorch?

username: staticfuzz
cooked: <p>Hi,<br>
I am new here. I’ve been trying to use stable video diffusion in ComfyUI on an Intel Core i9 MacBook Pro running Sonoma 14.3.1 with AMD Radeon Pro 5500M 8 GB. I come up against this error: RuntimeError: Conv3D is not supported on MPS. I can use the CPU instead, if I want to wait for a few hours instead of minutes, but this isn’t practical. I have tried the PyTorch Nightly release as well as some of the other suggestions on the Github page - <a href="https://github.com/comfyanonymous/ComfyUI/issues/2044" rel="noopener nofollow ugc">https://github.com/comfyanonymous/ComfyUI/issues/2044</a> but have not been able to get it to work using MPS. Are there plans for the Conv3D to be supported on MPS through PyTorch?<br>
Cheers</p>

username: Goncalo_Guedes
cooked: <p>Hi,</p>
<p>I have the same problem, did you find a solution?</p>

username: aisven
cooked: <p>Got the same problem.</p>
<p>2020 Intel MacBook Pro 16" with Touch-Bar<br>
with AMD Radeon Pro 5500M 8 GB</p>
Relative error greater than unit roundoff for torch.float16

username: Sahil_Bhola
cooked: <p>Converting a data type from torch.float64 to torch.float16 often results in a relative error greater than the unit roundoff. Here’s a sample reproduction code</p>
<p>num_exps = 100000<br>
urd = torch.tensor(torch.finfo(torch.float16).eps / 2.0, dtype=torch.float64) # Unit roundoff for FP16<br>
rel_error = torch.zeros(num_exps, dtype = torch.float64)</p>
<p>for ii in range(num_exps):<br>
val = torch.randn(1, dtype=torch.float64)<br>
val_half = val.to(dtype=torch.float16)<br>
rel_error[ii] = torch.abs((val - val_half) / val)</p>
<p>if torch.max(rel_error) &gt; urd:<br>
print(‘Max rel error :{} exceeded the urd : {}’.format(torch.max(rel_error), urd))<br>
fig, axs = plt.subplots()<br>
axs.hist(rel_error.numpy(), bins=100, label=‘$\delta$’, density=True)<br>
axs.axvline(urd, color=‘red’, label=‘$u$’)<br>
axs.legend()<br>
plt.show()</p>

username: HDCharles
cooked: <p>the dynamic range of fp16 is notorious bad to the point where we’ve had to swap to bf16 in a number of cases. I would assume that’s what you’re seeing.</p>
QuantStub with values in [-128,127]

username: Massimiliano_Datres
cooked: <p>Dear all,</p>
<p>I have trained a ResNet model using PACT. Now I want to convert it using the static quantization Pytorch package. Can I force torch.quantization.QuantStub() to map my input to a qint8 format, that means integer values between [-128,127], instead of values between [0, 256]?</p>
<p>Thanks in advance,</p>
<p>Max</p>

username: myshabako
cooked: <p>Use <code>qint8</code> for the <code>dtype</code> argument of the QConfig. e.g:</p>
<pre><code class="lang-auto">qconfig = QConfig(activation=MovingAverageMinMaxObserver.with_args(qscheme=torch.per_tensor_symmetric, dtype=torch.qint8), weight=PerChannelMinMaxObserver.with_args(dtype=torch.qint8, qscheme=torch.per_channel_symmetric))
</code></pre>
<p>Note that this will have a degraded accuracy</p>

username: Massimiliano_Datres
cooked: <p>Dear <a class="mention" href="/u/myshabako">@myshabako</a>,</p>
<p>thanks for the answer. Forcing the QConfig as you suggested produces the following error</p>
<blockquote>
<p>RuntimeError: expected scalar type QUInt8 but found QInt8</p>
</blockquote>
<p>during the QuantizedConv2D forward. Can you know why it is not supported?</p>
<p>Thanks in advance,</p>
<p>Max</p>

username: Vasiliy_Kuznetsov
cooked: <p>Hi <a class="mention" href="/u/massimiliano_datres">@Massimiliano_Datres</a> , you can try using the <code>qnnpack</code> backend which supports qint8 activations for some ops.  The fbgemm backend only supports quint8 activations.</p>

username: kechan
cooked: <p>was this your error:</p>
<p>RuntimeError: quantized::conv(FBGEMM): Expected activation data type QUInt8 but got QInt8</p>
<p>I was using get_default_qconfig(‘x86’)</p>

username: kechan
cooked: <p>I changed mine to ‘qnnpack’ but got:</p>
<p>File <a href="http://localhost:8080/lab/tree/vss/notebooks/vss_env/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py#line=194" rel="noopener nofollow ugc">/python3.10/site-packages/transformers/models/clip/modeling_clip.py:195</a>, in CLIPVisionEmbeddings.forward(self, pixel_values)</p>
<pre><code class="lang-auto">&gt; patch_embeds = patch_embeds.flatten(2).transpose(1, 2) 
&gt; class_embeds = self.class_embedding.expand(batch_size, 1, -1) 
&gt; embeddings = torch.cat([class_embeds, patch_embeds], dim=1).  # NotImpl Ex here.
&gt; embeddings = embeddings + self.position_embedding(self.position_ids)
&gt; return embeddings
</code></pre>
<p>NotImplementedError: Could not run ‘aten::qscheme’ with arguments from the ‘CPU’ backend.</p>

username: HDCharles
cooked: <p>that link is to a local file so i can’t see it.</p>
<p>have you tried following the</p>
<p>PTSQ API Example:</p>
<p>in <a href="https://pytorch.org/docs/stable/quantization.html" class="inline-onebox" rel="noopener nofollow ugc">Quantization — PyTorch 2.3 documentation</a></p>
Question on Handling Activation Tensors in PEFT Memory Management with LoRA in PyTorch

username: zbtrs
cooked: <p>Hello everyone,</p>
<p>I am currently working on PEFT memory management, specifically with the LoRA task. When I wrap a model using the Hugging Face PEFT library, it essentially freezes the backbone model’s parameters by setting <code>requires_grad=False</code>.</p>
<p>My first question is: Does PyTorch’s memory management strategy release activation tensors that are not used during the <code>backward</code> process? By “not used,” I mean those activation tensors that would have been used to calculate gradients for the backbone model’s weights, which are now frozen.</p>
<p>If PyTorch retains these activation tensors in memory, how can I manually prune the unnecessary ones? Is it possible to set them to <code>None</code> directly?</p>
<p>Thank you in advance for your help!</p>

username: Brock_Brown
cooked: <p>No additional memory should be used for the backward pass if <code>requires_grad</code> is <code>False</code>. From the <a href="https://pytorch.org/docs/stable/notes/autograd.html" rel="noopener nofollow ugc">Autograd documentation</a>:</p>
<blockquote>
<p>During the forward pass, an operation is only recorded in the backward graph if at least one of its input tensors require grad. During the backward pass (<code>.backward()</code> ), only leaf tensors with <code>requires_grad=True</code> will have gradients accumulated into their <code>.grad</code> fields.</p>
</blockquote>
<p>Also, if you ever want to manually relinquish memory of the graph from a tensor, you can detach a tensor with <code>some_tensor = some_tensor.detach()</code>.</p>

username: zbtrs
cooked: <p>Hi Brock_Brown!<br>
Thank you for your reply!<br>
Can I understand it this way: when a calculated activation tensor is not used to update a gradient, it will not appear in the computation graph, and thus PyTorch will release the memory for this tensor. In other words, PyTorch internally helps to prune the tensors that are not needed in the backward pass?<br>
And another question: Is this achieved through Python’s garbage collection (gc)?</p>

username: Brock_Brown
cooked: <p>Yup, it will not appear in the computation graph. Memory is relinquished through the garbage collector whenever there are no references to an object in Python.</p>

username: Brock_Brown
cooked: <p>Forgot a very important part here, the GPU does not clear its memory unless you have deleted the tensors (or just make sure there are no references to them) <em>and</em> you’ve cleared the cache with <code>torch.cuda.empty_cache()</code>.</p>

username: zbtrs
cooked: <p>Thank you very much!</p>
Pytorch dataloader very slow with hdf5 data

username: Jash_Jasani
cooked: <p>I’m have a very large dataset in hdf5 format which I can not load in memory all at once. I’m using custom dataset from torch here’s the code</p>
<pre><code class="lang-auto">import time
from utils import get_vocab_and_skipgrams
from torch.utils.data import Dataset
from torch.utils.data import DataLoader
import os
import h5py
import numpy as np
import torch

class CustomSkipGramDataset(Dataset):
    def __init__(self, filename, window_size, data_dir="training_data", data_exists=True):
        self.window_size = window_size
        self.filename = filename
        self.data_exists = data_exists
        self.vocab_path = os.path.join(data_dir, "vocab.npy")
        self.hdf5_path = os.path.join(data_dir, "skipgram.h5")
        
        if not data_exists:
            get_vocab_and_skipgrams(filename, data_dir)
        
        self.vocab = np.load(self.vocab_path, allow_pickle=True).tolist()
        self.vocab_size = len(self.vocab)
        self.hf = h5py.File(self.hdf5_path, "r")
        self.dataset = self.hf["positive_skips"]
        
    def __len__(self):
        return self.dataset.shape[0]
    
    def __getitem__(self, index):
        
        x, y = self.dataset[index]
        return x, y
</code></pre>
<p>Now when I’m loading it directly like this</p>
<pre><code class="lang-auto">with h5py.File("./training_data/skipgram.h5") as hf:
    dataset = hf["positive_skips"]
    for a in range(1,100):
        print(torch.tensor(dataset[a:100*a]))

</code></pre>
<p>it is indeed very fast compared to torch custom dataset. 100x faster almost. I know I’m doing something wrong.</p>

username: JuanFMontesinos
cooked: <p>Random suggestion, have you tried to disable multiprocessing? I’m not sure how well both would interact.</p>

username: Gillian
cooked: <p>There is certainly some troubles with pickling the CustomSkipGramDataset instances with multiprocessing.<br>
To work with HDF5 spirit with dataset, do the following:</p>
<ul>
<li>clear the <strong>init</strong> method from any high computation thing, don’t store large object in class attributes.</li>
<li>remove the <strong>getitem</strong> and bring the __iter__method instead.</li>
<li>code your __iter__method like the following:</li>
</ul>
<pre><code class="lang-auto">def __iter__(self):
  open_hdf5()
  prepare_data()
  for entry in data:
    yield item
</code></pre>
How to compute derivatives on multiple output in parallel

username: Isabella_Osei
cooked: <p>I’m working on a Physics Informed Neural Network that has two inputs and N outputs. The loss function contains high-order derivatives of the outputs with respect to the inputs x and y. Is it possible to compute the derivatives in parallel without a for loop?</p>
<pre><code class="lang-auto">def gradient(y, x, grad_outputs=None):
    if grad_outputs is None:
        grad_outputs = torch.ones_like(y)
    grad = torch.autograd.grad(y, [x], grad_outputs=grad_outputs, create_graph=True)[0]
    return grad


def compute_derivatives(x, y, u):
    dudx = gradient(u, x)
    dudy = gradient(u, y)

    dudxx = gradient(dudx, x)
    dudyy = gradient(dudy, y)

    dudxxx = gradient(dudxx, x)
    dudxxy = gradient(dudxx, y)
    dudyyy = gradient(dudy, y)

    dudxxxx = gradient(dudxxx, x)
    dudxxyy = gradient(dudxxy, y)
    dudyyyy = gradient(dudyyy, y)

    return dudxx, dudyy, dudxxxx, dudyyyy, dudxxyy
</code></pre>
<p>The code above works fine for a single output. For N outputs the shape of u is [batch_size, N] and I need to compute the derivatives for each column of u in parallel so that each derivative (ex. dudx, dudy…) shape matches the shape of u. Which is the most efficient way to do this? Thanks in advance.</p>

username: soulitzer
cooked: <p>Would this work for your case <a href="https://pytorch.org/docs/stable/generated/torch.func.vmap.html" class="inline-onebox" rel="noopener nofollow ugc">torch.func.vmap — PyTorch 2.3 documentation</a>?</p>

username: Isabella_Osei
cooked: <p>Thank you, I am going to have a look again. I already tried to implement, but maybe I missed something.</p>

username: Isabella_Osei
cooked: <p>Hi, I tested your suggestion about torch.func.vmap. I started with a simplified application (as below) to see how each output was printed out:</p>
<pre><code class="lang-auto">import torch
from functorch import vmap

x = torch.tensor([1.0, 2.0], requires_grad=True)
out = torch.stack([x * 2, x * 3], dim=0)

print('x:', x)
print('out:', out)


def single_gradient(out_row, x):
    grad_outputs = torch.ones_like(out_row)
    return torch.autograd.grad(out_row, [x], grad_outputs=grad_outputs, create_graph=True, retain_graph=True)[0]


batched_grad = vmap(single_gradient, (0, None))(out, x)

print('Batched Grads:', batched_grad)
</code></pre>
<p>Above I am creating a vector of two numbers and one matrix with shape 2x2.<br>
In theory, this should result in a differentiation of the matrix lines and give back another 2x2 matrix.</p>
<p>But I get this error:<br>
“RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn”</p>
<p>The error occurs when I call autograd, so it seems that using vmap affects the computational graph. Do you have possible ideas?</p>
Batch size in Rollout

username: putskan
cooked: <p>When performing <code>env.rollout</code>, it internally calls the policy module, with a sample at each step.<br>
It is in the shape of ovservation, without any <code>batch_size </code> dimension.<br>
To keep everything more consistent, is there a way to add a <code>batch_size</code> dimension of 1, instead of unsqueezing in the model itself?</p>
<p>Much appreciated!</p>

username: vmoens
cooked: <p>IIUC you want an env with a leading dim of 1 on each tensor, is that right?<br>
You should use this transform for this<br>
<a href="https://pytorch.org/rl/stable/reference/generated/torchrl.envs.transforms.BatchSizeTransform.html?highlight=batchsizetransform" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/rl/stable/reference/generated/torchrl.envs.transforms.BatchSizeTransform.html?highlight=batchsizetransform</a></p>
<p>This should do the trick:</p>
<pre data-code-wrap="python"><code class="lang-python">env = env.append_transform(BatchSizeTransform(reshape_fn=lambda x: x.unsqueeze(0))
</code></pre>
Detach the gradient of a part of parameters in a module

username: floatingpale
cooked: <p>Suppose I have a module a=nn.Linear(in,out). For example, I want to detach the gradient of a.weight.squeeze()[:10] and retain the rest part a.weight.squeeze()[10:] requires_grad=True. Then only a.weight,squeeze()[10:] can be updated. How can I achieve it?</p>
Pytorch quantized model to ONNX - quantized_decomposed::quantize_per_tensor Error

username: SHIVAJI_KANT
cooked: <p>Hi All, need a quick help!!</p>
<p>I am trying to convert a quantized pytorch model to ONNX format.<br>
This gives me following error:</p>
<p>UnsupportedOperatorError(torch.onnx.errors.UnsupportedOperatorError: ONNX Export failed on an operator with unrecognized namespace quantized_decomposed::quantize_per_tensor. If you are trying to export a custom operator, make sure you are registered it with the right domain and version.</p>
<p>Can you let me know why is this pytorch quantized_decomposed::quantize_per_tensor is not supported for ONNX format conversion.<br>
Or if there is a workaround</p>
<p>Thanks</p>

username: SHIVAJI_KANT
cooked: <aside class="quote no-group" data-username="SHIVAJI_KANT" data-post="1" data-topic="204954">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shivaji_kant/48/70856_2.png" class="avatar"> SHIVAJI_KANT:</div>
<blockquote>
<p>quantized_decomposed::quantize_per_tensor</p>
</blockquote>
</aside>
<p>I am using xnnpack_quantizer</p>

username: jerryzh168
cooked: <p>you’ll need to ask onnx team to add the op I think: <a href="https://github.com/onnx/onnx/issues" class="inline-onebox" rel="noopener nofollow ugc">Issues · onnx/onnx · GitHub</a></p>

username: David_Chen
cooked: <blockquote>
<p>UnsupportedOperatorError: ONNX Export failed on an operator with unrecognized namespace quantized_decomposed::quantize_per_tensor</p>
</blockquote>
<p>I have met the same question</p>