akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Bug when indexing tensors using an MPS device

joaquimgomez: <p>Hi there.</p>
<p>I detected this possible bug, but before raising an issue in the repo I’m asking here.</p>
<p>When executing the following code for a CPU or a CUDA device:</p>
<pre><code class="lang-auto">device = torch.device("cpu")

x = torch.tensor([[1, 0, 2, 0, -100, -100, -100],
                  [1, 0, 2, 0, -100, -100, -100]])
mask = torch.tensor([[True, True, True, True, False, False, False],
                     [True, True, True, True, False, False, False]])

x = x.to(device)
mask = mask.to(device)

print(x[mask])
</code></pre>
<p>x is <code>tensor([1, 0, 2, 0, 1, 0, 2, 0])</code>, what it’s correct.</p>
<p>But, when using MPS as a device, the result is <code>tensor([-100,    0,    2,    0,    1,    0,    2,    0], device='mps:0')</code>, which is not correct.</p>
<p>This does not happen when x is 1D nor when the mask is in the CPU. But it also happens using <code>torch.masked_select</code>.</p>
<p>If this is the expected behavior, could someone explain it to me, please? And is there some workaround?</p>
<p>I’m using PyTorch version 2.2.2 and a MacBook Pro with an M2 Max and macOS Sonoma 14.4.1.</p>
<p>Thanks.</p>
Meta learning weight update

shivam_singh: <p>I have a neural network. I calculated gradient for some layers with loss1. While calculating gradient I have a computational graph.  With that gradient I am getting an updated weight but haven’t lost the previous computational graph and gradient of weights are also in computational graph (with create_graph=True). with this updated weight I to get a loss2 and want update weights which are not used in this calculating loss2 but used in calculating loss1. How to do this ?</p>
How does PyTorch build to support various Linux distributions

Rhett-Ying: <p>PyTorch is supported on Linux distributions that use <a href="https://www.gnu.org/software/libc/" rel="noopener nofollow ugc">glibc</a> &gt;= v2.17. I am wondering how PyTorch achieve this? How does Pytorch library build? Is it built upon various base docker images of various Linux distributions with minimum version? Or it’s built upon a universal image such as UBI8?</p>

ptrblck: <p>The PyTorch binaries are built in a CentOS7 image providing <code>glibc2.17</code>.</p>

Rhett-Ying: <p>The default gcc version on CentOS7 is 7.3, so we need to upgrade it to gcc 9 manually for torch build?</p>
Torch Build failure

fhossein-quic: <p>My pytorch build fails after this patch was merged:<br>
<a href="https://github.com/pytorch/pytorch/commit/ea61c9cb299b6dfebc57dc9d8821c34321d568ab" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/commit/ea61c9cb299b6dfebc57dc9d8821c34321d568ab</a></p>
<p>Errors I get:<br>
<code>/Projects/pytorch/torch/csrc/distributed/c10d/ProcessGroupMPI.cpp:270:17: error: cannot initialize a variable of type 'MPI_Group' (aka 'int') with an rvalue of type 'std::nullptr_t'       MPI_Group worldGroup = nullptr;                 ^            ~~~~~~~ /Projects/pytorch/torch/csrc/distributed/c10d/ProcessGroupMPI.cpp:271:17: error: cannot initialize a variable of type 'MPI_Group' (aka 'int') with an rvalue of type 'std::nullptr_t'       MPI_Group ranksGroup = nullptr;</code></p>
<p>I build torch with clang and <code>USE_CUDA=OFF</code>. Is this an issue on my side or a bug?</p>
Tensor Indexing with Backpropagation

vbl: <p>I am trying to do a regression task on some graphs.<br>
The data I have is structured like this:</p>
<ul>
<li>Tensor of floats that stores all of the node features “x” of shape (number of nodes, number of node features)</li>
<li>Tensor of all edges  “edge_index” that stores the indices of start and end nodes for each edge of shape (2, number of edges)</li>
</ul>
<p>I want to be able to combine them into a tensor of shape (number of edges, 2*number of node features), i.e. combine the features of the nodes for every edge into a single tensor.<br>
For example:</p>
<pre><code class="lang-auto">x = tensor([[ 0,  1,  2],                      #4 nodes with 3 features each 
            [ 3,  4,  5],
            [ 6,  7,  8],
            [ 9, 10, 11]])



edge_index = tensor([[0, 1, 2, 3],             #4 edges. First edge goes from node0 to node1
                     [1, 2, 3, 0]])                      Last edge goes from node3 to node0



result = tensor([[ 0,  1,  2,  3,  4,  5],     #The combined tensor of shape (number of edges, 2*number of node features)
                 [ 3,  4,  5,  6,  7,  8],
                 [ 6,  7,  8,  9, 10, 11],
                 [ 9, 10, 11,  0,  1,  2]])
</code></pre>
<p>My first approach was to use <code>torch.index_select</code>, however after implementing it I realized that it doesn’t support backpropagation, therefore to me it was absolutely useless.</p>
<p>My second approach relied on slices:</p>
<pre><code class="lang-auto">output = output = torch.zeros([len(edge_index[0]),6], requires_grad = True)
for i in range(len(edge_index[0])):
    output[i]=torch.cat((x[edge_index[0][i].item():edge_index[0][i].item()+1],x[edge_index[1][i].item():edge_index[1][i].item()+1]),1)
</code></pre>
<p>This approach gave me the desired result and even supports backpropagation, however it is extremely slow. Processing 1500 nodes with 8000 edges takes nearly 4.5 seconds!</p>
<p>I want to find an approach that both supports backpropagation and is fast.</p>
<p>Any suggestions would be greatly appreciated!<br>
Thank you in advance!</p>

ptrblck: <p><code>torch.index_select</code> does support backpropagation:</p>
<pre><code class="lang-python">x = torch.randn(3, 3, requires_grad=True)
out = torch.index_select(x, 0, torch.tensor([0, 2]))
out.mean().backward()
print(x.grad)
&gt; tensor([[0.1667, 0.1667, 0.1667],
          [0.0000, 0.0000, 0.0000],
          [0.1667, 0.1667, 0.1667]])
</code></pre>

Amine08: <p>Question deleted. See <a href="https://discuss.pytorch.org/t/variable-update-in-torch-cat-torch-vstack-nn-zeropad2d-torch-index-select-and-torch-roll/124823">124823</a></p>

niu_yuanzhuo: <p>Hi, Say the column index is obtained though a network, and the loss is computed based on the selected columns from the matrix. Can the selected indexes of a tensor itself, but not the selected elements of indexes differentiable in PyTorch? So that I can select different indexes of a tensor by back propagation.</p>

ptrblck: <p>Integer tensors are not differentiable as described <a href="https://discuss.pytorch.org/t/attributeerror-nonetype-object-has-no-attribute-data/56732/31">here</a>.</p>
Covariance Kernel with Derivatives in Any Dimension

alegresor: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/3/137b445cc2c11af53a4adf6a5a35549d4bdc9da5.png" data-download-href="https://discuss.pytorch.org/uploads/default/137b445cc2c11af53a4adf6a5a35549d4bdc9da5" title="Screenshot 2024-04-25 at 11.08.10 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/137b445cc2c11af53a4adf6a5a35549d4bdc9da5_2_690x264.png" alt="Screenshot 2024-04-25 at 11.08.10 AM" data-base62-sha1="2MlayQh34k0mgy1kSLZn3zyFtHf" width="690" height="264" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/137b445cc2c11af53a4adf6a5a35549d4bdc9da5_2_690x264.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/137b445cc2c11af53a4adf6a5a35549d4bdc9da5_2_1035x396.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/137b445cc2c11af53a4adf6a5a35549d4bdc9da5_2_1380x528.png 2x" data-dominant-color="F6F6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2024-04-25 at 11.08.10 AM</span><span class="informations">1414×542 29.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The following code works, but is slow. How can I speed this up? Thanks!</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import time

class Kernel:
    def __call__(self,x,z,alpha,beta):
        x,z,alpha,beta = torch.atleast_2d(x),torch.atleast_2d(z),torch.atleast_2d(alpha),torch.atleast_2d(beta)
        n,m,d = len(x),len(z),len(x[0])
        assert x.shape==(n,d) and z.shape==(m,d) and alpha.shape==(n,d) and beta.shape==(m,d)
        return torch.tensor([[self._kd(x[i],z[j],alpha[i],beta[j],d) for j in range(m)] for i in range(n)])
    def _kd(self,x,z,alpha,beta,d):
        y = self.k(x,z)
        for i in range(d):
            for j in range(alpha[i]):
                y = torch.autograd.grad(y,x,create_graph=True)[0][i]
            for j in range(beta[i]):
                y = torch.autograd.grad(y,z,create_graph=True)[0][i]
        return y 

class GaussianKernel(Kernel):
    def __init__(self,l=1.):
        self.l = l
    def k(self,x,z):
        return torch.exp(-((x-z)**2).sum()/self.l)

kernel = GaussianKernel()
nx = 100

# no derivatives, takes about .5 sec
x = torch.rand(nx,1,requires_grad=True)
alpha = torch.zeros(nx,1,dtype=torch.int)
t0 = time.time()
kmat = kernel(x,x,alpha,alpha)
print("time %.1e"%(time.time()-t0))

# with derivatives, takes about 10 sec 
n4 = nx//4; assert (nx/4)%1==0
alpha_deriv = torch.tensor([0]*n4+[1]*n4+[2]*n4+[3]*n4,dtype=torch.int)[:,None]
t0 = time.time()
kmat = kernel(x,x,alpha_deriv,alpha_deriv)
print("time %.1e"%(time.time()-t0))
</code></pre>
Serving Torch_TersorRT converted model with Triton

valerkabvv: <p>Hi,<br>
Following the appropriate <a href="https://pytorch.org/TensorRT/tutorials/serving_torch_tensorrt_with_triton.html" rel="noopener nofollow ugc">topic</a>, Im trying to serve my converted model with Nvidia Triton, but getting error message.</p>
<pre><code class="lang-auto">Deployment Response Error, Status Code: 400, Reason: load failed for model 'SwinT-pt': version 1 is at UNAVAILABLE state: Internal: failed to load model 'SwinT-pt': [Error thrown at core/runtime/TRTEngine.cpp:62] Expected (cuda_engine.get() != nullptr) to be true but got false
Unable to deserialize the TensorRT engine
;
</code></pre>
<p>Model converted to torch_tensorrt with this code</p>
<pre><code class="lang-auto">exp_program = torch.export.export(model, tuple(inputs))
trt_gm = torch_tensorrt.dynamo.compile(exp_program, inputs, enabled_precisions=[torch.float], output_format="torchscript", version_compatible = True)
</code></pre>
<p>Model configuration file config.pbtxt is</p>
<pre><code class="lang-auto">name: "SwinT-pt"
platform: "pytorch_libtorch"
max_batch_size: 2
dynamic_batching {
    max_queue_delay_microseconds: 2000
}
input [
    {
        name: "input__0"
        data_type: TYPE_FP32
        dims: [2, 256, 256]
    }
]
output [
    {
        name: "output__0"
        data_type: TYPE_FP32
        dims: [4]
    }
]


instance_group [
    {
        count: 1
        kind: KIND_GPU
        gpus: [0]
    }
]

model_warmup [
    {
        name : "warmup_sample"
        batch_size: 2
        count: 5
        inputs {
            key: "input__0"
            value: {
                data_type: TYPE_FP32
                dims: [2, 256, 256]
                random_data: true
             }
        }
    }
]
</code></pre>
<p>Also model can be loaded through python api inside the triton docker container<br>
Have you any ideas what can caused that? Thank you</p>
Grid sample and align corners

Wesley_Neill: <p>In the code below, I’ve created a simple image:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/b/fbd415e4c96454b450a371f7986345280e169eca.png" alt="image" data-base62-sha1="zVMeE1udRhLLeHg2nFM0eEz3u30" width="269" height="307"></p>
<p>and then applied grid_sample to it using scaled integer row/col coordinates. The goal is to reproduce the input image exactly, and I achieve this using <code>align_corners=True</code>. However, I can’t think geometrically how to set up the same problem to work with <code>align_corners=False</code>. I want to understand this, as I prefer to leave default values as is in practice.</p>
<p>Here’s my example:</p>
<pre><code class="lang-auto">image = torch.arange(0., 10., requires_grad=True).expand(12, 10)
image = image[None, None]

row_coords = torch.linspace(0, 11, 12)
col_coords = torch.linspace(0, 9, 10)
row_col_coords = torch.cartesian_prod(row_coords, col_coords)

image_dims = torch.tensor(image.shape)[-2:]
ratio_spanned = (row_col_coords) / (image_dims - 1)
scaled = ratio_spanned * 2 
normalized_coords = -1 + scaled

normalized_coords = normalized_coords.flip(dims=(-1,))
normalized_coords = normalized_coords[None, :, None, :]

intensities = F.grid_sample(input=image, grid=normalized_coords, align_corners=True)
plt.imshow(intensities.view(12, 10).squeeze().detach())
</code></pre>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/b/fbd415e4c96454b450a371f7986345280e169eca.png" alt="image" data-base62-sha1="zVMeE1udRhLLeHg2nFM0eEz3u30" width="269" height="307"></p>
<p>But with <code>align_corners=False</code>:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/b/2b578c627c04e0f6cd7db67adcac12e6f60eab5b.png" alt="image" data-base62-sha1="6bq6n1r7ln8q4ANIKqYfxiQwbTt" width="257" height="304"></p>
<p><strong>What is the correct way to set up the dimensions and scaling such that default values work?</strong></p>

li-fang: <p>I had the same problem and I found that the normalization of grid coordinates should be different.</p>
<p>normalized_coords = (2 * row_col_coords + 1) / image_dims - 1</p>
v2.ToImage now working with ndarray

h-0-0: <p>Hi, I’ve come across an issue where when I try and apply ToImage to a ndarray of the general form (number of samples, number of channels, height, width) ie. an array of greyscale images I get an error saying that ‘number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 4 is not equal to len(dims) = 3’.</p>
<p>The following code should reproduce the error:</p>
<pre><code class="lang-auto">import numpy as np 
import torch
from torchvision.transforms import v2

n_samples = 100
n_channels = 1
height = 20
width = 20
# Create an np array with random entries of size (n_samples, n_channels, height, width)
X = np.random.randn(n_samples, n_channels, height, width)
# Create transform that will be applied to the data
trans = v2.Compose([v2.ToImage(), v2.ToDtype(torch.float32, scale=True)])
# Apply the transform to the data
X_torch = trans(X)
</code></pre>
<p>It works if I apply the transform separately in the array, but I was under the impression (from the documentation) that you should be able to apply it to a batch of images.</p>

h-0-0: <p>Found the issue. It assumes the ndarray has format (samples, height, width, channels), if given in this format it works fine.</p>
"BLEU: 1.3885571752318349 TER: 102.44788993634243" (wonky BLEU and TER scores)

Cian: <p>If I’m not mistaken, BLEU and TER are both supposed to be a number 0-1, with higher numbers better for BLEU and lowers numbers better for TER.</p>
<p>I’m not sure if the error is from this code itself, or if something got seriously messed up in the actual corpora, but here’s what I ran:</p>
<pre><code class="lang-auto">import sacrebleu
def evaluate_translation(reference_path, hypothesis_path):
    with open(reference_path, 'r', encoding='utf-8') as ref_file:
        refs = [ref_file.read().split('\n')]
    with open(hypothesis_path, 'r', encoding='utf-8') as hyp_file:
        hyps = hyp_file.read().split('\n')
    bleu = sacrebleu.corpus_bleu(hyps, refs)
    ter = sacrebleu.corpus_ter(hyps, refs)
    print(f"BLEU: {bleu.score}")
    print(f"TER: {ter.score}")
evaluate_translation('/(PATH)/eng-fra-test.txt-filtered.fr.subword.desubword', '/(PATH)/eng-fra.fr.translated.desubword')
</code></pre>
<p>And it’s in the title, but here are the results:</p>
<pre><code class="lang-auto">BLEU: 1.3885571752318349
TER: 102.44788993634243
</code></pre>
<p>The above is for English to French. When I tried French to English before, it spat out a perfectly normal BLEU score of about 0.57, but still with a wonky TER of about 112. Because the BLEU score came out perfectly fine, I have no idea where the problem is coming from.</p>
Why does the word with the index 0 in the vocabulary also have the index 0 in the embedding?

martinfloo: <p>I was just wondering how the embedding layer knows what index the different words should have in the vocabulary. In other words, why does all indexes in the vocabulary and embeddings match?</p>

divinho: <p>Your question is unclear. Why would they not be the same?</p>

vdw: <p>The embedding layer doesn’t know. It’s your job to ensure that it matches :).</p>
Le: "b8[64, 2048, 7, 7]" = torch.ops.aten.le.Scalar(relu_48, 0); relu_48 = None

sabiha_afroz: <p>I used torch.compile and torch._dynamo to generate fx graphs for resnet50 model. I got this line from fx_graph_transformed.py.</p>
<p>Q1: what is “b8”? Is it a data type? From the PyTorch website I found out bfloat16 is one data type. But nothing about b8.</p>
<p>Q2: how do I locate this torch.ops.aten.le.Scalar operation in pytorch source code. I tried " grep -rni “torch.ops.aten.le.Scalar” ". It returned nothing.</p>
<p>Q3: add: “i64<span class="chcklst-box fa fa-square-o fa-fw"></span>” = torch.ops.aten.add.Tensor(primals_164, 1) . Here, the output tensor has i64 datatype and the dimension is empty. Is there any element in this output tensor?</p>
<p>Q4: When I am using torch.compiler and torch._dynamo for a model training, it generates FX graphs in three directories. 1) model_backward, 2) model_forward, 3) model_inference. Directory 1 contains the FX graphs for backward propagation operations, directory 2 contains the FX graphs for forward propagation operations. What about the third directory? Is it about parameter updates after backward propagation? Is there any documentation to understand these generated FX graphs outputs? FX graph documentation “<a href="https://pytorch.org/docs/stable/fx.html" class="inline-onebox" rel="noopener nofollow ugc">torch.fx — PyTorch 2.2 documentation</a>” talks about what things I could do with the FX graphs. No detail about the generated output files understanding. (I have attached one screenshot)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/d/7ddf738cf7ceb2860c6674da64e82f3e5747ab71.png" data-download-href="https://discuss.pytorch.org/uploads/default/7ddf738cf7ceb2860c6674da64e82f3e5747ab71" title="Screenshot (582)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/d/7ddf738cf7ceb2860c6674da64e82f3e5747ab71.png" alt="Screenshot (582)" data-base62-sha1="hXwqq9m7RMJWvqkZBLSyvJa8n2p" width="316" height="500" data-dominant-color="222527"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (582)</span><span class="informations">387×612 10.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Any kind of help is appreciated.</p>

sabiha_afroz: <ol>
<li>B8 is a boolean tensor</li>
<li>add: “i64<span class="chcklst-box fa fa-square-o fa-fw"></span>”, is a scalar tensor</li>
</ol>
Best practice for running legacy pytorch code on newer GPU?

sad_robot: <p>Hi I’ve come into this problem quite frequently and it is rather annoying. I’ll be trying to inspect a paper’s results that is like 4 years old, and rather than working I will need to update their whole code-base to work with breaking pytorch interface changes so that I can use a modern version of pytorch that works with the new GPU.</p>
<p>Is there a better solution to this problem than rewriting the codebase(s) to match modern pytorch?<br>
P.S. A perfect solution in tensorflow is `import tensorflow.v2 as tf’. Idk what to do in pytorch.</p>