akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Torch Vision version compatibility with Triton Server and Pytorch

magesh: <p>Hello All, I am currently using Nvidia Triton Server v21.08 and now I want to build a pytorch model which will use torchvision library to transform data.</p>
<p>Before I begin, need some clarification on which version of pytorch and torchvision should I use for Trition Server 21.08… Can someone help with this version compatibility query please</p>

ptrblck: <p>You could check which <code>torchvision</code> version was released around Aug. 2021 and use this one.<br>
Based on <a href="https://github.com/pytorch/vision/releases">the release notes</a> it was <code>0.10.1</code> moving to the <code>0.11.0</code> release in Oct 2021.</p>
Failed to load image Python extension

Ranim_Saidi: <p>Hello I am getting this error I tried different torch and torchvision versions and nothing worked. Any suggestion please?</p>
<p>env/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory<br>
warn(f"Failed to load image Python extension: {e}")</p>

ptrblck: <p>This message should only be a warning or do you see any failures afterwards?<br>
I believe the pip wheels did not ship with <code>torchvision.io</code> extensions, which is why this warning is raised.</p>
AUTOGRAD.GRAD Related Error: The Occurence of Error Seemed To Be Affected By GRAD_OUTPUT Values

untitled47: <p>Hello everyone,</p>
<p>This post is an extention to</p>
<p><a href="https://discuss.pytorch.org/t/second-order-derivative-with-nan-value-runtimeerror-function-sigmoidbackwardbackward0-returned-nan-values-in-its-0th-output/173260" class="inline-onebox">Second Order Derivative with Nan Value - RuntimeError: Function 'SigmoidBackwardBackward0' returned nan values in its 0th output</a>.</p>
<p>Suppose: <code>g1 = torch.autograd.grad(loss, params)</code>, <code>g2 = torch.autograd.grad(g1, params, grad_outputs)</code>, where <code>params</code> denotes a list of parameters of a model, and hence <code>g1</code> would also be a list.</p>
<p>My new finding is that:</p>
<ol>
<li>With <code>torch.autograd.set_detect_anomaly(True)</code>, only when setting <code>grad_outputs = params</code>, the calculation of <code>g2</code> will be error free, other value of <code>grad_outputs</code> will lead to an error as shown below, even if <code>grad_outputs</code> is set to a value that is equal to a slight perturbation of <code>params</code>. Where <code>*</code> denotes <code>Mul</code>, <code>Sigmoid</code>, <code>Mm</code>, meaning that the error still persist when I tried to modify the model a little bit.</li>
</ol>
<pre><code class="lang-auto">Function '*Backward0' returned nan values in its 1th output.
</code></pre>
<ol start="2">
<li>Situation in 1 seedmed to only bother the model I am currently dealing with, a relatively large contrastive Text-Image model.</li>
</ol>
Problems with yolov5s from hub

Tulipian: <p>I hoped that I can use yolov5s from hub as plug in play in my standard code that works with pytorch resnet model, but somehow it does not work and I cannot figure out what’s the issue. I need to train a model first.</p>
<p><a href="https://pytorch.org/hub/ultralytics_yolov5/" rel="noopener nofollow ugc">Here is the link</a></p>
<p>The standard training piece of code looks like</p>
<pre><code class="lang-auto">        for i, (imgs, annotations) in enumerate(data_loader):
            imgs = list(img.to(device) for img in imgs)
            annotations = [{k: v.to(device) for k, v in t.items()} for t in annotations]
            loss_dict = model(imgs, annotations)
</code></pre>
<p>If i use it with yolov5s</p>
<pre><code>model = torch.hub.load('ultralytics/yolov5', 'yolov5s', classes=num_classes, pretrained=False, autoshape=False)
</code></pre>
<p>It first fails because image tensors are in list, it seems it wants a single image tensor…</p>
<pre><code class="lang-auto">...
  File "C:\Users\Windows/.cache\torch\hub\ultralytics_yolov5_master\models\yolo.py", line 208, in forward
    return self._forward_augment(x)  # augmented inference, None
  File "C:\Users\Windows/.cache\torch\hub\ultralytics_yolov5_master\models\yolo.py", line 212, in _forward_augment
    img_size = x.shape[-2:]  # height, width
AttributeError: 'list' object has no attribute 'shape'
</code></pre>
<p>When I provide single image tensor and anno it now fails with</p>
<pre><code class="lang-auto">...
  File "C:\Users\Windows/.cache\torch\hub\ultralytics_yolov5_master\models\common.py", line 313, in forward
    return torch.cat(x, self.d)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 22 but got size 21 for tensor number 1 in the list.
</code></pre>
<p>Depending on a run the numbers are different with “expected size to be X, but got X-1”.<br>
I cannot understand why this happens and how to fix it.</p>
<p>Any ideas where to look for?</p>

shnmka: <p>I’ve been trying to do the same thing that you did and came across the same exact errors. I tried to to just pass the image into the model and it does give me 3 tensors as output. But that doesnt make sense because my model is in train mode and it should take in annotations along with image  tensor. Let me know if you could find a workaround.</p>
What are the causes/solutions of nccl unpredictable behavior?

Ascrypto: <p>I have an issue where the same script with the same settings, in terms of memory and number of GPUs,…etc runs correctly (I can see outputs and errors) at one point, but then when I rerun it, it suddenly hangs (no output, no errors). It has slowed me down significantly since I have no clue what the issue is.</p>
<ul>
<li>The port number is updated for each execution.</li>
<li>I do wait for some time before rerunning it, and I do execute only one job at a time  (just in case).</li>
<li>I have a proper initialization and closing of nccl connections in the code.</li>
<li>For export NCCL_DEBUG=INFO, it stops at<br>
[0] NCCL INFO comm 0x7f7b7c002e10 rank 0 nranks 1 cudaDev 0 busId 1000 - Init COMPLETE,<br>
but none of the print statements in the code gets executed afterward.</li>
</ul>
<p>Is there something I could do to not get stuck at this? Am I missing something? The issue above also happens sometimes when executing one code: part of the code gets executed and then it hangs on the rest.</p>
How to utilize GPUs to accelerate my code?

imxtx: <p>Hi, I have a function <code>render_texture</code> that implements the <a href="https://www.javatpoint.com/computer-graphics-z-buffer-algorithm" rel="noopener nofollow ugc">z-buffer algorithm</a> to render texture, but it’s very slow. I need to backpropagate through this function. How can I accelerate it by utilizing GPU’s parallelization?</p>
<pre><code class="lang-python">def render_texture(vertices, colors, triangles, h, w, c=3, device="cpu"):
    """render mesh by z buffer. vertices,colors, and triangles are tensors
    Args:
        shape of vertices: 3 x n vertices
        shape of colors: 3 x n vertices
        shape of triangles: 3 x n triangles
        h: height
        w: width
    """
    # initial
    image = torch.zeros((h, w, c)).to(device)

    depth_buffer = torch.zeros([h, w]) - 999999.0
    # triangle depth: approximate the depth to the average value of z in each
    # vertex(v0, v1, v2), since the vertices are closed to each other
    tri_depth = (
        vertices[2, triangles[0, :]]
        + vertices[2, triangles[1, :]]
        + vertices[2, triangles[2, :]]
    ) / 3.0
    tri_tex = (
        colors[:, triangles[0, :]]
        + colors[:, triangles[1, :]]
        + colors[:, triangles[2, :]]
    ) / 3.0

    for i in range(triangles.shape[1]):
        tri = triangles[:, i]  # 3 vertex indices

        # the inner bounding box
        umin = max(int(torch.ceil(torch.min(vertices[0, tri]))), 0)
        umax = min(int(torch.floor(torch.max(vertices[0, tri]))), w - 1)

        vmin = max(int(torch.ceil(torch.min(vertices[1, tri]))), 0)
        vmax = min(int(torch.floor(torch.max(vertices[1, tri]))), h - 1)

        if umax &lt; umin or vmax &lt; vmin:
            continue

        for u in range(umin, umax + 1):
            for v in range(vmin, vmax + 1):
                if tri_depth[i] &gt; depth_buffer[v, u] and isPointInTriangle(
                    [u, v], vertices[:2, tri].cpu().numpy()
                ):
                    depth_buffer[v, u] = tri_depth[i]
                    image[v, u, :] = tri_tex[:, i]
    return image

def isPointInTriangle(point, tri_points):
    """Judge whether the point is in the triangle
    Method:
        http://blackpawn.com/texts/pointinpoly/
    Args:
        point: [u, v] or [x, y]
        tri_points: three vertices(2d points) of a triangle. 2 coords x 3 vertices
    Returns:
        bool: true for in triangle
    """
    tp = tri_points

    # vectors
    v0 = tp[:, 2] - tp[:, 0]
    v1 = tp[:, 1] - tp[:, 0]
    v2 = point - tp[:, 0]

    # dot products
    dot00 = np.matmul(v0.T, v0)
    dot01 = np.matmul(v0.T, v1)
    dot02 = np.matmul(v0.T, v2)
    dot11 = np.matmul(v1.T, v1)
    dot12 = np.matmul(v1.T, v2)

    # barycentric coordinates
    if dot00 * dot11 - dot01 * dot01 == 0:
        inverDeno = 0
    else:
        inverDeno = 1 / (dot00 * dot11 - dot01 * dot01)

    u = (dot11 * dot02 - dot01 * dot12) * inverDeno
    v = (dot00 * dot12 - dot01 * dot02) * inverDeno

    # check if point in triangle
    return (u &gt;= 0) &amp; (v &gt;= 0) &amp; (u + v &lt; 1)
</code></pre>
Policy outputs the same thing for any state

Viktor_Todosijevic: <p>So I trained a PPO model using torchrl and what’s happening is that douring training the model has very little variation in it’s actions but during validation it’s the exact same vector of actions every time, no variation whart so ever.</p>
<p><strong>I just wanted to ask if this is a common problem in RL or if I should be looking for a bug in my code?</strong></p>
<p>Here’s some more details: my observation space is of shape (309, ) and the action space is of shape (103,) both continuous, but actions being limited [0, 1]. I do train them as [0, 1], I heard it’s better to do [-1, 1] so maybe that’s the issue.</p>
<p>This is my PPO config:</p>
<pre><code class="lang-auto">name: ppo

actor:
  width: 512
  number_hidden: 3

critic:
  width: 512
  number_hidden: 3

clip_epsilon: 0.2
critic_coef: 1.0
entropy_eps: 1.0e-4
frames_per_batch: 1024
gamma: 0.99
lmbda: 0.95
learning_rate: 3.0e-3
loss_critic_type: smooth_l1
max_grad_norm: 1.0
num_epochs: 256
sub_batch_size: 512
</code></pre>

vmoens: <p>Hi <a class="mention" href="/u/viktor_todosijevic">@Viktor_Todosijevic</a><br>
Can you show how you build your model?<br>
I’d be glad to see what the policy looks like</p>
How to use CTPN detect result & RCNN recognize result on mobile?

charleybin: <p>Hi, I am new to pytorch, I’m doing it on a OCR project, it should works on mobile platform. I’m having some difficult to use it. can anyone help please !!</p>
<p>[Question]:<br>
1.I wanna know how to use my CTPN model detect the regions from giving picture,  whether my code is correct ?<br>
2. how to use output data, every thing is Tensor,  how to found the region result from those Tensor result ?<br>
3. A further question about this, I suppose the predict result of the RCNN detection is a tuple array that included some float array or tensor inside.  so,  how to convert those float data to a  text character ?</p>
<p>[My Steps]:</p>
<ol>
<li>checked out a  CTPN open source project and trained a model on my machine with Nvidia 3090 video card. ( there is a small set of labels and images inside of the project.)</li>
<li>I converted the checkpoint file to a mobile optimized format, and I have successful loaded this model file on my Android phone.</li>
<li>I picked up a picture from training set and pushed it into my android phone.</li>
</ol>
<p>Below is the code : (java)</p>
<p>// load picture file<br>
Bitmap bmp = BitmapFactory.decodeFile(picturePath.getAbsolutePath());</p>
<p>// convert picture to a tensor object.<br>
final Tensor inputTensor = TensorImageUtils.bitmapToFloat32Tensor(bmp, TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, TensorImageUtils.TORCHVISION_NORM_STD_RGB);</p>
<p>// load CTPN model file.<br>
Module mDetector=LiteModuleLoader.load(Utils.assetFilePath(this,“ctpn_v3_mobile_ep27.ptl”));</p>
<p>// detect the label regions.<br>
IValue detValue = mDetector.forward(IValue.from(inputTensor));</p>
<p>// output data is a tuple.<br>
IValue[] tuple = detValue.toTuple();<br>
for (int i = 0;i &lt; tuple.length; i++) {<br>
IValue val = tuple[i];<br>
Tensor tens = val.toTensor();<br>
float[] data = tens.getDataAsFloatArray();<br>
//////////////////////////////// then,  how to use the result data ???<br>
}</p>
SoA methods to restrict regression predictions to a closed interval

josmi9966: <p>I would like to predict a numerical value from the range 0 &lt;= y &lt;= 1, where the values are approximately evenly distributed in that interval, and more importantly, the values 0 and 1 actually occur quite often (unlike the theoretical case of a continuous distribution, the values to predict are obviously from a final set and the interval border values have a non-zero probability of occurring)</p>
<p>I will see values from that interval in the training set and I expect values from that interval (including the interval boundaries) to also occur in the output.</p>
<p>Is there any scientific insight into which approach would be best for this? The main recommendation I have seen so far are:</p>
<ul>
<li>sigmoid function: will restrict the output to 0…1 but warps the input in a way where exponentially bigger and bigger inputs are needed to get close to 0 or 1 and those values can actually never be reached</li>
<li>clamping: just pretend nothing is special and clamp the output</li>
<li>penalize out of interval values in the cost function, maybe combine with clamping</li>
<li>I believe there are more advanced techniques to train a NN with constraints, but I am not sure how exactly they work and which would apply here.</li>
</ul>
<p>I assume the basic approach should apply to any NN architecture where the target variable value is from a closed interval, ultimately I want to apply this to a convolutional RNN for predicting time series data where the prediction is sensor measurement from a fixed range.</p>
How to update a tensor in a loop?

imxtx: <p>Hi, there. I’m writing a function for texture rendering that needs to update the rendered image in a loop. I have to deal with the z-buffer in a loop because it stores depth information that decides whether a pixel occurs front or back. Here is the code (in Numpy):</p>
<pre><code class="lang-python">def render_texture(vertices, colors, triangles, h, w, c=3):
    """render mesh by z buffer
    Args:
        vertices: 3 x nver
        colors: 3 x nver
        triangles: 3 x ntri
        h: height
        w: width
    """
    # initial
    # print(vertices.shape, colors.shape, triangles.shape)
    # exit(0)
    image = np.zeros((h, w, c))

    depth_buffer = np.zeros([h, w]) - 999999.0
    # triangle depth: approximate the depth to the average value of z in each vertex(v0, v1, v2), since the vertices are closed to each other
    tri_depth = (
        vertices[2, triangles[0, :]]
        + vertices[2, triangles[1, :]]
        + vertices[2, triangles[2, :]]
    ) / 3.0
    tri_tex = (
        colors[:, triangles[0, :]]
        + colors[:, triangles[1, :]]
        + colors[:, triangles[2, :]]
    ) / 3.0

    for i in range(triangles.shape[1]):
        tri = triangles[:, i]  # 3 vertex indices

        # the inner bounding box
        umin = max(int(np.ceil(np.min(vertices[0, tri]))), 0)
        umax = min(int(np.floor(np.max(vertices[0, tri]))), w - 1)

        vmin = max(int(np.ceil(np.min(vertices[1, tri]))), 0)
        vmax = min(int(np.floor(np.max(vertices[1, tri]))), h - 1)

        if umax &lt; umin or vmax &lt; vmin:
            continue

        for u in range(umin, umax + 1):
            for v in range(vmin, vmax + 1):
                if tri_depth[i] &gt; depth_buffer[v, u] and isPointInTri(
                    [u, v], vertices[:2, tri]
                ):
                    depth_buffer[v, u] = tri_depth[i]
                    image[v, u, :] = tri_tex[:, i]
    return image
</code></pre>
<p>I want to rewrite the code with PyTorch tensor to optimize the input through backpropagation. I know that we can’t do an in-place update for a tensor that requires grad. How should I deal with the tensor update in the for loop?</p>

imxtx: <p>I figured it out. Just replace <code>np</code> with <code>torch</code>.</p>
Pytorch not working with Nvidia 4090

Sumit_Anil_Kshirsaga: <p>NVIDIA GeForce RTX 4090 with CUDA capability sm_89 is not compatible with the current PyTorch installation.<br>
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.<br>
If you want to use the NVIDIA GeForce RTX 4090 GPU with PyTorch, please check the instructions at <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a></p>
<p>My OS is Ubuntu 18.04.01<br>
Please help me solve this issue…<br>
I can provide more information, if you tell me what Info I should provide</p>

ptrblck: <p>Based on the error message you have installed PyTorch binaries with CUDA 10.2, which is incompatible with your 4090.<br>
Install the latest or nightly release with CUDA 11.7 or 11.8 (I would recommend using the nightly release with CUDA 11.8 for the best performance) using the install commands from <a href="https://pytorch.org/get-started/locally/">here</a>.</p>

Sumit_Anil_Kshirsaga: <p>How did you figure out it was cuda 10.2 earlier?<br>
I did the following and solved my issue:<br>
conda remove pytorch<br>
conda remove cudatoolkit<br>
Then the command from pytorch instructions from your link<br>
Thank you</p>

ptrblck: <aside class="quote no-group" data-username="Sumit_Anil_Kshirsaga" data-post="3" data-topic="173054">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sumit_anil_kshirsaga/48/57562_2.png" class="avatar"> Sumit_Anil_Kshirsaga:</div>
<blockquote>
<p>How did you figure out it was cuda 10.2 earlier?</p>
</blockquote>
</aside>
<p>The supported architectures were reported as:</p>
<pre data-code-wrap="python"><code class="lang-python">The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
</code></pre>
<p>which we used in all CUDA 10 builds. Unless you were using a custom build and set the architectures manually, your wheels were most likely using CUDA 10.2. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
Python CNN for Image recognition

dev2y: <p>Hi, I wanted to my own model and I took this example from github,</p>
<p><a href="https://github.com/ajinkya98/PyTorchCNN/blob/master/train.py" rel="noopener nofollow ugc">github.com</a></p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.utils.data import DataLoader
import torchvision.transforms as transforms
from Model import CNN
from Dataset import CatsAndDogsDataset
from tqdm import tqdm


device = ("cuda" if torch.cuda.is_available() else "cpu")

transform = transforms.Compose(
        [
            transforms.Resize((356, 356)),
            transforms.RandomCrop((299, 299)),
            transforms.ToTensor(),
            transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
        ]
    )

</code></pre>
<p>This file has been truncated. <a href="https://github.com/ajinkya98/PyTorchCNN/blob/master/train.py" rel="noopener nofollow ugc">show original</a></p>
<p>I replaced the model with the following.</p>
<blockquote>
<p>model = nn.Sequential(<br>
nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, stride=1, padding=1),<br>
nn.ReLU(),<br>
nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3, stride=1, padding=1),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2),<br>
nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, stride=1, padding=1),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2),<br>
nn.Flatten(),<br>
nn.Linear(in_features=64 * 74 * 74, out_features=256),<br>
nn.ReLU(),<br>
nn.Linear(in_features=256, out_features=1)<br>
).to(device)</p>
</blockquote>
<p>But I am getting the following error at</p>
<blockquote>
<p>loss = criterion(outputs, labels).</p>
</blockquote>
<blockquote>
<p>ValueError: Using a target size (torch.Size([32])) that is different to the input size (torch.Size([32, 1])) is deprecated. Please ensure they have the same size.</p>
</blockquote>
<p>Thank you,</p>

ptrblck: <p>Your <code>criterion</code> is <code>nn.BCELoss()</code> which expects a model output and target in the same shape.<br>
You should thus <code>.unsqueeze</code> the tensor in <code>dim1</code> and the error should disappear:</p>
<pre data-code-wrap="pthon"><code class="lang-plaintext">labels = labels.unsqueeze(1)
</code></pre>
<p>Also, I would recommend replacing <code>nn.BCELoss</code> with <code>nn.BCEWithLogitsLoss</code> for better numerical stability. <code>nn.BCEWithLogitsLoss</code> expects raw logits so remove the <code>torch.sigmoid</code> used on the model output.</p>

dev2y: <p>Thank you. <img src="https://discuss.pytorch.org/images/emoji/apple/pray.png?v=12" title=":pray:" class="emoji" alt=":pray:" loading="lazy" width="20" height="20"></p>
<p>I changed <code>nn.BCELoss</code> with <code>nn.BCEWithLogitsLoss</code> and added <code>labels = labels.unsqueeze(1)</code></p>
<p>I am now getting the following error at like <code>loss.backward()</code></p>
<p><code>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</code></p>
<p><strong>Update</strong>:</p>
<p><code>for name, param in model.named_parameters():         param.requires_grad = True</code></p>
<p>I set it true and it seems to be working. Thanks</p>
Why USE_CUDA must be 0 when XLA_CUDA=1

qipengh: <p>when I use pytorch/XLA on GPUS with docker，I found torch.cuda.is_available()==False。why？IF i want to use XLA and cuda， like pytorch_lightning，pl.Trainer with (accelerator=‘gpu’,strategy=‘xla’,）. It will be error of that not found GPUS.</p>

JackCaoG: <p>Thanks for raising this issue. I think there are two part of this issue</p>
<ol>
<li>We disabled the pytorch wheel with <code>USE_CUDA=0</code> since there was a old bug that we can’t have both pytorch and pytorch/xla both trying to grab the CUDA device. That issue might or might not still be the case.</li>
<li>The lighting’s naming of <code>accelerator</code> is a bit misleading. What you really want to do is to set <code>(accelerator=‘tpu’,strategy=‘xla’,）</code> since <code>accelerator=‘tpu’</code>  really just means <code>accelerator=‘xla’</code> then you can manually set XLA device to be GPU with <code>GPU_NUM_DEVICES=1</code>. We have never tested this use case, if you run into any issue, please file a feature request to pytoch lighting. We can work with them to enable Pytorch/XLA GPU using lighting.</li>
</ol>

adrianwaelchli: <p>That’s a good observation, thanks for raising this. We have a feature request here: <a href="https://github.com/Lightning-AI/lightning/issues/16130" class="inline-onebox" rel="noopener nofollow ugc">Integrating pytorch XLA when using multiple GPUs · Issue #16130 · Lightning-AI/lightning · GitHub</a> that will support accelerator=gpu + strategy=xla.</p>
<p><a class="mention" href="/u/qipengh">@qipengh</a> I suggest you comment on that issue with your use case. To raise the attention and so that the priority can be bumped and eventually be worked on :)) Others have also already asked for this.</p>
<p>EDIT: I just realized you already did! Sorry</p>

qipengh: <p>OK，Thanks. I will try it, the reply is helpful.</p>
Inductor vs. Extending Dispatcher?

kirayz: <p>First of all, great job on PyTorch2! I have had lots of fun playing with the new features, and thanks a lot for the great work!</p>
<p>I am currently looking at inductor to see if I can connect PT2 to a customized backend + FPGA-style hardware, but have been having a hard time trying to find the right way to do it. My questions are as follows:</p>
<ol>
<li>
<p>CoreATen vs. Prims IR: My understanding is that type promotion and broadcasting is handled at Prims level, a lower level than CoreATen. Hypothetically, if I am generating accelerator code from mid-level CoreATen graph (before lowering is called to get inductor IR), should I handle the type promotion and broadcasting myself? If so, where can I look up these rules?</p>
</li>
<li>
<p>Organizing code for target-specific code generation: What is the convention to organize codegen-specific functions to maximize reusing implementations given in PT2?<br>
For example, let’s say that we have a dumb hardware that can only run addmm.</p>
</li>
</ol>
<ul>
<li>If we want to do codegen at fxGraph level, what would be the file and function we need to override to emit this dumb_hardware.addmm?</li>
<li>If we want to do codegen at inductor IR level (loop-based IR), what would be the file and function to override for running dumb_hardware.addmm? Are there places for controlling the loop style, e.g., reduction loop vs. systolic array loops vs. SIMD loops, etc…</li>
</ul>
<p>Any pointers will be greatly appreciated. Thanks so much!</p>
Problem with in-place op

Holop_yolo: <p>Trying to find in-place operations but don’t see anything wrong. Clone() or detach() don’t work also for <strong>hdn</strong><br>
Here is error:<br>
<strong>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1130, 120]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead.</strong></p>
<p>here is Rnn’s class:</p>
<pre><code class="lang-auto">import torch.nn as nn



class RNN_One_To_Many(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(RNN_One_To_Many, self).__init__()

        self.hidden_size = hidden_size

        self.i2h = nn.Linear(input_size + hidden_size, hidden_size)
        self.i2o = nn.Linear(input_size + hidden_size, output_size)
        self.softmax = nn.LogSoftmax(dim=1)

    def forward(self, input, hidden):
        combined = torch.cat((input, hidden), 1)
        hidden = self.i2h(combined)
        output = self.i2o(combined)
        output = self.softmax(output)
        return output, hidden

    def initHidden(self, batch_size):
        return torch.zeros(batch_size, self.hidden_size)
</code></pre>
<p>And my train loop:</p>
<pre><code class="lang-auto">model = RNN_One_To_Many(1010, 120, 1010)
lr = 0.01

cr = torch.nn.CrossEntropyLoss()

opt = torch.optim.Adam(model.parameters(), lr)

with torch.autograd.set_detect_anomaly(True):

    for i, numwords in enumerate(train_d):

        opt.zero_grad()
        hdn = model.initHidden(1)
        out, hdn = model(numwords[0], hdn)
        dummytensor = torch.zeros((1,1010))
        
        #numwords.shape(valueofwordsInSent,1,Maxvaluewords) onehotencoded
        for i1 in range(1,numwords.shape[0]):
            hdn_copy = hdn.clone()
            out, hdn = model(dummytensor, hdn_copy)
            loss = cr(out, numwords[i1])
            loss.backward()
            opt.step()
            if(i % 100 == 0):
                print(loss.item())


</code></pre>

ptrblck: <p>You would need to <code>.detach()</code> the hidden state since you are currently trying to backpopagate through it multiple times.</p>
Which version of pytorch support the function torch.distributed.ring_exchange()?

1579534387: <p>Which version of pytorch support the function torch.distributed.ring_exchange()?</p>

ptrblck: <p>I don’t think this method is exposed but is used in <code>dist.batch_isend_irecv</code>.<br>
CC <a class="mention" href="/u/kwen2501">@kwen2501</a> to correct me if I’m wrong.</p>

kwen2501: <p><code>torch.distributed</code> does not have an API called <code>ring_exchange()</code>.</p>
<p>But you can express a ring-exchange pattern using <code>torch.distributed.batch_isend_irecv</code> <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>Here is an example:</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/master/torch/distributed/distributed_c10d.py#L1436-L1445">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/master/torch/distributed/distributed_c10d.py#L1436-L1445" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/master/torch/distributed/distributed_c10d.py#L1436-L1445" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/master/torch/distributed/distributed_c10d.py#L1436-L1445</a></h4>



    <pre class="onebox"><code class="lang-py">
      <ol class="start lines" start="1436" style="counter-reset: li-counter 1435 ;">
          <li>&gt;&gt;&gt; send_tensor = torch.arange(2) + 2 * rank</li>
          <li>&gt;&gt;&gt; recv_tensor = torch.randn(2)</li>
          <li>&gt;&gt;&gt; send_op = dist.P2POp(dist.isend, send_tensor, (rank + 1)%world_size)</li>
          <li>&gt;&gt;&gt; recv_op = dist.P2POp(dist.irecv, recv_tensor, (rank - 1 + world_size)%world_size)</li>
          <li>&gt;&gt;&gt; reqs = batch_isend_irecv([send_op, recv_op])</li>
          <li>&gt;&gt;&gt; for req in reqs:</li>
          <li>&gt;&gt;&gt;     req.wait()</li>
          <li>&gt;&gt;&gt; recv_tensor</li>
          <li>tensor([2, 3])     # Rank 0</li>
          <li>tensor([0, 1])     # Rank 1</li>
      </ol>
    </code></pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
Transformer outputting a constant vector in binary classification

Cai_Py: <p>I am new to PyTorch and I’m attempting to train a transformer model on biological sequences to output a binary classification. I have a functional training regiment in the sense that it iterates without errors, however my loss function never decreases and my validation accuracy is constant at 50%. Inspecting the tensor output from one forward pass shows that it is a tensor of constant values. I’m assuming something is wrong with my forward method (perhaps my max pooling?).</p>
<p>My input is of shape (n, 100, 4), where n is the number of samples, 100 is the sequence length, and 4 is the one-hot encoding of DNA.</p>
<p>Here is my model definition</p>
<pre><code class="lang-auto">class NeuralNetwork(nn.Module):
	'''
	Build a neural network transformer for one-hot encoded DNA sequences
	'''
	def __init__(self):
		super(NeuralNetwork, self).__init__()
		self.transformer1 = nn.TransformerEncoderLayer(
			d_model= 4,
			nhead= 2,
			batch_first= True,
			dim_feedforward= 1024
		)
		self.transformer2 = nn.TransformerEncoderLayer(
			d_model= 4,
			nhead= 2,
			batch_first= True,
			dim_feedforward= 1024
		)
		self.transformer3 = nn.TransformerEncoderLayer(
			d_model= 4,
			nhead= 2,
			batch_first= True,
			dim_feedforward= 1024
		)
		self.linear = nn.Linear(
			in_features= 4,
			out_features=1
		)

	def forward(self, x):
		x= self.transformer1(x)
		x= self.transformer2(x)
		x= self.transformer3(x)
		x= torch.max(x, dim=1)[0]
		logits = self.linear(x)
		return torch.flatten(logits)
</code></pre>
<p>I am using a BCEwithLogitsLoss and the Adam optimizer. If I print the value of  <code>output= model(input)</code> I get a constant tensor.</p>
<p>Let me know if I need to include more details, I am attempting to not overclog the post. I posted this in NLP as I assumed you are the resident experts on transformers. Please correct me if I should move it.</p>

ptrblck: <p>I can properly overfit random samples using your model:</p>
<pre><code class="lang-python">model = NeuralNetwork()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()

x = torch.randn(8, 100, 4)
target = torch.randint(0, 2, (8,)).float()

for _ in range(1000):
    optimizer.zero_grad()
    out = model(x)
    loss = criterion(out, target)
    loss.backward()
    optimizer.step()
    print("loss {:.6f}".format(loss.item()))

print(out)
# tensor([-3.4665,  2.0541, -3.1483,  4.0016, -3.1309, -3.1514,  3.8889,  4.5529],
#        grad_fn=&lt;ReshapeAliasBackward0&gt;)
print(target)
# tensor([0., 1., 0., 1., 0., 0., 1., 1.])
</code></pre>
<p>so you might need to experiment more with the hyperparameters of your training.</p>
Yolo Custom Training Help: RuntimeError: Sizes of tensors must match except in dimension 1

ArkLade: <p>Hello all,<br>
I was trying out YoloV8 custom training for segmentation from Roboflow tutorial.<br>
I started with loading my labeled dataset from Roboflow.</p>
<pre><code class="lang-auto">!mkdir {HOME}/datasets
%cd {HOME}/datasets

!pip install roboflow

from roboflow import Roboflow
rf = Roboflow(api_key="xxxxxxxxxxxx")
project = rf.workspace("suns-d").project("building-component-detection")
dataset = project.version(1).download("yolov8")
</code></pre>
<p>Then, I used the following script to start the training.</p>
<pre><code class="lang-auto">%cd {HOME}

!yolo task=segment mode=train model=yolov8s-seg.pt data={dataset.location}/data.yaml epochs=10 imgsz=640
</code></pre>
<p>But “<strong>RuntimeError: Sizes of tensors must match except in dimension 1</strong>” shows up and not sure how to deal with it. Help is much appreciated.</p>
<p>Below is my current Colab script that I am working on (reference:<a href="https://github.com/roboflow/notebooks" class="inline-onebox" rel="noopener nofollow ugc">GitHub - roboflow/notebooks: Set of Jupyter Notebooks linked to Roboflow blog posts and used in our YouTube videos.</a>).</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/1t2Erep0E-n0hPCsT-pp98whEA_V4oRWg?usp=sharing">
  <header class="source">
      <img src="https://ssl.gstatic.com/colaboratory-static/common/cbc39b2d1c5f58b7a7c4a9fa346421a9/img/favicon.ico" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1t2Erep0E-n0hPCsT-pp98whEA_V4oRWg?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/drive/1t2Erep0E-n0hPCsT-pp98whEA_V4oRWg?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


ArkLade: <p>All of my labeled images are 640x640</p>
Update weights of network by differential equation

Christian: <p>Hi,<br>
I have a problem with weight updating. I want to update the weights by differential equation, which means weight is updating time-dependently. How to implement it as the weight is also updated by gradient decent ( gradient decent goes after differential equation update).</p>
<p>Thank you so much in advance. : )</p>

Christian: <p>Sorry, I solved it. Thank you</p>
Indexing 4D tensor

theycallmefm: <p>Hi, I want to implement the <code>torch.gather</code> function, but I am confused about indexing.</p>
<p>In my case, I have 4D tensor which has the size of <code>[B, N, K, C]</code><br>
I want to sample S points from N. I have an indices tensor that has size of <code>[B, S]</code><br>
In the end, I want a tensor that has size of <code>[B, S, K, C]</code></p>
<p>How can I achieve this?</p>

ptrblck: <p>This should work:</p>
<pre><code class="lang-python">B, N, K, C, S = 2, 3, 4, 5, 6

x = torch.randn(B, N, K, C)
idx = torch.randint(0, N, (B, S))

out = x[torch.arange(x.size(0)).unsqueeze(1), idx]
print(out.shape)
# torch.Size([2, 6, 4, 5]) = [B, S, K, C]
</code></pre>

theycallmefm: <p>It seems to be working, thank you so much!</p>

Rane90: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you for your response.</p>
<p>I have a follow-up question: How do I index a more “inner” dimension?</p>
<p>Say, for the exact same setting only that I would like to index dimension 2 for <code>x</code>.</p>
<pre><code class="lang-auto">x = torch.randn(B, N, K, C)
idx = torch.randint(0, S, (B, N, S))
</code></pre>
<p>And slightly more related to my use case: what would I need to do if I have an index tensor  <code>idx.shape=(B, S)</code> which I want to use for indexing <code>dim 2 (K)</code>. Meaning that for each index in <code>dim 0 (B)</code>, I want to index the same values in <code>dim 2 (K)</code>, for all indices in <code>dim 1 (N)</code>.</p>
<p>Thank you in advance</p>

ptrblck: <p>I don’t know what your expected output shape would be as the <code>idx</code> tensor should most likely contain values in <code>[0, K]</code>, but this might work:</p>
<pre><code class="lang-python">x = torch.randn(B, N, K, C)
idx = torch.randint(0, K, (B, N, S))

out = x[torch.arange(x.size(0))[:, None, None], torch.arange(x.size(1))[:, None], idx]
print(out.shape)
# torch.Size([2, 3, 6, 5]) = [B, N, S, C]
</code></pre>

Rane90: <p>Yes, this seems to work. Thank you very much <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
How can I use the CudaCachingAllocator API?

fabian_schutze: <p>I would like to limit the max GPU memory pytorch has available to a fraction of total GPU memory. (At deployment many processes use GPU memory and I would like to ensure that deployed pytorch models do not consume to much resources).</p>
<p>Let’s say at inference, the following code is used (from the <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="noopener nofollow ugc">docs</a> )</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt; // One-stop header.

#include &lt;iostream&gt;
#include &lt;memory&gt;

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr &lt;&lt; "usage: example-app &lt;path-to-exported-script-module&gt;\n";
    return -1;
  }


  torch::jit::script::Module module;
  try {
    // Deserialize the ScriptModule from a file using torch::jit::load().
    module = torch::jit::load(argv[1]);
  }
  catch (const c10::Error&amp; e) {
    std::cerr &lt;&lt; "error loading the model\n";
    return -1;
  }

  std::vector&lt;torch::jit::IValue&gt; inputs;
  inputs.push_back(torch::ones({1, 3, 224, 224}));

  // Execute the model and turn its output into a tensor.
  at::Tensor output = module.forward(inputs).toTensor();
  std::cout &lt;&lt; output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) &lt;&lt; '\n';

  std::cout &lt;&lt; "ok\n";
}
</code></pre>
<p>The CudaCachingAllcator API seems to permit setting a limit of the total GPU memory, see <a href="https://github.com/pytorch/pytorch/blob/f84f89b1c3f2bc74512e7a7b05ae6185164a9b3e/c10/cuda/CUDACachingAllocator.h#L187" class="inline-onebox" rel="noopener nofollow ugc">pytorch/CUDACachingAllocator.h at f84f89b1c3f2bc74512e7a7b05ae6185164a9b3e · pytorch/pytorch · GitHub</a> :</p>
<pre><code class="lang-auto">
class CUDAAllocator : public Allocator {
 public:
  virtual void* raw_alloc(size_t nbytes) = 0;
  virtual void* raw_alloc_with_stream(size_t nbytes, cudaStream_t stream) = 0;
  virtual void raw_delete(void* ptr) = 0;
  virtual void init(int device_count) = 0;
  virtual bool initialized() = 0;
  virtual void setMemoryFraction(double fraction, int device) = 0;
</code></pre>
<p>Can somebody explain how I could use the <code>CudaAllocator</code> class to limit total GPU consumption in the example above?</p>

ptrblck: <p>Something like this might work:</p>
<pre><code class="lang-python">auto allocator = c10::cuda::CUDACachingAllocator::get();
allocator-&gt;setMemoryFraction(fraction, device);
</code></pre>

fabian_schutze: <p>Thanks for the comments, <a class="mention" href="/u/ptrblck">@ptrblck</a> . That was excellent inspiration. The following works in the sample program:</p>
<pre><code class="lang-auto">c10::cuda::CUDACachingAllocator::init(1);
c10::cuda::CUDACachingAllocator::setMemoryFraction(0.01, 0);
</code></pre>
<p>(Note that you have to initialize the number of available GPUs not the device id.)</p>
How to modify densenet121 model to take input with 1 channel?

Ruby_Shrestha: <p>I wanted to use densenet121 with grayscale images, so looking to change it to accept 1 channel input. I tried the following :</p>
<pre><code class="lang-auto">model = torchvision.models.densenet121(pretrained=pretrained)
        model.features[0] = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
        num_ftrs = model.classifier.in_features 
        model.classifier = nn.Linear(num_ftrs, n_classes)
</code></pre>
<p>But getting error: “RuntimeError: stack expects each tensor to be equal size, but got [1, 224, 224] at entry 0 and [4, 224, 224] at entry 17”</p>
<p>Is there any other way I can modify densenet121 to accept 1 channel input?</p>

ptrblck: <p>Your code works fine for me:</p>
<pre><code class="lang-python">model = torchvision.models.densenet121(pretrained=True)
model.features[0] = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)

x = torch.randn(2, 1, 224,224)
out = model(x)
print(out.shape)
# torch.Size([2, 1000])
</code></pre>
Does Pytorch >1.12 support aarch64?

nomaue: <p>I have aarch64 cpu, and cuda11.4.<br>
so I use</p>
<blockquote>
<p>conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch</p>
</blockquote>
<p>command but i got the following error.</p>
<pre><code class="lang-auto">PackagesNotFoundError: The following packages are not available from current channels:

  - torchaudio==0.12.1
  - cudatoolkit=11.3

Current channels:

  - https://conda.anaconda.org/pytorch/linux-aarch64
  - https://conda.anaconda.org/pytorch/noarch
  - https://repo.anaconda.com/pkgs/main/linux-aarch64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-aarch64
  - https://repo.anaconda.com/pkgs/r/noarch

</code></pre>
<p>is this error related to aarch64 architecture?</p>
<p>and when I type nvidia-smi, It shows</p>
<blockquote>
<p>NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6<br>
and 2 of NVIDIA  A100 GPU</p>
</blockquote>
<p>but for nvcc --version, I got</p>
<blockquote>
<p>Cuda compilation tools, release 11.4, V11.4.100<br>
Build cuda_11.4.r11.4/compiler.30188945_0</p>
</blockquote>
<p>does this inconsistency of CUDA version cause any error like the above?<br>
Thank you.</p>

ptrblck: <p>I don’t think ARM binary builds are provided (with the exception for Mac) and I guess you are using an ARM server CPU?<br>
If so, you might need to build PyTorch from source or use the <a href="https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch">NGC container</a>.</p>

nomaue: <p>Thank you.<br>
yes, I’m using ARM server CPU. then can I build PyTorch from source on Conda environment?</p>

ptrblck: <p>Yes, you can build PyTorch from source, too, if needed.</p>
Is there any way to detect when a backward pass reaches a certain node?

zhouxian: <p>Hi, I am trying to integrate torch’s autograd with a custom library, and I wonder if it’s possible to do the following:</p>
<pre><code class="lang-auto">a = torch.randn(5, 3, requires_grad=True)

l = a.sum()
l.backward()
</code></pre>
<p>When I call <code>l.backward()</code>, in my understanding the backward pass will go all the way from <code>l</code> back to <code>a</code>. In our library, I would like this <code>l.backward()</code> call to trigger some behavior of <code>a</code> itself. I am trying to implement a class that inherits torch tensor and use it to replace <code>a</code>, and i want it to do something when the backward pass reaches <code>a</code>. Is it possible?</p>
<p>Something like this</p>
<pre><code class="lang-auto">class CustomTensor(torch.Tensor):

    def reached_this_tensor(self):
        print('reached')

a = CustomTensor([1,2,3])
l = a.sum()
l.backward()
</code></pre>
<p>And it will print <code>reached</code>.</p>

ptrblck: <p><code>tensor.register_hook</code> might work:</p>
<pre><code class="lang-python">a = torch.randn(5, 3, requires_grad=True)
a.register_hook(lambda grad: print("reached a with grad {}".format(grad)))

l = a.sum()
l.backward()
# reached a with grad tensor([[1., 1., 1.],
#         [1., 1., 1.],
#         [1., 1., 1.],
#         [1., 1., 1.],
#         [1., 1., 1.]])
</code></pre>

zhouxian: <p>Oh wow!! Thanks so much!</p>
How to slicing libtorch forward output

donkey95: <p>hello, im new to this… i need help</p>
<p>this is my c++ code</p>
<pre><code>  // [0]Define the input tensor
  torch::Tensor veh_input2 = torch::tensor({
      {{5.006278301940284, 0.0786753188073942, 4.411626434845282, 0.09598564608132709, 4.52527292334112, 0.1367866935617143},
       {4.344146508992793, 0.034440922157513695, 4.472077232158242, 0.04202762418462361, 4.399348505369012, 0.05233431582289311},
       {5.800142409494782, 0.11272599320992382, 5.238293982154694, 0.15335229366842879, 4.269588880340237, 0.22455373622631702},
       {4.18455135464626, 0.03927927361417172, 4.165869740437117, 0.045566222570332576, 4.237983947536861, 0.055740630185644596}}
  });

  // [1]Run the model
  auto output = veh_model.forward({veh_input2});

  std::cout &lt;&lt; "==========[1]============";
  std::cout &lt;&lt; typeid(output).name() &lt;&lt; std::endl;
  std::cout &lt;&lt; output &lt;&lt; std::endl;

  ============ program die here !! ===================

  // [2]Extract the outputs
  torch::Tensor mu_veh = output.toTuple()-&gt;elements()[0].toTensor();
  torch::Tensor sigma_veh = output.toTuple()-&gt;elements()[1].toTensor();
</code></pre>
<p>and this is my output</p>
<p>==========[1]============<br>
typeid(output).name() ==&gt; struct c10::IValue</p>
<p>[(1,.,.) =<br>
5.3061  15.4691<br>
4.2534  15.5291<br>
5.9763  13.6928<br>
4.2134  15.4633<br>
[ CPUFloatType{1,4,2} ], (1,.,.) =<br>
2.5224  3.6053<br>
2.0312  4.4862<br>
2.6079  4.7206<br>
1.9234  4.2305<br>
[ CPUFloatType{1,4,2} ]]</p>
<p>i want to access each value of tensor vector like “5.3061”<br>
why this c++ code didnt work?</p>
<pre><code>  // [2]Extract the outputs
  torch::Tensor mu_veh = output.toTuple()-&gt;elements()[0].toTensor();
  torch::Tensor sigma_veh = output.toTuple()-&gt;elements()[1].toTensor();
</code></pre>
<p>if you know, let me know thanks</p>
How to build a sparse net using Pytorch

nahidalam: <p>Trying to understand how to use Pytorch to build a sparse neural net. Lets say I have the code of a Vision Transformer. I now want to create a sparsed version of this ViT. How do I get started? I see pytorch has a <code>torch.sparse</code> in beta <a href="https://pytorch.org/docs/stable/sparse.html" class="inline-onebox" rel="noopener nofollow ugc">torch.sparse — PyTorch 1.13 documentation</a></p>