akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Problems using DataParallel

Seungjun99: <p>Hi, I just started studying pytorch recently, and want to train a model with multi GPUs, so tried an example using DataParallel.</p>
<p>But after the model is assigned to GPUs, the training does not proceed.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/3/d3389115b534606419bd0ba6cb5138a3a68debb9.png" data-download-href="https://discuss.pytorch.org/uploads/default/d3389115b534606419bd0ba6cb5138a3a68debb9" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/3/d3389115b534606419bd0ba6cb5138a3a68debb9.png" alt="image" data-base62-sha1="u8xWh8e9RNgU5l33OPjXflG3MIF" width="500" height="500" data-dominant-color="292929"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">646×646 21.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I have 8 GTX 1080Ti GPUs.</p>
<p>This is the example I tried.<br>
<a href="https://github.com/chi0tzp/pytorch-dataparallel-example/blob/master/main.py" rel="noopener nofollow ugc">Example</a></p><aside class="onebox githubblob" data-onebox-src="https://github.com/chi0tzp/pytorch-dataparallel-example/blob/master/main.py">
  <header class="source">

      <a href="https://github.com/chi0tzp/pytorch-dataparallel-example/blob/master/main.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/chi0tzp/pytorch-dataparallel-example/blob/master/main.py" target="_blank" rel="noopener nofollow ugc">chi0tzp/pytorch-dataparallel-example/blob/master/main.py</a></h4>


      <pre><code class="lang-py">import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
from torch.utils.data import Dataset, DataLoader


class RandomDataset(Dataset):
    def __init__(self, size, length):
        self.len = length
        self.data = torch.randn(length, size)

    def __getitem__(self, index):
        return self.data[index]

    def __len__(self):
        return self.len


class Model(nn.Module):
</code></pre>



  This file has been truncated. <a href="https://github.com/chi0tzp/pytorch-dataparallel-example/blob/master/main.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I got <em><strong>Let’s use 8 GPUs!</strong></em> message but no more output.</p>
<pre><code class="lang-auto">python test.py
Let's use 8 GPUs!

</code></pre>
<p>This is my torch version.</p>
<pre><code class="lang-auto">pip show torch
Name: torch
Version: 2.2.2+cu118

</code></pre>
<p>Is there anything else I need to configure to use DataParallel?</p>
Is the Concurrency and Synchronization knowledge learned in the Operating System course quite essential for deep learning model training or inference?

woongjoonchoi: <p>I would like to try multi-GPU training, but I wonder if I can get good results without understanding synchronization and concurrency.</p>

ptrblck: <p>It’s hard to tell what’s “essential”, but I would claim you can certainly benefit from Operating System knowledge as well as understanding asynchronous execution in DL.</p>

woongjoonchoi: <p>Thank you for replying. !!  It was a bit difficult to understand asynchronous execution in DL at a low level, so I am studying OS knowledge at the same time, and I am glad that it will be helpful.</p>
Quantization <> Runtime Error: Didn't find engine for operation quantized::linear_prepack NoQEngine

Harsh_Mittal: <p>Hi there!</p>
<p>I was playing around with the quantization &amp; there’s a run-time issue I’m facing while running it locally, while it works on google colab.</p>
<p>Here’s an error statement: Didn’t find engine for operation quantized::linear_prepack NoQEngine</p>
<p>I checked the torch version which is 2.2.1 in my case (local machine), while it’s 2.2.1+cu121 in the Google colab case.</p>
<p>I’d appreciate a help.</p>
<p>Thanks!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/2/e25f8117ea878e58098b03bebe078af58d827950.png" data-download-href="https://discuss.pytorch.org/uploads/default/e25f8117ea878e58098b03bebe078af58d827950" title="Screenshot 2024-05-09 at 10.09.46 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e25f8117ea878e58098b03bebe078af58d827950_2_690x475.png" alt="Screenshot 2024-05-09 at 10.09.46 AM" data-base62-sha1="wiAwzaEajk16AHZ015Et5ijhe3S" width="690" height="475" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e25f8117ea878e58098b03bebe078af58d827950_2_690x475.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e25f8117ea878e58098b03bebe078af58d827950_2_1035x712.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e25f8117ea878e58098b03bebe078af58d827950_2_1380x950.png 2x" data-dominant-color="252625"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2024-05-09 at 10.09.46 AM</span><span class="informations">2374×1636 505 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Can ranks use different batch patterns when calling P2P communication?

constroy: <p>Two ranks do send and recv to exchange data.<br>
Rank 0 groups them together, while rank 1 calls them separately.<br>
Is the behaviour undefined?</p>
<pre data-code-wrap="python"><code class="lang-python">send_tensor = torch.arange(2, dtype=torch.float32, device='cuda') + 2 * rank
recv_tensor = torch.randn(2, dtype=torch.float32, device='cuda')

if rank == 0:
    send_op = dist.P2POp(dist.isend, send_tensor, 1)
    recv_op = dist.P2POp(dist.irecv, recv_tensor, 1)
    reqs = dist.batch_isend_irecv([send_op, recv_op])
    for req in reqs:
        req.wait()
else:
    send_op = dist.P2POp(dist.isend, send_tensor, 0)
    recv_op = dist.P2POp(dist.irecv, recv_tensor, 0)
    reqs = dist.batch_isend_irecv([send_op])
    reqs += dist.batch_isend_irecv([recv_op])
    for req in reqs:
        req.wait()
</code></pre>
<p>I got an <code>ncclInternalError</code>. Is this defined by NCCL?</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "test_comm.py", line 30, in &lt;module&gt;
    reqs = dist.batch_isend_irecv([send_op])
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/distributed_c10d.py", line 1865, in batch_isend_irecv
    p2p_op.op(p2p_op.tensor, p2p_op.peer, p2p_op.group, p2p_op.tag)
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/distributed_c10d.py", line 1810, in _coalescing_manager
    work = group._end_coalescing(device)
torch.distributed.DistBackendError: NCCL error in: /root/pytorch/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:3608, internal error - please report this issue to the NCCL developers, NCCL version 2.20.5
ncclInternalError: Internal check failed.
Last error:
Message truncated : received 4096 bytes instead of 2048

</code></pre>
How to use index_copy_ while ignoring -1?

youkaichao1: <p>I have a code snippet to use index copy:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
torch.set_default_device('cpu')
data = torch.zeros((5, 5))
part = torch.tensor([[1, 2, 3, 4, 5], [1, 2, 3, 4, 5]], dtype=torch.float32)
index = torch.tensor([-1, 3])
data.index_copy_(0, index, part)
print(data)
</code></pre>
<p>And I want <code>-1</code> to be ignored, rather than throwing an error. Is it possible?</p>
<p>I can use <code>mask = index != -1</code>, and use <code>data.index_copy_(0, index[mask], part[mask])</code> , but <code>part[mask]</code> has variable length, and does not work for cudagraph.</p>

ptrblck: <p>You might need to expand the tensor copying all “unused” values into this additional index and then slice the output before returning it.</p>

youkaichao1: <p>Can you give a concrete example?</p>

ptrblck: <pre data-code-wrap="python"><code class="lang-python">data = torch.zeros((5+1, 5)) # add "ignore" row at the end
ignore_index = data.size(0) - 1

part = torch.tensor([[1, 2, 3, 4, 5], [1, 2, 3, 4, 5]], dtype=torch.float32)
index = torch.tensor([ignore_index, 3])
data.index_copy_(0, index, part)
print(data)
# tensor([[0., 0., 0., 0., 0.],
#         [0., 0., 0., 0., 0.],
#         [0., 0., 0., 0., 0.],
#         [1., 2., 3., 4., 5.],
#         [0., 0., 0., 0., 0.],
#         [1., 2., 3., 4., 5.]])

data = data[:-1] # remove last row
print(data)
# tensor([[0., 0., 0., 0., 0.],
#         [0., 0., 0., 0., 0.],
#         [0., 0., 0., 0., 0.],
#         [1., 2., 3., 4., 5.],
#         [0., 0., 0., 0., 0.]])
</code></pre>

youkaichao1: <p>Got it. Thanks! That’s one possible solution, although I have to reserve one slot for that padding.</p>
torch._export.verifier.SpecViolationError: Operator torch._ops.aten.linalg_vector_norm.default is not Aten Canonical

nbansal90: <p>I am trying to create a Executorch Program using the steps mentioned at <a href="https://pytorch.org/executorch/stable/getting-started-setup.html" rel="noopener nofollow ugc">Setting Up Executorch</a>. I made the changes successfully according to my model… But as I am executing the export script… I am getting this error:</p>
<pre><code class="lang-auto">torch._export.verifier.SpecViolationError: Operator torch._ops.aten.linalg_vector_norm.default is not Aten Canonical.
</code></pre>
<p>Looking back at my model, the only operator which might have caused this issue seems to be:<br>
<code>q = torch.nn.functional.normalize(q, dim=-1)</code>, But I am not sure how exactly do I deal with this error and am stuck at this step.</p>
<p>What could be a probable work-around for this case. Any suggestions/help is appreciated!</p>
<p>Regards!</p>
Join Join the PyTorch 2024 Summer Docathon!

svekars: <p><strong><img src="https://lh7-us.googleusercontent.com/udfo-LiOH8oukQltzTxb_ZiIJXSfHEE5lINhTZZ9i3dQuW_klnb_9duen-IEuWOt-yZmDHpL2iNVQPdjlVX6inppV7DlTbZgSnPHfEZrEvGiHALeL-1_x7MhwohDyUm94LZ7wjVCodB1X2i740b8AP0" alt="" width="624" height="315" role="presentation"></strong></p>
<p>We’re excited to invite you to the PyTorch Summer Docathon, happening from June 4 to June 18, 2024. This event is open to everyone, whether you’re an experienced PyTorch developer or a beginner. Participating in the docathon is an excellent opportunity to contribute to the PyTorch project.</p>
<p>We will be focusing on various tasks to enhance the existing documentation in the pytorch/pytorch and pytorch/tutorials repositories, as well as in some of the libraries.</p>
<p>This event is a wonderful platform to connect with other PyTorch community members and learn from each other.</p>
<p>Sounds interesting? Confirm your participation <a href="https://community.linuxfoundation.org/events/details/lfhq-pytorch-foundation-presents-pytorch-docathon-june-4-20th-2024/?__hstc=132719121.074cf0f91adace3800e276889124a78e.1708732899717.1715185360644.1715191872699.160&amp;__hssc=132719121.15.1715191872699&amp;__hsfp=3211688760" rel="noopener nofollow ugc">here</a>.</p>

ptrblck:
What is wrong with the custom dataloader, the get item function is never executed even after all possible debugs? can anyone share their insights?

Chinmay_Sahu: <pre><code class="lang-auto">class myNormalize:
    def __call__(self, data):
        print("Normalization function")
        img_normalized = data / 255.0  # Normalize to [0, 1]
        return img_normalized

class myToTensor:
    def __call__(self, data):
        print("myToTensor function")
        image = data
        return torch.tensor(image).permute(2, 0, 1)

class myResize:
    def __init__(self, size_h=256, size_w=256):
        self.size_h = size_h
        self.size_w = size_w
    
    def __call__(self, data):
        print("Resize function")
        image = data
        return T.functional.resize(image, (self.size_h, self.size_w))


class InferenceDataset():
    def __init__(self, data_path, transform=None):
        self.data_path = data_path
        self.imgs = os.listdir(data_path)
        self.data = []
        for img_name in self.imgs:
            img_path = data_path + img_name
            self.data.append(img_path)
        self.transform = transform
    def __len__(self):
        return len(self.data)
    def __getitem__(self, index):
        print(f"Fetching item at index {index}")
        img_path = self.data[index]
        print(f"Fetching item at path: {img_path}")
        img = np.array(Image.open(img_path).convert('RGB'))
        img = self.transform(img)
        
        return img, img_path

# Define the test transformer (composed of myNormalize, myResize, and myToTensor)
test_transformer = transforms.Compose([
    myNormalize(),  # Normalize to [0, 1]
    myResize(size_h=256, size_w=256),  # Resize to desired dimensions
    myToTensor(),  # Convert to PyTorch tensors
])

path_to_test_data = "../images/"

# Create an instance of the test dataset with the specified transformer
testing_dataset = InferenceDataset(path_to_test_data, transform= test_transformer)
print("Dataset Length:", len(testing_dataset))
# Create the test loader using the test dataset
test_loader = DataLoader(
    testing_dataset,
    batch_size=1,
    shuffle=False,
    num_workers=1  # Adjust this value based on your system configuration
)
</code></pre>

ptrblck: <p>Your code works fine for me after removing the undefined data folder:</p>
<pre data-code-wrap="python"><code class="lang-python">class InferenceDataset():
    def __init__(self, data_path, transform=None):
        self.data_path = data_path
        self.transform = transform
        
    def __len__(self):
        return 10

    def __getitem__(self, index):
        print(f"Fetching item at index {index}")        
        return torch.randn(3, 224, 224), torch.randint(0, 10, (1,))
</code></pre>
<p>You might want to derive your custom dataset from <code>torch.utils.data.Dataset</code> and also check the <code>len</code> of it to make sure valid images are found.</p>

Chinmay_Sahu: <p>I identified the error. It was not working because of num of workers set to 1 when I am using it in CPU. I set it to 0 and it worked.</p>
Network does not pick up target from input

AleTL: <p>Hello!</p>
<p>This is sort of a general machine learning question that I am not able to solve. I want to predict a 1-dimensional target variable. To do so, I use a neural net. I feed 100 features that should help the model to predict the target variable. Along those features, I feed the target itself, so my input is 101-dimensional. I train my network but it does not learn to pick up the target. What is interesting is that if I constrain my input to just 11 dimensions (10 random + target), it works. I have not done more tests but I don’t understand why this happens.</p>
<p>Any idea? The model is very simple, just a neural net with some activations and linear layers, without dropout or any sort of batch/layer norm. I have tried varying the capacity of my model but it still behaves this way. I tried models from 100Mi parameters to a few thousand parameters.</p>
<p>Thanks!</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/aletl">@AleTL</a>,</p>
<p>Can you share a minimal reproducible example?</p>
PyTorch on ppc64l2 cpu-only systems

eknath: <p>I have installed PyTorch on P9 (ppc64le cpu-only). Since binaries are not available for this system, I installed it from scratch. Conda list shows:<br>
torch                     2.4.0a0+git7ddf57e           dev_0    <br>
However, when I run an LLM using this, it runs extremely slow  (110x slow).  Can someone suggest what could be wrong and what measures I should take to rectify.</p>
Torchvision errors

Jim_Papavasileiou: <p>Hi there. I have a problem when running a linux bash python script.First i get “did you have libjpeg or libpng installed before building torchvision from source” as error and i saw that i need to have torch and torchvision same dependecies.If i install same dependecies and solve the problem then i get “operator torchvision::nms does not exist”. Can someone help me? Thanks in advance!</p>

ptrblck: <p>Which <code>torch</code> and <code>torchvision</code> versions are you using and how did you install them?</p>

shu: <p>Hello, I have also encountered the same issue. My Torch version is 2.2.2, and my torchvision version is 0.18. I would greatly appreciate it if you could answer my question.</p>

AidanGoldfarb: <aside class="quote no-group" data-username="Jim_Papavasileiou" data-post="1" data-topic="201606">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jim_papavasileiou/48/69575_2.png" class="avatar"> Jim_Papavasileiou:</div>
<blockquote>
<p>operator torchvision::nms does not exist</p>
</blockquote>
</aside>
<p>I had the same <a href="https://discuss.pytorch.org/t/runtimeerror-operator-torchvision-nms-does-not-exist/192829">issue</a>, turns out I had installed the CPU only binaries when I was trying to run on the GPU.</p>

Jim_Papavasileiou: <p>I have torch version 2.2.0 installed by “conda install pytorch”</p>
How to correctly backpropogate manual gradients

Aaron_Thomas: <p>Hi all,<br>
For my use case, I need to manually calculate the gradient w.r.t to the parameters of my network and do backpropagation.</p>
<p>This is how I’m currently calculating the loss and doing backprogation on the parameters:</p>
<pre><code class="lang-auto">network_optimizer.zero_grad()
network_loss = criterion(data)
gradients = network.get_gradient() #Manual gradient calculation

for param in network.parameters():
     param.grad = gradients
     param.backward(gradients)

network_optimizer.step()
</code></pre>
<p>Given my gradient calculation is correct and both the gradients tensor and prams tensor are the same shape. Does the above correctly assign the gradients to the parameters and do backpropogation w.r.t the parameters?</p>
<p>Any insight would be greatly appreciated!</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/aaron_thomas">@Aaron_Thomas</a>,</p>
<p>If you’re manually calculating gradients, why you don’t you define your own gradients within a <code>torch.autograd.Function</code> object? Then you don’t have to manually place the gradients on the <code>.grad</code> of your network’s parameters, autograd will do that for you?</p>

Aaron_Thomas: <p>Hi <a class="mention" href="/u/alphabetagamma96">@AlphaBetaGamma96</a> thank you for your response, im relatively new to pytorch, how could I write the autograd function to handle the gradients in this usecase?</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/aaron_thomas">@Aaron_Thomas</a>,</p>
<p>There’s an example tutorial in the docs here: <a href="https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html" class="inline-onebox" rel="noopener nofollow ugc">PyTorch: Defining New autograd Functions — PyTorch Tutorials 2.3.0+cu121 documentation</a></p>
How to normalize bounding boxes in Detectron2

Kamal_Moha: <p>I’m using detectron2 for object detection and solving this <a href="https://zindi.africa/competitions/local-ocean-conservation-sea-turtle-face-detection/" rel="noopener nofollow ugc">zindi challenge</a> whereby I’m tasked to detect and get the bounding boxes of the face of sea turtle(s)</p>
<p>I would like to know how to <strong>normalize</strong> the bounding boxes when creating the submission csv file. I’m able to create a decent model that draws the bounding boxes around the face of the turtle, but when I submit it to zindi, I get a low score.</p>
<p>And the reason for that is because most likely, I’m not normalizing the bbox coordinates properly.</p>
<p>For now, I’m using the below code to normalize my bbox for submission;</p>
<pre><code class="lang-auto">new_lst = [val[0]/img_w, val[1]/img_h, val[2]/img_w, val[3]/img_h]

x, y, w, h = new_lst[0], new_lst[1], new_lst[2], new_lst[3]
</code></pre>
<p>whereby;</p>
<p>val = list of actual bounding boxes for example, <code>val = [207.4368, 185.5899, 368.4472, 312.9465]</code></p>
<p>img_w and img_h is the width &amp; height of the image respectively. For example, <code>img_h = 384, img_w = 512</code></p>
<p>In colab, the model has performed well with the below metrics;<br>
<code>{'AP': 80.98161104144435, 'AP50': 98.71082301103036, 'AP75': 98.4998592575604, 'APs': 85.34653465346534, 'APm': nan, 'APl': nan}</code></p>
<p>but when I submit my csv file, my zindi leaderboard score is low because of the incorrect submission format. I’m certain the issue is with normalizing of the bounding boxes and that’s where I need help.</p>
<p>Here are some of my questions;</p>
<ol>
<li>Am I correctly normalizing the bounding boxes for submission?</li>
<li>Is there a better way to normalize the bounding boxes for better score on zindi?</li>
</ol>
<p>Kindly help me calculate the normalized bounding box. Thanks</p>
Reverse Fusion of Node Pairs in Scheduler

NiklasZ: <p>Hi,</p>
<p>I’m interested in PyTorch’s operator fusion (still relatively new, so please feel free to correct me). I have been reading the Scheduler’s <a href="https://github.com/pytorch/pytorch/blob/main/torch/_inductor/scheduler.py" rel="noopener nofollow ugc">source code</a> and noticed a strange case where it seems to consider the fusability of a pair of nodes <code>node1, node2</code> in the normal order: <code>self.can_fuse(node1, node2)</code> but also the reverse order: <code>self.can_fuse(node2, node1)</code>. Specifically, I’m referring to this code here:</p>
<pre data-code-wrap="python"><code class="lang-python">def check_all_pairs(nodes):
    for node1_index, node1 in enumerate(nodes):
        for node2 in nodes[node1_index + 1 :]:
            key = (node1, node2)
            if key in seen:
                continue
            seen.add(key)

            if self.can_fuse(node1, node2):
                possible_fusions.append(key)
            elif (node2.is_template() or node2.is_foreach()) and self.can_fuse(
                node2, node1
            ):
                # foreach fusions and epilogue fusions are order dependent
                possible_fusions.append((node2, node1))
</code></pre>
<p><a href="https://github.com/pytorch/pytorch/blob/main/torch/_inductor/scheduler.py#L1979C1-L1993C64" rel="noopener nofollow ugc">source</a></p>
<p>I inspected this function on some trivial functions via <code>torch.compile()</code> and it seems that <code>nodes</code> is typically a set containing a graph node <code>n</code> and all the nodes it outputs to <code>m_1,...m_k</code> (it’s probably not that simple, so feel free to correct me there). So we have a set (<code>n</code>, <code>m_1</code>, … <code>m_k</code>). So in the <code>self.can_fuse(node1, node2)</code> case we have:</p>
<ol>
<li>Check whether we can fuse <code>n</code> with some <code>m_i+1</code> (vertical fusion?)</li>
<li>Check whether we can fuse <code>m_i</code> with <code>m_i+1</code> (horizontal fusion?)<br>
for some <code>i</code> in <code>1...k</code>.</li>
</ol>
<p>Now my question is, when is <code>self.can_fuse(node2, node1)</code> meaningful? From my understanding this could be wrong as fusing operations in reverse can affect the overall function’s correctness.</p>
<p>Would appreciate someone’s insight here. Many thanks!</p>
Setting learning rate for Stochastic Weight Averaging

thorstenwagner: <p>Hi there,</p>
<p>I’m experimenting with stochastic weight averaging.</p>
<p>What I have done so far:</p>
<ul>
<li>Initialized the network with my current best model</li>
<li>Reduced the learning rate I used for my current best model by one order of magnitude.</li>
<li>Train for 20 epochs using EMA weighting with the following setup:<br>
<code>AveragedModel(self.model, multi_avg_fn=get_ema_multi_avg_fn(0.999))</code></li>
</ul>
<p>However, this strategy did not improve the model.</p>
<p>Now I’m wondering if the strategy I chose makes sense. My questions are</p>
<ul>
<li>Should I use a higher learning rate instead?</li>
<li>Should I average over more epochs?</li>
<li>Should I use equal weight averages instead of EMA? Or just a different alpha?</li>
</ul>
<p>I would be very grateful if someone could share their experience <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>Thanks a lot!<br>
Thorsten</p>
RuntimeError: [srcBuf length] > 0 INTERNAL ASSERT FAILED

roberto_bruzzese: <p>This error happens on Apple M3 Max Pro   during execution of the following python code :</p>
<pre><code class="lang-auto"> def forward(self, node_features, edge_index, edge_weight, batch):
        node_features = super().forward(node_features, edge_index, edge_weight, batch)
        return self.downstream_layers(node_features)
</code></pre>
<p>RuntimeError: [srcBuf length] &gt; 0 INTERNAL ASSERT FAILED at “/Users/runner/work/_temp/anaconda/conda-bld/pytorch_1712608633180/work/aten/src/ATen/native/mps/OperationUtils.mm”:383, please report a bug to PyTorch. Placeholder tensor is empty! "</p>
<p>How can I solve or turnaround it ?</p>

ptrblck: <p>Could you create an issue on GitHub, please?</p>

roberto_bruzzese: <p>I hace created it. On <a href="https://github.com/pytorch/pytorch/issues/81180" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/issues/81180</a></p>
Runtime Error on local rank using torch library

SRK195: <p>I am trying to finetune the llama model using the torch library with the cmd -  tune run lora_finetune_single_device --config llama2/7B_lora_single_device epochs=1</p>
<p>I get an Error - “RuntimeError: The local rank is larger than the number of available GPUs.”</p>
<p>I am on a Mac with Intel Core i7 and 32GB RAM. Please advise.</p>
Saving exported program from transformers library

mhs4670go: <p>Hello.</p>
<p>I’m trying to export pt2 file from llm model from transformers library and save it.</p>
<pre data-code-wrap="python"><code class="lang-python">from transformers import LlamaForCausalLM, LlamaConfig
import torch
from torch.export import export

config = LlamaConfig()
config.num_hidden_layers = 1

m = LlamaForCausalLM(config)

with torch.no_grad():
    ep = export(m, (m.dummy_inputs['input_ids'],))

torch.export.save(ep, 'llama.pt2')
</code></pre>
<p>But, when <code>torch.export.save</code> was called, it raised serialization error.</p>
<pre data-code-wrap="bash"><code class="lang-bash">NotImplementedError: No registered serialization name for &lt;class 'transformers.modeling_outputs.CausalLMOutputWithPast'&gt; found. Please update your _register_pytree_node call with a `serialized_type_name` kwarg.
</code></pre>
<p>Is there anything that I should do for the saving pt2 file?</p>
Llama2 runner for android Error

vjayd: <p>etdump library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt<br>
bundled_program library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt<br>
flatccrt library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt<br>
mpsdelegate library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt<br>
qnn_executorch_backend library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt<br>
vulkan_backend library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt<br>
quantized_kernels library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt<br>
quantized_ops_lib library is not found.<br>
If needed rebuild with the proper options in CMakeLists.txt</p>
PyTorch all_to_all_single will cause memory leak?

Bob_Liao: <p>For example, I will call <code>dist.all_to_all_single</code> in a for-loop, this will cause memory footprint increasing like:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/8/98d41a2d9480a3c0829000ecde691888031d67a9.png" data-download-href="https://discuss.pytorch.org/uploads/default/98d41a2d9480a3c0829000ecde691888031d67a9" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/8/98d41a2d9480a3c0829000ecde691888031d67a9.png" alt="image" data-base62-sha1="lNZ09BS5lOzQAtnpJ6p2UUmU6sF" width="545" height="500" data-dominant-color="464742"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1018×933 20.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>But if I comment the line which calls all_to_all_single, the memory footprint increaed slowly like below:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/8/6832cceaeb3bde0ef16ec53443e0e7bfd051154e.png" data-download-href="https://discuss.pytorch.org/uploads/default/6832cceaeb3bde0ef16ec53443e0e7bfd051154e" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/8/6832cceaeb3bde0ef16ec53443e0e7bfd051154e.png" alt="image" data-base62-sha1="eRMuMcfR7dT4hfEcmAQ8qsOtac6" width="492" height="500" data-dominant-color="484742"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">974×988 216 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So dist.all_to_all_single will cause memory leak?</p>
Converting GloVe word Embedding .txt to .pt

SecretAsianMan92: <p>Hello, I am new to python and pytorch. I am trying to use the DJL library to train an RNN to classify documents but I am missing a word embedding. I found the GloVe Word Embedding and I have downloaded it to my system as a .zip that contains the .txt file. However, when I direct DJL to load this word embedding it is throwing  FileNotFoundException: glove_6B_50d.pt file not found in:… I am guessing that I need to take the .txt out of the zip and convert it to a .pt file but I am not sure on how to do that conversion. Is this the correct assumption or is my assumption wrong? If my assumption is correct, is there an example I can take a look at to perform this conversion?</p>

ptrblck: <p>There is no standard approach to convert any text file to a serialized PyTorch file.<br>
The <code>.pt</code> ending is often used for files serialized via <code>torch.save</code>, but it also not a requirement.<br>
You can save any objects via <code>torch.save</code> and it’s also unclear what exactly the text file contains, so we would need to get more details about its content and what is expected in the <code>.pt</code> file.</p>

SecretAsianMan92: <p>The file contains many lines that start with the word followed by x amount of floats and all are separated by a single space. So for example:<br>
Hello -1.23242 0.21342134 … -0.1231<br>
…<br>
World 1.4354 -0.5432 … 0.7542</p>
<p>In regards to what the .pt needs, I am not sure. I will have to ask around to get that information.</p>

ptrblck: <p>You could use e.g. <code>pandas</code> to transform the text file to the desired numpy arrays and then convert them to tensors. However, you would still need to know what the target <code>.pt</code> is supposed to contain as e.g. <code>string</code>s are not a valid PyTorch <code>dtype</code>.</p>
UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu

mazai-xiangji: <h2><a name="system-1" class="anchor" href="#system-1"></a>System:</h2>
<p>Ubuntu22.04</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/d/5df786a465c5c61bcd9a668892ec147aaf4563c6.png" data-download-href="https://discuss.pytorch.org/uploads/default/5df786a465c5c61bcd9a668892ec147aaf4563c6" title="图片"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/d/5df786a465c5c61bcd9a668892ec147aaf4563c6.png" alt="图片" data-base62-sha1="dpgJ3dUAdv2xd0Nvk3462V4kBOm" width="611" height="500" data-dominant-color="232324"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">图片</span><span class="informations">666×545 13.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<h2><a name="problem-2" class="anchor" href="#problem-2"></a>Problem</h2>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/8/c82a67e4520fe7268e18d8d254d2514c0b39734a.png" data-download-href="https://discuss.pytorch.org/uploads/default/c82a67e4520fe7268e18d8d254d2514c0b39734a" title="图片"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/8/c82a67e4520fe7268e18d8d254d2514c0b39734a.png" alt="图片" data-base62-sha1="syKlAjrO53GRE5xunThfosKAOQG" width="690" height="65" data-dominant-color="212121"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">图片</span><span class="informations">1266×121 4.79 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I have tried NVIDIA drivers 525, 535, and 550, and encountered the same error with each. However, when I reverted to driver 515, everything worked again. I’m really not sure what the issue is. I installed PyTorch through conda, so there should not be any incompatibility issues with the CUDA version.</p>

ptrblck: <aside class="quote no-group" data-username="mazai-xiangji" data-post="1" data-topic="202299">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mazai-xiangji/48/69498_2.png" class="avatar"> mazai-xiangji:</div>
<blockquote>
<p>have tried NVIDIA drivers 525, 535, and 550, and encountered the same error with each. However, when I reverted to driver 515, everything worked again.</p>
</blockquote>
</aside>
<p>I guess your install routine failed while trying to upgrade the drivers. You could run a simple CUDA sample as a quick smoke test after installing new drivers to make sure your setup is working properly.</p>
Why did my diffusion network with file size of only 7mb blow up the video memory of the A100？

a1176552537: <p>At the beginning, I used the architecture of U-Net+ residual +selfattention, which is the basic model of ddip (four layers of downsampling, each layer outputs the image of the feature dimension [128,256,256,512], with an intermediate layer, and the total parameter size is about 600mb. During the sampling process, I input an image [1,1900,1900] with less than 40Gb of video memory. (Black font as shown)<br>
Later, I changed the basic convolutional layers (resblocks) of encoder and decoder into three convolutional layers (wa), three layers of downsampling, the average output feature dimension of each layer is 32, and the total model size is about 7mb. During the sampling process I also entered an image [1,1900,1900], indicating that I needed 180Gb of video memory.<br>
I want to know why this is and how to solve this problem?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/9/29967b912df65628a49297f8acc4366b642aaa34.png" data-download-href="https://discuss.pytorch.org/uploads/default/29967b912df65628a49297f8acc4366b642aaa34" title="mmexport1715053464084"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/9/29967b912df65628a49297f8acc4366b642aaa34_2_690x448.png" alt="mmexport1715053464084" data-base62-sha1="5VTZ0jj6FDhALAaI36omcqoqVZq" width="690" height="448" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/9/29967b912df65628a49297f8acc4366b642aaa34_2_690x448.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/9/29967b912df65628a49297f8acc4366b642aaa34_2_1035x672.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/9/29967b912df65628a49297f8acc4366b642aaa34_2_1380x896.png 2x" data-dominant-color="FEFEFE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">mmexport1715053464084</span><span class="informations">1981×1287 178 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Forward activations will most likely use the large amount of memory as explained <a href="https://discuss.pytorch.org/t/resnet-50-takes-10-13gb-to-run-with-batch-size-of-96/117402/2">here</a>.</p>
Runtime error: NotImplementedError: Cannot copy out of meta tensor; no data!

goelayu: <pre><code class="lang-auto">from transformers import LlamaConfig, LlamaForCausalLM

c = LlamaConfig(&lt;path to config.json&gt;)
with torch.device('meta'):
    m = LlamaForCausalLM(c)
    
w = torch.load(&lt;path to weights.bin file&gt;)
m.load_state_dict(w, assign=True)
m.to("cuda:0") //throws error
</code></pre>
<p>The last line throws the following error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/goelayus/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2692, in to
    return super().to(*args, **kwargs)
  File "/home/goelayus/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1152, in to
    return self._apply(convert)
  File "/home/goelayus/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 802, in _apply
    module._apply(fn)
  File "/home/goelayus/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 802, in _apply
    module._apply(fn)
  File "/home/goelayus/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 802, in _apply
    module._apply(fn)
  [Previous line repeated 2 more times]
  File "/home/goelayus/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 849, in _apply
    self._buffers[key] = fn(buf)
  File "/home/goelayus/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1150, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!
</code></pre>
<h3><a name="expected-behavior-1" class="anchor" href="#expected-behavior-1"></a>Expected behavior</h3>
<p>The model should be copied to the GPU device.</p>
... RuntimeError: mat1 dim 1 must match mat2 dim 0

Hansel77: <p>Hi there, so I am currently trying to learn about Few-Shot Learning, and I was running this code as one of the baselines for my project. I encountered the error when I was trying to change the backbone_model from “conv-4” to ResNet.</p>
<p>Here’s the error mentioned:<br>
Traceback (most recent call last):<br>
File “pretrain.py”, line 113, in <br>
logits = model(data)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py”, line 165, in forward<br>
return self.module(*inputs[0], **kwargs[0])<br>
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/home/jovyan/DiffKendall/Models/models/DiffKendall.py”, line 75, in forward<br>
return self.fc(x)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/linear.py”, line 94, in forward<br>
return F.linear(input, self.weight, self.bias)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py”, line 1753, in linear<br>
return torch._C._nn.linear(input, weight, bias)</p>
<p>I checked that my input is (128, 3, 84, 84) but I still cannot figure out what is wrong with the code. Any help is appreciated🙏🏼</p>
<p>I’ve also provided the link to the whole code here:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/kaipengm2/DiffKendall">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/kaipengm2/DiffKendall" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/c8c383530e780fa55d018907ac3a2f5abe23ee5f814645a1d0e27dd90b2f08ab/kaipengm2/DiffKendall" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/kaipengm2/DiffKendall" target="_blank" rel="noopener nofollow ugc">GitHub - kaipengm2/DiffKendall</a></h3>

  <p>Contribute to kaipengm2/DiffKendall development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


ptrblck: <p>The issue is raised in <code>self.fc</code> so you might need to check the activation shape and make sure the feature dimension matches the <code>in_features</code> in the linear layer.</p>
AttributeError: module 'torch.utils.data' has no attribute 'random_split

TMY: <p>Hello ,<br>
I am getting error “AttributeError: module ‘torch.utils.data’ has no attribute 'random_split”</p>
<p>in following code.</p>
<p>train_dataset, test_dataset = torch.utils.data.random_split(full_dataset, [train_size, test_size])</p>
<p>Anybody know the reason ?<br>
My torch.utils.data does not suggest random_split in import section even.</p>

ptrblck: <p>Which PyTorch version are you using?<br>
You can check it using <code>print(torch.__version__)</code>.<br>
The current stable version is <code>1.1.0</code>, so if you are using an older one, I would recommend to install the latest version following the instructions <a href="https://pytorch.org/get-started/locally/" rel="nofollow noopener">from the website</a>.</p>

TMY: <p>Thank you verymuch.<br>
I upgraded the version and now it is working.</p>
<p>Thanks…</p>

kkjl: <p>Hello, I have exactly the same issue with 2.3.0+cpu of PyTorch Version. After splitting : train_loader, val_loader = torch.utils.data.random_split(full_loader.dataset, [train_size, test_size]) I have AttributeError: ‘int’ object has no attribute ‘to’. Coulp you please help to fix it?</p>

AlphaBetaGamma96: <aside class="quote no-group" data-username="kkjl" data-post="4" data-topic="44668">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/51bf81/48.png" class="avatar"> kkjl:</div>
<blockquote>
<p>AttributeError: ‘int’ object has no attribute ‘to’. Coulp you please help to fix it?</p>
</blockquote>
</aside>
<p>Your variable isn’t a tensor, but is an integer (or type <code>int</code>), make sure to cast the integer to a <code>torch.tensor</code> object and you can then pass it to the GPU.</p>
Check flavour of cuda getting installed in nvcr.io/nvidia/pytorch:22.09-py3

rishabhjainps: <p>Hi,</p>
<p>We are trying to see which flavour of cuda is installed on image <code>nvcr.io/nvidia/pytorch:22.09-py3</code></p>
<p>flavors of Cuda -</p>
<p>Three flavors of images are provided: <a href="https://hub.docker.com/r/nvidia/cuda/" class="inline-onebox" rel="noopener nofollow ugc">Docker</a></p>
<ul>
<li><code>base</code>: Includes the CUDA runtime (cudart)</li>
<li><code>runtime</code>: Builds on the <code>base</code> and includes the [CUDA math libraries], and [NCCL]. A <code>runtime</code> image that also includes [cuDNN] is available.</li>
<li><code>devel</code>: Builds on the <code>runtime</code> and includes headers, development tools for building CUDA images. These images are particularly useful for multi-stage builds.</li>
</ul>
<p>Found DockerFile for <a href="https://github.com/pytorch/pytorch/blob/main/Dockerfile#L77" class="inline-onebox" rel="noopener nofollow ugc">pytorch/Dockerfile at main · pytorch/pytorch · GitHub</a>  which points to direct install of cuda, does not mention flavor gets installed. Any directions to check the installation will be helpful.</p>

ptrblck: <p>The NVIDIA NGC containers are devel containers and allow you to build your application in them or to use them as a base for your custom container. The linked <code>Dockerfile</code> is used to build the PyTorch container on docker.hub and ja unrelated.</p>
I am unable to Utlize my AMD GPU on my HP Omen Laptop

Nikhil_S: <p>The reason why I contacted you is because I have AMD ryzen 9 processor and AMD Raedon GPU. I wish to run pytorch, which is essential for deep learning models for my MSc project. When I searched the documentation, link: <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a></p>
<p>RoCm is something that I am allowed download to use pytroch, I have downloaded this, unfortunately when I ask this in my code, torch.cuda.is_available(), it always returns false, that means it is not using my GPU even though I have GPU,</p>
<p>Regarding my GPU version please refer to the snippet,</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/7/c76b55776ebf6eb32cb7a0b758cf183c9ebc0b9c.png" data-download-href="https://discuss.pytorch.org/uploads/default/c76b55776ebf6eb32cb7a0b758cf183c9ebc0b9c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c76b55776ebf6eb32cb7a0b758cf183c9ebc0b9c_2_509x500.png" alt="image" data-base62-sha1="ss8YEJRVPpwdK3lUy6CodPQuCo4" width="509" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c76b55776ebf6eb32cb7a0b758cf183c9ebc0b9c_2_509x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c76b55776ebf6eb32cb7a0b758cf183c9ebc0b9c_2_763x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c76b55776ebf6eb32cb7a0b758cf183c9ebc0b9c_2_1018x1000.png 2x" data-dominant-color="272726"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1066×1047 74 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>As per the AMD GPU support list: <a href="https://docs.amd.com/en/docs-5.3.0/release/gpu_os_support.html" class="inline-onebox" rel="noopener nofollow ugc">GPU and OS Support (Linux) — ROCm 5.3.0 Documentation Home</a></p>
<p>Pytorch is not using my GPU, even tensorflow is not using my GPU. How do I fix this? Please escalate the issue to the relevant team as this is important. Please note laptop is HP Omen.</p>

nforryan: <p>I believe this happens because only CUDA capable devices can be utilised by PyTorch. Here’s a list of them: <a href="https://developer.nvidia.com/cuda-gpus" class="inline-onebox" rel="noopener nofollow ugc">CUDA GPUs - Compute Capability | NVIDIA Developer</a><br>
Sadly I don’t think there’s any AMD GPUs as CUDA is Nvidia’s technology.</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/nikhil_s">@Nikhil_S</a>,</p>
<p>I’d recommend reading this thread on using AMD GPUs with PyTorch: <a href="https://discuss.pytorch.org/t/how-to-run-torch-with-amd-gpu/157069/6" class="inline-onebox">How to run torch with AMD gpu? - #6 by AlphaBetaGamma96</a></p>