akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Will it be faster if I convert the jpg datasets into pt datasets?

username: jack_S
cooked: <p>I am going to train ImageNet on a single GPU, but it is too large to put all of them in the memory for CPU. So, I’m just thinking what if I convert all the jpg files into <code>transforms.Resize</code>d pt datasets, as I don’t need to go through <code>PIL.Image.open</code> and <code>transforms.Resize</code>. Will this be faster? Or there’re some quicker methods?<br>
Owing to the large size of the dataset, I don’t have much time to test all methods that I can imagine. Therefore, any help is appreciated. <img src="https://discuss.pytorch.org/images/emoji/apple/innocent.png?v=12" title=":innocent:" class="emoji" alt=":innocent:" loading="lazy" width="20" height="20"></p>

username: ptrblck
cooked: <p>You wouldn’t have to profile the entire dataset, but just a subset estimating the expected performance. E.g. while storing tensors could save some processing in e.g. resizing you should consider if you want to use static resizing during training or a random crop followed by resizing. Also, the size difference might also be large comparing JPEG encoded images against raw tensors as described <a href="https://discuss.pytorch.org/t/best-way-to-save-and-load-lots-of-tensors/170767/2">here</a>.</p>

username: jack_S
cooked: <p>Thanks for your suggestions!</p>
Is Autocast Failing to Cast Gradients?

username: kheyer
cooked: <p>If I <code>autocast</code> to fp16, should I expect gradients to be computed in fp16 as well?</p>
<p>I’ve noticed that when I explicitly call <code>.half()</code> on a model, gradients will be computed in fp16. But when I use <code>autocast</code>, the gradients are computed in fp32. Is this the expected behavior?</p>
<p>Example:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import torch.nn as nn

torch.set_default_device('cuda')

model = nn.Linear(8,1)
opt = torch.optim.SGD(model.parameters(), lr=1e-3)

x = torch.randn(12, 8, dtype=torch.float16)

with torch.autocast(device_type='cuda', dtype=torch.float16, enabled=True):
    output = model(x)
    loss = output.mean()
    loss.backward()
    
print(model.weight.grad.dtype) # gradient is computed in fp32 despite autocast
# &gt; torch.float32

opt.zero_grad()

model.half()

with torch.autocast(device_type='cuda', dtype=torch.float16, enabled=True):
    output = model(x)
    loss = output.mean()
    loss.backward()
    
print(model.weight.grad.dtype) # after manually calling .half(), gradient is computed in fp16
# &gt; torch.float16
</code></pre>

username: ptrblck
cooked: <p>Yes, this is expected behavior as the weights of the model won’t be transformed and thus the <code>wgrads</code> will have the same <code>dtype</code>, which is <code>float32</code> by default.</p>
Calculated values goto INF with unknown reason

username: Daniel_W
cooked: <p>Hello,</p>
<p>I want to implement an invertible neural net with the loss function related to the output and the jacobian determinant of  the network’s output wrt its input. The original version of the invertible neural net gives analytical expressions on the jacobian determinant while I want to use torch’s autograd to calculate this jacobian determinant now to make it more general.</p>
<p>I implemented these two versions of codes. The version with analytically calculated jacobian works well but the version with autograd somehow fails (it runs but some of the values are INF). I tried to debug it by comparing the jacobian determinants but the weird thing is that 1. initiallly, the batched jacobian determinants align 2. Aftering training for a few iterations, for the batched jacobian determinants, some still align with the analytical values while some go to INF which confused me.</p>
<p>some results like the following: 1. use autodiff<br>
tensor([[    inf],<br>
[77.4460],<br>
[    inf],<br>
[    inf],<br>
[    inf],<br>
[73.4892],<br>
[81.2950],<br>
[    inf],<br>
[73.0275],<br>
[    inf]],</p>
<ol start="2">
<li>analytical, tensor([134.4002,  77.4460, 111.4590, 135.3076, 104.6687,  73.4892,  81.2950,<br>
99.3901,  73.0275,  97.8689], device=‘cuda:0’, grad_fn=)</li>
</ol>
<p>Anyone possibly knows the reason? if the code is needed, I could also try to show it. Thank you!</p>
Exporting `squeeze` function is not understandable

username: ha2ka
cooked: <p>I don’t understand the logic behind the parsing of the <code>squeeze</code> function when doing an ONNX export. For example, in the following sample script, the exported operators behavior changes depending on whether or not the <code>GridSample</code> function is inserted.</p>
<p>I am seeking an explanation or a link to a reference document regarding the above.</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import torch.nn as nn
import torch.nn.functional as F

import onnx
from onnxruntime.tools import pytorch_export_contrib_ops

pytorch_export_contrib_ops.register()


class Model(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x):
        a = x.new_zeros((x.size(0), x.size(1), x.size(2)))
        y = F.pad(x, [2, 2])
        x = F.interpolate(a[None], size=y.shape[-2:])
        # x = torch.nn.functional.grid_sample(x, torch.randn(1, 30, 30, 2))
        x = x.squeeze(0)
        return x


def deploy():
    M = Model().to("cpu").eval()
    X = torch.randn(20, 16, 50, 100)

    traced = torch.jit.trace(M, X)

    torch.onnx.export(
        traced,
        X,
        "tmp.onnx",
        verbose=False,
        do_constant_folding=True,
        opset_version=16,
        input_names=["input"],
    )

    model = onnx.load("tmp.onnx")
    print(onnx.helper.printable_graph(model.graph))


if __name__ == "__main__":
    deploy()
</code></pre>
<p>Thank you.</p>
"No module named 'torch'" using crontab

username: Ahmad_Yeaseen_Khan
cooked: <p>My Python code works fine if I run it from the command line by:</p>
<pre><code> $ python3 detect_people_main.py
</code></pre>
<p>Now I want to run the code from crontab using:</p>
<pre><code>@reboot sleep 30 &amp;&amp; /usr/bin/python3 /home/pip/Downloads/yolov7/detect_people_main.py &gt;&gt; /home/pip/Downloads/yolov7/logfile.log 2&gt;&amp;1 &amp;
</code></pre>
<p>while printing the logs in <code>logfile.log</code>. But the code does not work from crontab. And the log is :</p>
<pre><code>Traceback (most recent call last):
  File "/home/pip/Downloads/yolov7/detect_people_main.py", line 6, in &lt;module&gt;
    import torch
ModuleNotFoundError: No module named 'torch'
</code></pre>
<p>I am using Jetson NANO to run the code. Any suggestions to solve the issue?</p>

username: ptrblck
cooked: <p>I would guess your cron job might use another Python environment which does not have <code>torch</code> installed. You could check if e.g. <code>which python3</code> (which seems to work) is pointing to another location than <code>/usr/bin/python3</code>.</p>

username: Ahmad_Yeaseen_Khan
cooked: <p>I was running the code on reboot using crontab from the sudo user. Somehow my Python from sudo user could not find Pytorch. After switching from <code>sudo crontab -e</code> to only <code>crontab -e</code> I solved the issue for Pytorch.</p>
Runtime error after trained using anchor box transform

username: Khizer_Zakir
cooked: <p>RuntimeError                              Traceback (most recent call last)<br>
Cell In[91], line 2<br>
1 # Call the function with the modified parameters<br>
----&gt; 2 postprocess_and_export_predictions(model, test_given_loader, device, predictions, use_dems, postprocess=True, threshold=0)</p>
<p>Cell In[81], line 29, in postprocess_and_export_predictions(model, test_given_loader, device, predictions_path, use_dems, postprocess, threshold, p_threshold, all_means, all_stds)<br>
26 data_batch = data_batch.to(device)<br>
28 # Predict and apply a threshold for binary predictions<br>
—&gt; 29 outputs = model(data_batch)<br>
30 predictions = torch.sigmoid(outputs) &gt; p_threshold<br>
32 for idx, prediction in enumerate(predictions):<br>
33     # Verification based on the red band</p>
<p>File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)<br>
1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]<br>
1517 else:<br>
 → 1518     return self._call_impl(*args, **kwargs)</p>
<p>File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1527, in Module._call_impl(self, *args, **kwargs)<br>
1522 # If we don’t have any hooks, we want to skip the rest of the logic in<br>
1523 # this function, and just call forward.<br>
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks<br>
1525         or _global_backward_pre_hooks or _global_backward_hooks<br>
1526         or _global_forward_hooks or _global_forward_pre_hooks):<br>
 → 1527     return forward_call(*args, **kwargs)<br>
1529 try:<br>
1530     result = None</p>
<p>Cell In[68], line 67, in UNetModified.forward(self, input)<br>
64 x4_0 = self.conv4_0(self.pool(x3_0))<br>
66 # Use the pad_and_concat function for each concatenation step<br>
—&gt; 67 x3_1 = self.conv3_1(pad_and_concat(self.up(x4_0), x3_0))<br>
68 x2_2 = self.conv2_2(pad_and_concat(self.up(x3_1), x2_0))<br>
69 x1_3 = self.conv1_3(pad_and_concat(self.up(x2_2), x1_0))</p>
<p>Cell In[68], line 32, in pad_and_concat(x1, x2)<br>
28 x1 = F.pad(x1, [diffX // 2, diffX - diffX // 2,<br>
29                 diffY // 2, diffY - diffY // 2])<br>
31 # Concatenating along the channel dimension<br>
—&gt; 32 return torch.cat([x2, x1], dim=1)</p>
<p>RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 43 but got size 44 for tensor number 1 in the list.</p>
Pytorch Not clearing up Memory even after manually deleting the tensors

username: Abhishek_Varghese
cooked: <p>I’m iterating through batches and deleting tensors once processed. But yet torch is not clearing up the memory. The model is Conv1D based feed forward network. Here’s the code and a screenshot of memory snapshot :<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/8/b87c811dc1efb57ba02dbc0c23a49432d9ad81cd.png" data-download-href="https://discuss.pytorch.org/uploads/default/b87c811dc1efb57ba02dbc0c23a49432d9ad81cd" title="Untitled"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b87c811dc1efb57ba02dbc0c23a49432d9ad81cd_2_417x500.png" alt="Untitled" data-base62-sha1="qk2Bru5kjXcn4aGcpe9fbOFAJI1" width="417" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b87c811dc1efb57ba02dbc0c23a49432d9ad81cd_2_417x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b87c811dc1efb57ba02dbc0c23a49432d9ad81cd_2_625x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/8/b87c811dc1efb57ba02dbc0c23a49432d9ad81cd.png 2x" data-dominant-color="EFEBE9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Untitled</span><span class="informations">717×859 41.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The memory screenshot clearly shows even after variable is deleted, somehow, the tensors are still in cuda. What am I doing wrong here?</p>

username: ptrblck
cooked: <p>You are appending the output of the model to a <code>list</code>, which also stores the computation graphs and thus also intermediate tensors needed for the gradient computation. Deleting tensors manually won’t work as the <code>list</code> still stores a reference to the output and computation graph.</p>
How to obtain a result of 0 from CrossEntropyLoss?

username: Wencan
cooked: <p>I want to test if my code correctly calculates the loss value. I think I first need to get a correct fixed loss value from CrossEntropyLoss.<br>
But …<br>
Why is the output of the following code not 0?</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

inputs = torch.tensor([[1., 0., 0.],
                      [0., 1., 0.],
                      [0., 0., 1.]])
labels = torch.tensor([0, 1, 2])

criterion = nn.CrossEntropyLoss()

loss = criterion(inputs, labels)

print(loss)

</code></pre>

username: ptrblck
cooked: <p><code>nn.CrossEntropyLoss</code> expects logits (unbound), not probabilities, so pass large/small values instead of ones and zeros:</p>
<pre data-code-wrap="python"><code class="lang-python">inputs = torch.tensor([[100., 0., 0.],
                      [0., 100., 0.],
                      [0., 0., 100.]])
labels = torch.tensor([0, 1, 2])

criterion = nn.CrossEntropyLoss()

loss = criterion(inputs, labels)

print(loss)
# tensor(0.)
</code></pre>

username: Atefe_Hassani
cooked: <p>Hi <span class="mention">@ptrblck_de</span>,</p>
<p>How do we push forward or back a loss curve to be similar to a source loss curve and train at the same pace in Pytorch?</p>
<p>Thanks in advance</p>

username: ptrblck
cooked: <p>I don’t fully understand your use case. Could you describe what exactly you are working on?</p>
Freezing layers when using torch.compile

username: Tom_Bush
cooked: <p>Hello,</p>
<p>I am finetuning Wav2Vec by placing an MLP on top of its representations. I am doing this by freezing Wav2Vec for the first few epochs of training and then unfreezing it. I implement this by setting requires_grad=False for all Wav2Vec parameters. I would also like to use torch.compile to speed up training. When I try to do this, after unfreezing the Wav2Vec module (i.e. setting requires_grad=True) I get the following error when I pass inputs into my model:</p>
<p>RuntimeError: addmm(): functions with out=… arguments don’t support automatic differentiation, but one of the arguments requires grad.</p>
<p>I am pretty confused about this but I am thinking that the problem is that torch.config is building the computational graph such that the the Wav2Vec parameters don’t require gradient calculations and is hence unable to handle the epochs when I unfreeze the model. My question is how I should go about handling this - should I compile the model twice (e.g. once for the training epochs where the Wav2Vec layer is frozen and then re-compile when I unfreeze it) or am I missing something?</p>
<p>I am sorry if this is a silly question but I am really struggling to figure it out</p>
<p>Thanks in advance if anyone can point me towards anything that helps</p>

username: bdhirsh
cooked: <p>That sounds like a bug with torch.compile.</p>
<p>(1) what version of PyTorch are you using? Can you try running your example with a nightly and see if you still get the same error? (There have generally been a lot of bug fixes over the last few months).</p>
<p>(2) if you still get the error on a nightly, do you have a reproducer script that shows the issue? That would be very helpful, you can file an issue with the repro on GitHub.</p>

username: Tom_Bush
cooked: <p>Yep, using a new version of PyTorch seems to fix the issue. I was on 2.1 and running in 2.2 seems to fix the problem. Thanks!</p>
(W-F+2P)/S+1 works out to be a decimal

username: Zhangruiqi111
cooked: <p>If you have data with a height of 10, a width of 10, a number of channels of 3, and a lot size of 1, after nn.Conv2d(in_channels=3, out_channels=64, kernel_size=5, stride=2, padding=1), If the theoretical output has a height and width of 4.5, but the actual output has a height and width of 4, is it possible to show that nn.Conv2d is rounded down?</p>

username: ptrblck
cooked: <p>I’m unsure how you want to “show” the rounding, but the docs are giving you the output sizes using the floor function <code>⌊x⌋</code>, which means you will round down to the next Integer.</p>
Solution for Excercise in NLP tutorial

username: gtm2122
cooked: <p>I was trying to think of how to implement the exercise mentioned here <a href="http://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html" rel="nofollow noopener">http://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html</a></p>
<p>To augment the POS tagger I would have to find charecter level representation of each word. So I was wondering how would the ground truth look like ? For example for the word The, if I passed that to nn.Embedding(number_of_unique_letters,embed_dim) and then passed that to an nn.LSTM and then finally to a fully connected layer, what would the loss compare against ?</p>
Compatibility checks of CNN model on both image and numerical data

username: Jackson_brown
cooked: <p>Hi,</p>
<p>I’m having some confusion with the custom dataset. So, I have one folder that contain some image file and another folder contain the numerical value properties of some physics informed mechanical properties in csv. So, to find which properties is good for which physics properties, to do it, what model architecture can help with using the image folder files along with the numerical value. Does CNN works, or something need to look with UNet or DMD etc model? Any suggestions how to use a ML model for such custom dataset to handle, load and perform those evaluation. Thanks.</p>
ThreadPoolExecutor GPU utilization is 100%

username: Mahdi_Morafah1
cooked: <p>Hello,</p>
<p>I want to train several clients (models) in parallel. Here is my sequential training of the clients:</p>
<pre><code class="lang-auto">`for idx in idxs_users:
clients[idx].set_state_dict(copy.deepcopy(w_glob))
loss = clients[idx].train(is_print=False)`
</code></pre>
<p>Now I have changed my code to run the for-loop in parallel using concurrent.futures.ThreadPoolExecutor. Here is my code:</p>
<pre><code class="lang-auto">
 jobs, results = [], []
        #max_len = sum(len(user) for user in idxs_users) - 4
        torch.set_num_threads(1)
        torch.set_num_interop_threads(1)
        max_len = 10
        streams = [torch.cuda.Stream() for _ in range(max_len)]

        #torch.cuda.synchronize()
        # Execute tasks with limited concurrency
        with concurrent.futures.ThreadPoolExecutor(max_workers=min(max_len, os.cpu_count() - 1)) as executor:
            for i, (cn, idx) in enumerate([(cn, idx) for cn in range(len(idxs_users)) for idx in idxs_users[cn]]):
                stream = streams[i % max_len]  # Select stream
                #torch.cuda.synchronize()
                with torch.cuda.stream(stream):
                    jobs.append(executor.submit(train_client, clients[cn][idx], copy.deepcopy(w_glob[cn])))

        concurrent.futures.wait(jobs)
        for stream in streams:
            with torch.cuda.stream(stream):
                torch.cuda.synchronize()

        loss_locals = [job.result() for job in jobs]
</code></pre>
<p>However, the problem is that my code stuck here for hours and hours. My GPU utilization is 100% and cpu is also very high. My guess is that all task want to access GPU at the same time and this causes my code getting stuck. Could you please help? If there is any suggesting how to solve this issue or how to parallelize my client trainings and avoid for-loop?<br>
Thank you.</p>
Exporting Custom Cuda Op to Onnx

username: eduardsps
cooked: <p>I have custom cuda op that I want to export to onnx and later tensorRT. Original implementation of this op  accepted input tensors as references:</p>
<pre data-code-wrap="c++"><code class="lang-c++">void my_custom_cuda_op(
    bool attribute_1,
    const torch::Tensor &amp;input_1, const torch::Tensor &amp;input_2, const torch::Tensor &amp;input_2, torch::Tensor &amp;output)
{
</code></pre>
<p>When exporting to onnx it does not trace the op properly. If I don’t pass the output tensor but create it within the op, then op shows up in onnx graph. I guess that issue is that op is inplace?</p>
When should you *not* use custom_{fwd/bwd}?

username: garrett361
cooked: <p>From the <a href="https://pytorch.org/docs/stable/notes/amp_examples.html#functions-with-multiple-inputs-or-autocastable-ops" rel="noopener nofollow ugc">autograd and amp docs</a>, I gather that the <code>@custom_fwd</code> and <code>@custom_bwd</code> decorators let your custom autograd <code>Function</code> compose nicely w/ mixed precision. In what scenarios, then, would you <em>not</em> wrap your <code>Function</code>’s methods with these decorators?</p>
<p>I’m asking because I was a little surprised that custom <code>Function</code>s and <code>amp</code> don’t compose by default, but I’m sure there’s a good reason for it. Thanks!</p>
Efficient way to create Dataset object from synthetic data

username: kirk86
cooked: <p>Hi folks,</p>
<p>I have some function <code>sample_fn</code> creating some data on the fly calling a lot of <code>torch.rand</code> functions inside it.</p>
<p>What I’d like to do is create a <code>Dataset</code> object from it ensuring:</p>
<ol>
<li>
<p>It is efficient in terms of generating and loading the data</p>
</li>
<li>
<p>Ensures that there’s some diversity in the generated data (e.g., using <code>torch.rand</code> without passing a <code>rng</code> generator on every call).</p>
</li>
</ol>
<p>So far, my attempt is the following:</p>
<pre><code class="lang-auto">#set seed for reproducibility

random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)

# do some other stuff

# create dataset
class SyntheticDataset(tud.Dataset):

    def __init__(self, sample_fn, nb_samples, seq_len):
        self.nb_samples = nb_samples
        self.samples = sample_fn(nb_samples, seq_len) # do I need to pass rng here for 2?

    def __getitem__(self, idx):
        X = self.samples['input'][idx]
        y = self.samples['output'][idx]
        return X, y

    def __len__(self):
        return len(self.y)

# train model
</code></pre>
<p>Is there a better way to achieve those two goals?</p>
Failed to install version 2

username: nickums
cooked: <p>Excuting the install statement to upgrade to version 2 given in ‘pytorch getting started’ has simply upgraded me from 1.12 to 1.13.   Is it because I need a higher version of cuda?</p>
<p>pip install --upgrade --extra-index-url <a href="https://download.pytorch.org/whl/cu113" rel="noopener nofollow ugc">1</a> “torch==1.12.1+cu113” torchvision==0.13.1+cu113 torchaudio==0.12.1</p>

username: b30
cooked: <p>I believe you’re explicitly telling pip to install <code>torch==1.12.1+cu113</code>. Use something like<br>
<code>pip install torch==2.1.2+cu118 torchaudio torchvision --extra-index-url https://download.pytorch.org/whl/cu118</code> instead. This will install <code>torch 2.1.2</code> and latest compatible versions of <code>torchaudio</code> and <code>torchvision</code>.</p>

username: nickums
cooked: <p>I had made a mess of the command, Now installed as shown with cu118 tho with various dependency problems,<br>
[many thanks for your  helpful response,</p>
Mobile NVIDIA RTX A5500 compatibility

username: rynczakd
cooked: <p>Can the NVIDIA RTX A5500 laptop graphics card be used to work with PyTorch?</p>
<p>I checked the official <a href="https://developer.nvidia.com/cuda-gpus#compute" rel="noopener nofollow ugc">NVIDIA documentation</a> and I found that the last CUDA-Enabled NVIDIA RTX Mobile GPU is RTX A5000 - therefore, I wonder if this card is compatible with CUDA and CuDNN versions so that it can be used for neural network training?</p>
<p>I will be grateful for any advice.</p>

username: ptrblck
cooked: <p>Yes, the RTX A5500 is an Ampere GPU with compute capability 8.6 and is supported in all of our current stable and nightly binary builds.</p>

username: rynczakd
cooked: <p>Will these cards be compatible in both desktop and mobile versions?<br>
What is the reason that they are not on the official Nvidia website?</p>

username: ptrblck
cooked: <p>There is no difference in compute capability between workstation and mobile GPUs. I’m not sure what exactly you are looking for, but <a href="https://www.nvidia.com/en-us/design-visualization/rtx-a5500/">here</a> is the link to the product.</p>

username: rynczakd
cooked: <p>I am looking for Mobile RTX A5500 and the problem was that I couldn’t find this GPU here:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/6/16fba0f781e4e6d2d5028849ee16acb6ee4df11c.png" data-download-href="https://discuss.pytorch.org/uploads/default/16fba0f781e4e6d2d5028849ee16acb6ee4df11c" title="NVIDIA"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/6/16fba0f781e4e6d2d5028849ee16acb6ee4df11c_2_690x326.png" alt="NVIDIA" data-base62-sha1="3hjCdTo2SiBYfI8QsZDT4jnmxIg" width="690" height="326" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/6/16fba0f781e4e6d2d5028849ee16acb6ee4df11c_2_690x326.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/6/16fba0f781e4e6d2d5028849ee16acb6ee4df11c_2_1035x489.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/6/16fba0f781e4e6d2d5028849ee16acb6ee4df11c_2_1380x652.png 2x" data-dominant-color="E7E8E6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">NVIDIA</span><span class="informations">1696×802 75.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So I wasn’t sure if this card would be CUDA compatible and I could use it for AI in PyTorch. Just to make sure again - this card (Mobile RTX A5500) will be CUDA compatible?</p>

username: snowwolf_snowwolf
cooked: <p>yes i got same problem , installed cuda 12.1  but not worked</p>
Same Google drive dataset download error for several weeks now

username: Sam_Lerman
cooked: <pre><code class="lang-auto">dataset = torchvision.datasets.celeba.CelebA(root=dataroot, download=True)
</code></pre>
<p><code>RuntimeError: The daily quota of the file img_align_celeba.zip is exceeded and it can't be downloaded. This is a limitation of Google Drive and can only be overcome by trying again later.</code></p>
<p>For weeks now, same error.</p>

username: ptrblck
cooked: <p>CC <a class="mention" href="/u/pmeier">@pmeier</a> this issue seems to be related to <a href="https://github.com/pytorch/vision/issues/2262">this one</a> and I’m unsure if there is a workaround besides trying to manually download the data as described <a href="https://github.com/pytorch/vision/issues/2262#issuecomment-1235752527">here</a>.</p>

username: pmeier
cooked: <p>Hey <a class="mention" href="/u/sam_lerman">@Sam_Lerman</a>. Unfortunately, there is nothing we (<code>torchvision</code>) can do about it. Google Drive is a subpar way to host a dataset, but since we only provide a wrapper around what the authors provided, we have to live with it. On of the limitations is that each file on Google Drive has a daily quota and if that is met, no one can download the file for some time. And it seems you are hitting that.</p>
<blockquote>
<p>For weeks now, same error.</p>
</blockquote>
<p>If that is true, maybe try to download at a different time of the day? I don’t know when this quota is reset, but if that for example at 0:00 UTC+0 and you are in UTC-9 or the like, the quote might already be met by other people around the globe before you get a chance to try during “normal” hours. I just tried 12:00 UTC+1 and the download works. So it is not an issue with <code>torchvision</code>.</p>

username: Sam_Lerman
cooked: <p>I’ve tried downloading at all hours and just tried at 12:22 AM UTC. Still the same error. I’m thinking there’s a DoS attack or something.</p>

username: barista
cooked: <p>Same issue on my side: 2:11 PM Singapore, February 2024.</p>
Speedup forward/backward propagation

username: Estos
cooked: <p>Hello, I’m kind-off new into pytorch. I’ve been trying to implement a forward method that takes a tensor say V of n values in [0,1], picks the top k indices referring to the top-k elements in the tensor, then outputs a  tensor of the same dimension as V with the top_k values set to 1, and the rest set to 0. k is a string (in [0,1]) given to the forward method. My implementation is as follows. It seems to work but seems a bit slow, is there another implementation that could offer speedups? Here’s the code :</p>
<blockquote>
<pre><code class="lang-auto">class Top_k(torch.autograd.Function):
    @staticmethod
    def forward(ctx, V,k):

        a = V.view(1,-1).squeeze()
        ind = torch.topk(a , round(float(k) * a .shape[0]))[1]
        a = a*0
        a[ind] = a[ind] + 1
        a = a.view(V.shape)
        ctx.save_for_backward(V)
        return a

    @staticmethod
    def backward(ctx, grad_output):
        pvals = ctx.saved_tensors
        return pvals[0] * grad_output, None
</code></pre>
</blockquote>
<p>Thanks</p>
Threading/Pipelining support

username: samskalicky
cooked: <p>I see the overview about this in the doc: <a href="https://pytorch.org/executorch/stable/runtime-overview.html#threading-considerations" class="inline-onebox" rel="noopener nofollow ugc">ExecuTorch Runtime Overview — ExecuTorch documentation</a></p>
<p>But im curious if there are more use-cases that are being considered for Executorch. For example, Im particularly interested in the case where the accelerator has an asynchronous interface:</p>
<ul>
<li>enqueue execution for a given set of inputs (maybe takes a callback or returns an ID/handle)</li>
<li>either calls callback when execution completes, or some mechanism for checking status of ID/handle</li>
<li>accelerator has multiple input buffers to overlap copying inputs with previous execution</li>
<li>accelerator has multiple output buffers to overlap copying outputs with subsequent execution</li>
</ul>
<p>And how this might work if the model is entirely supported by accelerator (single blob), versus partially supported and only a subgraph is executed by accelerator.</p>
<p>Does Executorch support overlapping compute/data-movement when subgraph is executing on accelerator?</p>

username: Martin_Yuan
cooked: <p>ExecuTorch does not have assumption on hardware specification (like how acclerator works, or data is moved). However, we provide entry points so that async is possible. i.e., users can have an async call to move data inside it’s own accelerator delegate, and have a custom wait operator later when the moved data is ready.</p>