akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Setting transforms attribute of dataset not working after dataset has been initialized

username: Megh_Bhalerao
cooked: <p>Hello, I want to set the transform attribute of a Dataset object once I have already instantiated the object - I try to do that using the setattr method of python, but it does not work - below is a minimum working example.</p>
<pre><code class="lang-auto">import torch
import numpy as np
from torch.utils.data import Dataset
import torchvision.transforms as transforms
from torch.utils.data import TensorDataset

trans = transforms.Compose([transforms.Resize(size=224), transforms.CenterCrop(size=(224, 224)), transforms.ToTensor(), transforms.Normalize(mean=(0.48145466, 0.4578275, 0.40821073), std=(0.26862954, 0.26130258, 0.27577711))])
images = torch.randn(50000, 3, 32, 32)
labels = torch.randint(9, (50000,1))
dset = TensorDataset(images, labels)
setattr(dset, 'transform', trans)
for idx, (x, y) in enumerate(dset):
    print(x.shape)
    break
</code></pre>
<p>The output of this snippet gives me -<br>
<code>torch.Size([3, 32, 32])</code></p>
<p>but rather, we should be getting<br>
<code>torch.Size([3, 224, 224])</code></p>
<p>why is setting attribute externally not working?</p>
<p>Please let me know if I am missing anything and thanks for your time!</p>

username: ptrblck
cooked: <p><code>TensorDataset</code> doesn’t have and use a <code>self.transform</code> attribute and will just directly index the passed tensors as seen <a href="https://github.com/pytorch/pytorch/blob/4c584acc5d87ece9b236424cef6474c453e8d4b3/torch/utils/data/dataset.py#L207-L208">here</a>.<br>
So technically you are creating this attribute, which is just never used.</p>

username: Megh_Bhalerao
cooked: <p>Thank you! But, isnt <code>TensorDataset</code> inherited from <code>Dataset</code>, and hence it must have the <code>transform</code> attribute?</p>

username: ptrblck
cooked: <p>It it inherited from <code>Dataset</code> but this base class also doesn’t define the <code>transform</code> attribute as seen in <a href="https://github.com/pytorch/pytorch/blob/4c584acc5d87ece9b236424cef6474c453e8d4b3/torch/utils/data/dataset.py#L41-L72">its definition</a>.</p>

username: Megh_Bhalerao
cooked: <p>Oh, thank you, that clears it!</p>

username: ptrblck
cooked: <p>With that being said, the easiest approach to apply a transformation on each tensor would be by creating a custom <code>Dataset</code>.</p>
Loss function that optimises uncertainty from MCdropout

username: hello_planet
cooked: <p>Hello,</p>
<p>I have a question regarding MC dropout. I understand that we can estimate uncertainty by taking repeated forward passes at test time with dropout enabled and grad turned off.</p>
<p>My question is, what if we wanted to use a loss function that considers uncertainty… How do we ensure that the uncertainty calculated at test time is optimized in the loss function?</p>
<p>Thank you</p>
<p>Here’s how I have calculated the uncertainty</p>
<pre data-code-wrap="mc_predictions"><code class="lang-plaintext">with torch.no_grad():
    for _ in range(100):
        mc_y_pred = model(X_train)
        mc_predictions.append(mc_y_pred)

mc_predictions = torch.stack(mc_predictions)
sigma = torch.std(mc_predictions, dim=0)

#Here y_pred could be the mean of mc_predictions
y_pred = model(X_train_batch)

loss = some_loss_func(y_true, y_pred, sigma) 
##backwards pass
optimizer.zero_grad()
loss.backward()
#update weights
optimizer.step()</code></pre>
I have a problem about intel R lris R Xe Graphics

username: johncn9
cooked: <p>File “\wav2liphp\lib\site-packages\torch\cuda_<em>init</em>_.py”, line 239, in _lazy_init<br>
raise AssertionError(“Torch not compiled with CUDA enabled”)<br>
AssertionError: Torch not compiled with CUDA enabled</p>
<p>My PC is Intel R lris Xe Graphics ,help me</p>
PyTorch - how to select from bounding boxes (3D tensor) based on desired indices (2D tensor)?

username: cdahms
cooked: <p>Suppose I have a tensor <code>batch_bboxes</code> of shape <code>(BATCH_SIZE, 10000, 4)</code>, each bbox is represented by 4 float values.<br>
I also have a tensor <code>best_batch_idxs</code> of shape <code>(BATCH_SIZE, 100)</code> where for each batch these are the 100 bboxes with the highest confidence scores that I’m trying to select.</p>
<p>The following program using a loop would work:</p>
<pre><code class="lang-auto">import torch

BATCH_SIZE = 8

batch_bboxes = torch.randn(size=(BATCH_SIZE, 10000, 4), dtype=torch.float32)

print('\n' + 'batch_bboxes.shape = ' + str(batch_bboxes.shape) + '\n')

batch_best_idxs = torch.randint(low=0, high=10000, size=(BATCH_SIZE, 100)).type(torch.int64)

print('\n' + 'batch_sorted_idxs.shape = ' + str(batch_best_idxs.shape) + '\n')

batch_selected_bboxes = torch.full(size=(BATCH_SIZE, 100, 4), fill_value=-9999.9999, dtype=torch.float32)
for batch_idx in range(BATCH_SIZE):
    bboxes = batch_bboxes[batch_idx]
    best_idxs = batch_best_idxs[batch_idx]
    selected_bboxes = bboxes[best_idxs]
    batch_selected_bboxes[batch_idx] = selected_bboxes
# end for

print('\n' + 'batch_selected_bboxes.shape = ' + str(batch_selected_bboxes.shape) + '\n')
</code></pre>
<p>If I was willing to sacrifice readability I could change the for loop part to:</p>
<pre><code class="lang-auto">for batch_idx in range(BATCH_SIZE):
    batch_selected_bboxes[batch_idx] = batch_bboxes[batch_idx][batch_best_idxs[batch_idx]]
# end for
</code></pre>
<p>But this is still slow since it’s using a loop.  Is there a way to do this without a loop, instead using some combination of <code>torch.index_select</code>, <code>torch.gather</code>, or fancy indexing?  It seems it should be possible to do this without a loop but so far I haven’t been able to work out how, any suggestions?</p>
RetinaNet (from torchvision) fine tuning

username: merfan
cooked: <p>Hi all, I want to fine-tune retinanet from torchvision using the pipeline from <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a>, but a couple of issues happening. For the model, I am altering the pretrained model with the following code for one object class:</p>
<pre><code class="lang-auto">model = torchvision.models.detection.retinanet_resnet50_fpn(pretrained=True)
num_anchors=model.head.classification_head.num_anchors
in_channels=model.head.classification_head.conv[0].in_channels
model.head=RetinaNetHead(in_channels, num_anchors, 2)
</code></pre>
<p>The issue is the evaluation gives absolute zero on every metric and loss shows nan randomly for the same data.</p>

username: Dicko
cooked: <p>Hello there, I was wondering if you managed to solve this issue? I am having similar troubles.</p>
<p>I have read that others changed the learning rate and opted for SGD instead of Adam, but I wanted to use ADAM.</p>

username: merfan
cooked: <p>I got some preliminary results using Adam. But I tried out some learning rates. I suggest doing a small grid search for a few epochs.</p>

username: Dicko
cooked: <p>Ah cool, thank you for your reply.<br>
So from your experience, working with ADAM was ok and the solution could be trying different learning rates. Also, thank you for your suggestion. I’ll try and see how to implement a grid search.</p>
[Solved]Simple question about keep dim when slicing the tensor

username: vinthony
cooked: <p>Hi,<br>
I am new in pyTorch however, I use torch in previous.</p>
<p>Here is a question bother me that how to slice the tensor and keep their dims in pytorch?</p>
<p>In torch I could write down like that:</p>
<pre><code>val = torch.rand(4,3,256,256);
val_keep = val[{{},{1},{},{}}] # output is (4x1x256x256)
val_notkeep = val[{{},1,{},{}}] # output is (4x256x256)
</code></pre>
<p>however, it seems python transformed the dims automaticly in pytorch?</p>
<pre><code>val = torch.rand(4,3,256,256);
val_notkeep = val[:,1,:,:] #output is (4x256x256)
</code></pre>
<p>so how to slice the tensor and keep the dims in pytorch?</p>

username: tom
cooked: <p>Hi,</p>
<p>in Python (in addition to pytorch this also works with lists, numpy arrays) you can do this by using a “one-element slice”</p>
<pre><code class="lang-auto">val_keep = val[:,1:2,:,:]
</code></pre>
<p>Best regards</p>
<p>Thomas</p>

username: vinthony
cooked: <p>thanks a lot! it works great!</p>

username: jon
cooked: <p>Is this the recommended practice? What if you have many dimensions? Thanks!</p>

username: Plagon
cooked: <p>Another option is to use <code>unsqueeze</code>:</p>
<pre><code class="lang-python">import torch

t = torch.ones(10, 5, 2)
t[:, 1, ...].unsqueeze(1)
# or
t.moveaxis(1, 0)[1].unsqueeze(1) 
</code></pre>
<p>Note that you can use the ellipsis <code>...</code> to instead of <code>:</code> for the remaining dimensions.</p>

username: lucasvw
cooked: <p>And yet another way of doing it, which has my (personal) preference:</p>
<pre><code class="lang-auto">torch.randn(8,1024,256)[0].shape
&gt; torch.Size([1024, 256])

torch.randn(8,1024,256)[[0]].shape
&gt; torch.Size([1, 1024, 256])
</code></pre>
<p>Or more general:</p>
<pre><code class="lang-auto">torch.randn(8,1024,256)[:,0,:].shape
&gt; torch.Size([8, 256])

torch.randn(8,1024,256)[:,[0],:].shape
&gt; torch.Size([8, 1, 256])
</code></pre>

username: MrCrHaM
cooked: <p>Just read a <a href="https://stackoverflow.com/a/18183182/12996771" rel="noopener nofollow ugc">comment</a> regarding to this kind of slicing for <code>numpy.ndarray</code>, which claimed that this operation would <strong>copy</strong> the data.</p>
<p>Is it true though? Especially for PyTorch?</p>
Select in images from 2d coordinates and compute gradient with respect to 2d coordinates

username: HassanLH
cooked: <p>Hello,</p>
<p>I have in my code 2d coordinates computed from variables I want to optimize. I have some images (which are distance maps and are considered ground truth so constant) and I wish to select in those images the pixels by indexing with my 2d coordinates and get gradient with respect to my variables.<br>
To my understanding, functions like select_index and gather only compute gradient with respect to the input and not the index.</p>
<p>Is there a way to do this in pytorch please ?</p>
Converting model to nnapi, failing at serializing

username: JosephJ
cooked: <p>I’m following <a href="https://pytorch.org/tutorials/prototype/nnapi_mobilenetv2.html" rel="noopener nofollow ugc">(Beta) Convert MobileNetV2 to NNAPI — PyTorch Tutorials 2.0.1+cu117 documentation</a> to get my model to work on android gpu device but I’m using</p>
<pre><code class="lang-auto">        traced = torch.jit.script(model, input_tensor)
</code></pre>
<p>Instead of trace, as it’s supposed to be more suitable for my model architecture (YoloX), I’ve removed the head since it’s supposed to be iffy to add it too, and converted depthwise convs to regular 2dconve both for support and as it’s supposed to run better on gpu operations, but I’m getting the following error when running the following line:</p>
<pre><code class="lang-auto">    nnapi_model = torch.backends._nnapi.prepare.convert_model_to_nnapi(traced, input_tensor)

</code></pre>
<pre><code class="lang-auto">  retval_count) = serializer.serialize_model(model, inputs, return_shapes)
  File "/envs/yolox/lib/python3.9/site-packages/torch/backends/_nnapi/serializer.py", line 677, in serialize_model
    self.add_node(node)
  File "/envs/yolox/lib/python3.9/site-packages/torch/backends/_nnapi/serializer.py", line 871, in add_node
    adder(self, node)
  File "/envs/yolox/lib/python3.9/site-packages/torch/backends/_nnapi/serializer.py", line 848, in &lt;lambda&gt;
    self.add_conv2d(node),
  File "/envs/yolox/lib/python3.9/site-packages/torch/backends/_nnapi/serializer.py", line 1779, in add_conv2d
    return self.add_conv2d_common(
  File "/envs/yolox/lib/python3.9/site-packages/torch/backends/_nnapi/serializer.py", line 1979, in add_conv2d_common
    assert kern_d == in_c
AssertionError
</code></pre>
<p>I’ve added a print to those, and I’m getting kern_d=32, in_c=64</p>
<p>Any ideas how to correct this issue?<br>
Thanks.</p>
Batch mode of Jacobian calculation: how can I change my output shape as the output shape of torch.autograd.functional.jacobian

username: S_M
cooked: <p>Hi all I am trying to calculate Jacobian matrix in the form of batch mode as my input ‘x’ has 128x1000 dimension where 128 is the batch size and 1000 is the number of classes. When I used torch.autograd.functional.jacobian it gave me ‘cuda out of memory’ error. So the batch mode was preferred. the output of the torch.autograd.functional.jacobian function in lower size is like [batch_size, n_classes, batch_size, n_classes] so I need to change my output like this 4D while my output is [batch_size, n_classes, n_classes].<br>
my code is as follows;</p>
<pre><code class="lang-auto">import torch
import torch.nn.functional as F

def gradient(y, x, grad_outputs=None):
    """Compute dy/dx @ grad_outputs"""
    if grad_outputs is None:
        grad_outputs = torch.ones_like(y)
    grad = torch.autograd.grad(y, [x], grad_outputs = grad_outputs, create_graph=True)[0]
    return grad

def jacobian(y, x):
    """Compute dy/dx = dy/dx @ grad_outputs; 
    for grad_outputs in [1, 0, ..., 0], [0, 1, 0, ..., 0], ...., [0, ..., 0, 1]"""
    jac = torch.zeros(y.shape[0], x.shape[0]) 
    for i in range(y.shape[0]):
        grad_outputs = torch.zeros_like(y)
        grad_outputs[i] = 1
        jac[i] = gradient(y, x, grad_outputs = grad_outputs)
    return jac

def divergence(y, x):
    div = 0.
    for i in range(y.shape[-1]):
        div += torch.autograd.grad(y[..., i], x, torch.ones_like(y[..., i]), create_graph=True)[0][..., i:i+1]
    return div

def laplace(y, x):
    grad = gradient(y, x)
    return divergence(grad, x)

x = torch.rand(128,1000)
w = torch.rand(1000, 1000)
b = torch.rand(1000)
j=[]
for i in range(128):
    x_p = torch.tensor(x[i,:], requires_grad=True)
    y = F.linear(x_p, w, b)
    j_p=jacobian(y,x_p)
    j.append(j_p)

j=torch.stack(j,0)        

</code></pre>
<p>Thanks.</p>
How do I use grid_sample to translate a frames pixels using a Flow Field?

username: Bled_Clement
cooked: <p>I have used the RAFT model in pytorch to calculate optical flow between two frames. Here is the code for that:</p>
<pre><code>    noise_img = noise_img.to(device)
    clean_img = clean_img.to(device)
    return_index = noise_img.size(1) // 2
    aligned_frames = torch.zeros((noise_img.size(0), noise_img.size(1), noise_img.size(2), noise_img.size(3), noise_img.size(4)))
    aligned_frames[:, return_index, :, :, :] = noise_img[:, return_index, :, :, :]

    for idx in range(noise_img.size(1)):
            if not idx == return_index:
                    curr_frame = noise_img[:, idx, :, :, :]
                    ref_frame = noise_img[:, return_index, :, :, :]
                    curr_transf, ref_transf = transforms(curr_frame, ref_frame)
                    curr_flow = mc_model(curr_transf, ref_transf)[-1] # Take the final flow prediction
                    aligned_frames[:, idx, :, :, :] = align_frames(curr_transf, curr_flow)
</code></pre>
<p>In the above I am passing two frames through mc_model (RAFT) to return an optical flow map. In the final line I am trying to map the current frame to be aligned with the reference frame. Below is the function I use:</p>
<pre><code>def warp_flow(img, flow):
    flow_permute = torch.permute(flow, (0, 2, 3, 1))
    remapped = torch.nn.functional.grid_sample(img, flow_permute)
    return remapped
</code></pre>
<p>Unfortunately, <code>remapped</code> when saved as an image, does not return a coherent image. Most images are zero with some looking like bright waves. I’m missing a step in using <code>curr_flow</code> but I don’t quite understand what.</p>
<p>Thank you.</p>
DDP - Dockerfile example?

username: raphasramos
cooked: <p>Do you have a dockerfile that works with DDP using torch==1.12.1+cu113 and torchvision==0.13.1+cu113? Or maybe another version of torch with CUDA &lt;= 11.4?</p>
<p>Want to make sure that mine is not missing anything</p>

username: eqy
cooked: <p>Do you have a specific restriction on building your own container or would using a prebuilt container work for you? If so you could check out the 21.10 NGC container which should have CUDA 11.4: <a href="https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel_21-10.html" class="inline-onebox" rel="noopener nofollow ugc">PyTorch Release 21.10 - NVIDIA Docs</a></p>

username: raphasramos
cooked: <p>Prebuilt container works for me. Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
Runtime Error:Found dtype Long but expected Float

username: Saghar
cooked: <p>Hi when i use MSELOSS instead of NLLLOSS , i get this Error</p>
<p>import torch<br>
import torch.nn as nn<br>
import torch.optim as optim</p>
<p>model = nn.Sequential(<br>
nn.Linear(3072, 512),<br>
nn.Tanh(),<br>
nn.Linear(512, 2),<br>
nn.LogSoftmax(dim=1))</p>
<p>learning_rate = 1e-2</p>
<p>optimizer = optim.SGD(model.parameters(), lr=learning_rate)</p>
<p>loss_fn = nn.MSELoss()</p>
<p>n_epochs = 100</p>
<p>for epoch in range(n_epochs):<br>
for img, label in cifar2:<br>
out = model(img.view(-1).unsqueeze(0))<br>
leb = torch.tensor([label])<br>
loss = loss_fn(out, leb)</p>
<pre><code>    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

print("Epoch: %d, Loss: %f" % (epoch, float(loss)))
</code></pre>

username: ptrblck
cooked: <p>As the error message explains floating point tensors are expected for the model output and target while you are most likely still passing the target tensor as a <code>LongTensor</code> to <code>nn.MSELoss</code>.<br>
Transform it via <code>label = label.float()</code> and the error should be gone.<br>
The more interesting question would be what the model output and targets contain and how <code>MSELoss</code> should be used in this case as it seems you might still pass class indices to the loss function and I would then expect to see a shape mismatch next.</p>
Normalization statistics for fine-tuning a pretrain model

username: TeDataPro
cooked: <p>Hello <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>My objective is to <strong>fine-tune a model which was pretrain on ImageNet dataset</strong>.</p>
<p>In my previous work, I’ve always used normalization statistics of ImageNet dataset to <strong>“fully benefit from pretrained weights”</strong>, both at inference and at fine-tuning.</p>
<p>However in my current work, my <strong>data is significantly different from ImageNet images</strong>.<br>
In the way that, after performing normalization to my data with ImageNet statistics, my images are very dark and an important part of information seem to be lost.</p>
<p>To give you an idea of this difference :</p>
<ul>
<li>Mean of ImageNet : [0.485, 0.456, 0.406]</li>
<li>Mean of my dataset : [0.321, 0.373, 0.303]</li>
</ul>
<p>On this post <a href="https://discuss.pytorch.org/t/should-we-use-our-normalization-for-training-a-pretrained-model/34905">Should we use our normalization for training a pretrained model? - PyTorch Forums</a>, it is recommended to use statistics of our fine-tuning dataset, but this is not what I usually do as I said before.</p>
<p>My question is : which normalization statistics should I use, and why ? The ImageNet ones ? Or my dataset ones ?</p>
<p>Thank you very much for your answer and have a nice day !</p>
<p><strong>Update :</strong><br>
Maybe to give you more information for understanding my problem, when I display images without clipping (i.e. just after normalization), images look very dark. When I display images with linear mapping ( <a href="https://discuss.pytorch.org/t/why-do-images-look-weird-after-imagenet-normalization/92071">Why do images look weird after (Imagenet) normalization? - vision - PyTorch Forums</a> ), my images are correctly displayed, but it is not what is given to the neural network. So, is the information still meaningfull even if images displayed without clipping are very dark and seem to have less information ?</p>

username: eqy
cooked: <p>The point of e.g., subtracting the mean and dividing by the standard deviation is to “center” the distribution, among other things. You would not be centering your training data if you used the ImageNet statistics, so you should use the statistics of your dataset.</p>
<p>However, in practice, I suspect the model would be able to learn around the “mistake” even if you used the ImageNet statistics during finetuning.</p>
How to implement 'Matrix exponential times a vector'?

username: naiman
cooked: <p>The documentation <a href="https://pytorch.org/tutorials/intermediate/parametrizations.html" rel="noopener nofollow ugc">here</a> explains how to regularize the weights of different layers. Specifically, there is an example of matrix_exp operation over the weights of a layer. I am interested in implementing a matrix exponential times a vector, that is, exp(tW)z where t is a batch of different time steps (shape: b x 1) and W are the parameters of the layer (say a linear layer with d x d), and z is a latent representation of a batch, of a shape: b x d.<br>
Using the link above, I can implement exp(tW) for a specific t for the whole batch (1x1) via building a new module and then feeding it with my batch of latent representations z. Is it possible to do all in one operation?</p>
<p>In particular, I am interested in <a href="https://eprints.maths.manchester.ac.uk/1591/1/alhi11.pdf" rel="noopener nofollow ugc">this implementation</a>; Computing the Action of the Matrix Exponential, with an Application to Exponential Integrators by Al-Mohy et al.</p>

username: eqy
cooked: <p>You might consider seeing if your module (even if implemented as multiple operations) can be fused into a single operation via e.g., <code>torch.compile</code>: <a href="https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">torch.compile Tutorial — PyTorch Tutorials 2.0.1+cu117 documentation</a>.</p>
Problems getting PyTorch up and running on my desktop

username: Garrettk
cooked: <p>I’m looking to get started in PyTorch on my machine. I’ve done some trivial work in TensorFlow, but their lack of support for anything other than NVidia GPUs is annoying. I have both an NVidia GPU and AMD GPU installed in my desktop, along with a CPU. I wanted to make sure I could get everything to work. To get an idea of performance, I decided to go through the standard performance tests here:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/pytorch/benchmark">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/pytorch/benchmark" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/070e1f0163322d73698d0139da626c28449e475c8a0fe7565b30ece5984c26a1/pytorch/benchmark" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/pytorch/benchmark" target="_blank" rel="noopener nofollow ugc">GitHub - pytorch/benchmark: TorchBench is a collection of open source...</a></h3>

  <p>TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance. - GitHub - pytorch/benchmark: TorchBench is a collection of open source benchmarks used to evaluate PyTorc...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I tried running python3 install.py but that errored out with:</p>
<pre><code class="lang-auto">Error for /home/garrett/devel/benchmark/torchbenchmark/models/BERT_pytorch:
---------------------------------------------------------------------------
running develop
/usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-369467.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python3.10/dist-packages/
&lt;snip&gt;
</code></pre>
<p>Requiring root write access to run some perf tests is dumb. At the same time, running “–help” on the install script doesn’t provide a way to specify a different installation directory, despite the error text suggesting otherwise.</p>
<p>So. How can I safely run some basic performance test using every combination of my GPUs and CPU for performance comparisons?</p>
Calculating meanAP for segmentation

username: David_Jitca
cooked: <p>Hello!<br>
I need to calculate the meanAP for segmentation of brain tumors. I have the ground truth, a matrix with 0 and 1, and the predicted segmentation also a matrix 0 and 1. I ve calculate IOU adn Dice score, but I need to calculate mAP. I ve used mAP (mean average precision function from sk-learn but I obtain nan). Do you how to calculate this score for segmentatiion easier, I ve found something on internet but only for detection!<br>
Thanks.</p>

username: Mirali33
cooked: <p>Hello <a class="mention" href="/u/david_jitca">@David_Jitca</a>,</p>
<p>Did you find any helpful solution??</p>
<p>I am also working on a segmentation task and I am not sure how to calculate mAP between 2 masks (i.e., ground truth and prediction).</p>
<p>It would be helpful if anyone can point out to useful code or give some insights.</p>
Weird behavior of torch.multinomial()

username: Heizkessel
cooked: <p>The <a href="https://pytorch.org/docs/stable/generated/torch.multinomial.html" rel="noopener nofollow ugc">pytorch docs on the multinomial distribution</a> torch.multinomial( <em>input</em> , <em>num_samples</em> , <em>replacement=False</em> , * , <em>generator=None</em> , <em>out=None</em>) say about the <em>input</em> argument:</p>
<blockquote>
<p><strong>input</strong> (<a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor" rel="noopener nofollow ugc"><em>Tensor</em></a>) – the input tensor containing probabilities</p>
</blockquote>
<p>and they further say:</p>
<blockquote>
<p>The rows of <code>input</code> do not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.</p>
</blockquote>
<p>Unfortunately, it is not clear what is meant by “weights” and how they are transformed into probabilities, but even so, this shouldn’t be a problem if <em>input</em> sums to 1, so it should be interpreted as the probabilities. Then why does the following happen?</p>
<pre><code class="lang-auto">$ torch.multinomial(torch.tensor([0, 1/2, 1/2, 0]), 4)
tensor([2, 1, 0, 3])
</code></pre>
<p>Apparently it first fraws the “possible” options, and then due to the default statement “replacement_False”, it moves on to the other two somehow. But I don’t see why options 0 and 3 are drawn at all, since they started out with a probability of 0. I also don’t understand how “replacement=False” works when all that is provided are the probabilities of the different options.</p>
<p>Similarly, according to the docs, <code>torch.multinomial(torch.tensor([0, 10, 3, 0], dtype=torch.float), 4)</code> should result in an error, while in my case, it doesn’t:</p>
<pre><code class="lang-auto">$ torch.multinomial(torch.tensor([0, 10, 3, 0], dtype=torch.float), 4)
tensor([1, 2, 0, 3])
</code></pre>
<p>So why does this happen?</p>
<p>I’m using pytorch 2.0.0</p>

username: ptrblck
cooked: <p>From the docs:</p>
<blockquote>
<p>When drawn without replacement, <code>num_samples</code> must be lower than number of non-zero elements in <code>input</code> (or the min number of non-zero elements in each row of <code>input</code> if it is a matrix).</p>
</blockquote>
<p>so it seems this is a regression and should be fixed.</p>
<p>EDIT: already tracked <a href="https://github.com/pytorch/pytorch/issues/50034">here</a>.</p>
NotImplementedError: Could not run ‘torchvision::nms’ with arguments from the ‘CUDA’ backend

username: ilia
cooked: <p>i installed the torchvision from source and i get this error<br>
command - torchvision.ops.nms(boxes, scores, iou_thres)<br>
Error - NotImplementedError: Could not run ‘torchvision::nms’ with arguments from the ‘CUDA’ backend.<br>
if I install it via pip, then there is no error, but I don’t have access to VideoReader from torchvision<br>
command - torchvision.set_video_backend(“video_reader”)<br>
Error - RuntimeError: video_reader video backend is not available. Please compile torchvision from source and try again</p>
<p>I want to use VideoReader and not get errors “backend is not available”<br>
in both cases I work with cuda and it is always active.  (I work with yolo7)</p>
<p>I know that I can solve this problem if I transfer the data to the cpu. But it takes a long time and VideoReader becomes useless</p>
<p>pip freeze:<br>
torch==2.0.1<br>
torchaudio==2.0.2<br>
torchvision==0.15.2a0+fa99a53</p>
<hr>
<p>when installing from torchvision source i set FORCE_CUDA=1 and after that got build error.<br>
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.</p>
<p>In my case, cuda was downloaded to docker (tritonserver). I don’t understand what to do in this case.</p>

username: ptrblck
cooked: <p>Did you try to set the <code>CUDA_HOME</code> env variable as suggested in the error message for the source build?</p>

username: ilia
cooked: <p>thank you very much for your reply, I really appreciate it!  the problem is that I don’t know what path to point to the CUDA_HOME variable if cuda is in docker.  Usually they indicate the path “/usr/local/cuda”, but this option does not suit me.  If you use the installation via pip, then there is no such problem.</p>

username: ptrblck
cooked: <p><code>CUDA_HOME</code> should point to the locally installed CUDA toolkit, which is needed for a source build with GPU support. Yes, the pip wheels don’t need a locally installed CUDA toolkit as we bundle all dependencies in them.</p>
Error while using Conv2d

username: rtadd
cooked: <p>Hello.<br>
I’m learning pytorch and I run into an error trying to use Conv2D onto the gpu.</p>
<pre><code class="lang-auto">import torch as trc
import os
os.environ["HSA_OVERRIDE_GFX_VERSION"]= "10.3.0"
gpu = trc.device("cuda" if trc.cuda.is_available() else "cpu")

ex1 = trc.zeros(1, 1, 5, 5)
ex1[0, 0, :, 2] = 1

conv1 = trc.nn.Conv2d(in_channels=1, out_channels=1, kernel_size=3)
res1 = conv1(ex1)
print(res1)  #shows result

ex2 = ex1.to(gpu)
conv2 = conv1.to(gpu)

res2 = conv2(ex2) #error here
print(res2)
</code></pre>
<p>When I run the operation on the cpu there is no problem. But trying to do the same on the gpu gives the following error:</p>
<blockquote>
<p>MIOpen(HIP): Error [Compile] ‘hiprtcCompileProgram(prog.get(), c_options.size(), c_options.data())’ naive_conv.cpp: HIPRTC_ERROR_COMPILATION (6)<br>
MIOpen(HIP): Error [BuildHip] HIPRTC status = HIPRTC_ERROR_COMPILATION (6), source file: naive_conv.cpp<br>
MIOpen(HIP): Warning [BuildHip] /tmp/comgr-8c2e12/input/naive_conv.cpp:39:10: fatal error: ‘limits’ file not found<br>
<span class="hashtag-raw">#include</span>  // std::numeric_limits<br>
^~~~~~~~<br>
1 error generated when compiling for gfx1030.<br>
terminate called after throwing an instance of ‘miopen::Exception’<br>
what():  /long_pathname_so_that_rpms_can_package_the_debug_info/data/driver/MLOpen/src/hipoc/hipoc_program.cpp:304: Code object build failed. Source: naive_conv.cpp<br>
Aborted (core dumped)</p>
</blockquote>
<p>I should clarify that I’m running an AMD gpu and the ROCm version of PyTorch on linux. So far everything I’ve tried has worked, and I’ve managed to train deep NN using the gpu.</p>
<p>Honestly, I have no idea what that error means. I don’t know if there’s something wrong with the code or if this is an issue related to PyTorch/ROCm/HIP/AMD.<br>
If anyone could help me with this or at least point me in the right direction, I would appreciate it.</p>

username: eqy
cooked: <p>This github issue seems to have a similar error <a href="https://github.com/RadeonOpenCompute/ROCm/issues/1889" class="inline-onebox" rel="noopener nofollow ugc">AMD Radeon RX 6800 - HIPRTC_ERROR_COMPILATION · Issue #1889 · RadeonOpenCompute/ROCm · GitHub</a>, and it appears the solution is to install the missing <code>libstdc++-12-dev</code> package.</p>

username: rtadd
cooked: <p>It fixed the problem.<br>
Thank you very much!</p>
Help with Chatbot

username: DevBadger10
cooked: <p><strong>I am not well-versed in PyTorch enough to even SLIGHTLY comprehend or even slightly adapt the tutorial.</strong><br>
Hi! I’m a beginner, so please explain this to me simply. How would I make a chatbot that comes up with unique, new text based off of an input, and how would I then converse with it?<br>
My training data is in a .csv file structured like this:</p>
<div class="md-table">
<table>
<thead>
<tr>
<th>Input 1</th>
<th>Output 1</th>
</tr>
</thead>
<tbody>
<tr>
<td>Input 2</td>
<td>Output 2</td>
</tr>
</tbody>
</table>
</div>

username: DevBadger10
cooked: <p>And could I maybe locally run a finetuned model of something else?</p>

username: J_Johnson
cooked: <p>Most language model chatbots are developed in the following steps:</p>
<ol>
<li>Train the tokenizer and model with next token prediction(i.e. hide the next word in the sequence and the model is tasked with predicting it);</li>
<li>Since the outputs are probabilities(each token is assigned a value of 0 to 1), we can take the top 10% and choose a random token from those. This ensures the model doesn’t wind up overfitting on one response.</li>
<li>Additional training/fine-tuning via supervised learning, RLHF or a similar reinforcement learning training method.</li>
</ol>
<p>There is an entire platform built on PyTorch and in partnership with PyTorch which specializes in chatbot models. That is <a href="https://huggingface.co/learn/nlp-course/chapter1/1" rel="noopener nofollow ugc">HuggingFace</a>. They have online courses for NLP(Natural Language Processing) and Reinforcement Learning. They also have a simple(r) to use interface for downloading pretrained models and deploying them. And plenty of trained models, so you don’t have to start from scratch. I would suggest you start there.</p>

username: DevBadger10
cooked: <p>Okay, thanks. I’ll check that out.</p>

username: DevBadger10
cooked: <p>HuggingFace-Course where have you been all my life? To all those watching, this solution is amazing.</p>
Torch.cuda.is_available() returns false after suspend

username: r2d2bol
cooked: <p>This is not repeatable behavior, but I have observed that sometimes after waking up from suspend, torch.cuda.is_available() returns false.  If I run nvidia-smi, it detects all GPUs on the system.  If I reboot, then torch.cuda.is_available() returns true again.</p>
<p>Is there any way to make torch detect cuda without rebooting?</p>
<p>Some details about the system:</p>
<ul>
<li>ubuntu 20.04</li>
<li>cuda 10.2 (installed via conda)</li>
<li>nvidia driver 450.66</li>
<li>gpu is rtx 2080</li>
</ul>

username: ptrblck
cooked: <p>You could try to reload the nvidia kernel module as described <a href="https://discuss.pytorch.org/t/cuda-runtime-error-999/69658/12">here</a>.</p>

username: jtchilders
cooked: <p>I see a similar issue and I cannot reset or reload the driver because my Ubuntu Xorg is using it to drive the monitor.<br>
<code>5.11.0-37-generic #41~20.04.2-Ubuntu SMP</code></p>
<pre><code class="lang-auto">| NVIDIA-SMI 460.91.03    Driver Version: 460.91.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 207...  Off  | 00000000:01:00.0  On |                  N/A |
| N/A   35C    P8     3W /  N/A |    378MiB /  7974MiB |      3%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1007      G   /usr/lib/xorg/Xorg                 35MiB |
|    0   N/A  N/A      2032      G   /usr/lib/xorg/Xorg                191MiB |
|    0   N/A  N/A      2167      G   /usr/bin/gnome-shell               30MiB |
|    0   N/A  N/A     17865      G   ...AAAAAAAAA= --shared-files       76MiB |
|    0   N/A  N/A     22353      G   ...AAAAAAAAA= --shared-files       30MiB |
+-----------------------------------------------------------------------------+```</code></pre>

username: ptrblck
cooked: <p>My GPU is also used to visualize the desktop and the linked commands work fine.<br>
However, I’m not familiar with your setup, so a restart might be unavoidable.</p>

username: Pavils_Jurjans
cooked: <p>I can confirm that the kernel module reloading, as described by <a class="mention" href="/u/ptrblck">@ptrblck</a> works for me on Ubuntu 22.04.</p>
How to combine a custom loss to a PyTorch loss by weights?

username: Haalum
cooked: <p>I have two loss functions, one is a custom with both forward() and backward() called <code>MyMSELoss </code>. Another loss function is directly from PyTorch <code>mseloss </code>.<br>
I want to do a weighted sum of the losses to get <code>combinedloss</code>. When I multiplied <code>torch_mse</code> by 2, both the loss and gradients are changed accordingly but for the <code>my_mse </code>, they don’t change when it is multiplied by 2.</p>
<p><strong>So my question is how to put a weight on the custom loss <code>my_mse </code> when combining the two losses??</strong></p>
<p>Here is the demo version of my code. Thanks in advance.</p>
<pre><code class="lang-auto">import torch
from torch.autograd import Function
import torch.nn.functional as F

class MyMSELoss(Function):
    @staticmethod
    def forward(ctx, y_pred, y):    
        ctx.save_for_backward(y_pred, y)
        return ( (y - y_pred)**2 ).mean()
    
    @staticmethod
    def backward(ctx, grad_output):
        y_pred, y = ctx.saved_tensors
        grad_input = 2 * (y_pred - y) / y_pred.shape[0]        
        return grad_input, None
    
mseloss = F.mse_loss

# Usage example
y_pred = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = torch.tensor([2.0, 3.0, 4.0])

torch_mse = mseloss(y_pred, y)
my_mse = MyMSELoss.apply(y_pred, y)
combinedloss = torch_mse * 2 + my_mse * 2
combinedloss.backward()

# Access the gradients
gradients = y_pred.grad
print('loss', combinedloss)
print('gradients', gradients)
</code></pre>

username: ptrblck
cooked: <p>You are not using <code>grad_output</code> in your custom <code>backward</code> method, which will be scaled, and are thus not seeing any effect of the incoming gradient.</p>
What is another method for Jacobian calculation instead of using torch.autograd.functional.jacobian

username: S_M
cooked: <p>Hi all I am trying to calculate Jacobian of a matrix which is 1000by1000. When torch.autograd.functional.jacobian is used the cuda arises out of memory error. I need another some method to scape from cuda out of memory error.<br>
Like numpy calculation or any other sample code to calculate Jacobian matrix.</p>
<p>Any help would be appreciated.</p>
‘CUDA error: out of memory’ after some epochs

username: Shashank_Priyadarshi
cooked: <p>Hi, I am facing this error of CUDA out-of-memory error, I tried resolving it by reducing the batch size but still I am getting the same out-of-memory error after one epoch.<br>
I am using Imagenet100 dataset having shape as [64, 3, 224, 224]<br>
training script -</p>
<pre><code class="lang-auto">def train(args, model, device, train_loader, optimizer, epoch):
    model.train()
    for batch_idx, (data, target) in enumerate(train_loader):
        data, target = data.to(device), target.to(device)
        optimizer.zero_grad()
        output = model(data)
        loss = F.nll_loss(output, target)
        loss.backward()
        optimizer.step()
        if batch_idx % args.log_interval == 0:
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}\tt : {}'.format(
                epoch, batch_idx * len(data), len(train_loader.dataset),
                100. * batch_idx / len(train_loader), loss.item(), model.sensor.t))
            if args.dry_run:
                break
</code></pre>
<p>model used -</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.convnext = models.convnext_tiny()
        self.fc1 = nn.Linear(1000, 100)

    def forward(self, x):
        x = self.convnext(x)
        x = self.fc1(x)
        output = F.log_softmax(x, dim=1)
        return output
</code></pre>

username: ptrblck
cooked: <p>The code doesn’t indicate any obvious error, but you could check if the memory usage increases in each iteration or where exactly the GPU runs out of memory.</p>

username: Shashank_Priyadarshi
cooked: <p>I am running this in Colab and the GPU usage is around 6.9 GB. After one epoch I get the output as torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 8.97 GiB (GPU 0; 14.75 GiB total capacity; 10.00 GiB already allocated;…</p>
Why is NVidia seemingly going along with the MSFT lock-in push?

username: aifartist
cooked: <p>This is in regards to the big announcement a few days ago.  I wanted to see if it could achieve the same performance I can with 90+ it/s on Ubuntu with a 4090 using my own TRT compile pipeline.  However, the directml packages they said to install for Olive don’t exist on Linux.  And even if they did I have since discovered they need MSFT DirectX.  The demo just doesn’t work on Linux.  Then I hear perhaps it was intended to be that way.  Thus, …</p>
<p>Why does the new announced driver which is supposed to squeeze more usage from the GPU not have a Linux version?<br>
Why does this new announcement require directml which requires MSFT’s directX?<br>
Why does the Olive Stable Diffusion demo not work on Linux?<br>
AI books website tutorials are 98%(?) pytorch/cuda based.  How does directml fit in with that?<br>
Does MSFT really think it can renege on their commitment to embrace Linux?<br>
Doesn’t NVidia have the expertise to produce drivers and the lowest level tools to achieve the highest possible performance without needing MSFT’s DirectX?  After all NVidia owns the hardware.<br>
Can’t NVidia do in PYTorch what MSFT is doing without vendor lock-in?<br>
Should open software require it be run on Windows?</p>
<p>Am I wrong about a few of these?  Perhaps but something smells suspect.</p>
Block reduce with different sized blocks

username: cmartin
cooked: <p>I would like to be able to compute a reduction over different sized blocks within a larger matrix efficiently. I am working with point set data, so each individual data point is a 2d tensor that can have a varying number of rows, but they all have the same feature dimension (ie all are <code>[n, d]</code> where <code>d</code> is constant).</p>
<p>Suppose I have 3 tensors:</p>
<pre><code class="lang-python">A = torch.tensor([1., 2., 3.]).reshape(3,1).repeat(1,3) # 3x3
B = torch.tensor([4., 5.]).reshape(2,1).repeat(1,3) # 2x3
C = torch.tensor([6., 7., 8., 9.]).reshape(4,1).repeat(1,3) # 4x3
</code></pre>
<p>The number of items in each point set can dramatically differ, so having padded rows is not preferred (and this is my current implementation), so what I’m doing is stacking each individual point set, since they have the same feature dimension:</p>
<pre><code class="lang-python">stacked = torch.vstack([A,B,C])
</code></pre>
<p>Now here is my problem. I need to compute set-set pairwise distances as a function of item-item pairwise distances. The item-item distances are easy since I can just do:</p>
<pre><code class="lang-python"># NxN where N is the total number of items, ie 9 in this ex
item_dists = torch.cdist(stacked, stacked) 
</code></pre>
<p>But the item-item distances now have blocks that correspond to all possible pairwise distance matrices when considering each point set (A, B, C). For example, the A-A block is located at positions <code>[0:3, 0:3]</code> since its items are the first compared. Then the A-B block is located at <code>[0:3, 3:5]</code> since B has the next two items in <code>stacked</code>.</p>
<p>The problem I am facing is how to efficiently perform a reduction over these blocks of different sizes. The simplest solution is to use a for loop that can extract these blocks, and then do the reduction like this:</p>
<pre><code class="lang-python">sizes = [3, 2, 4]
# [0, 3, 5, 9]
ptr = list(itertools.accumulate(sizes, initial=0))
batch_size = len(sizes)

# all combinations: [(A, B), (A, C), (B, C)] 
# ie:               [(0, 1), (0, 2), (1, 2)]
indices = list(
    itertools.combinations(range(batch_size), r=2)
)
item_dists = torch.cdist(stacked, stacked) 
for i, j in indices:
    x_start = ptr[i]
    x_end = ptr[i+1]
    y_start = ptr[j]
    y_end = ptr[j+1]
    block = item_dists[x_start:x_end, y_start:y_end]
    # block reduction
</code></pre>
<p>However, this loop is prohibitively slow. Any thoughts for a better implementation, or does the loop just need to come out of python?</p>

username: cmartin
cooked: <p>Not sure if there is a better solution, but compiling the loop gives substantial speed ups that I am satisfied with. I used numba to do this since this calculation does not need autograd</p>
Is there a way to get QR supported through magma for torch+rocm5.4.2?

username: rndompc
cooked: <p>Hello,</p>
<p>Torch Version: 2.0.1+rocm5.4.2</p>
<p>I am trying to train a rnn with torch 2.0.1+rocm5.4.2 but I keep getting the this error:</p>
<pre><code class="lang-auto">  File "train.py", line 297, in &lt;module&gt;
    generate_init_weight(model, init_weight_name) 
  File "site-packages/lightning_utilities/core/rank_zero.py", line 27, in wrapped_fn
    return fn(*args, **kwargs)
  File "/model/src/trainer.py", line 165, in generate_init_weight
    mm = model.generate_init_weight()
  File "/model/src/model.py", line 597, in generate_init_weight
    nn.init.orthogonal_(m[n], gain=gain * scale)
  File "site-packages/torch/nn/init.py", line 484, in orthogonal_
    q, r = torch.linalg.qr(flattened)
RuntimeError: Calling torch.orgqr on a CUDA tensor requires compiling PyTorch with cuSOLVER. Please use PyTorch built with cuSOLVER support.
</code></pre>
<p>I thought an installation of magma would fix it but it looks like even when magma is in my installation, it isn’t being used for QR.</p>
<p>Is there a way top get the  torch.nn.init.orthogonal_() function with hipSolver or rocSolver or Magma?</p>
<p>Thanks!</p>