akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Why are optimizer state_dict params integers and not tensors?

jramapuram: <p>Trying to remap a saved optimizer to a new param_group and trying to check valid shaping of tensors between the optimizers, but realized that the params are integers and not tensors. How should I think of this? The docs say optimizer.state_dict()[“param_groups”][*][“params”] should be tensors.</p>
<pre><code class="lang-python">&gt; torch.optim.SGD(nn.Linear(3, 2).parameters(), 0.1).state_dict()["param_groups"]

[{'lr': 0.1,
  'momentum': 0,
  'dampening': 0,
  'weight_decay': 0,
  'nesterov': False,
  'maximize': False,
  'foreach': None,
  'differentiable': False,
  'params': [0, 1]}]  # &lt;-- ints ¯\_(ツ)_/¯ 
</code></pre>
<p>What is the recommended way to validate that when I restore an optimizer that the parameters from <code>param_group_orig</code> match <code>param_group_new</code> [eg: different layerwise LR] ? I guess there is some link to the momentum terms as well  somwehere behind the scenes?</p>

eqy: <p>From a cursory glance at the base <code>Optimizer</code> class I think the explanation is that these are indices forming a mapping rather than the raw tensors themselves: <a href="https://github.com/pytorch/pytorch/blob/18587cb31f484e4a9c4d4653c654ff27c71e9e2a/torch/optim/optimizer.py#L336" class="inline-onebox" rel="noopener nofollow ugc">pytorch/optimizer.py at 18587cb31f484e4a9c4d4653c654ff27c71e9e2a · pytorch/pytorch · GitHub</a></p>
<p>This seems to make sense as the linear layer you defined would have two param tensors: a weight and a bias.</p>
Model summary info with libtorch(c++ frontend)

DongkyuLee: <p>Hi</p>
<p>I have question about libtorch api.</p>
<p>In pytorch with python, I can use torchinfo.summary function to show model summary which includes parameters, flow, and pass sizes etc.</p>
<p>However, I can’t find out similar function when I use libtorch(c++ frontend), is there any similar functions in libtorch?</p>
<p>Here is example for torchinfo.summary(refer to <a href="https://github.com/TylerYep/torchinfo" class="inline-onebox" rel="noopener nofollow ugc">GitHub - TylerYep/torchinfo: View model summaries in PyTorch!</a>)</p>
<pre><code class="lang-auto">================================================================================================================
Layer (type:depth-idx)          Input Shape          Output Shape         Param #            Mult-Adds
================================================================================================================
SingleInputNet                  [7, 1, 28, 28]       [7, 10]              --                 --
├─Conv2d: 1-1                   [7, 1, 28, 28]       [7, 10, 24, 24]      260                1,048,320
├─Conv2d: 1-2                   [7, 10, 12, 12]      [7, 20, 8, 8]        5,020              2,248,960
├─Dropout2d: 1-3                [7, 20, 8, 8]        [7, 20, 8, 8]        --                 --
├─Linear: 1-4                   [7, 320]             [7, 50]              16,050             112,350
├─Linear: 1-5                   [7, 50]              [7, 10]              510                3,570
================================================================================================================
Total params: 21,840
Trainable params: 21,840
Non-trainable params: 0
Total mult-adds (M): 3.41
================================================================================================================
Input size (MB): 0.02
Forward/backward pass size (MB): 0.40
Params size (MB): 0.09
Estimated Total Size (MB): 0.51
================================================================================================================
</code></pre>
<p>Thank you</p>
How to fill in a batch of tensor with another tensor according to mask

Y_Y: <p>Hi all,<br>
Suppose I have the following code</p>
<pre><code class="lang-auto">to_learn = nn.Parameter(torch.randn([1, 3, 10, 10]))
</code></pre>
<p>In the forward, I have a mask where each mask masks different areas of a tensor</p>
<pre><code class="lang-auto">
def forward(x, mask)
    bs = x.shape[0]
    new= torch.zeros([bs, 3, 64, 64]).cuda()
    # fill in new with to_learn where mask is 1
    ...
    return x + new
</code></pre>
<p>The shapes of <code>mask</code> and <code>x</code> are [bs, 3, 64, 64].<br>
How can I efficiently fill in the tensor <code>new</code> with the <code>to_learn</code> tensor only at locations where <code>mask==1</code>?</p>
<p>Thank you for any help.</p>

eqy: <p>You don’t need to allocate a new tensor for this as a separate step, as an out of place operation should achieve the same thing, e.g.,</p>
<pre><code class="lang-auto">new = torch.mul(to_learn, mask)
</code></pre>
<p>A similar approach is used by the pruning utilities:<br>
<a href="https://pytorch.org/tutorials/intermediate/pruning_tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/pruning_tutorial.html</a></p>

Y_Y: <p>Hi thank you for the suggestion. But this will cause size not matching issue.<br>
<code>RuntimeError: The size of tensor a (64) must match the size of tensor b (10) at non-singleton dimension 3</code><br>
Do you happen to know another way?</p>

eqy: <p>Ah, I interpreted the question as intending to mask <code>to_learn</code>, but if the mask is intended for the input, you can do</p>
<pre><code class="lang-auto">x = torch.mul(x, mask)
</code></pre>

Y_Y: <p>Thank you. There was a typo in my original post I have fixed.</p>
<p>My purpose is to fill in a batch of image <code>x</code> with <code>to_learn</code> when <code>mask</code> is 1. Suppose there is an image of a dog. The mask gives me a segment of the dog’s head. I want to fill in the head area with <code>to_learn</code>.</p>
<p>Using the suggested method will not fill in <code>x</code> with <code>to_learn</code> right?</p>

eqy: <p>No, it achieves the goal of masking the input preventing the parameters in <code>to_learn</code> from accessing the masked values.</p>

Y_Y: <p>Thank you. In this case, x is masked. Then how do we fill in <code>to_learn</code> into masked area of <code>x</code> considering <code>x</code> and <code>to_learn</code> in different shape?</p>

eqy: <p>If you mean how to predict the output with a masked input <code>x</code>, then this is doable with a common operation like a convolution.</p>
What is the best number of input size for GRU in MLP-GRU model?

peony: <p>I am doing action recognition. Each of my frames has a size of [3, 50] and I feed them into the MLP whose output is then fed to my GRU. I wonder if the input_size  to my GRU (or the output of the MLP) should be [3x50].</p>
<p>Thank you in advance.</p>
BiLSTM incorrect on MPS?

FrankVerhoef: <p>Hi, I been testing various NLP models on MPS, and I observe various strange things. It seems that bidirectional LSTM on MPS is not working correctly.</p>
<p>See the code below:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

print("Torch version: ", torch.__version__)
torch.manual_seed(123)
embeddings = torch.rand(3,1,2)  # L, B, E

model = nn.LSTM(input_size=2, hidden_size=2, bidirectional=True)
print(model(embeddings)[0])

model.to("mps")
print(model(embeddings.to("mps"))[0])

</code></pre>
<p>Results are below. CPU above, MPS below. How can results on CPU be different than on MPS?<br>
==&gt; I am missing something?<br>
I’m testing on MacBook Pro with M1PRO, 32GB, 10-core CPU, 16-core GPU.</p>
<pre><code class="lang-auto">Torch version:  2.0.0.dev20230205
tensor([[[ 0.0054,  0.2338, -0.1167,  0.2263]],
        [[ 0.0200,  0.3305, -0.0967,  0.2075]],
        [[ 0.0422,  0.3740, -0.0590,  0.1553]]], grad_fn=&lt;CatBackward0&gt;)
tensor([[[0.2325, 0.2447, 0.4149, 0.4394]],
        [[0.3524, 0.3702, 0.3516, 0.3785]],
        [[0.4139, 0.4346, 0.2306, 0.2531]]], device='mps:0',
       grad_fn=&lt;CatBackward0&gt;)
</code></pre>

ptrblck: <p>Thanks for reporting the issue! I believe this could also be a valid bug in the MPS backend so could you also create a GitHub issue for it as described <a href="https://discuss.pytorch.org/t/incorrect-mean-of-slice-of-tensor-on-mps/172355/2">here</a>?</p>
Skipping embedding layers of pre-trained model GPT2 by huggingface

DHARMENDRA_PRAJAPAT: <p>This is the architecture of gpt2 language model.<br>
GPT2LMHeadModel(<br>
(transformer): GPT2Model(<br>
(wte): Embedding(50257, 768)<br>
(wpe): Embedding(1024, 768)<br>
(drop): Dropout(p=0.1, inplace=False)<br>
(h): ModuleList(<br>
(0): Block(<br>
(ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)<br>
(attn): Attention(<br>
(c_attn): Conv1D()<br>
(c_proj): Conv1D()<br>
(attn_dropout): Dropout(p=0.1, inplace=False)<br>
(resid_dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
(ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)<br>
(mlp): MLP(<br>
(c_fc): Conv1D()<br>
(c_proj): Conv1D()<br>
(dropout): Dropout(p=0.1, inplace=False)<br>
)<br>
)<br>
…more blocks are there</p>
<p>I am using gpt2 pre-trained model by huggingface.<br>
I want to use GATs for embedding and want to incorporate that embedding with embedding provided by gpt2 pretrained model.<br>
so my question is, can we directly give input to the block 0 by skipping the embedding layer part of this model.</p>
NNPACK SpatialConvolution_update

Onyekwelu_Michael: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/6/3630445ee3b0c75ef5b7bc781c330ae123000687.png" data-download-href="https://discuss.pytorch.org/uploads/default/3630445ee3b0c75ef5b7bc781c330ae123000687" title="Screenshot 2023-02-12 at 14.00.43"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/3630445ee3b0c75ef5b7bc781c330ae123000687_2_689x301.png" alt="Screenshot 2023-02-12 at 14.00.43" data-base62-sha1="7Jnc65bfCNn6IeCmhRHcErQyfLp" width="689" height="301" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/3630445ee3b0c75ef5b7bc781c330ae123000687_2_689x301.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/3630445ee3b0c75ef5b7bc781c330ae123000687_2_1033x451.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/6/3630445ee3b0c75ef5b7bc781c330ae123000687.png 2x" data-dominant-color="252727"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-02-12 at 14.00.43</span><span class="informations">1344×588 108 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I get this error while trying to. train a GAN model. It ran on a window system in my lab, but gives this error on my personal mac (Platform: ‘macOS-13.2-arm64-arm-64bit’)</p>
Question about SNR formulation in diffusion model

Yolkandwhite: <p>It is very simple question/</p>
<p>I don’t understand why (x_0)**2 is not on numerator in eq.109 below.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/2/5227338df71ea70bdde63b019a21af85adf2c843.png" data-download-href="https://discuss.pytorch.org/uploads/default/5227338df71ea70bdde63b019a21af85adf2c843" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/2/5227338df71ea70bdde63b019a21af85adf2c843_2_690x86.png" alt="image" data-base62-sha1="bIL8KBBOqhPuao1ux3ey7pvq5yP" width="690" height="86" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/2/5227338df71ea70bdde63b019a21af85adf2c843_2_690x86.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/2/5227338df71ea70bdde63b019a21af85adf2c843_2_1035x129.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/2/5227338df71ea70bdde63b019a21af85adf2c843_2_1380x172.png 2x" data-dominant-color="F3F2F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1740×217 38.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I think it has to be like this.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/0/b065959ce724f39382f4f5ae46c751a142881e47.png" alt="image" data-base62-sha1="patG3S4GGhKsuRjV7u32Dd9t2Ll" width="208" height="117"></p>
<p>Can anyone explain why there’s no x_0 term on numerator??</p>
How to use torch.distributions.Gumbel in C++ API?

lichangh20: <p>I want to use torch.distributions.Gumbel in C++ by using libtorch, so I try the following code:</p>
<pre data-code-wrap="C"><code class="lang-C">    torch::distributions::Gumbel gumbel_distribution(norm_weight, torch::ones_like(norm_weight));
    auto weight_phi = gumbel_distribution.rsample();
</code></pre>
<p>However, when I try to compile the code, it reports error like the following:</p>
<blockquote>
<p>error: name followed by “::” must be a class or namespace name</p>
</blockquote>
<p>So later I ask ChatGpt on how to implement it, and it shows me the following code:</p>
<pre data-code-wrap="C++"><code class="lang-C++">  #include &lt;torch/torch.h&gt;
  #include &lt;torch/distributions/gumbel.h&gt;
  double location = 0.0;
  double scale = 1.0;
  torch::distributions::Gumbel gumbel(location, scale);
</code></pre>
<p>When I try to compile this code, my laptop reports that the file “torch/distributions/gumbel.h” does not exist! Later I ask ChatGpt the same question again, and it gives me the following answer:</p>
<blockquote>
<p>The torch::distributions::Gumbel class I mentioned in my previous answer is not available in the C++ API of PyTorch as of version 1.10.0.</p>
</blockquote>
<p>So I am wondering if I can use torch.distributions.Gumbel through libtorch now?Many thanks!</p>
How to print more predicted categories using ResNet50

jagac: <p>I followed the documentation here <a href="https://pytorch.org/vision/stable/models.html" class="inline-onebox" rel="noopener nofollow ugc">Models and pre-trained weights — Torchvision main documentation</a> and was wondering how can we print for example the top 3 predicted categories for ResNet50?</p>

ptrblck: <p>You could use <code>torch.topk</code> on the model outputs and check the <code>.indices</code> attribute.</p>

jagac: <p>worked perfectly thank you!!</p>
Error in VAE Architecture

PyTorch2603: <p>Hi. I’ve been trying to write a VAE for a project, but I found that the loss is stabilizing at a pretty high value. I wanted to know if I was just making a simple error in my VAE.<br>
This is the architecture:<br>
class VAE(nn.Module):<br>
def <strong>init</strong>(self, z_dim):<br>
super(VAE, self).<strong>init</strong>()</p>
<pre><code>    # Encoder
    self.encode=nn.Sequential(
    nn.Linear(22,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720)
    )
    self.fc_mu = nn.Linear(720, z_dim)
    self.fc_logvar = nn.Linear(720, z_dim)
    # Decoder
    self.decode=nn.Sequential(
    nn.Linear(z_dim, 720),
    nn.ReLU(),
    nn.Linear(720, 720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,720),
    nn.ReLU(),
    nn.Linear(720,22)
    )
    nn.init.kaiming_uniform_(self.fc_mu.weight)
    nn.init.kaiming_uniform_(self.fc_logvar.weight)
    self.encode.apply(weights_init)
    self.decode.apply(weights_init)
</code></pre>
<p>Is there any issue that is immediately apparent that could be causing this error? Thanks!</p>
How to get the current gpu number without the input tensors when using DataParallel

Jason_Zan: <p>Hi,<br>
I’m using <code>DataParallel</code> to do multi-gpu training. I need initialize a <code>torch.zeros</code> tensor in my model <code>forward</code> function, which will be added by several logits (calculated by input tensor).The code is as belows:</p>
<pre><code class="lang-auto">        linear_logit = torch.zeros([X.shape[0], 1]).to(self.device)
        ...
        linear_logit += sparse_feat_logit
        ...
        linear_logit += dense_value_logit
</code></pre>
<p>The parallel part code:</p>
<pre><code class="lang-auto">        model = torch.nn.DataParallel(model, device_ids=[0,1])
</code></pre>
<p>Then I got this error:<br>
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!</p>
<p>This is because that the input data and <code>sparse_feat_logit</code> is on cuda:1, but the <code>linear_logit</code> is on cuda:0 (<code>self.device</code> is cuda:0, since the model need to be in cuda:0 before running <code>torch.nn.DataParallel</code>).</p>
<p>I tried to get the current gpu number by input tensors, but I got other errors because some of the input data maybe empty sometimes.</p>
<p>I would like to know how to get the current gpu number without the input tensors. Maybe there is a function or property inside the model? Thank you!</p>

ptrblck: <aside class="quote no-group" data-username="Jason_Zan" data-post="1" data-topic="172298">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jason_zan/48/57202_2.png" class="avatar"> Jason_Zan:</div>
<blockquote>
<p>I tried to get the current gpu number by input tensors, but I got other errors because some of the input data maybe empty sometimes.</p>
</blockquote>
</aside>
<p>This sounds generally like a valid approach. If the input tensors are empty, you could use a registered parameter and use its device via: <code>next(self.parameters()).device</code>.</p>

Jason_Zan: <p>It’s very helpful. Thank you! I really appreciate your help!</p>
Will horovod be compatible with pytorch 2.0

KonArtist: <p>If we have horovod running with pytorch 1.x, will it be possible to upgrade to pytorch 2.0 and run the code as usual. And would it be compatible with a compiled graph?</p>

kumpera: <p>There are no backward compatibility breaking changes in PyTorch 2.0 when it comes to running distributed jobs.</p>
<p>Whether you’ll be able to use torch.compile or not depends on the model itself.</p>

KonArtist: <p>Thanks for clarifying</p>
Can some body help me, I can't load my custom model

Wisarut_HAKAEN: <p>I’m using torch 1.11.0+cu113</p>
<hr>
<p>import torch<br>
path = ‘yolov5s.pt’<br>
model = torch.hub.load(‘.’, ‘custom’, path=path, source=‘local’)<br>
print(model(‘images\1.jpg’))</p>
<hr>
<p>I got<br>
File “.\load3.py”, line 3, in <br>
model = torch.hub.load(‘.’, ‘custom’, path=path, source=‘local’)<br>
File “C:\Users\bukob\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\hub.py”, line 404, in load<br>
model = _load_local(repo_or_dir, model, *args, **kwargs)<br>
File “C:\Users\bukob\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\hub.py”, line 430, in _load_local<br>
hub_module = _import_module(MODULE_HUBCONF, hubconf_path)<br>
File “C:\Users\bukob\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\hub.py”, line 76, in _import_module<br>
spec.loader.exec_module(module)<br>
File “”, line 779, in exec_module<br>
File “”, line 915, in get_code<br>
File “”, line 972, in get_data<br>
FileNotFoundError: [Errno 2] No such file or directory: ‘.\hubconf.py’</p>

ptrblck: <p><code>hubconf.py</code> seems to be defined in the Yolo repository of I’m not mistaken, so you might need to install the repo or make sure the scripts are oin your python path.</p>

Wisarut_HAKAEN: <p>yeah you’re right I have no hubconf.py then I use this</p>
<hr>
<p>import torch<br>
import cv2</p>
<p>path = ‘weaponsdex.103_9kIMG_16batch_300e.pt’<br>
model = torch.hub.load(‘ultralytics/yolov5’, ‘custom’, path)</p>
<p>img = cv2.imread(“images/1.jpg”)<br>
result = model(img)</p>
<p>print(result)</p>
<hr>
<p>and it works, THX for your helpful answer</p>

Jason_Zan: <p>Hi ptrblck, Could you help me please?</p><aside class="quote quote-modified" data-post="1" data-topic="172298">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jason_zan/48/57202_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-get-the-current-gpu-number-without-the-input-tensors-when-using-dataparallel/172298">How to get the current gpu number without the input tensors when using DataParallel</a> 
  </div>
  <blockquote>
    Hi, 
I’m using DataParallel to do multi-gpu training. I need initialize a torch.zeros tensor in my model forward function, which will be added by several logits (calculated by input tensor).The code is as belows: 
        linear_logit = torch.zeros([X.shape[0], 1]).to(self.device)
        ...
        linear_logit += sparse_feat_logit
        ...
        linear_logit += dense_value_logit

The parallel part code: 
        model = torch.nn.DataParallel(model, device_ids=[0,1])

Then I got this erro…
  </blockquote>
</aside>
Pytorch taking forever to install

Bhavay_Malhotra: <p>Hello community, I’m trying to install pytorch on a linux server using the following command<br>
“conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia”</p>
<p>However, after a few failed steps, it’s stuck at a particular step. I tried to run it all night long using the tmux session but in the morning too it was stuck at that particular step.<br>
Here is a snapshot of the process.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/6/a636088be127b67c5f1d96a26d01e957870e4fa7.png" data-download-href="https://discuss.pytorch.org/uploads/default/a636088be127b67c5f1d96a26d01e957870e4fa7" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/6/a636088be127b67c5f1d96a26d01e957870e4fa7.png" alt="image" data-base62-sha1="nIn1ChbMgFzdFdiPZBWk3EKzEXR" width="690" height="71" data-dominant-color="15355D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">985×102 4.34 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Could you try to create a new virtual environment as it seems <code>conda</code> might have trouble with the conflict resolution?<br>
If possible, you could could also try to use <code>mamba</code> instead which is way faster in this step in my experience.</p>

Bhavay_Malhotra: <p>Hey, thanks for reply.<br>
I tried to create 2-3 environments and installed pytorch still same issue.<br>
I also reinstalled conda and tried it but same issue is still present.<br>
I’ll look for mamba, Thanks</p>
How to convert the value of AverageMeter to int or float?

peony: <p>I stored the average of my losses over each epoch:</p>
<p><code>losses = AverageMeter()</code></p>
<p>and now I wish to plot them, so I did something like:</p>
<p><code> plt.plot((float(losses))</code></p>
<p>but I get this error:</p>
<p><code>TypeError: float() argument must be a string or a number, not 'AverageMeter'</code></p>
<p>If so, how do we convert AverageMeter to a number?</p>
<p>Thank you in advance</p>

ptrblck: <p>I don’t know which implementation of <code>AverageMeter</code> you are using but I would assume you could access its internal attributes such as <code>.val</code> etc.</p>
Capturing CUDA graph with Stream Priority

Zhaodong_Chen: <p>Hi!</p>
<p>I have a question about how can I preserve the priority of streams when capturing the CUDA graph. I have a set of kernels running on two streams with priority -1 and 0. However, my profiling result shows that this priority is not captured.</p>
<p>I also tried to change the flag for <code>cudaGraphInstantiateWithFlag</code> to <code>cudaGraphInstantiateFlagUseNodePriority</code>, but seems that it doesn’t do the job.</p>
<p>Thanks!</p>
Torch2.0 with dataparallel

y-zheng18: <p>It seems torch2.0 with torch.nn.DataParallel improves little. Codes look like:</p>
<pre><code class="lang-auto">model_opt = torch.nn.DataParallel(model, device_ids=device_ids)
model_opt = torch.compile(model_opt)
</code></pre>
<p>When using single GPU, i.e., <code>device_ids=[0]</code> will lead to a 1.5 faster training, but for multi-gpu it falls back to the non-compile speed.</p>
I can't compile a very basic example using libtorch due to a linking error for the file mkl_intel_ilp64.lib

dcagritan: <p>Dear all,<br>
I want to build torch_cluster and to do that I was using precompiled libtorch. However this gave me a linking error with the mkl_intel_ilp64.lib. I’m on a Windows machine and have already installed MKL libraries from Intel’s website. After this, I wanted to try to create an very basic example case as in:<a href="https://pytorch.org/cppdocs/installing.html" class="inline-onebox" rel="noopener nofollow ugc">Installing C++ Distributions of PyTorch — PyTorch master documentation</a>, however I faced the same issue. It says something like "LINK : fatal error LNK1181: cannot open the file ‘mkl_intel_ilp64.lib’<br>
Do you have any idea why this problem is occuring? I’ve been fighting with this for a couple of days and any help would be appreciated.<br>
Thank you very much in advance and have a good day!<br>
Regards</p>
CUDA Out of Memory Error in Pytorch

Noorulain_Islam: <p>I am trying to run my model on jupyter notebook using pytorch_geometric framework. But, I am facing this error. When I train my model, I am facing this error. Is there anyway to fix this error?</p>
<h2>
<a name="error-is-this-one-1" class="anchor" href="#error-is-this-one-1"></a>Error is this one:</h2>
<p>OutOfMemoryError                          Traceback (most recent call last)<br>
 in <br>
----&gt; 1 model.forward(train_data[“feature”], train_data[“edge_index”], train_data[“edge_weight”] )</p>
<p>6 frames<br>
/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/message_passing.py in <strong>lift</strong>(self, src, edge_index, dim)<br>
234             try:<br>
235                 index = edge_index[dim]<br>
 → 236                 return src.index_select(self.node_dim, index)<br>
237             except (IndexError, RuntimeError) as e:<br>
238                 if index.min() &lt; 0 or index.max() &gt;= src.size(self.node_dim):</p>
<p>OutOfMemoryError: CUDA out of memory. Tried to allocate 25.04 GiB (GPU 0; 14.76 GiB total capacity; 1.73 GiB already allocated; 12.11 GiB free; 1.87 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF</p>

ptrblck: <p>You could try to decrease the batch size the reduce the memory usage. Alternatively, <code>torch.utils.checkpoint</code> could also help but I don’t know how well PyG is supported.</p>
Is PyTorch code blocking?

Sam_Lerman: <p>Does a call to something like <code>loss.backwards()</code> cause the CPU code to block?</p>
<pre><code class="lang-auto">loss.backwards()
print(5)
</code></pre>
<p>Does <code>5</code> get printed concurrently with or after <code>loss.backwards()</code> is finished executing?</p>
<p>I tried running the following example to test it myself:</p>
<pre><code class="lang-auto">import torch
from torch import nn
print('net')
net = nn.Linear(100000, 1)
print('loss')
loss = net(torch.ones(10000, 100000)).mean()
print('back')
loss.backward()
print(5)
</code></pre>
<p>It’s hard to tell for me. Seems like at least the net creation is blocking.</p>
<p>Edit: I would also like to know the logic of this when the model is on GPU.</p>
<pre><code class="lang-auto">loss = net.to('cuda')(torch.ones(10000, 100000).to('cuda')).mean()
</code></pre>

ptrblck: <p>Since your code is purely performed on the CPU each call would be “blocking”.<br>
Onc you move the model to the GPU its execution will be asynchronous and the CPU will only block if you are explicitly synchronizing the code or if data is moved explicitly or implicitly to the CPU.</p>
Hist_index or scatter and reduce, but not histc or scatter_reduce

kunaltyagi: <p>I’m looking to express the following op in a nice manner:</p>
<pre data-code-wrap="py"><code class="lang-plaintext"># compute index[i, ...] for all data[i, ...]
hist_value[i, index[i, ...]] += data[i, ...]
</code></pre>
<p>This is almost like the <code>scatter_reduce_</code> with add operation, but that would require:</p>
<ul>
<li>additional memory allocation for <code>hist_value</code> <strong>AND</strong> <code>index</code> to be of the same shape as <code>data</code>
</li>
<li>additional reduction (<code>sum</code>) over the dimensions we had to add</li>
</ul>
<p>It’s almost like <code>histc</code>, but not quite, since</p>
<ul>
<li>
<code>indices</code> are known</li>
<li>Can’t use the default algorithm for bucket calculation since the bucket index computation is slightly tricky</li>
</ul>
<p>I know the min/max/number of buckets so an API similar to <code>histc</code> might be the best fit here (but I’m not sure)</p>
<p>I don’t know if this is doable via Einstein notation. What do you people think?</p>
Question about DETR gradient backpropagation to object queries. 

smartbarbarian: <p>Assume we only have 4 object queries and the batch size is one, only the last decoder layer will output loss, and negative labels will not be calculated into the loss. If we set tgt_mask like<br>
[[False, False, True, True],<br>
[False, False, True, True],<br>
[True, True, False, False],<br>
[True, True, False, False],]<br>
And only the first object query matches the True label within one step during training, the last two object queries will not be updated during gradient back propagation. But I found that all object queries have been updated. Can anyone clarify it? Thank you so much.</p>

Peter_Vennerstrom: <p>DETR assigns all queries as either positive or negative samples and they are supervised with a classification loss.  Only the queries assigned as positive samples are supervised with regression losses related to the bounding box prediction.</p>
<p>Each query will be updated by classification loss and the subset of queries assigned as positive samples (foreground) will also be updated by regression losses.</p>

smartbarbarian: <p>Thank you so much for replying the post! I just updated the post. Because I can’t modify it when it was pending state. In my expirement, I ignored negative samples and only calculate loss caused by positive samples by using torch.gather. And only caluate loss of the last decoder layer output. So the other object queries should not be updated theoretically. But the result make me confused.</p>

smartbarbarian: <p>I found it may be caused by the momentum in optimzer. So it’s is correct such tgt masks can prevent interactions among object queries?</p>

Peter_Vennerstrom: <p>tgt_mask will prevent interactions with the tgt sequence, but they will be still be supervised by DETR loss functions by default.  To prevent specific queries from being supervised, the loss functions will need the same per query masks.</p>
<p>Does your system work in an autoregressive manner?</p>

smartbarbarian: <p>It is used for trajectory prediction. A lot of papers uses multiple decoders to ensemble results and achieve a better performance. So I was thinking that could we use tgt_mask to make a single large decoder runnning like multiple small decoders.</p>
Custom loss: TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first

nikiguo93: <p>Hi, my custom loss looks like this:</p>
<p>class CKALoss(torch.nn.Module):<br>
“”"<br>
“”"<br>
def <strong>init</strong>(self):<br>
super(CKALoss, self).<strong>init</strong>()</p>
<pre><code>def forward(self, yhat,local_Y):
    ground_truth = local_Y
    pred = yhat
    m = ground_truth.size()[0]
    cka = CKA(m=m, kernel='linear')
    #rho = cka.compare(pred.detach().cpu().numpy(), ground_truth.detach().cpu().numpy())
    rho = cka.compare(pred, ground_truth) #centered kernel alignment similarity
    rho = torch.from_numpy(np.asarray(rho))
    loss = torch.log(1-rho)
    return loss
</code></pre>
<p>Then I got the error message “TypeError: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.”<br>
Can anybody know how to fix it?</p>

khushi-411: <p>Hi, there!</p>
<blockquote>
<p>TypeError: can’t convert cuda:0 device type tensor to numpy.</p>
</blockquote>
<p>The error says that you initialized your tensor in CUDA memory and need to convert the tensor to CPU tensor before changing to numpy arrays. FYI: NumPy does not have CUDA support.</p>
<p>You can initialize your tensor in the CPU memory from the beginning. Something like this:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.tensor([1, 2, 3], device="cpu")
</code></pre>
<p>This would resolve your error and make your code much cleaner <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>A small question: Why are you converting tensors to NumPy arrays? I think there’s no need; PyTorch has full support.</p>
<p>Thank you!</p>

Alvaro-Ciudad: <p>There is a tensor that its in cuda and not in cpu. Numpy does not support cuda operations so you need to transform it back to cpu. Probably it is the rho value, just stick a .cpu() inside of the call np.assarray</p>

kyeongjun1007: <p>try below if you want to use your gpu</p>
<p>add this code<br>
<code>device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')</code></p>
<p>and attach .to(device) to your model and local_Y tensor or numpy array like this</p>
<pre><code class="lang-auto">model = Model(~~).to(device)
local_Y = ~~.to(device)
</code></pre>
<p>If it’s not work, CKA could make this error. find cuda setting of CKA class…</p>
Object Detection - RuntimeError: stack expects each tensor to be equal size

Imraj: <p>Hi,</p>
<p>I created a custom dataset for object detection named ReceiptDataset  as below</p>
<pre><code class="lang-auto">class ReceiptDataset(torch.utils.data.Dataset):
  def __init__(self, train_dir,width,height,labels,transforms=None):
    self.images = os.listdir(train_dir)
    self.width = width
    self.height = height
    self.train_dir = train_dir
    self.labels = labels
    self.transforms = transforms

  def __getitem__(self,idx):
    img_name = self.images[idx]
    img_path = os.path.join(self.train_dir,img_name)
    img = cv2.imread(img_path)
    img_res = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32)
    img_res = cv2.resize(img_res,(self.width,self.height), cv2.INTER_AREA)

    annot = self.labels[str(img_name)]

    lbls = []
    boxes = []
    target = {}

    box_wt, box_ht, _ = img_res.shape
    print(f"img_res shape: {img_res.shape}")

    for item in annot:
      x,y,wt,ht,lbl = item

      x_min = x - wt/2
      x_max = x + wt/2
      y_min = y - ht/2
      y_max = y + ht/2

      x_min_new = int(x_min * box_wt)
      x_max_new = int(x_max * box_wt)
      y_min_new = int(y_min * box_ht)
      y_max_new = int(y_max * box_ht)

      boxes.append([x_min_new,x_max_new,y_min_new,y_max_new])

      lbls.append( classes.index(str(lbl)) )

    print(f"dls_lbls: {lbls}")

    boxes = torch.as_tensor(boxes, dtype=torch.int64)
    lbls = torch.as_tensor(lbls, dtype=torch.int64)

    target["boxes"]  = boxes
    target["labels"] = lbls
    target["image_id"] = torch.as_tensor(idx)

    if self.transforms:
      trans = self.transforms(image=img_res,
                              bboxes = target["boxes"],
                              labels=lbls
                              )
      img_res = trans["image"]
      target["boxes"] = torch.Tensor(trans["bboxes"])


    return img_res, target


  def __len__(self):
    return len(self.images)
</code></pre>
<p>and I created an instance with:</p>
<p><code>train_dataset = ReceiptDataset("label-detector/images",width,height,plabels)</code></p>
<p>and my training snippet is :</p>
<pre><code class="lang-auto">from engine import train_one_epoch, evaluate

for epoch in range(num_epochs):
  train_one_epoch(model,optim,train_loader,device,epoch,print_freq=2)

  lr_scheduler.step()

  evaluate(model,test_loader,device)
</code></pre>
<p>but anytime I run the training loop, I’m getting a runtime error:</p>
<p><code>RuntimeError: stack expects each tensor to be equal size, but got [11,4] at entry 0 and [9,4] at entry 1</code></p>
<p>There are 17 classes in total and each image has a minimum of 4 annotations.<br>
I noticed the problem seems to be coming from my labels list/tensor in the dataset class, the size of the labels list/tensor varies based on the number of annotated items in an image, but I can’t seem to figure out a way to fix this.</p>
<p>Thanks!</p>
Can't import Optim.lr_scheduler

kyeongjun1007: <p>Hi. I’m KyeongJun</p>
<p>My question is simple.<br>
When I’m trying to import learning rate scheduler,</p>
<p><code>from torch.optim.lr_scheduler import LambdaLR (or StepLR etc.)</code></p>
<p>it works well.</p>
<p><code>import torch.optim.lr_scheduler.LambdaLR (or StepLR etc.)</code></p>
<p>but this.</p>
<p>What’s wrong with the second code?<br>
I’m using torch version 3.13.0+cu117.</p>
<p>Thank you.</p>
How to know which qconfigs were ignored in quantization?

111357: <p>As the description in <a href="https://github.com/pytorch/pytorch/tree/master/torch/ao/quantization/fx#13-quantdequantstub-and-observerfakequantize-insertion" rel="noopener nofollow ugc">docs</a>, the invalid <code>qconfig</code> will be ignored.</p>
<blockquote>
<p>QuantDeQuantStubs are inserted based on the <code>qconfig_mapping</code> provided by users. Also we have a backend_config that specifies the configs that are supported by the backend. In this step, we will</p>
</blockquote>
<blockquote>
<ul>
<li>Check if <code>qconfig_mapping</code> is compatible with <code>backend_config</code> or not, if user requested a qconfig that is not compatible with <code>backend_config</code>, we’ll not insert observers for the operator, the config <strong>would just be ignored</strong>.</li>
</ul>
</blockquote>
<p>Is there any warning information telling us which <code>qconfig</code> were ignored and why?</p>

Vasiliy_Kuznetsov: <p>Hi <a class="mention" href="/u/111357">@111357</a> , today the main way you could debug this is by inspecting the model after quantization is applied.</p>
<p>cc <a class="mention" href="/u/jerryzh168">@jerryzh168</a>  for request on warnings when qconfig is ignored due to incompatibility.</p>

jerryzh168: <p>yeah don’t print this right now I think, but this half we may add this support</p>

111357: <p>Hi Vasiliy and Jerry <a class="mention" href="/u/vasiliy_kuznetsov">@Vasiliy_Kuznetsov</a> <a class="mention" href="/u/jerryzh168">@jerryzh168</a> , thx for your reply. Is possible to provide the code snippet or the link about the ignore logic? I want to know more. Many thx <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"></p>