akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Tutuorials about how to design the models able to export

username: WeNeedMore
cooked: <p>Any tutorials of pytorch, to teach us how could we develop model able to export by</p>
<pre><code class="lang-auto">torch.jit.script(model, img)
torch.jit.trace(model, img)
</code></pre>
<p>Not all of the models can export by those cmd directly, do the communties have some tutorials to give us some recommendations of writing “model easy to deploy”?Thanks</p>

username: marksaroufim
cooked: <p>The torchscript tutorial is not bad <a href="https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Introduction to TorchScript — PyTorch Tutorials 1.10.1+cu102 documentation</a></p>
<p>Generally you want to avoid for loops or if conditions or custom operations</p>
Shipping a desktop application with the CUDA binaries?

username: MarkJG
cooked: <p>Is it safe to ship a desktop application that bundles Pytorch CUDA? Are there any legal implications? Referring to the CUDA binaries (cusolver64_10.dll, cusparse64_10.dll and so on)</p>

username: marksaroufim
cooked: <p>Not a lawyer so worth reading the NVIDIA EULA <a href="https://docs.nvidia.com/cuda/eula/index.html" class="inline-onebox" rel="noopener nofollow ugc">EULA :: CUDA Toolkit Documentation</a></p>
Multiple models inference time on the same GPU

username: Jun_Bai
cooked: <p>I used two processes to load two models on a single GPU. but I found the inference time for one process one model is almost similar with two processes two models. I just want to know how to run two models to make the inference in parallel on a single GPU.</p>

username: marksaroufim
cooked: <p>AFAIK GPUs are asynchronous machines so you shouldn’t need to manage multiple processes loading to the GPU, just load it as normal and let PyTorch and NVIDIA handle how to make it efficient</p>
Torch to ONNX conversion going wrong

username: Darshan
cooked: <p>Hi Pytorch Team,</p>
<p>I am converting one PyTorch Model(yoloact) trained on my custom dataset to ONNX, the conversion is completing successfully, but in the ONNX model one of the node is not getting connected to the graph.<br>
I have attached the picture of the graph below.</p>
<p>Having said that, the pytorch model Inference is working fine, but not sure why one of the node is not getting connected to the graph in the onnx model.</p>
<p>Can you please assist me to solve this error? Would be a great help from yourside.</p>
<p>I am attaching the screenshot of the graph below:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/2/02ff6c3bebd5e3de7873f87bea571edd6272d7fd.png" data-download-href="https://discuss.pytorch.org/uploads/default/02ff6c3bebd5e3de7873f87bea571edd6272d7fd" title="Screenshot from 2021-05-18 12-17-13"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/2/02ff6c3bebd5e3de7873f87bea571edd6272d7fd_2_690x219.png" alt="Screenshot from 2021-05-18 12-17-13" data-base62-sha1="qwc8r50bxdNu90bsMPHCV6YxFH" width="690" height="219" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/2/02ff6c3bebd5e3de7873f87bea571edd6272d7fd_2_690x219.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/2/02ff6c3bebd5e3de7873f87bea571edd6272d7fd_2_1035x328.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/2/02ff6c3bebd5e3de7873f87bea571edd6272d7fd.png 2x" data-dominant-color="F7F7F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2021-05-18 12-17-13</span><span class="informations">1341×426 28.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Thanks,<br>
Darshan</p>

username: Darshan
cooked: <p>Looking forward to the Reply.</p>
<p>Regards,<br>
Darshan</p>

username: novice96
cooked: <p>hey how do you convert Pytorch to Onnx , I’ve tried but no lucks.</p>

username: marksaroufim
cooked: <p>Hi Darshan could you share your code? It might be possible that you have an unsused constant</p>
Deploy model with pytorch custom operator to onnx to tensorrt?

username: li_bi
cooked: <p>Is there possible to deploy a model on Nvidia embedded platform (For example: Nividia AGX xavier , Nvidia drive px2, Nvidia dirve xavier) ,this model contains  Sparse convolution pytorch custom operator C++ implement?<br>
In this way : pytorch pth model → onnx model → tensorrt engine.<br>
So the problem becomes to sparse-conv-opt pytorch custom opt → onnx custom opt → tensorrt plugin.<br>
So far as I know, pytorch custom opt → onnx custom opt is supported by offical, but the last step: onnx custom opt → tensorrt plugin.<br>
Is there anybody deployed model in this way successfully? Not limited to sparse convolution.</p>

username: marksaroufim
cooked: <p>Hi <a class="mention" href="/u/li_bi">@li_bi</a> since this is an ONNX specific question your best bet is to open a github issue on an ONNX repo like <a href="https://github.com/onnx/onnx" class="inline-onebox" rel="noopener nofollow ugc">GitHub - onnx/onnx: Open standard for machine learning interoperability</a></p>
Conversion to ONNX: how to get ScatterND?

username: Htess
cooked: <p>I need to do some kind of channel indexation/insertion using scatter functions, but the element-wise scatter available in Pytorch leads to a Scatter ONNX function that is very cumbersome (as it requires indexing every single element of the tensor) while I only need to index channels.<br>
The ONNX function ScatterND would allow that (which would be a lot less memory-expensive, since it requires only storing channels indices) but no equivalent exist in Pytorch (to my knowledge).</p>
<p>I was planning to implement a custom function, that would be converted into a ScatterND, but maybe you know one more straightforward way to get this operator.</p>

username: marksaroufim
cooked: <p><a class="mention" href="/u/htess">@Htess</a> If you can’t find the op you’re looking for here <a href="https://github.com/pytorch/pytorch/blob/master/torch/onnx" class="inline-onebox" rel="noopener nofollow ugc">pytorch/torch/onnx at master · pytorch/pytorch · GitHub</a> in any of the symbolic opset files you may need to implement it yourself</p>
<p>There’s multiple ways of supporting the op yourself <a href="https://pytorch.org/docs/stable/onnx.html#example-alexnet-from-pytorch-to-onnx" class="inline-onebox" rel="noopener nofollow ugc">torch.onnx — PyTorch 1.10 documentation</a></p>
<p>And if you want deeper feedback your best bet is to open up a PR on github and tag members of the ONNX team</p>
Loss function precision with AMP

username: BogdanSavkovic
cooked: <p>In Transfer Learning tutorial, to be able to guarantee the preciseness of loss function calculation, regarding potential difference in sizes between the last batch and other batches, we introduce running loss:</p>
<p><code>running_loss += loss.item() * inputs.size(0)</code></p>
<p>I would like to keep this approach even when using autocast and grad_scaler. In this case:</p>
<pre><code class="lang-auto">with autocast():
    outputs = model(batch_inputs)
    loss = torch.nn.CrossEntropyLoss(outputs)

scaler.scale(loss).backward()
scaler.step(optimizer)
scaler.update()
</code></pre>
<p>what happens to loss?</p>
<p>Should it be unscaled before <code>running_loss += loss.item() * inputs.size(0)</code> and reported that way? Also, where  would that part of code be implemented?</p>
<p>Thanks!</p>

username: BogdanSavkovic
cooked: <p>Code sample:</p>
<pre><code class="lang-auto">def train_one_epoch(dataloader, model, optimizer, scaler, device):
   
    model.train()
    running_loss = 0.0
   
    for inputs, labels in dataloader:
         inputs, labels = inpunts.to(device), labels.to(device)
        
        optimizer.zero_grad()
        with torch.cuda.amp.autocast():
            outputs = model(inputs)
            loss = nn.CrossEntropyLoss(outputs)

        scaler.scale(loss).backward()
        scaler.step(optimizer)
        scaler.update()
    
        # is this correct or do we need unscaling somewhere?
        running_loss += loss.item() * inputs.size(0)

    return running_loss        
</code></pre>

username: ptrblck
cooked: <p>You could unscale the loss my multiplying it with <code>inv_scale = 1./scaler.get_scale()</code> before calling <code>scaler.update()</code>.</p>

username: BogdanSavkovic
cooked: <p>Can’t I just use loss before scaling it?</p>
<p>Also, torch docs says that crossentropy is calculated on fp32 data anyways. Does it mean that amp benefits do not apply while calculating loss this way?</p>

username: ptrblck
cooked: <p>Yes, using the loss before scaling should also work.</p>
<p>Also yes, <code>nn.CrossEntropyLoss</code> needs the higher precision of <code>float32</code> for its stability.</p>
Data loader is too slow

username: OBouldjedri
cooked: <p>I am working on a data set that I stored in pickle extention the data set is set as this :<br>
train data:<br>
classe1:<br>
instence1.pickle<br>
instance2.pickle<br>
…<br>
classe2:<br>
…</p>
<p>I think I am having a big delay in training because of this :</p>
<pre><code>class CustomDataset_train(Dataset):
    def __init__(self):
    
   
      self.data_path = dataset_dir_train
      file_list = glob.glob(self.data_path + "*")
      self.data = []
      for class_path in file_list:
          class_name = class_path.split("\\")[-1]
        
          for data_path in glob.glob(class_path + "/*.pickle"):
            
              self.data.append([data_path, class_name])
          
  
      self.class_map = {'class1 ': 0,
                        'class2':1,
                       ......
                      
                      
                      }
    
 
  def __len__(self):
    
      return len(self.data)
  def __getitem__(self, idx):
  
    data_path, class_name = self.data[idx]

    data = pickle.load(file=open(data_path, "rb"))
  
    
    class_id = self.class_map[class_name]
    
    
    fused_tensor_data=torch.from_numpy(data)
    
   
    
 
    
    return (fused_tensor_data, class_id)
</code></pre>
<p>the data is stored as a numpy array with the shape of 250,164,3 in every pickle file . is there a solution to make it faster ? something like DataFolder</p>

username: Eta_C
cooked: <p>Large single file may be a better choice.</p>
<pre><code class="lang-python">class CustomDataset_train(Dataset):
    def __init__(self, dir, filename="train.pth"):
        if os.path.isfile(filename):
            self.data = torch.load(filename)
        else:
            class_map = {'class1 ': 0,  'class2':1, ... }
            file_list = glob.glob(dir + "*")
            self.data = []
            for class_path in file_list:
                class_name = class_path.split("\\")[-1]
                for data_path in glob.glob(class_path + "/*.pickle"):
                    self.data.append((pickle.load(file=open(data_path, "rb")), class_map[class_name]))
            torch.save(filename, self.data)

    def __getitem__(self, idx):
        return self.data[idx]    
</code></pre>

username: Oussama_Bouldjedri
cooked: <p>what would be the reason for this late ? loading every single file alone? in numpy shape</p>
Channel-wise multiplication Conv Layer and Linear Layer

username: CarlosDanielBez
cooked: <p>Hi, sorry for the inconvenience, I’m new to pytorch.</p>
<p>I would like to build a custom network that integrates two types of data (tabular and image). In this network, the output of a fully connected layer (tabular data input) multiplies the output of a convolutional network layers. For this, the number of neurons in the output is equal to the number of channels in the conv network (channel wise multiplication).</p>
<p>my init looks like this (note: i have a Conv function outside the class.)</p>
<pre><code class="lang-auto">        self.conv1 = conv_block(3, 16)

        self.conv2 = conv_block(16, 32)

        self.conv3 = conv_block(32, 64)

        self.ln1 = nn.Linear(64 * 26 * 26, 16)

        self.relu = nn.ReLU()

        self.batchnorm = nn.BatchNorm1d(16)    #Na predição eliminar o Batch

        self.dropout = nn.Dropout2d(0.5)

        self.ln2 = nn.Linear(16, 3)      

        self.ln4 = nn.Linear(5, 16)

        self.ln5 = nn.Linear(16, 16)     

        self.ln6 = nn.Linear(16, 32)    

        self.ln7 = nn.Linear(32, 64)   

        self.relu = nn.ReLU()
</code></pre>
<p>and forward pass like this:</p>
<pre><code class="lang-auto">def forward(self, img, tab):

        tab = self.ln4(tab)

        tab = self.relu(tab)

        tab = self.ln5(tab)

        tab = self.relu(tab)

        #Entra a Primeira Mul

        img = self.conv1(img)

        img.mul(tab[1:])
       
        tab = self.ln6(tab)

        tab = self.relu(tab)

        #Entra a Segunda Mul

        img = self.conv2(img)

        img = torch.mul(img, tab)

        tab = self.ln7(tab)

        tab = self.relu(tab)

        #Entra a Terceira Mul 

        img = self.conv3(img)

        img = torch.mul(img, tab)

        img = img.reshape(img.shape[0], -1)

        img = self.ln1(img)

        img = self.relu(img)

        img = self.batchnorm(img)

        img = self.dropout(img)

        img = self.ln2(img)
        
        return img
</code></pre>
<p>but everything i try is generating incompatibility in multiplication</p>
<p>RuntimeError: The size of tensor a (111) must match the size of tensor b (31) at non-singleton dimension 2</p>
<p>just for curiosity, the initial multiplications are Image Conv Layer [32, 16, 111, 111] and<br>
Tabular Output first linear ([32, 16])</p>
<p>Can someone help me. It works on Keras using <strong>multiply</strong> method.</p>

username: ptrblck
cooked: <p>You would need to <code>unsqueeze</code> the missing dimension so that PyTorch can broadcast the smaller tensor:</p>
<pre><code class="lang-python">conv_output = torch.randn(32, 16, 111, 111)
a = torch.randn(32, 16)

out = conv_output * a
# &gt; RuntimeError: The size of tensor a (111) must match the size of tensor b (16) at non-singleton dimension 3

out = conv_output * a[:, :, None, None] # works
</code></pre>

username: CarlosDanielBez
cooked: <p>It worked. The solution worked well. Thank you so much!</p>
Pretrained Pytorch Model Train and Validation accuracy on ImageNet?

username: jS5t3r
cooked: <p>I executed the script underneath and I get a train accuracy of 96% and a test accuracy of 77%. I just want to know if this is correct? Do I change the normalization or something else?<br>
Link to model: <a href="https://pytorch.org/vision/stable/_modules/torchvision/models/resnet.html#wide_resnet50_2" class="inline-onebox" rel="noopener nofollow ugc">torchvision.models.resnet — Torchvision 0.11.0 documentation</a></p>
<pre><code class="lang-python">#!/usr/bin/env python3
import pdb
import os, sys
import torch
import torchvision
import torch.nn as nn
import torchvision.transforms as transforms
from torch.utils.data import DataLoader, TensorDataset
import torchvision.datasets as datasets
import torchvision.models as models

def clean_accuracy(data_loader, model, wanted_samples=2000):
    clean_dataset = []; correct = 0; total = 0; i = 0
    for images, labels in data_loader:
        images = images.cuda()
        labels = labels.cuda()
        outputs = model(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()
        if (predicted == labels):
            clean_dataset.append([images, labels])
        i = i + 1
        if i % 500 == 0:
            acc = (wanted_samples, i, 100 * correct / total)
            print('INFO: Accuracy of the network on the %d test images: %d, %d %%' % acc)
        if len(clean_dataset) &gt;= wanted_samples:
            break

model = models.wide_resnet50_2(pretrained=True)
model.cuda(); model.eval()

mean = [0.485, 0.456, 0.406] # https://pytorch.org/vision/stable/models.html#wide-resnet
std  = [0.229, 0.224, 0.225]
normalization = [transforms.Normalize(mean=mean, std=std)]
transform_list = transforms.Compose([transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor()] + normalization)
IMAGENET_PATH  = "/home/DATA/ImageNet" # imagenet 2012
TRAIN_PATH     = os.path.join(IMAGENET_PATH, 'train') # contains subfolders
TEST_PATH      = os.path.join(IMAGENET_PATH, 'val') # contains subfolders 50k samples

train_loader  = torch.utils.data.DataLoader(datasets.ImageFolder(TRAIN_PATH , transform_list), batch_size=1, shuffle=True, num_workers=4, pin_memory=True)
test_loader   = torch.utils.data.DataLoader(datasets.ImageFolder(TEST_PATH, transform_list), batch_size=1, shuffle=True, num_workers=4, pin_memory=True)

clean_accuracy(train_loader, model, wanted_samples=2000)
clean_accuracy(test_loader, model, wanted_samples=2000)
</code></pre>
Parallel for loop computation

username: Kevinzhu
cooked: <p>I’m currently doing mix-coding with MATLAB and Python. Unfortunately in MATLAB ‘parfor’ function cannot call Python function if it returns a python object. So I would like to move the parallel for loop into my python function. Is there any descent way that I can do parallel for loop in python? I’m kinda new to python coding so maybe it is a very basic problem.</p>
<p>Kevin</p>

username: cbalioglu
cooked: <p>Note sure about the nature of your loop, but if it is compute-bound, I would avoid using Python since its infamous GIL (global interpreter lock) will prevent more than one thread to run at the same time. If your loop is IO-bound you might want to check out Python’s <code>asyncio</code> or <code>concurrent</code> packages.</p>
PyG dataloder with previously stored data

username: AleTL
cooked: <p>Hello everyone!</p>
<p>I’m working with a really huge dataset of small graphs. What I’ve done so far is creating a Dataset class that stores the torch_geometric.data.Data() graph objects in a processed folder. What I want is to be able to read those files with a Dataset class without generate them again. How can I do it? It seems pretty silly but I’m wasting a lot of time and I don’t find the way.</p>
<p>The idea is to generate the Data() objects if they haven’t been previously generated and just read it if they already exists. I’m trying to do so creating a Dataset without download and process but it doesn’t work.</p>
<p>Thank you!</p>

username: ptrblck
cooked: <p>I don’t know which format is used to store the generated <code>torch_geometric</code> data but assume you can also load each sample with <code>torch_geometric</code> in a similar way.<br>
If so, I would guess you could write a custom <code>Dataset</code> as described <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">here</a> and load each sample in the <code>__getitem__</code>.</p>

username: AleTL
cooked: <p>Hi!</p>
<p>Thank you for your answer! I’ve tried that but it doesn’t work because a standard Pytorch Dataset needs just tensor data, it’s not able to handle torch_geometric.data.Data() format.<br>
I’m now trying to do it with a PyG Dataset, that can obviously handle PyG.Data objects.</p>
<p>Thanks!</p>

username: AleTL
cooked: <p>In case someone is curious, what I’ve finally done is to load the graph files without any dataset, just like it appears on the PyG tutorial.</p>
<blockquote>
<p>from torch_geometric.data import Data<br>
from torch_geometric.loader import DataLoader</p>
<p>data_list = [Data(…), …, Data(…)]<br>
loader = DataLoader(data_list, batch_size=32)</p>
</blockquote>
How to get multiple Tensors returned from CUDA extension

username: kevin-ssy
cooked: <p>Hi there,</p>
<p>I want to transfer two tensors into my self-defined CUDA extension and return them back after some operations. However, as C++ only support to return one single variable, this seems to be a bit hard to implement. I tried to use boost tuple to combine them, but got an error shown below:</p>
<p>undefined symbol: _ZN5boost6python23throw_error_already_setEv</p>
<p>I also tried to use data_ptr() to get the address of the tensor as an int value, however, I don’t know how to cast the int value into a Tensor pointer in my cuda program. Do you know how to deal with that?</p>

username: albanD
cooked: <p>Hi,</p>
<p>Internally, <code>using variable_list = std::vector&lt;Variable&gt;;</code> is used.<br>
I am not sure if pybind will convert that properly though <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

username: MLAI
cooked: <p>As <a class="mention" href="/u/alband">@albanD</a> already pointed out. You can use <code>std::vector&lt;torch::Tensor&gt;</code> to wrap multiple tensors in a vector. Pybind11 will convert that properly to a list of torch tensors.</p>
<p>This Dummy function just outputs N boring (all-zeros) torch tensors.</p>
<pre><code class="lang-auto">#include &lt;torch/extension.h&gt;
using namespace std;

std::vector&lt;torch::Tensor&gt; Dummy(int const N) {
    std::vector&lt;torch::Tensor&gt; outputs;
    auto out = torch::zeros({1,2}, torch::dtype(torch::kFloat32));
    for (int n=0; n&lt;N; n++)
        outputs.push_back(out.clone());
    return outputs;
}
PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
  m.def("Dummy", &amp;Dummy, "Dummy function.");
</code></pre>
<p>After building, we can use it in Python as</p>
<pre><code class="lang-auto">import torch
import dummyextension

outputs = dummyextension.Dummy(3)
print(outputs)
&gt;&gt; [tensor([[0., 0.]]), tensor([[0., 0.]]), tensor([[0., 0.]])]
</code></pre>
<p>Of course pythons list unpacking will work as well:</p>
<pre><code class="lang-auto">out1, out2 = dummyextension.Dummy(2)
</code></pre>
Reduction mean weighted for NLL loss Multi-Label classification CF RecSys

username: marathon.our
cooked: <p>Hi, I have implemented a Variational AutoEncoder for Collaborative Filterting (user-item) and since I have very sparse data, where some items are very popular and some are not, I’ve calculated the weights for them and try to use them with reduction=mean for NLLLoss (it’s actually CE because I use LogSoftmax right before), however pytorch doesn’t work with multi label classification<br>
For example I have labels in format</p>
<pre><code class="lang-auto">[[0, 1, 0, 0, 0],
 [1, 0, 0, 0, 0],
 [0, 0, 0, 1, 0]]
</code></pre>
<p>which could be interpreted as <code>[1, 0, 3]</code> and it will work with nn.NLLLoss, but I can have <code>[[0,1,0,1,0]]</code> labels, meaning what user bought 2 out of 6 items and I need to classify both for this sample.<br>
So I implement loss myself as<br>
<code>loss = -outputs * targets * weight</code><br>
which is the unreduced (i.e. with <code>reduction</code> set to <code>'none'</code> ) loss and it matches nn.NLLLoss for <code>reduction='none'</code><br>
But I want to get  <code>reduction</code>=<code>'mean'</code> loss, and when I use the formula from official repo <code>loss.sum() / weight.sum()</code> the loss is not equal to <code>nn.NLLLoss(weight=weight, reduction='mean')</code> I tried just getting the mean of my loss and results seem more correlating but anyway different.<br>
My question is what formula should I use to implement NLL loss with weights and with reduction mean to match nn.NLLLoss for simple samples like <code>[0,0,1,0,0]</code> so I can then use this formula for CF samples like <code>[0,1,0,1,0]</code> (refers to item 1 and item 3 are present)</p>
<p>simple reproducible test</p>
<pre><code class="lang-auto">import torch

size = (3, 5)
kwargs = dict(
    reduction='mean',
    weight=torch.randn(size=(1, 5))
)
outputs = torch.randn(*size, requires_grad=True)
targets = torch.tensor([1, 0, 3])
targets_one_hot = torch.tensor([[0, 1, 0, 0, 0],
                                [1, 0, 0, 0, 0],
                                [0, 0, 0, 1, 0]])


def nll_loss(outputs, targets: torch.Tensor, weight, reduction):
    loss = -outputs * targets * weight
    loss = loss.sum(dim=-1, keepdim=True)
    if reduction == 'none':
        return loss
    elif reduction == 'sum':
        return loss.sum()
    elif reduction == 'mean':
        return loss.sum() / weight.sum()


src_loss_value = torch.nn.functional.nll_loss(outputs, targets, **kwargs)
own_loss_value = nll_loss(outputs, targets_one_hot, **kwargs)
assert torch.allclose(src_loss_value, own_loss_value)
</code></pre>
<p>Any ideas would be great, thank you :).</p>
Get input params from layer with a jit model

username: Milton_Rugues
cooked: <p>Hello,</p>
<p>I would like to have the input parameters of a layer.<br>
For example for a convolution layer conv2D, I would like to have the padding, the stride, etc.</p>
<p>The model I use is a torch::jit::script::Module. I searched the internet and the docs to see where to get all this, but I found nothing.</p>
<p>Thank you very much</p>

username: AlexWilkinsonnn
cooked: <p>Maybe you want <code>dump</code>:<br>
<a href="https://pytorch.org/cppdocs/api/structtorch_1_1jit_1_1_module.html#_CPPv4NK5torch3jit6Module4dumpEbbb" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/cppdocs/api/structtorch_1_1jit_1_1_module.html#_CPPv4NK5torch3jit6Module4dumpEbbb</a></p>
How to share a nn.LSTM layer for several sequence?

username: Alobal_Blizz
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/2/4231eda1e92b79943dc7971d72c5cf02c2ce44e4.png" alt="2022-02-22 17-05-32 的屏幕截图" data-base62-sha1="9rAtZRCoohYUBaevxd89Vv3Byio" width="394" height="217"></p>
<p>e.g. in PointLSTM model, there are 4 sequences i.e. (x;f)_nt , they need to compute through a shared LSTM layer with a shared state, and output independent 4 states i.e. (h,c)_nt.</p>
<p>But I don’t know how to  share a LSTM layer for 4 different sequences, or how to compute through LSTM but not update its state, which can achieve the shared-LSTM.</p>
Storing data in a map-like instruction with a pair for the key

username: Nil_MSh
cooked: <p>Hi! I have trained resnet model and I’ve assigned a single integer to each of my neurons. I want to sort and find 5 of most valuable and least valuable neurons based on that single integer. How should I store my data so I can sort and have the layer and id of those neuron + that integer value?</p>
Runtime warning loss.backward resize

username: frakkerman
cooked: <p>I am using the C++ API.<br>
When using either l1_loss or mse_loss I get an annoying warning when calling loss.backward(). Small code snippet:</p>
<pre><code class="lang-auto">  opt.zero_grad();		
  auto forward = model-&gt;forward(minibatch.features);
  torch::Tensor loss = torch::mse_loss(forward, minibatch.labels);//torch::Tensor loss = torch::l1_loss(forward, minibatch.labels);
  loss.backward();
  opt.step();
</code></pre>
<p>The runtime warning I get is as follows:</p>
<pre><code class="lang-auto">[W ..\..\aten\src\ATen\native\Resize.cpp:23] Warning: An output with one or more elements was resized since it had shape [64, 1], which does not match the required output shape [64, 64].This behavior is deprecated, and in a future PyTorch release outputs will not be resized unless they have zero elements. You can explicitly reuse an out tensor t by resizing it, inplace, to zero elements with t.resize_(0). (function resize_output_check)

</code></pre>
<p>So, in this case my minibatch size is 64, the features and labels are all correctly dimensioned. I do not understand why it is expecting a [64,64] output shape.<br>
I searched for this online, found one topic (<a href="https://github.com/pytorch/examples/issues/819" class="inline-onebox" rel="noopener nofollow ugc">DCGAN C++ warning after PyTorch update · Issue #819 · pytorch/examples · GitHub</a>) that deals with this problem, but no good solution seems to be proposed.</p>
<p>Am I overlooking something in the mse/l1 loss functions? Is this a known issue? Is there a way to change the verbosity of warning printing in the C++ API? FYI: the code runs without errors and the neural net seems to fit properly. I use LibTorch version 1.10.2+cpu.</p>

username: ptrblck
cooked: <p>What shapes do the <code>forward</code> and <code>minibatch.labels</code> have?<br>
I’m wondering if you are broadcasting in <code>mse_loss</code> (which could yield wrong results, if that’s not intended).</p>

username: frakkerman
cooked: <p>In the case of a minibatch size of 64 <code>forward</code> is a <code>CPUFloatType{64,1}</code>, so it contains a prediction for every sample in the minibatch. <code>minibatch.labels</code> is a <code>CPUFloatType{64}</code>.</p>
<p>So, you are completely right, I was broadcasting in <code>mse_loss</code>. I fixed this by making <code>minibatch.labels</code> a <code>CPUFloatType{64,1}</code>, so it has the same shape as <code>forward</code>.</p>
<p>Thanks for pointing me in the right direction!</p>
DQN tutorial with no cheat

username: alexis-jacq
cooked: <p>In the DQN tutorial (<a href="https://github.com/pytorch/tutorials/blob/master/Reinforcement%20(Q-)Learning%20with%20PyTorch.ipynb">https://github.com/pytorch/tutorials/blob/master/Reinforcement%20(Q-)Learning%20with%20PyTorch.ipynb</a>), at a point, it is suggested to crop the image around the object of interest, while we expect the algorithm to be able to extract this feature. If I try to remove this “cheating” trick (commented in the following code)…</p>
<pre><code class="lang-python">resize = T.Compose([T.ToPILImage(), T.Scale(40, interpolation=Image.CUBIC), T.ToTensor()])

# This is based on the code from gym.
screen_width = 600
def get_cart_location():
    world_width = env.x_threshold * 2
    scale = screen_width / world_width
    return int(env.state[0] * scale + screen_width / 2.0) # MIDDLE OF CART
    
    
def get_screen():
    screen = env.render(mode='rgb_array').transpose((2, 0, 1)) # transpose into torch order (CHW)
    # Strip off the top and bottom of the screen
    '''
    # this is the trick :
    screen = screen[:, 160:320]
    view_width = 320
    cart_location = get_cart_location()
    if cart_location &lt; view_width // 2:
        slice_range = slice(view_width)
    elif cart_location &gt; (screen_width - view_width // 2):
        slice_range = slice(-view_width,None)
    else:
        slice_range = slice(cart_location - view_width // 2, cart_location + view_width // 2)
    # Strip off the edges, so that we have a square image centered on a cart
    screen = screen[:, :, slice_range]
    '''
    # Convert to float, rescare, convert to torch tensor (this doesn't require a copy)
    screen = np.ascontiguousarray(screen, dtype=np.float32) / 255
    screen = torch.from_numpy(screen)
    # Resize, and add a batch dimension (BCHW)
    return resize(screen).unsqueeze(0)

env.reset()
plt.imshow(get_screen().squeeze(0).permute(1, 2, 0).numpy(), interpolation='none')
plt.show()
</code></pre>
<p>… I can’t obtain any learning. I tried several different combinations of parameters, and I also try to change the structure of the network, but no way to reach any acceptable result. I there a way to make it work ?</p>

username: apaszke
cooked: <p>That’s the primary reason why the input is cropped <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
That’s how it is with RL, it’s quite unstable.</p>

username: alexis-jacq
cooked: <p>Haha, I take your response as a challenge! I will find a way.</p>

username: arc_HITMAN
cooked: <p>Hey, Did you find a way to make the agent learn without the trick?</p>
Conditional operation on a NxM tensor based on values of another NxM tensor(labels)

username: yegane
cooked: <p>Hi,<br>
I have an NxM matrix (A) and another NxM matrix (B). The values of B are labels for values of A.<br>
If I have C number of labels in B. Is there a simple GPU-friendly operation that takes the average of all elements of A that have the same labels in B, and then maps them into a vector with the size of Cx1? Can I make it without loops?<br>
A=[5 6 7 9 10]<br>
B=[0 0 0 1  1]<br>
(5+6+7)/3=6<br>
(9+10)=9.5<br>
C=[6 9.5]</p>

username: KFrank
cooked: <p>Hi Yegane!</p>
<aside class="quote no-group" data-username="yegane" data-post="1" data-topic="144706" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yegane/48/10618_2.png" class="avatar"> yegane:</div>
<blockquote>
<p>the average of all elements of A that have the same labels in B,</p>
</blockquote>
</aside>
<p>One approach is to extend both <code>A</code> and <code>B</code> along a new “class” dimension<br>
(using <code>expand()</code> and <code>one_hot()</code>, respectively).  The one-hotted version<br>
of the labels tensor is used as a class mask for the values tensor so that<br>
per-class sums and counts can be computed, and from them, averages.</p>
<p>Here is such a function that computes the per-class averages, together<br>
with some test cases:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)

_ = torch.manual_seed (2022)

# could be simpler if data has a fixed dimensionality or shape

def class_average (data, labels):
    assert  data.shape == labels.shape
    nClass = labels.max() + 1
    dims = [-1] * data.dim() + [nClass]
    dex = data.unsqueeze (-1).expand (dims)
    lex = torch.nn.functional.one_hot (labels)
    sums = (lex * dex).sum (list (range (data.dim())))
    cnts = lex.sum (list (range (data.dim())))
    return  sums / cnts   # nan for labels that don't occur

A = torch.tensor ([5, 6, 7, 9, 10])
B = torch.tensor ([0, 0, 0, 1, 1])
clAB = class_average (A, B)

print ('A:', A)
print ('B:', B)
print ('class_average (A, B):', clAB)

m = 3
n = 5
nClass = 4

data = torch.randn (m, n)
labels = torch.randint (nClass, (m, n))
cldl = class_average (data, labels)

print ('class_average (data, labels):', cldl)

cl_missing_label = class_average (torch.arange (4), torch.tensor ([0, 1, 2, 4]))
print ('class_average (torch.arange (4), torch.tensor ([0, 1, 2, 4])):', cl_missing_label)
</code></pre>
<p>Here is the test output:</p>
<pre><code class="lang-plaintext">1.10.2
A: tensor([ 5,  6,  7,  9, 10])
B: tensor([0, 0, 0, 1, 1])
class_average (A, B): tensor([6.0000, 9.5000])
class_average (data, labels): tensor([ 0.4032,  0.1719,  0.1518, -1.1578])
class_average (torch.arange (4), torch.tensor ([0, 1, 2, 4])): tensor([0., 1., 2., nan, 3.])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>
Sample from Mixture Density

username: indrajitsg
cooked: <p>I am trying to sample from a mixture density:</p>
<pre><code class="lang-auto">import torch
from torch.distributions.mixture_same_family import MixtureSameFamily
from torch.distributions.categorical import Categorical
from torch.distributions.normal import Normal

# Set values for the mixture
alphas = [0.6, 0.3, 0.1]
means = [30, 60, 120]
sigmas = [5, 3, 1]

gm = MixtureSameFamily(mixture_distribution=Categorical(probs=torch.tensor(alphas)),
                       component_distribution=Normal(loc=torch.tensor(means), scale=torch.tensor(sigmas)))

# Draw 1e5 random samples from the mixture distribution
torch.manual_seed(seed=42)
samples = gm.sample(sample_shape=torch.Size([100]))
</code></pre>
<p>But I am getting this error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "mdn_test.py", line 27, in &lt;module&gt;
    samples = gm.sample(sample_shape=torch.Size([100]))
  File "/data/miniconda3/envs/py38/lib/python3.8/site-packages/torch/distributions/mixture_same_family.py", line 167, in sample
    comp_samples = self.component_distribution.sample(sample_shape)
  File "/data/miniconda3/envs/py38/lib/python3.8/site-packages/torch/distributions/normal.py", line 64, in sample
    return torch.normal(self.loc.expand(shape), self.scale.expand(shape))
RuntimeError: "normal_kernel_cpu" not implemented for 'Long'
</code></pre>
<p>Please help - I am not able to understand what this error means.<br>
Regards,<br>
Indrajit</p>

username: ptrblck
cooked: <p>Pass the <code>means</code> and <code>sigmas</code> as floating point values:</p>
<pre><code class="lang-python">means = [30., 60., 120.]
sigmas = [5., 3., 1.]
</code></pre>
<p>and it should work.</p>

username: indrajitsg
cooked: <p>Awesome! Thank you so much!</p>
How to turn parameter to buffer?

username: grudloff
cooked: <p>In the context of having a trained model. Is there a way to turn a parameter into a buffer to keep it fixed, as an alternative to freezing that parameter?</p>

username: grudloff
cooked: <p>Seems this can be done like this:</p>
<pre><code class="lang-python">for name, param in module.named_parameters(recurse=False):
    delattr(module, name) # Unregister parameter
    module.register_buffer(name, param) 
</code></pre>
<p><code>recurse</code> must be set to false, because in case there is a submodule with its own parameters, doing this would fail. Instead, this should be repeated for all submodules.</p>
<p>Edit:<br>
Here is a function that does this:</p>
<pre><code class="lang-python">def param_to_buffer(module):
    """Turns all parameters of a module into buffers."""
    modules = module.modules()
    module = next(modules)
    for name, param in module.named_parameters(recurse=False):
        delattr(module, name) # Unregister parameter
        module.register_buffer(name, param)
    for module in modules:
        param_to_buffer(module)
</code></pre>
Resuming Training with New Parameters

username: Erdi
cooked: <p>Hi all</p>
<p>Assume that we are performing hyper-parameter optimization(grid or random search for example). In the first trial, we grab a set of parameters(learning rate, batch size etc) , train and save the model. In the second trial, if we grab <em>different set of parameters</em>, load previously saved model and start training, is it reasonable to expect better final performance from the second trial? This seems to be just like transfer learning.<br>
If so, then I can save the best model in each trial and start training in the next trial by loading this best model. I tried on a simple binary classification and it seems it is working. So my question, is it reasonable to start the next trial (having different parameters) on the top of the previously saved ‘best’ model?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Erdi" data-post="1" data-topic="144634">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/erdi/48/15065_2.png" class="avatar"> Erdi:</div>
<blockquote>
<p>So my question, is it reasonable to start the next trial (having different parameters) on the top of the previously saved ‘best’ model?</p>
</blockquote>
</aside>
<p>No, I don’t think so as it would invalidate your hyperparameter search.<br>
The goal of a grid or random search of hyperparameters is to find the best (or any suitable) set of these parameters and use them to train the actual model.<br>
In your use case each new run wouldn’t give you any information about the performance of this particular hyperparameter set since the model was already pretrained in the last run.</p>
PyTorch Installation - Error - " Could Not Find Version "

username: amajor
cooked: <p>im using pip version 21.3.1 ,</p>
<p>When i enter the commands from the website to install PyTorch, it states Error below,</p>
<p>Error: Could not find a version that satisfies the requirement torch==1.10.2+cu113 (from versions: none)<br>
Error: No matching distribution found for torch==1.10.2+cu113</p>
<p>What is wrong, how do i correct this ?</p>
<p>Also, can someone confirm that you can install PyTorch with 32 bit ? i assume this is fine with python pip, but i notice on the website it says with anaconda it must be 64 bit.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="amajor" data-post="1" data-topic="144623">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/f1d935/48.png" class="avatar"> amajor:</div>
<blockquote>
<p>Also, can someone confirm that you can install PyTorch with 32 bit ?</p>
</blockquote>
</aside>
<p>No, I don’t think that’s possible and you would need to install a Python 64bit version between 3.6 and 3.9. I don’t know if Python 3.10 is fully supported already, but the binaries are at least not built for it in the latest stable release.</p>

username: amajor
cooked: <p>Thanks for your response,</p>
<p>If that’s true for the 32 bit with pip, it needs to be stated on the website, it is not there. it is only stated there for the anaconda.</p>

username: ptrblck
cooked: <p>Thanks for pointing this out! Would you be interested in submitting a fix for this? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
About pytorch's memory management

username: eedalong
cooked: <pre><code class="lang-auto">    auto options = torch::TensorOptions()
                           .dtype(at::kFloat)
                           .device(torch::kCUDA, current_device);
    auto res = torch::empty(res_shape, options);
</code></pre>
<p>When I create a CUDA tensor in C++, does this automatically use pytorch’s CudaCachingAllocator?</p>

username: ptrblck
cooked: <p>Yes, creating tensors on the GPU will use the caching allocator.</p>
Exact displacement of each pixel on using grid_sample

username: nik-u
cooked: <p>I have input image and grid passed in torch.nn.functional.grid_sample(). Now if I have a random pixel location (x, y) from the input image, how can I find out its location in the output of grid_sample().</p>
<p>Would this be sufficient for finding new location of pixel:<br>
ix = ((ix + 1) / 2) * (IW-1);<br>
iy = ((iy + 1) / 2) * (IH-1);<br>
as mentioned in <a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c" class="inline-onebox" rel="noopener nofollow ugc">pytorch/SpatialGridSamplerBilinear.c at f064c5aa33483061a48994608d890b968ae53fb5 · pytorch/pytorch · GitHub</a></p>
DataParallel with test batch size = 1

username: theGuyWithBlackTie
cooked: <p>I have trained a model using <code>DataParallel</code> that used 2 GPUs and <code>train_batch_size = 16</code>. Now for certain unavoidable reasons, I need to test the model with <code>test_batch_size = 1</code>. My question is; is it possible to do that?</p>
<p>Because, with <code>test_batch_size = 1</code>, I’m getting below error:</p>
<p><code>ValueError: not enough values to unpack (expected 2, got 1)</code></p>
<p>So, is this error due to <code>test_batch_size = 1</code> and if yes how can I test model with <code>test_batch_size=1</code> considering I’ve used <code>DataParallel</code> and <code>train_batch_size &gt; 1</code>?</p>

username: ptrblck
cooked: <p>You could use the internal model directly via <code>model.module</code> and pass the sample directly to it.<br>
However, the error message suggests the error is caused while unpacking an object and <code>nn.DataParallel</code> is able to accept a single sample in the common use case as seen here:</p>
<pre><code class="lang-python">import torch
import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(1, 1)

    def forward(self, x):
        print('forward on device {}'.format(x.device))
        out = self.fc(x)
        return out

model = MyModel()
model = nn.DataParallel(model).cuda()

x = torch.randn(8, 1).cuda()
out = model(x)

print('single sample')
x = torch.randn(1, 1).cuda()
out = model(x)
</code></pre>
<p>Output:</p>
<pre><code class="lang-python">forward on device cuda:0
forward on device cuda:1
forward on device cuda:2
forward on device cuda:3
forward on device cuda:4
forward on device cuda:5
forward on device cuda:6
forward on device cuda:7
single sample
forward on device cuda:0
</code></pre>