akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
RNN:RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed

username: lhsICT
cooked: <p>Which I wanted was a software that return ‘2’ when I give float list’1,2,3,4’as input.<br>
However, it made error message</p>
<blockquote>
<p>import torch<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
import numpy as np<br>
import pandas as pd<br>
import matplotlib.pyplot as plt<br>
import datetime<br>
import torch.optim as optim</p>
<p>torch.manual_seed(1)</p>
<p>input_size=1<br>
hidden_size=1<br>
learning_rate=0.1<br>
x_data=[[1,2,3,4]]<br>
x_one_hot=[[[1],[2],[3],[4]]]<br>
y_data=[[2]]</p>
<p>x=torch.FloatTensor(x_one_hot)<br>
y=torch.LongTensor(y_data)</p>
<p><span class="hashtag-raw">#declare</span> RNN<br>
rnn=torch.nn.RNN(input_size,hidden_size,batch_first=True)</p>
<p><span class="hashtag-raw">#loss</span>&amp;optimizer setting<br>
criterion=torch.nn.CrossEntropyLoss()<br>
optimizer=optim.Adam(rnn.parameters(),learning_rate)</p>
<p><span class="hashtag-raw">#start</span> training<br>
for i in range(100):<br>
optimizer.zero_grad()<br>
outputs,_status=rnn(x)<br>
<span class="hashtag-raw">#print</span>(outputs.shape)<br>
<span class="hashtag-raw">#print</span>(outputs.view(-1,input_size).shape)<br>
<span class="hashtag-raw">#print</span>(y.view(-1).shape)<br>
print(outputs[0].shape)<br>
print(y[0].shape)<br>
loss=criterion(outputs[0],y[0])<br>
loss.backward()<br>
optimizer.step()<br>
result=outputs.data.numpy().argmax(axis=2)<br>
print(“prediction:”,result)</p>
</blockquote>
<p>and this is an error message<br>
RuntimeError: Assertion `cur_target &gt;= 0 &amp;&amp; cur_target &lt; n_classes’ failed.<br>
I need some help.Thank you.</p>
Def _sync_transform(self, img, mask):

username: achref
cooked: <p>Hello<br>
np.unpackbits get 24 bits for Cityscapes  we reverse the order (total 19 classes) for SBD  datasets we reverse the order (total 20 classes)</p>
<p>For ADE20K that has 150 classes what i do for getting 150 bits.</p>
<p>def _sync_transform(self, img, mask):<br>
<span class="hashtag">#hy</span> modified this function<br>
# random crop crop_size<br>
crop_size = self.crop_size<br>
w, h = img.size<br>
x1 = random.randint(0, w - crop_size)<br>
y1 = random.randint(0, h - crop_size)<br>
img = img.crop((x1, y1, x1+crop_size, y1+crop_size))<br>
mask = mask.crop((x1, y1, x1+crop_size, y1+crop_size))<br>
<span class="hashtag">#np</span>.unpackbits get 24 bits, we extract [:,:5:] and reverse the order (total 19 classes), i.e. [:,:,-1:-20:-1]<br>
mask = np.unpackbits(np.array(mask), axis=2)[:,:,-1:-20:-1]<br>
mask = torch.from_numpy(np.array(mask)).float()<br>
mask = mask.transpose(0, 1).transpose(0, 2) <span class="hashtag">#channel</span> first<br>
# return img, self._mask_transform(mask)<br>
return img, mask</p>
<p>Thank you</p>

username: ptrblck
cooked: <p>As far as I see it in the <a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.unpackbits.html">docs</a>, only <code>np.uint8</code> is supported by <code>np.unpackbits</code>, so you might need to use <a href="https://docs.scipy.org/doc/numpy/user/basics.byteswapping.html">byte swapping</a> to get the underlying representation.</p>
<p>What is your use case that you need to hack around the bits?</p>

username: achref
cooked: <p>Hi<br>
Thank you for your response.</p>
<p>DFF algorithm estimate for each pixel at the boundry the probabilities belongs to each class.<br>
I’ve understood  each bit if equal 1 then this pixel correspond to Class_i<br>
but until now it is complicate to modify mask = np.unpackbits(np.array(mask), axis=2) to extract more than 24 bits.</p>
Does Pytorch support fused/folded BatchNormalization?

username: Shisho_Sama
cooked: <p>Hello everyone, hope you are having a great day.<br>
I’m curious to know whether Pytorch (as of latest version) have support for <strong>Fused</strong> <strong>BatchNormalization</strong>.<br>
Basically <code>FusedBatchNormalization</code> is simply the fusion of <code>BatchNormalization</code> into precdeding convolutional neural network since, the parameters after  training are fixed and can thus be used as constants.<br>
Based on tensorflow’s documentation, it provides 12% to 30% boost of performance at inference time which is a considerable gain. <a href="https://web.archive.org/web/20180620042504/https://www.tensorflow.org/performance/performance_guide" rel="noopener nofollow ugc">Link</a></p>
<blockquote>
<p>Fused batch norm combines the multiple operations needed to do batch normalization into a single kernel. Batch norm is an expensive process that for some models makes up a large percentage of the operation time. Using fused batch norm can result in a 12%-30% speedup.</p>
<p>There are two commonly used batch norms and both support fusing. The core <a href="https://web.archive.org/web/20180620042504/https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization" rel="noopener nofollow ugc"> <code>tf.layers.batch_normalization</code> </a> added fused starting in TensorFlow 1.3.</p>
</blockquote>
<p>I know we had a Pr back in 2017 which was rejected. but I dont know if we have it implemented or not!<br>
Any update in this regard is greatly appreciated.</p>
<p><strong>Update :</strong><br>
Here is a Pytorch implementation from <strong>Intel’s</strong> <strong>NervanaSystems</strong> : <a href="https://github.com/NervanaSystems/distiller/blob/4717596112b70600bb3ac54c7a23a55abfea113e/distiller/model_transforms.py#L102" rel="noopener nofollow ugc">Folded_batch_normalization</a></p>

username: vadimkantorov
cooked: <p>PyTorch currently has <code>torch.nn.utils.fuse_conv_bn_eval</code> (and I think some ConvBnRelu fusion with kernels only in quantized setting) which you must call manually (for details look at the source code, it’s really simple code), but it’s <a href="https://github.com/pytorch/pytorch/issues/28757" rel="nofollow noopener">not documented yet</a></p>
Can pytorch1.0.0 in win10 with cuda9.0 use tensorboard？

username: Hong_Cheng
cooked: <p>Now I am using pytorch1.0 in win10 with cuda9.0. I’d like to use tensorboard to visualize the training progress and check the network structure.</p>
<p>However, I find pytorch1.3 have the official support for tensorboard.<br>
But pytorch1.3 need cuda9.2.</p>
<p>What should I do?</p>

username: peterjc123
cooked: <p>If you have the driver version &gt;= 398.26, then you could install pytorch 1.3. cuda 9.2 here doesn’t refer to the cuda version at the user side, but it means the version of cuda used during build.</p>

username: Hong_Cheng
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/a/3a0a2496f70209d28897701180cdca250a3ec133.png" alt="image" data-base62-sha1="8hrqjqQ2jxzZBry9gxEi7CqVk0b" width="673" height="334"><br>
God, I use search tool in win10, and I find the nvidia-smi!<br>
Here the driver version is 432, I’ll try to install pytorch1.3 in my PC.</p>
<p>Thanks a lot</p>

username: Hong_Cheng
cooked: <aside class="quote no-group" data-username="Hong_Cheng" data-post="4" data-topic="65747">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hong_cheng/48/10639_2.png" class="avatar"> Hong_Cheng:</div>
<blockquote>
<p>0, and I find the nvidia-smi!<br>
Here the driver version is 432, I’ll try to install pytorch1.3 in my PC.</p>
<p>Thanks a lot</p>
</blockquote>
</aside>
<p>It works.<br>
I install pytorch1.3.1 with cuda10 and cudnn7.5 in the GTX1080<br>
And we need to install the tensorboard with version higher 1.14.0</p>
torch.Size([]) (target size != input size)

username: Ronnie1107
cooked: <p>Hi, i’m trying to create a linear regression neural network. It’s my first time using pytorch, and i’m usinge multiple inputs. However i keep stubling into a problem where my target size is different to input size at the criterion function. My output has the size of [1] and the target one [], which is where i got stuck, as i don’t understand how it can be that size - it contains a number (i’m using this dataset <a href="https://www.kaggle.com/uciml/pima-indians-diabetes-database" rel="nofollow noopener">https://www.kaggle.com/uciml/pima-indians-diabetes-database</a> ). I tried browsing for this problem but nothing really helped me. Sorry if this is a stupid question or i’m doing something wrong, i never worked with pytorch before.<br>
Here is the code:</p>
<pre><code class="lang-auto">pdTrain = pd.read_csv("train.csv", header=None)
pdTest = pd.read_csv("test.csv", header=None)
tmpTrain = pdTrain.values
tmpTest = pdTest.values
trainDataset = torch.from_numpy(tmpTrain).float()
testDataset = torch.from_numpy(tmpTest).float()

batch_size = 100
n_iters = 30
epochs = n_iters / (len(trainDataset) / batch_size)
epochs = int(epochs)

train_loader = torch.utils.data.DataLoader(dataset=trainDataset,
                                           batch_size=batch_size,
                                           shuffle=True)
test_loader = torch.utils.data.DataLoader(dataset=testDataset,
                                          batch_size=batch_size,
                                          shuffle=False)

class LinearRegression(nn.Module):
    def __init__(self,input_size,output_size):
        super(LinearRegression,self).__init__()
        self.linear = nn.Linear(input_size,output_size)

    def forward(self,x):
        out = self.linear(x) 
        return out

input_size = 8
output_size = 1
learning_rate = 0.002
model = LinearRegression(input_size,output_size)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(),lr=learning_rate)

for epoch in range(epochs):
    epoch += 1
    for i in enumerate(train_loader):
        for j in i[1]:
            inputs = j[0:8]
            result = j[8]
            optimizer.zero_grad()
            outputs = model(inputs)
            loss = criterion(outputs, result)  #problem
            loss.backward()
            optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss))
</code></pre>

username: albanD
cooked: <p>Hi,</p>
<p>A Tensor with no dimension (size []) means it is a scalar.<br>
You can make them the same size by doing <code>.squeeze()</code> on the output.</p>

username: Ronnie1107
cooked: <p>Thanks, that helped with the size issue, however the loss seems to be shrinking for a couple of instances, then it goes to infinity and then to nan, where it stays for the rest of the runtime.</p>
<p>EDIT: changing the learning rate helped with that. However the loss seems to be jumping around, not descreasing too much, and it gets way outside 0-1 range</p>
Understanding 3 dimensions of input in nn.LSTM(input, hidden)?

username: shwetank
cooked: <p>I am facing problem to understand how the first dimension of input in nn.LSTM(input,hidden) is working internaly. I am doing time-series problem, and passing a seq of 10 numbers.<br>
please help me to understand this. i am also sharing the similar link to help you to understand.</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://stackabuse.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/" target="_blank" title="02:34PM - 25 October 2019" rel="nofollow noopener">Stack Abuse – 25 Oct 19</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail">

<h3><a href="https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/" target="_blank" rel="nofollow noopener">Time Series Prediction using LSTM with PyTorch in Python</a></h3>

<p>Time series data, as the name suggests is a type of data that changes with time. For instance, the temperature in a 24-hour time period, the price of various products in a month, the stock prices of a particular company in a year. Advanced deep...</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
How to create heatmap by given coordinates in mat file in pytorch

username: Shriya_Kak
cooked: <p>I am working on one network, for that the input is image and heatmap corresponding to 21 landmarks for face alignment. I want to preprocess the data. Could anyone suggest any method of how to generate heatmap from the given x,y coordinate in mat file in PyTorch.</p>

username: ptrblck
cooked: <p>Could you explain your use case a bit more?<br>
Do you have heatmaps or coordinates stored in the <code>mat</code> file?<br>
Would you like to create a model, which predicts one or the other and are stuck in creating the targets?</p>

username: Shriya_Kak
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,<br>
I have coordinates in the mat file for the 21 landmarks for every image. The input to the network is an image(RGB) concatenated with heatmaps of these 21 landmarks. I am trying to train a network which will generate 68 landmarks using the above input.</p>

username: ptrblck
cooked: <p>Thanks for this information.<br>
Since you have the landmarks, how would you like to create the heatmaps?<br>
Do you want to plot these landmarks as single pixels into a single channel image and concatenate it with the input or are you thinking about another approach?</p>

username: Shriya_Kak
cooked: <p>The landmarks as a single-pixel need to be plotted in 3 channels i.e RGB and concatenated with the image.</p>

username: ptrblck
cooked: <p>In that case, simple indexing should work fine:</p>
<pre><code class="lang-python">N, H, W = 12, 10, 10
landmarks = torch.randint(0, H, (N, 2))

tmp = torch.zeros(N, 3, H, W)
rows, cols = landmarks.split(1, 1)
tmp[torch.arange(N), :, rows, cols] = 1.

image = torch.randn(N, 3, H, W)
image = torch.cat((image, tmp), 1)
print(image.shape)
&gt; torch.Size([12, 6, 10, 10])
</code></pre>

username: Shriya_Kak
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>. I’ll try this and update.</p>
nn.CTCLoss negative loss

username: HuynhSang
cooked: <p>Hello everyone, I wonder if someone could help me with this. I created a mini test with pytorch.nn.CTCLoss, and i don’t know why it return negative.</p>
<pre><code class="lang-auto">import torch
from torch import nn

print(torch.__version__)    # 1.3.1

# Initialize random batch of input vectors, for *size = (T,N,C)
input = torch.FloatTensor([[[0.1, 0.6, 0.1, 0.1, 0.1], [0.1, 0.1, 0.6, 0.1, 0.1]]]).transpose(0, 1)
input_lengths = torch.IntTensor([2])

labels = torch.IntTensor([[1, 2]])
label_sizes = torch.IntTensor([2])

ctc_loss = nn.CTCLoss()
loss = ctc_loss(input, labels, input_lengths, label_sizes)
print(loss.item())  # -0.6000000238418579
</code></pre>

username: ptrblck
cooked: <p><code>nn.CTCLoss</code> expects log probabilities as the input as described in the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CTCLoss">docs</a>.<br>
If you call ìnput = input.log_softmax(2)`, you’ll get a positive loss value.</p>

username: HuynhSang
cooked: <p>I got it. Thank for your help</p>
Getting CUDA error even though I'm putting tensors on GPU

username: seankala
cooked: <p>Hello. I’m currently trying to run a deep learning model using PyTorch. This is a rather common question, but I’m not able to figure out what the problem is with my code.</p>
<p>Right now, the model block that I’m running looks like this:</p>
<pre><code class="lang-auto">self.linear = nn.Sequential(nn.Linear(in_features=2048, out_features=512),
                            nn.ReLU(),
                            nn.Linear(in_features=512, out_features=256),
                            nn.ReLU(),
                            nn.Linear(in_features=256, out_features=128),
                            nn.ReLU(),
                            nn.Linear(in_features=128, out_features=64))
</code></pre>
<p>And I have a tensor that’s of shape <code>(10248, 2048)</code>.</p>
<p>When I initially ran the program, I got a</p>
<pre><code class="lang-auto">*** RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_addmm
</code></pre>
<p>I initially thought no big deal, I must have forgotten to put the tensors on GPU. But I noticed I had in fact done that with the <code>tensor = tensor.to('cuda')</code> code. Since this didn’t work I tried changing it to <code>tensor = tensor.cuda()</code> and that still didn’t work.</p>
<p>I tried placing a <code>pdb.set_trace()</code> line to debug it, and after checking the tensor I noticed that it indeed is on GPU. That is, when I print it out, it looks like:</p>
<pre><code class="lang-auto">tensor([[0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.]], device='cuda:0')
</code></pre>
<p>What might I have overlooked in this sitaution? Honestly, I’m not sure how to properly interpret the error message either as I can’t find what <code>_th_addmm</code> means. Any tips are appreciated. Thanks in advance.</p>

username: Eta_C
cooked: <p>Did you forget to put the model on GPU?<img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>

username: seankala
cooked: <p>Hello. I wish that were the case, but I also put the model on the GPU with the same method.</p>

username: ptrblck
cooked: <p>Could you post your code here so that we can have a look?<br>
Based on your description, it should work, and we cannot debug any further without seeing the code.</p>
It is fine to eval net with BN layers with batchsize > 1?

username: Naruto-Sasuke
cooked: <p>It is fine to eval net with BN layers with batchsize &gt; 1?</p>

username: ptrblck
cooked: <p>Yes. During evaluation (after <code>model.eval()</code> was called), the running estimates for the mean and variance will be used for every sample, so the batch size doesn’t matter.</p>
Some questions regarding autoregressive model

username: vainaijr
cooked: <ol>
<li>
<p>Does autoregressive mean that the output of our model is combined with current input and this combination acts as input to our model for next time step?</p>
</li>
<li>
<p>If this is not the case, then what is autoregressive model, if this is the case, then what is done to take care, that one wrong prediction does not destroy all the future time step predictions, as if one wrong prediction is made, then it would get combined with input and act as input for next time step, which would lead to further incorrect predictions.</p>
</li>
<li>
<p>What is a way to predict variable length sequence, and combine this variable length sequence with input sequence to again make variable length sequence predictions, let us say for pixels?</p>
</li>
</ol>

username: ptrblck
cooked: <ol>
<li>
<p>An autoregressive model uses previous output(s) as the new input. The <a href="https://en.wikipedia.org/wiki/Autoregressive_model">Wikipedia article</a> gives you some more information.</p>
</li>
<li>
<p>From the same article:</p>
</li>
</ol>
<blockquote>
<p>Intertemporal effect of shocks<br>
In an AR process, a one-time shock affects values of the evolving variable infinitely far into the future. […] Continuing this process shows that the effect of <img src="https://wikimedia.org/api/rest_v1/media/math/render/svg/9e900f9bee793f99d10877ef108da074cbca60ce" alt="arepsilon _{1}" width="2" height="2"> never ends, although if the process is <a href="https://en.wikipedia.org/wiki/Stationary_process">stationary</a> then the effect diminishes toward zero in the limit.</p>
</blockquote>
<p>(I removed a large portion of the text due to the format.)</p>
<p>In DL setups teacher forcing might be used during training.<br>
However, I’m not sure if there is any technique applicable during inference.</p>
<ol start="3">
<li>I’m not familiar a lot with autoregressive models in the DL domain. The “classical” AR models use a single output as the (scaled) input to the model, if I’m not mistaken (my work in the signal processing domain dates back a little further <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> ).</li>
</ol>
Is there a converter for keras to pytorch?

username: set92
cooked: <p>I’m trying to learn pytorch and seeing codes of others helps me to see how to do things, and i was trying to do the kaggle competition of quora questions because it was a competition I started but never finished it, but I can’t find much in pytorch, I found some codes in github but there are difficult to understand</p>
<p>And I found <a href="https://github.com/shagunsodhani/PyDataConf2017/blob/master/notebook/SiameseNetwork.ipynb" rel="nofollow noopener">this one in Keras</a>, and I think it has good explanations but I wanted to see the code in pytorch to learn how to the models, because in the <a href="https://github.com/galsang/BIMPM-pytorch/blob/master/model/BIMPM.py" rel="nofollow noopener">harder examples</a> they use a lot of layers, methods, and I can’t understand it.</p>

username: shivsd
cooked: <p>Did you find any converter from keras to pytorch ?</p>

username: amit_js
cooked: <p>What exactly you want to convert? pretrained model file?</p>

username: Eta_C
cooked: <p>Maybe <a href="https://github.com/microsoft/MMdnn" rel="nofollow noopener">MMdnn</a>? I am not sure.</p>
MaskGAN implementation in PyTorch

username: bhushans23
cooked: <p>Do we have maskGAN implementation in PyTorch?<br>
MaskGAN is used for language generation using filling in the blanks approach.</p>

username: Rahul_Raj
cooked: <aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="32" height="32">
      <a href="https://github.com/jerinphilip/MaskGAN.pytorch" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars3.githubusercontent.com/u/727292?s=400&amp;v=4" class="thumbnail onebox-avatar" width="400" height="400">

<h3><a href="https://github.com/jerinphilip/MaskGAN.pytorch" target="_blank" rel="nofollow noopener">jerinphilip/MaskGAN.pytorch</a></h3>

<p>Pytorch implementation of MaskGAN. Contribute to jerinphilip/MaskGAN.pytorch development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
Using libtorch and pcl causing segmentation fault

username: harishkool
cooked: <p>I am using libtorch and pcl for a c++ project. Below is the code I am trying, I am getting segmentation fault.</p>
<p>Versions:<br>
Libtorch :  1.3.0<br>
PCL      : 1.8</p>
<blockquote>
<p><span class="hashtag-raw">#include</span> &lt;pcl/point_cloud.h&gt;<br>
<span class="hashtag-raw">#include</span> &lt;pcl/io/pcd_io.h&gt;<br>
<span class="hashtag-raw">#include</span> &lt;pcl/point_types.h&gt;<br>
<span class="hashtag-raw">#include</span> &lt;pcl/common/common.h&gt;<br>
<span class="hashtag-raw">#include</span> &lt;torch/torch.h&gt;</p>
<p>void load_pcd(){</p>
<pre><code>pcl::PointCloud&lt;pcl::PointXYZ&gt;::Ptr cloud(new pcl::PointCloud&lt;pcl::PointXYZ&gt;); 
if (pcl::io::loadPCDFile&lt;pcl::PointXYZ&gt;("pcd_path",*cloud)== -1){
    PCL_ERROR ("Couldn't read file test_pcd.pcd \n");
    return ;
}
else{
    std::cout&lt;&lt;"data loaded"&lt;&lt;std::endl;
}
</code></pre>
<p>}<br>
int main(){<br>
load_pcd();<br>
return 0;<br>
}</p>
</blockquote>
<p>Cmake file:</p>
<blockquote>
<p>cmake_minimum_required(VERSION 3.0 FATAL_ERROR)<br>
project(load)<br>
set(CMAKE_CXX_STANDARD 11)<br>
find_package(PCL 1.2 REQUIRED)<br>
find_package(Torch REQUIRED)<br>
include_directories(${PCL_INCLUDE_DIRS})<br>
link_directories(${PCL_LIBRARY_DIRS})<br>
add_definitions(${PCL_DEFINITIONS})<br>
add_executable (load get_pcd_lst.cpp)<br>
target_link_libraries (load ${PCL_LIBRARIES})<br>
target_link_libraries(load ${TORCH_LIBRARIES})</p>
</blockquote>
Optimize CUDA kernel

username: billiout
cooked: <p>I am new to CUDA programming and I am trying to create a custom CUDA kernel for an operation I am working on but I can’t figure how to optimize my code. Below is a simple example of my own ceiling function and it seems to be much slower than the internal function implemented in Pytorch. I have noticed that the latency comes when I write the output_value to the output tensor inside the kernel. Running Pytorch ceiling operation for 100,000 iterations on a tensor with shape [100, 50, 100] takes 7.22s where mine takes 9.75s.</p>
<blockquote>
<p><span class="hashtag-raw">#include</span> &lt;torch/extension.h&gt;</p>
<p><span class="hashtag-raw">#include</span> &lt;ATen/cuda/CUDAContext.h&gt;<br>
<span class="hashtag-raw">#include</span> &lt;ATen/cuda/CUDAApplyUtils.cuh&gt;</p>
<p><span class="hashtag-raw">#include</span> &lt;THC/THC.h&gt;<br>
<span class="hashtag-raw">#include</span> “THC/THCDeviceUtils.cuh”</p>
<p><span class="hashtag-raw">#include</span> &lt;cuda.h&gt;<br>
<span class="hashtag-raw">#include</span> &lt;cuda_runtime.h&gt;</p>
<p><span class="hashtag-raw">#include</span> &lt;assert.h&gt;</p>
<p><span class="hashtag-raw">#include</span> &lt;cmath&gt;</p>
<p>/**<br>
input: [T, B, R]<br>
output: [T, B, R]<br>
**/<br>
template &lt;typename scalar_t&gt;<br>
__global__ void MethodKernel(<br>
const at::PackedTensorAccessor32&lt;scalar_t, 3, torch::RestrictPtrTraits&gt; input,<br>
scalar_t* output){</p>
<pre><code>const int length = input.size(0);
const int batchSize = input.size(1);
const int r_dim = input.size(2);

const int index = blockDim.x * blockIdx.x + threadIdx.x;
const int rIdx = index % r_dim;
const int batchIdx = (index / r_dim) % batchSize;
const int tokenIdx = ((index / r_dim) / batchSize);

if (batchIdx &lt; batchSize and tokenIdx &lt; length and rIdx &lt; r_dim) {
    scalar_t output_value = std::ceil(input[tokenIdx][batchIdx][rIdx]);
   
    output[(tokenIdx * batchSize + batchIdx) * r_dim + rIdx] = output_value;
}
</code></pre>
<p>}</p>
<p>at::Tensor method_encoder_forward(at::Tensor input) {</p>
<pre><code>const int length = input.size(0);
const int batchSize = input.size(1);
const int r_dim = input.size(2);

const dim3 blockSize(1024);
const dim3 gridSize(
    (length*batchSize*r_dim + blockSize.x - 1) / blockSize.x);

auto output = at::zeros_like(input);


AT_DISPATCH_FLOATING_TYPES_AND_HALF(output.type(), "method_encoder_forward", ([&amp;] {
    
    auto inputAcsr = input.packed_accessor32&lt;scalar_t, 3, torch::RestrictPtrTraits&gt;();
    
    MethodKernel &lt;&lt;&lt;gridSize, blockSize, 0, at::cuda::getCurrentCUDAStream()&gt;&gt;&gt; (
            inputAcsr, output.data&lt;scalar_t&gt;());

}));

AT_CUDA_CHECK(cudaGetLastError());

return output;
</code></pre>
<p>}</p>
<p>PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {<br>
m.def(“method_encoder_forward”, &amp;method_encoder_forward, “Method Encoder Forward”);<br>
}</p>
</blockquote>
Size mismatch in torch.autograd.backward()

username: 11130
cooked: <p>Hello, everyone! I want to calculate per-example gradients. But when I run the code snippet (linear regression to 0) below, I got RuntimeError: invalid gradient at index 0 - got [10] but expected shape compatible with [100].</p>
<pre><code class="lang-auto">n_sample = 100
n_feature = 10
A=torch.randn(n_sample, n_feature)
x=torch.randn(n_feature)
grad_tensors = [x] * n_sample
b=torch.matmul(A,x)
losses = b ** 2
losses.requires_grad = True

print(losses.size())
torch.autograd.backward(tensors=(losses, ), grad_tensors=grad_tensors)
print(grad_tensors)

</code></pre>
<p>losses.size()=torch.Size([100]), i.e, the number of samples. But why does it expect shape compatible with [100].</p>
Should I use cuda tensors for indexing if the data is on cuda?

username: czxttkl
cooked: <p>Suppose an nn.module model has some indexing work happen in forward():</p>
<pre><code class="lang-auto">def forward(self, data):
    # some indexing works, e.g.,:
    index = torch.randint(4, (4,))  # should I add index = index.cuda()?
    self.linear(data[:, index])
    ...
</code></pre>
<p>If this model is on GPU, for the best performance, should I also move <code>index</code> to cuda? Or it doesn’t matter? Why?</p>

username: albanD
cooked: <p>If <code>data</code> is on the GPU, you <strong>have to</strong> move the indices to the GPU. Otherwise, it will raise an error.</p>

username: czxttkl
cooked: <p>As an example shown below, I don’t move index to the GPU but the code still runs. My question is: Will adding <code>index = index.cuda()</code> help improve performance? and why?</p>
<pre><code class="lang-auto">class MyModule(nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()
        self.linear = nn.Linear(1000, 200)
        
    def forward(self, data):
        index = torch.randint(1000, (1000,))  # should I add index = index.cuda()?
        return self.linear(data[:, index])
    
a = MyModule().cuda()
b = torch.randn(1000, 1000).cuda()
a(b)
</code></pre>

username: albanD
cooked: <p>Ho,</p>
<p>I guess in this case, it considers the Tensor as just a sequence of numbers.<br>
Moving the indices to CUDA will definitely be faster.</p>

username: czxttkl
cooked: <p>Thanks. You are right.</p>
Is it possible to plugin extra logic when serving a serialized model

username: sleepy
cooked: <p>Hi,</p>
<p>I am just wondering: it is possible to plugin extra logic between layers of the neural network model (serialized) during serving?</p>
<p>Any help or idea would be greatly appreciated.</p>
<p>Best,</p>

username: ptrblck
cooked: <p>Could you explain your use case a bit more?<br>
I.e. what does serving mean in the context and what would you like to add?<br>
Are you referring to some deployment software / platform or just a plain PyTorch model, you would like to change?</p>

username: sleepy
cooked: <p>Thanks for the reply.</p>
<p>Yeah, sure. I am referring to the prediction process once the serialized model (say, a DNN model) is loaded.</p>
<p>I am not sure whether pytorch serving supports extra logic injection between each of the neural network layer (during the prediction process). I am suspecting that JIT does not provide such API. Does it require fundamental changes to the source code to enable such a feature?</p>
<p>Best,</p>
How to modify the internal calculation process of LSTM in pytorch-v1.1.0?

username: zwd2016
cooked: <p>I want to modify the calculation process inside the LSTM. However, when I queried the _VF.lstm() method, no corresponding python implementation was found. Then I found the C++ implementation at this address (i.e., <a href="https://github.com/pytorch/pytorch/tree/master/aten/src/ATen/native/RNN.cpp" rel="nofollow noopener">https://github.com/pytorch/pytorch/tree/master/aten/src/ATen/native/RNN.cpp</a>) on GitHub. My question is which files need to be modified under the local PyTorch directory.</p>
Standard benchmarks/good datasets for transfer learning

username: James_Howard
cooked: <p>Hi,</p>
<p>I wish to investigate various methods which can assist with transfer learning. Surprisingly, I cannot see any de facto standard datasets for assessing transfer learning performance, like we have with ImageNet for training image classifiers from scratch.</p>
<p>My hope was to use networks pre-trained on ImageNet.</p>
<p>I’ve found a few interesting datasets - Stanford dogs seems an excellent candidate because obviously dogs are well represented in ImageNet, but I was thinking of something more general?</p>
<p>Many thanks</p>
In pytorch, view(), permute(), contiguous() and other functions do operation in-place or allocate new memory block?

username: PiggySensen
cooked: <p>I really want to know that in Pytorch, functions such like view(), permute(), contiguous() operate the Tensor in-place or they will allocate new memory block to store the result.<br>
Recently, I encounter that my data needs lot of memory, so I need to reduce memory usage in the forward process, and knowing the answer of the question above is of great help.<br>
Thanks a lot</p>

username: albanD
cooked: <p>Hi,</p>
<p>It should be mentioned in the doc.</p>
<ul>
<li>view never allocates memory (but only works on contiguous Tensor)</li>
<li>permute never allocates memory (but returns a non-contiguous Tensor)</li>
<li>contiguous allocates only if the input is non-contiguous. Otherwise it is a no-op.</li>
</ul>

username: PiggySensen
cooked: <p>Thanks, you help a lot</p>
Unable to import anything from torch._C

username: F-Schlatt
cooked: <p>I’m getting the most peculiar error since today. Somehow something broke with my pytorch installation on a linux server. When I try to import anything from <code>torch._C</code> (and consequently <code>import torch</code> doesn’t work at all), it stalls and never continues. I’m unable to interupt the import and need to completely kill the process. I’ve completely reinstalled anaconda but to no avail. Does anyone have any idea?</p>

username: ptrblck
cooked: <p>Could you check for multiple installations and delete all of them (using <code>pip</code> and <code>conda</code>)? Rerun the uninstall commands at least twice.<br>
Are you trying to import PyTorch from the build folder accidentally?</p>

username: colesbury
cooked: <p>Also try getting a backtrace from GDB (<a href="https://discuss.pytorch.org/t/pytorch-cpu-hangs-on-nn-linear/17748/2">instructions here</a>) and share it here.</p>

username: F-Schlatt
cooked: <p>Thanks for all the suggestions, but the problem resolved itself somehow overnight and everything is working fine again. Since it’s a shared university server maybe someone else had a process open that was interfering somehow?! I have no idea. In case it ever shows up again I’ll continue this thread.</p>
Freeze some parts of the resnet model

username: oasjd7
cooked: <p>This is a part of the Resnet block.</p>
<pre><code class="lang-auto">DataParallel(
  (module): DataParallel(
    (module): Sequential(
      (0): Sequential(
        (0): BasicBlock(
          (downsample): Downsample(
            (avg): AvgPool2d(kernel_size=2, stride=2, padding=0)
          )
          (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu1): ReLU(inplace)
          (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu2): ReLU(inplace)
          (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (1): BasicBlock(
          (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu1): ReLU(inplace)
          (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu2): ReLU(inplace)
          (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (2): BasicBlock(
          (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu1): ReLU(inplace)
          (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu2): ReLU(inplace)
          (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (3): BasicBlock(
          (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu1): ReLU(inplace)
          (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu2): ReLU(inplace)
          (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
      )
      (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace)
    )
  )
)

</code></pre>
<p>I want to freeze first BasicBlock (0) and (1)<br>
then, Is this way works fine?</p>
<pre><code class="lang-auto">for param in resnet.module.module[0][0].parameters():
        param.requires_grad = False
for param in resnet.module.module[0][1].parameters():
        param.requires_grad = False
</code></pre>
Embedding weight is not being updated on training

username: rajkuma2011
cooked: <p>Below is my model code and expecting to have embedding weight updated as training is progressing but it is not being updated. Only embedding which is being updated is of index = 0 which is ‘unk’ word. Am I missing anything in the model ? Could it be possible someone to help on this ? I am struggling for last few days.</p>
<pre><code class="lang-python">class FeedForwardModel(nn.Module):
    def __init__(self, embedding_matrix, embed_dim):
        super().__init__()
        self.embedding = nn.EmbeddingBag.from_pretrained(torch.FloatTensor(embedding_matrix))
        self.fc1 = nn.Linear(embed_dim, 10)
        self.fc2 = nn.Linear(10, 1)
        self.output = nn.Sigmoid()
        self.init_weights()
        
    def __init__(self, num_embeddings, embed_dim):
        super().__init__()
        self.embedding = nn.EmbeddingBag(num_embeddings, embedding_dim)
        self.fc1 = nn.Linear(embed_dim, 10)
        self.fc2 = nn.Linear(10, 1)
        self.output = nn.Sigmoid()
        self.embedding.weight.data.uniform_(-0.5, 0.5)
        self.embedding.weight.requires_grad = True
        self.init_weights()
                  
        
    def forward(self, input, offsets):
        #print(input)
        #print(self.embedding.weight[1:3])
        embedded = self.embedding(input, offsets)
        h1 = F.tanh(self.fc1(embedded))
        h2 = self.fc2(h1)
        return self.output(h2)
    
    def init_weights(self):
        initrange = 0.5
        self.fc1.weight.data.uniform_(-initrange, initrange)
        self.fc2.weight.data.uniform_(-initrange, initrange)
        self.fc1.bias.data.zero_()
        self.fc2.bias.data.zero_()
</code></pre>

username: ptrblck
cooked: <p>Could you check your <code>input</code> for all unique values?<br>
If you are passing all valid embedding indices, you’ll get valid gradients in the complete <code>weight</code> parameter of <code>self.embedding</code>:</p>
<pre><code class="lang-python">
num_embeddings = 10
embedding_dim = 100
model = FeedForwardModel(num_embeddings, embedding_dim)

x = torch.arange(num_embeddings)
offsets = torch.tensor([0, 5])

output = model(x, offsets)
output.mean().backward()

print(model.embedding.weight.grad)
</code></pre>
<p>However, if you pass e.g. only zeros as <code>input</code>, only <code>weight.grad[0]</code> will be filled with non-zero values.</p>
Dataloader Error

username: Sven_Sven
cooked: <p>I am using the dataloader class to load files and then sending them to a GPU (Nvidia K80). The Intel processor has the Haswell architecture.</p>
<pre><code>image_loader = torch.utils.data.DataLoader(dataset, batch_size=4, shuffle=True, num_workers=4)
for x in enumerate(image_loader, 0):
    x = x.to(device)
</code></pre>
<p>However, this gives the error:</p>
<p><code>RuntimeError: could not unlink the shared memory file /torch_18607_1504223053</code></p>
<p>What could the issue be? It does not appear that any files (paths to images) are duplicated in my dataset, which would cause the dataloader to load the same image simultaneously in two parallel processes.</p>

username: ptrblck
cooked: <p>Is your code running fine with <code>num_workers=0</code>?<br>
The error message points to a failing exit call, so it might be a red herring and your code might crash before the unlike error is raised.</p>

username: Sven_Sven
cooked: <p>I have not observed the error with <code>num_workers=0</code>. However, the error doesn’t always appear when running with <code>num_workers=4</code> either. My guess is it is due to the dataloader shuffling the objects being loaded and one specific combination of objects not working together. Is this a possibility?</p>
<p>Also, are you referring to an exit call by the dataloader workers? And what is the unlike error?</p>

username: ptrblck
cooked: <p>Are you processing multiple samples inside your <code>Dataset</code>'s <code>__getitem__</code> method?<br>
If you are loading and processing single samples, I don’t think the error might be related to the shuffling, but that’s of course just a guess.</p>
<p>“unlike” was the autocorrected version of unlink. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
Result different between mobile and PC

username: Ten_Canmotcai
cooked: <p>I convert fastai to torchscript then run same converted model on both Android and PC(Macbook).<br>
Result is different.</p>
<p>Code convert:</p>
<pre><code class="lang-auto">import torch
example = torch.rand(1, 3, 64, 64)
traced_script_module = torch.jit.trace(torch_model, example)
traced_script_module.save("model.pt")
</code></pre>
<p>Run on PC:</p>
<pre><code class="lang-auto">img = cv2.imread('aaanx_7.jpg')
img = img/255.0
img = np.moveaxis(img, -1, 0)
img = np.array(np.expand_dims(img, axis=0))
traced_script_module.forward(torch.from_numpy(img).float())
</code></pre>
<p>Run on Android:</p>
<pre><code class="lang-auto">bitmap = BitmapFactory.decodeStream(getAssets().open("aaanx_7.jpg"));
Tensor inputTensor = TensorImageUtils.bitmapToFloat32Tensor(bitmap,
              new float[]{0.0f, 0.0f, 0.0f}, new float[]{1.0f, 1.0f, 1.0f});
Tensor outputTensor = module.forward(IValue.from(inputTensor)).toTensor();
</code></pre>
<p>Tensor output is different.<br>
Please help me understand that.<br>
Thanks</p>

username: eellison
cooked: <p>Is the output significantly different or does it just have small numerical differences? If it is significantly different please open an issue.</p>

username: Ten_Canmotcai
cooked: <p>It is significantly different</p>
<p>I train resnet50 to predict extended mnist and result is different number with all images</p>

username: Ten_Canmotcai
cooked: <p>I checking with input image and confuse with TensorImageUtils.bitmapToFloat32Tensor.<br>
It is little-endian and it seem to be not same with PC.<br>
I am not sure. Do you have any suggestion?</p>
<p>Thanks</p>
Best way to convolve on different channels with a single kernel?

username: TheShadow29
cooked: <p>Say I have a 2D signal which is basically 4 channels of 1d signals and each 1d signal is of shape 100. So my input is of shape 100x4. Now I have a single kernel, <code>torch.nn.Conv1d()</code> and I want to apply the same kernel to each of the channels individually. What is the most efficient way to do this?</p>
<p>The method I have come up is to use list, but I feel there should be more elegant way to do the same. My code is</p>
<pre><code class="lang-auto">out_list = []
for i in channels:
    # inp is of shape batch_size x channels (4) x vector len (100)
   out = conv(inp[:, [i], :])
    out_list.append(out)
o1 = torch.cat(out_list,0)
o1 = o1.view(num_channels, batch_size, vector_len)
o1 = o1.permute(1,0,2).contiguous()
return o1
</code></pre>
<p>Would be glad if there is a better implementation for the same (or if there is any bug in the presented code). Thank you</p>

username: ptrblck
cooked: <p>You could repeat the single kernel and use the functional API:</p>
<pre><code class="lang-auto">x = Variable(torch.randn(1, in_channels, 100))

# Gaussian kernel
kernel = Variable(torch.FloatTensor([[[0.06136, 0.24477, 0.38774, 0.24477, 0.06136]]]))
weights = kernel.repeat(1, in_channels, 1)
output = F.conv1d(x, weights)
</code></pre>
<p>Would this work for your use case?</p>

username: TheShadow29
cooked: <p>Will this also work if I want to learn the kernels as well? That is during the backward() method and updating weights with the optimizer, will this ensure that the kernels weights get updated by the average of the gradients in all the channels?</p>

username: ptrblck
cooked: <p>You can initialize it with the same weights for each input channel. The vanilla weight updates will change the weights without the constraint to have the same values.</p>
<p>If you need to backpropagate, the better method would be to change the view of the input channels, so that the channels are stored in the batch dimension, and you only use a single kernel.</p>
<p>This should work:</p>
<pre><code class="lang-auto">x = Variable(torch.randn(batch_size, in_channels, 100))

# Gaussian kernel
kernel = Variable(torch.FloatTensor([[[0.06136, 0.24477, 0.38774, 0.24477, 0.06136]]]))
output = F.conv1d(x.view(-1, 1, 100), kernel).view(batch_size, in_channels, 96)
</code></pre>

username: TheShadow29
cooked: <p>Sweet. This should. work.</p>
Runtime memory breakdown (activations, weights, etc.)

username: taltab
cooked: <p>Hello,<br>
I am looking for a way to link cuda memory allocations (which are done by the CUDACachingAllocator, at which point only the requested size is known) to the accompanying usage of memory. Ultimately this would help me get a breakdown of where the GPU memory is being consumed (ie. weights, activations, etc.). I have been looking for relevant metadata in c10/core/TensorImpl.h, c10/core/StorageImpl.h, and other places so that hopefully I could modify the source code and pass this information down to the allocation directly. Is there any way for me to find descriptions of why a certain amount of memory was allocated, or is there another way for me to accomplish my task?</p>
<p>Another place that I looked was in the nn.Module named_parameters(), but this did not give me all the relevant info, for example there is no information about what data structures the activations are using.<br>
Thank you.</p>
What is the correct way to measure the total execution time for a pytorch function running on GPU?

username: ahsabali
cooked: <p>Following is an example code showing what I am trying to measure. Here I am using time.perf_counter() to measure time. Is this the correct way to measure execution time in this scenario? If not, what is the correct way? My concern is, GPU evaluations are asynchronous and GPU execution might not be completed when ExecTime is measured below.</p>
<pre><code class="lang-auto">import torch
import torch.nn.functional as F
import time

Device = torch.device("cuda:0")
ProblemSize = 100
NumChannels = 5
NumFilters = 96
ClassType = torch.float32

X = torch.rand(1, NumChannels, ProblemSize, ProblemSize, dtype=ClassType).to(Device)
weights = torch.rand(NumFilters, NumChannels, 10, 10, dtype=ClassType).to(Device)
    
#warm up
Y = F.conv2d(X, weights)
Y = F.conv2d(X, weights)

#time
t = time.perf_counter()
Y = F.conv2d(X, weights)
ExecTime = time.perf_counter() - t
</code></pre>

username: ptrblck
cooked: <p>As you assume, you would have to synchronize the code before starting and stopping the timer via <code>torch.cuda.synchronize()</code>.</p>
<p>I would also recommend to add some warm-up iterations as well as time your operations in a loop and take the average execution time.</p>