akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Libtorch model inference

sanketgujar: <p>Hi, I am trying to load do inference in libtorch, but it gives me Segmentation fault (core dumped) after I pass the model predicts the outputs. I am using the example code and I am not sure why is it happening?</p>
<p>you can find the here: <a href="https://gist.github.com/sanketgujar/baf27851fb1b849ced78cf0668411dd0" rel="nofollow noopener">https://gist.github.com/sanketgujar/baf27851fb1b849ced78cf0668411dd0</a></p>

yf225: <p>Could you also provide your model definition code and how you export your model?</p>

sanketgujar: <p>Model Defination code : <a href="https://gist.github.com/81dff17eb270594637b5b356591bf8c1.git" rel="nofollow noopener">https://gist.github.com/81dff17eb270594637b5b356591bf8c1.git</a></p>
<p>I converted the model to Torch Script via tracing as shown in the tutorial<br>
<a href="https://pytorch.org/tutorials/advanced/cpp_export.html#step-4-executing-the-script-module-in-c" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/cpp_export.html#step-4-executing-the-script-module-in-c</a></p>

sanketgujar: <p>The debugger output:</p>
<p><em>Thread 1 “example-app” received signal SIGSEGV, Segmentation fault.</em><br>
<em>0x00007ffff608d9fe in ?? () from /usr/local/cuda-10.0/lib64/libcudart.so.10.0</em></p>
<p>I am still not sure what is the problem here. The model is doping the prediction but giving seg fault after.</p>
Shall I apply transforms.Normalize several times when using two model sequentially?

gcosne: <p>Dear PyTorch fellows,</p>
<p>To give a little bit of context, I’m working with Generative Models (Generative Adversarial Networks- GAN) achieving image to image translation. In my case input and output are images.</p>
<p>To help the model converge, I’m using a semantic segmentation model on the input and the output of the GAN and compute a cross Entropy loss on specific region so that the semantic of the image is kept during the translation.</p>
<p>My question is the following: since my GAN requires a different normalization from my semantic segmentation model:</p>
<pre><code class="lang-auto">transform_GAN = transforms.Compose([
                            transforms.RandomHorizontalFlip(),
                            transforms.Resize(256),
                            transforms.RandomCrop(256),
                            transforms.ToTensor(),
                            transforms.Normalize((0.5, 0.5, 0.5),
                                                 (0.5, 0.5, 0.5))
                            ])

    transform_seg = transforms.Compose(
            [
                 transforms.ToPILImage(),
                 transforms.Resize((512,512)),
                 transforms.ToTensor(),
                 transforms.Normalize((0.485, 0.456, 0.406),\
                                      (0.229, 0.224, 0.225))
            ])
</code></pre>
<p>Shall I denormalize the output of the first one before applying the transformation of the second ?</p>
<p>Thank you for your help,</p>
<p>Gautier</p>

Nikronic: <p>Hi,</p>
<p>I have not tried 2 different models, but in the similar context, I tried to normalize my images twice sequentially and then unnormalize them exactly in the reverse order but it did not give me the result I was looking for, So I decided to use <code>normalization-&gt;unnormalization</code> twice.<br>
But I think you do not need to convert back tensors to PIL images then do some preprocessing again and convert them to tensors. You can do those process without converting to PIL even if have not a class like <code>Resize</code> for tensors, because it would be easy to implement.</p>

gcosne: <aside class="quote no-group" data-username="Nikronic" data-post="2" data-topic="52063">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/71873_2.png" class="avatar"> Nikronic:</div>
<blockquote>
<p>do not need to convert back tensors to PIL images then do some preprocessing again and convert them to tensors. You can do those process without converting t</p>
</blockquote>
</aside>
<p>Update: thank you Nikronic for your answer, I went with the un-normalization and it leads to the good expected results.</p>
<p>Concerning the resize, I used <a href="https://pytorch.org/docs/stable/_modules/torch/nn/functional.html" rel="noopener nofollow ugc">torch.nn.functional.interpolate</a></p>
When I use APEX to train mixed precision modes

luodahei: <p>I want to use GN/IN in my model.But APEX only keep BN with FP32.So what should i do when use GN/IN with FP32 in APEX？Thanks very much。</p>

ptrblck: <p><code>F.instance_norm</code> uses <code>batch_norm</code> internally (<a href="https://github.com/pytorch/pytorch/blob/cfe94009966e815ca3fd9742ef577d4627b8b989/aten/src/ATen/native/Normalization.cpp#L451" rel="nofollow noopener">line of code</a>).<br>
<code>nn.GroupNorm</code> is defined as an FP32 method <a href="https://github.com/NVIDIA/apex/blob/574fe2449cbe6ae4c8af53c6ecb1b5fc13877234/apex/amp/lists/functional_overrides.py#L41" rel="nofollow noopener">here</a>.</p>
BatchNorm2d behaviour in video inputs (5d)

IliasPap: <p>When i use a 2d cnn for feature extraction of the video in a CNN-RNN architecture to pass the 5d tensor through the cnn it is reshaped as (batch_size<em>number_of_frames , C , H ,W ) 4d tensor so as batch_size = (batch_size</em>number_of_frames )<br>
Does BatchNorm2d affect performance or should I pass every frame at a time from the cnn ?</p>
PyTorch dimension conventions?

ahmetumutdurmus: <p>Hello,</p>
<p>Sorry if this is a duplicate question yet I couldn’t find a similar question.</p>
<p>Is there a reason why PyTorch uses [N, X] dimension format? That is the prebuilt layers take inputs with dimensionality of [BatchSize, NumberOfInputFeatures] rather than [NumberOfInputFeatures, BatchSize]. The Broadcasting semantics are also arranged accordingly.</p>
<p>Was this an arbitrary design choice or was it intentional because of some efficiency purpose? I am asking this as this is against the dimension conventions generally embraced in papers etc…</p>
<p>Thanks</p>

Nikronic: <p>Hi,</p>
<p>I have enrolled different courses and worked with some of the popular frameworks and it seems it is all about conventions. And the most used one is the method pytorch uses.</p>
<p>By the way, Andrew Ng uses same convetion has been used in PyTorch in computer vision and deep learning courses he instruct in Stanford university.</p>
<p>Good luck</p>
DataParallel backend problem

Aviv_Shamsian: <p>Hi All,<br>
How are you?<br>
I have recently installed two new GPUs and wanted to test them so I tried to run the cifar10 tutorial notebook and the following error came up:</p>
<pre><code class="lang-auto">RuntimeError: Expected object of backend CUDA but got backend CPU for argument #2 'target'
</code></pre>
<p>I thought the problem would be the assignment of the labels and inputs to the GPUs but I have done it (see the attached images below) I also checked that both of the GPUs are detectable both with <code>nvidia-smi</code> and <code>torch.cuda.device_count()</code></p>
<p>Maybe the problem is elsewhere? with the torch.nn.DataParallel?</p>
<p>Thank you very much for your help.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/8/8414f9bde40a9fff2bed3bebe8796aa72b20f588.png" data-download-href="https://discuss.pytorch.org/uploads/default/8414f9bde40a9fff2bed3bebe8796aa72b20f588" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/8/8414f9bde40a9fff2bed3bebe8796aa72b20f588_2_470x500.png" alt="image" data-base62-sha1="iQrXTmDYIJFlb9WueLazeSRkQIw" width="470" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/8/8414f9bde40a9fff2bed3bebe8796aa72b20f588_2_470x500.png, https://discuss.pytorch.org/uploads/default/original/2X/8/8414f9bde40a9fff2bed3bebe8796aa72b20f588.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/8/8414f9bde40a9fff2bed3bebe8796aa72b20f588.png 2x" data-dominant-color="F2F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">697×740 41.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Swarchal: <p>Try changing</p>
<pre><code class="lang-python">inputs.to(device)
labels.to(device)
</code></pre>
<p>to</p>
<pre><code class="lang-python">inputs = inputs.to(device)
labels = labels.to(device)
</code></pre>

Aviv_Shamsian: <p>thank you it’s actually works!</p>
Difference between PIL resize and OpenCV resize

LvJC: <p>Hi,</p>
<p>I want to mimic behavior of PIL.Image.resize(…,interpolation=PIL.Image.ANTIALIAS). What I would like to expect, is to have matching pixel vales (~99.9% exact match should be fine) . How can I do this with OpenCV ? I tried Opencv resize with available interpolation methods (including LANCZOS4) , but result does differ from the one I got from PIL.</p>
<p>Why I’m doing this? Well I 'm developing machine learning program in C that is using (infer) pre-trained model (done in Python+PIL with PIL.Image.ANTIALIAS). Even small differences in preprocessing of images has an impact on how machine learning model behave.</p>
<p>So either I find a way to achieve same resize behavior in OpenCV as used in PIL.Image.ANTIALIAS or how to use PIL in C.</p>
<p>So how can I get over this difference?</p>
<p>Thanks in advance for answering my question.</p>
<p>Please advice</p>

Amrit_Das: <p>In any case, you can easily convert PIL format to OpenCv format. Here’s how<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/amrit-das/helper_scripts/blob/master/cv-pil.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/amrit-das/helper_scripts/blob/master/cv-pil.py" target="_blank" rel="nofollow noopener">amrit-das/helper_scripts/blob/master/cv-pil.py</a></h4>
<pre><code class="lang-py">import cv2
from PIL import Image
import numpy as np
import sys

# PIL to openCV conversion
im = Image.open("/home/amrit/Desktop/SRM/codes/{}".format(sys.argv[1]))
im.show()
cv = cv2.cvtColor(numpy.array(im), cv2.COLOR_RGB2BGR)
cv2.imshow("cv",cv)
cv2.waitKey(0)

# OpenCV to PIL conversion
im = cv2.imread("/home/amrit/Desktop/SRM/codes/{}".format(sys.argv[1]))
cv = cv2.cvtColor(numpy.array(im), cv2.COLOR_BGR2RGB)
im = Image.fromarray(cv)
im.show()



</code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

Beatrice_Paige: <p>One major difference: The output of operations is completely different. In Pillow, ​you set radius, while in cv you set kernel size, which is literally diameter. You should use ksize=(7, 7) to achieve the same result.</p>

LvJC: <p>But <code>Dataloader</code> could only support PIL Image as its input.<br>
What I want to do is that:</p>
<ol>
<li>Use OpenCV to read, and preprocess image like <code>transforms</code>.</li>
<li>Use <code>data.Dataloader</code> to create train dataloader and test dataloader. But it cannot accept data that is resized by OpenCV.</li>
</ol>

Amrit_Das: <p>Read with opencv, convert it to PIL format, process the data and then you can revert it back to cv2 if you want.</p>

Beatrice_Paige: <p>ah neat. we must be on the same team.</p>

LvJC: <p>OK Finally, our team rewrites the whole torchvision to enable it to support opencv readings, including the <code>transform</code> file.<br>
Now it works well.</p>

SudhakarSekar17: <p><a class="mention" href="/u/lvjc">@LvJC</a>: How to check that transform file uses opencv for resizing an image?</p>
How to deploy the pytorch model?

XiaXuehai: <p>I am a novice.<br>
I don’t learn about caffe2,so I use the flask to deploy the model with python.<br>
But I don’t know which is better?Does caffe2 is faster?or if I optimize the inference code and the model,I can get the similar speed?</p>

ptrblck: <p>I think it depends on your use case.<br>
As far as I understand your current PyTorch model runs in a Flask app and works.<br>
Do you see any bottlenecks regarding the performance, i.e. do you expect a huge workload which your current approach cannot handle?</p>
<p>I would look for the current bottlenecks and optimize according to these.</p>

rqyang: <p>Here’s a tutorial for deploying a pytorch model: <a href="https://pytorch.org/tutorials/beginner/deploy_seq2seq_hybrid_frontend_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/deploy_seq2seq_hybrid_frontend_tutorial.html</a>. Hope it helps.</p>

Vania_Todorova: <p>what was your model? i m struggling to figure out how to deploy a linear regression model. do we need to save the weights ?</p>
(C++) Where ist the mnist source code, not header file?

RuFAI: <p>Hi, where can I find the source code that belongs to the mnist.h header file? I am interested in the specific implementation of the data loading procedure but as expected there is just the decleation of the used methods in the header file.</p>
<p>Would be glad to get some information</p>

dongfangduoshou123: <p>大哥在这呢。<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/api/src/data/datasets/mnist.cpp" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/api/src/data/datasets/mnist.cpp" target="_blank" rel="nofollow noopener">pytorch/pytorch/blob/master/torch/csrc/api/src/data/datasets/mnist.cpp</a></h4>
<pre><code class="lang-cpp">#include &lt;torch/data/datasets/mnist.h&gt;

#include &lt;torch/data/example.h&gt;
#include &lt;torch/types.h&gt;

#include &lt;c10/util/Exception.h&gt;

#include &lt;cstddef&gt;
#include &lt;fstream&gt;
#include &lt;string&gt;
#include &lt;vector&gt;

namespace torch {
namespace data {
namespace datasets {
namespace {
constexpr uint32_t kTrainSize = 60000;
constexpr uint32_t kTestSize = 10000;
constexpr uint32_t kImageMagicNumber = 2051;
constexpr uint32_t kTargetMagicNumber = 2049;
</code></pre>

  This file has been truncated. <a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/api/src/data/datasets/mnist.cpp" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
Cuda 10.1 error using transposeconv2d with output_padding 1

AlphabetMan: <p>Might be off topic, but I did not find where to report a bug on nvidia website. I tried training a GAN based on pix2pixhd architecture with Amp ‘O1’ opt_level with cuda 10.1 cudnn 7.6 pytorch nightly build ubuntu 18.04.<br>
Code breaks here:<br>
File “./samplegenerator.py”, line 263, in <br>
scaled_loss.backward()<br>
File “/usr/local/lib/python3.6/dist-packages/torch/tensor.py”, line 118, in backward<br>
torch.autograd.backward(self, gradient, retain_graph, create_graph)<br>
File “/usr/local/lib/python3.6/dist-packages/torch/autograd/<strong>init</strong>.py”, line 93, in backward<br>
allow_unreachable=True)  # allow_unreachable flag<br>
RuntimeError: _cublasOpFromChar input should be ‘t’, ‘n’ or ‘c’ but got `</p>
<p>This kind of code should reproduce the behaviour:</p>
<pre><code class="lang-auto">class Modelis(nn.Module):
   def __init__(self):
       super().__init__()
       self.conv = nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, 
                             stride=2, padding=1)
       self.deconv = nn.ConvTranspose2d(in_channels=256, out_channels=128, 
           kernel_size=3, stride=2, padding=1, output_padding=1)
   def forward(self, x):
       x = self.conv(x)
       x = self.deconv(x)
       return x

Criterion = nn.bcewithlogits()
netG = Modelis()
netG = netG.cuda()
optimizerG = optim.Adam(netG.parameters(), lr=0.001, betas=(0.5, 0.999))
netG, optimizerG = amp.initialize(netG, optimizerG, opt_level='O1') 

for i in range(100):
    batch = (torch.randn(8,128,16,16).cuda() - 0.5) * 2
    output = netG(batch)
    loss = Criterion(output, torch.ones_like(output))

    with amp.scale_loss(loss, optimizerG) as scaled_loss:
        scaled_loss.backward()
</code></pre>
<p>It works in fp32 mode. After experiments have found that backward breaks on nn.transposeconv2d when output_padding is 1 (not 0). Using pytorch docker container (cuda 10.0, cudnn 7, pytorch 1.0) both fp32 and fp16 works fine.</p>

ptrblck: <p>Thanks for reporting this issue. We’ll look into it.</p>

ptrblck: <p>We could reproduce this issue and are tracking it <a href="https://github.com/pytorch/pytorch/issues/23545" rel="nofollow noopener">here</a>.<br>
Thanks <a class="mention" href="/u/ngimel">@ngimel</a> for the support tracking down this issue.</p>
Freeing CUDA memory after forwarding tensors

Valiox: <p>Consider the following loop:</p>
<pre><code class="lang-auto">    for batch in dataloader:
        batch = batch.cuda()
        features = model(batch)
</code></pre>
<p>If forwarding a batch takes up almost the memory in my GPU (let’s say 7gb out of 8), then this loop will fail on the second iteration due to a OOM error.</p>
<p>This version won’t:</p>
<pre><code class="lang-auto">    for batch in dataloader:
        batch = batch.cuda()
        features = model(batch)
        del features
</code></pre>
<p>Even though <code>features</code> is only a very small tensor (a dozen values). What does <code>del</code> exactly do here? Why do I need to manually free the memory to achieve what I want? Is this good practice? If no, what is an alternative that doesn’t require to halve the batch size?</p>

ptrblck: <p>The whole computation graph is connected to <code>features</code>, which will also be freed, if you didn’t wrap the block in a <code>torch.no_grad()</code> guard.<br>
However, the second iteration shouldn’t cause an OOM issue, since the graph will be freed after <code>optimizer.step()</code> is called.</p>
<p>If you run out of memory after the training and in the first evaluation iteration, you might keep unnecessary variables alive due to Python’s function scoping as explained in <a href="https://discuss.pytorch.org/t/increase-the-cuda-memory-twice-then-stop-increasing/28462/2">this post</a>.</p>

Valiox: <p>Indeed, I understand now. The previous iteration of <code>features</code> remains in memory while <code>model(batch)</code> is evaluated the second time, and so there are points in the execution where two different graphs exist in memory.</p>
<p>You’re right, in inference mode I should be wrapping the call with <code>no_grad</code>.</p>
<p>Would this not happen in a training loop, as the additional backward pass would free the graph?</p>

ptrblck: <p>In a training loop you would usually reassign the output to the same variable, thus deleting the old one and store the current output.</p>
<p>If you are using e.g. different variables for the output, losses etc. in the training and validation loop, you would waste a bit of memory, which could be critical, if you are using almost the whole GPU memory.</p>
Pass training set through cuda network without overloading memory

spacemeerkat: <p>I have a neural network model that I have mapped to a GPU device (hence the CUDA network description in the topic title) and during training I wish to put the entire training set through the network. (For exploratory purposes, <em>not</em> for training)</p>
<p>If I was in evaluation mode for a test set I would map the saved (trained) model to a CPU device and pass the test set in as one giant batch.</p>
<p>However, if I try to do this during training with the CUDA network I overload the gpu memory. Is there a good way to temporarily stop the model relying on GPU memory within training just to put the training set through (without recording gradients)?</p>
<p>My current ideas are:</p>
<ul>
<li>find a way to temporarily map the network to the CPU without saving and reloading the model</li>
<li>put the training data through one example at a time (this is very slow and goes against the point of the original problem)</li>
</ul>
<p>Any help on this would be really appreciated! Apologies for not providing any code, I feel like it’s a problem that maybe has a generalized solution anyway.</p>

ptrblck: <p>I assume “overloading” means your GPU is running our of memory?<br>
If so, you could try to wrap this special forward pass in a <code>with torch.no_grad()</code> block so avoid storing the intermediate activations (which are not needed, if you don’t plan on calling <code>backward</code>).<br>
If that doesn’t help and you are still running out of memory, you could of course chunk the data in smaller batches, which should still be faster than passing each sample one by one.</p>
<aside class="quote no-group" data-username="spacemeerkat" data-post="1" data-topic="50951">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spacemeerkat/48/4442_2.png" class="avatar"> spacemeerkat:</div>
<blockquote>
<p>o temporarily map the network to the CPU without saving and reloading the model</p>
</blockquote>
</aside>
<p>You don’t have to save and reload the model in order to push it to the CPU.<br>
Just call <code>model.to('cpu')</code> to move all parameters to the CPU.</p>

spacemeerkat: <p>In the end I used the <code>model.to('cpu')</code> option and that worked for me, many thanks again! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
About creating extension and c++ front end

111120: <p>If I use c++ frontend, I need include &lt;torch/torch.h&gt;. If I create a extension for python, I need include &lt;torch/extension.h&gt;. This means I can debug cuda extension in c++, which used &lt;torch/torch.h&gt;. Then we only change torch.h to extension.h to create python extension. Is it right?</p>
ONNX to Pytorch Conversion

Subham_Tiwari: <p>There are libraries to convert PyTorch to ONNX. But is there some library to convert ONNX to Pytorch?</p>

Mazhar_Shaikh: <p>There is a feature request for this functionality. <a href="https://github.com/pytorch/pytorch/issues/21683" rel="nofollow noopener">link</a><br>
Please feel free to add on to why this may be useful. Also feel free to contribute by creating this feature yourself and submitting a pull request.</p>
Apply transformation to each channel of an image

lorenzo_fabbri: <p>My images have 6 channels. I would like to apply some transformations to them. The problem is that if I apply, for instance, a <code>RandomHorizontalFlip</code>, then the resulting image has 3 channels only.</p>
<p>I thus tried to use a Lambda function to apply the same transformation to each channel. I came up with the following class:</p>
<pre><code class="lang-auto">class HorizontalFlipChannels(object):
    
    def transform(self, img):
        
        transformed_channels = []
        
        for idx, channel in enumerate(img):
            print(idx)
            channel = transforms.ToPILImage()(channel)
            channel = transforms.RandomHorizontalFlip()(channel)
            channel = transforms.ToTensor()(channel)
            
            transformed_channels.append(channel)
        
        img = torch.cat(transformed_channels)
        
        return img
</code></pre>
<p>The <em>problem</em> is that if I print the index of the channel (when I iterate over the images in one batch), then I get something like the following</p>
<pre><code class="lang-auto">0
0
1
2
1
0
2
1
0
3
3
4
2
1
5
3
4
2
4
5
3
...
</code></pre>
<p>Is this normal? I was expecting to see something like <code>0 1 2 3 4 5 0 1 2 3 4 5 ...</code>.</p>

Oli: <p>Do you by any chance has 4 workers in your dataloader? If more than 0, then the program can do these things</p>

lorenzo_fabbri: <p>Yes, I have 4 workers. But I was wondering whether the <em>final</em> order of the channels for all the images will be the same.</p>

Oli: <p>Yes it will, so no worries <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> It’s just the multiprocessing that’s screwing with the printouts.</p>
Shuffle outside of batch

Yifan: <p>Is there any way to shuffle between batch but don’t interrupt the order within each batch?<br>
Many Thanks!</p>

smth: <p>i dont actually understand what you mean (the sentence is very confusing). Can you give an example?</p>

miguelvr: <p>You can do it at a data preprocessing stage:</p>
<p>List of batches -&gt; shuffle it -&gt; feed each batch to the model on the training loop</p>

stephen-song: <p>u can form batch data at Dataset and get a list of batched data, then use Dataloader to shuffle this list.</p>
Rotate a tensor at a given degree

AtenaNguyen: <p>Hi there,</p>
<p>I have an tensor, and I want to rotate it at a given degree, i.e. 0.05.<br>
How can I do it efficiently? I saw the torchvision.transforms.RandomRotation but this is used for data augmentation.</p>
<p>Any advice?</p>
<p>Thank you</p>

Balamurali_M: <p>You can do it in multiple ways.</p>
<p>Use can pil Library directly for rotation as pytorch uses pil for this kind of image operations.</p>
<p>Another simple way I can think of is use random rotation class itself. But give the min and max with the same value.</p>
<p>If you want to write.a proper class, copy the random rotation class source code and change the way you need it to be like getting the rotation angle alone.</p>

AtenaNguyen: <p>Thank you for suggestion,</p>
<p>The problem is this is the tensor as an intermediate layer in a network. Using PIL lib would requires to convert it back to numpy which is significantly slow.</p>
<p>Secondly, as I haven’t yet figure out how to use use random rotation class caused it only allow to use in the augmentation process.</p>
<p>Thirdly, making a custom class as modified of random rotation sounds reasonable. But I haven’t try yet. Just wonder why pyTorch does not support this function.</p>
<p>Thank you for your suggestions</p>
View tensor as larger tensor without copying memory

David_Colwell: <p>Hi all,</p>
<p>I have 2 tensors. They are quite large, so for the sake of simplicity i will give a simplified example. The small tensor is half the size of the large tensor in all dimensions (2,2) vs (4,4)</p>
<pre><code class="lang-auto">small = [[1,2],[3,4]]
large = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
</code></pre>
<p>I need to do a comparison between the 2. For the sake of this comparison, each item in the small tensor needs to be repeated so that it fills double the space in each dimension. It is basically a Nearest Neighbour upsampling. The catch is, if possible, i do not want to copy the memory, since these are really big tensors.</p>
<pre><code class="lang-auto">desired_small_tensor = [[1,1,2,2],[1,1,2,2],[3,3,4,4],[3,3,4,4]]
</code></pre>

Joel_Wu: <p>If you not willing to upsampling small tensor, then you may need slicing your large tensor to match the small one. Here is sample code:</p>
<pre><code class="lang-auto">sizes = small.shape
out = torch.zeros_like(large, dtype=torch.uint8)

for i in range(2):
    large_dim0 = large.narrow(0, i * sizes[0], sizes[0])
    out_dim0 = out.narrow(0, i * sizes[0], sizes[0])
    for j in range(2):
        large_dim1 = large_dim0.narrow(1, j * sizes[1], sizes[1])
        out_dim1 = out_dim0.narrow(1, j * sizes[1], sizes[1])
        
        out_dim1[:] = small &lt; large_dim1
</code></pre>

David_Colwell: <p>Thanks <a class="mention" href="/u/joel_wu">@Joel_Wu</a><br>
I think the slicing will probably be slow, since each dimension has &gt; 1000 elements. I’ll keep looking for a solution, but for now I’m upsampling the small one</p>
Dealing with floating point exceptions

kl_divergence: <p>I’ve been getting Floating point exception at different iterations in my code. I am not using shuffling, but even then this error arises randomly. There is no traceback of this error as well, so I’m unable to find the occurence of this error. I’ve tried pdb/(try-except) but they won’t work in in this case. I haven’t tried with fpectl, but not sure if it would be useful for this. Is there a better way to find the occurence (in the form of a traceback) of <code>Floating point exception (core dumped)</code> ?</p>

ptrblck: <p>Could you try to get a stack trace using <code>gdb</code>:</p>
<pre><code class="lang-auto">$ gdb --args python my_script.py
...
Reading symbols from python...done.
(gdb) run
...
(gdb) backtrace
...
</code></pre>

kl_divergence: <p>Thanks for replying. I got this:</p>
<pre><code class="lang-auto">During startup program terminated with signal SIGFPE, Arithmetic exception.
(gdb) backtrace
No stack.
</code></pre>
<p>I’ve done everything to remove bad samples, but still it persists.</p>
Help clarifying repackage_hidden in word_language_model

WenchenLi: <p>Hi,<br>
In the example of word_language_model, we have</p>
<pre><code class="lang-auto">def repackage_hidden(h):
    """Wraps hidden states in new Variables, to detach them from their history."""
    if type(h) == Variable:
        return Variable(h.data)
    else:
        return tuple(repackage_hidden(v) for v in h)
</code></pre>
<p>I dont think I fully understand what the “history” includes, can somebody helps clarify this?</p>
<p>Thanks!</p>

apaszke: <p>Every variable has a <code>.creator</code> attribute that is an entry point to a graph, that encodes the operation history. This allows autograd to replay it and differentiate each op. So each hidden state will have a reference to some graph node that has created it, but in that example you’re doing BPTT, so you never want to backprop to it after you finish the sequence. To get rid of the reference, you have to take out the tensor containing the hidden state <code>h.data</code> and wrap it in a fresh <code>Variable</code>, that has no history (is a graph leaf). This allows the previous graph to go out of scope and free up the memory for next iteration.</p>

jekbradbury: <p>I was going to add that <code>.detach()</code> does the same thing, but I checked the code and realized that I’m not at all sure about the semantics of <code>var2 = var1.detach()</code> vs <code>var2 = Variable(var1.data)</code>…</p>

apaszke: <p>Right now the difference is that <code>.detach()</code> still retains the reference, but it should be fixed.</p>
<p>It will change once more when we add lazy execution. In eager mode, it will stay as is (always discard the <code>.creator</code> and mark as not requiring grad). In lazy mode <code>var1.detach()</code> won’t trigger the compute and will save the reference, while <code>Variable(var1.data)</code> will trigger it, because you’re accessing the data.</p>

LaceyChen17: <p>So we do not need to repackage hidden state when making predictions ,since we don’t do a BPTT ?</p>

jdhao: <p>For any latecomers, <code>Variable</code> object does not have <code>creator</code> attribute any more, which is renamed to <code>grad_fn</code>. You can see <a href="https://github.com/pytorch/tutorials/issues/97">here</a> for more information.</p>

ratishsp: <p>Shouldn’t the code set <code>requires_grad=True</code> to the hidden state as shown below?<br>
As per my understanding, each bptt set should be able to have gradients computed for h.</p>
<pre><code class="lang-auto">def repackage_hidden(h):
    """Wraps hidden states in new Variables, to detach them from their history."""
    if type(h) == Variable:
        return Variable(h.data, requires_grad=True)
    else:
        return tuple(repackage_hidden(v) for v in h)
</code></pre>
<p>Thanks.</p>

alphadl: <p>it has already been updated to be compatible with the latest PyTorch version:</p>
<pre><code class="lang-auto">def repackage_hidden(h):
    """Wraps hidden states in new Tensors, to detach them from their history."""
    if isinstance(h, torch.Tensor):
        return h.detach()
    else:
        return tuple(repackage_hidden(v) for v in h)
</code></pre>
Where is torch::autograd::profiler?

jangbi: <p>I am using libtorch, c++ frontend, downloaded from the main page of <a href="https://pytorch.org/" rel="noopener nofollow ugc">https://pytorch.org/</a> and trying to profile layer by layer.</p>
<p>I’ve learn that in python i can use <code>torch.autograd.profiler.profile</code>. And i’ve read some website, including <a href="https://github.com/pytorch/pytorch/pull/16580/files#diff-81d29c29bbf9fb05edfa7fd70d189374" class="inline-onebox" rel="noopener nofollow ugc">Access profiler from cpp by zdevito · Pull Request #16580 · pytorch/pytorch · GitHub</a> and <a href="https://caffe2.ai/doxygen-c/html/structtorch_1_1autograd_1_1profiler_1_1_record_profile.html" class="inline-onebox" rel="noopener nofollow ugc">Caffe2 - C++ API: torch::autograd::profiler::RecordProfile Struct Reference</a></p>
<p>But when i use CLion to construct my code, use <code>torch::autograd::profiler::RecordProfile</code> , it report an error that</p>
<blockquote>
<p>no member named ‘profiler’ in namespace 'torch::autograd</p>
</blockquote>
<p>I turned to <a href="https://pytorch.org/cppdocs/api/library_root.html" class="inline-onebox" rel="noopener nofollow ugc">Library API — PyTorch main documentation</a>, and cannot find something about <code>profiler</code>, so how should i do to use the <code>profiler</code> in my project?</p>

tom: <p>You can include <code>torch/csrc/autograd/profiler.h</code>. I’m not entirely sure of the stability guarantee of the things in <code>torch/csrc/</code>, but it is in the distribution.</p>
<p>Best regards</p>
<p>Thomas</p>

jangbi: <p>Thanks for you reply, but now i have some question about the fields that displayed by profiler, since they are different from the python document.</p>
<pre><code class="lang-auto">{
  "name": "size",
  "ph": "X",
  "ts": 236152.000000,
  "dur": 1.000000,
  "tid": 0,
  "pid": "CPU Functions",
  "args": {}
},
</code></pre>
<p>What’s the meaning of <code>ph</code>, <code>ts</code>, <code>dur</code> and <code>tid</code>, and how should i explain their value?</p>
Compiling 32 bit (x86) version of LibTorch for windows 64

zarlior: <p>Hi,<br>
I have downloaded the compiled version of LibTorch 1.0 and saw that all DLLs are compiled for 64bit application.<br>
I have a 32bit application which I want to link to LibTorch so I need 32bit version DLLs.</p>
<p>Is there a way to download\compile it?</p>
<p>Thanks!</p>

peterjc123: <p>Sorry. It is not possible because pytorch is only ready for x64.</p>

Zhangpengtao: <aside class="quote no-group" data-username="zarlior" data-post="1" data-topic="36037">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/7ab992/48.png" class="avatar"> zarlior:</div>
<blockquote>
<p>downloaded</p>
</blockquote>
</aside>
<p>Do you know how to get the 32bit version DLLs,or whether pytorch supports 32bit in the future?</p>

peterjc123: <p>Please refer to this PR. <a href="https://github.com/pytorch/pytorch/pull/17865" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/17865</a></p>
The derivative for 'target' is not implemented

SamChung_Hwang: <p>loss1 is ok, but loss2 makes the error “the derivative for ‘target’ is not implemented”<br>
Why the error occurs ?</p>
<p>import torch<br>
import torch.nn as nn<br>
from torch.autograd import Variable<br>
from torch.autograd import Variable<br>
import torch.nn.functional as F</p>
<p>m = nn.Sigmoid()<br>
loss = nn.BCELoss()</p>
<p>a = Variable(torch.Tensor([[1,2],[3,4]]), requires_grad=True)<br>
y = torch.sum(a**2)<br>
target = torch.empty(1).random_(2)<br>
label = Variable(torch.Tensor([10]), requires_grad=True)<br>
y.backward()<br>
print(a.grad)<br>
loss_fn = nn.BCELoss()<br>
loss1 = loss_fn(m(y), target)<br>
loss2 = loss_fn(m(y), label)</p>

ptrblck: <p>You are passing a target tensor, which requires gradients (<code>labels</code>) in the second use case.<br>
Since the derivative for the target is not supported for <code>nn.BCELoss</code>, you should detach it before calling <code>loss_fn</code>.</p>

SamChung_Hwang: <p>Thank you. It works.</p>
<p>import torch.nn as nn<br>
from torch.autograd import Variable<br>
import torch.nn.functional as F</p>
<p>m = nn.Sigmoid()<br>
loss = nn.BCELoss()</p>
<p>a = Variable(torch.Tensor([[1,2],[3,4]]), requires_grad=True)<br>
y = torch.sum(a**2)<br>
target = torch.empty(1).random_(2)<br>
<span class="hashtag">#label</span> = Variable(torch.Tensor([10]), requires_grad=True)<br>
label = torch.ones(1, requires_grad=True)<br>
y.backward()<br>
print(a.grad)<br>
loss_fn = nn.BCELoss()<br>
loss1 = loss_fn(m(y), target)<br>
loss2 = loss_fn(m(y), label.detach())</p>
The arguments of .backward()

Yifan_Xu: <p>I kind of understand that when there is no argument passed in .backward(), it means the gradient that we want to compute is a scaler tensor, otherwise there would be an error.  However, today I read a code that works with no argument in .backward() for a vector tensor. Does anyone knows why this works?</p>
<p>I paste a snippet here</p>
<pre><code class="lang-auto">loss = F.binary_cross_entropy_with_logits(xbhat, xb, size_average=False) / B
loss.backward()
</code></pre>
<p>where xbhat is predicted values and xb are original mnist data</p>

Mazhar_Shaikh: <p>Hi Yifan_Xu,<br>
Backward for a vector tensor certainly does not work without the input gradient.<br>
One reason the above code may be working is that the size_average argument has been deprecated in the current version of pytorch, in favour of the reduction argument, which has a default value of ‘mean’. Hence, the above code may unintentionally calculated gradient on a scalar tensor.<br>
Hope this helps.</p>

Yifan_Xu: <p>Hey Mazhar_Shaikh,</p>
<p>Thanks for your replay. I finally understand what was going on under the hood. I didn’t make it clear that this code was from a neural net. That means although xb is passed as a vector, it does not calculated as a vector. Say we have xb = [x1, x2, x3] and a simple MLP that outputs xbhat = g(w0b+w1x1 + w2x2 + w3x3). As you easily tell, x1, x2, x3 is done separately not as a whole. That’s why using .backward() without argument is legal here.</p>
Using dropout in evaluation mode

spacemeerkat: <p>Is there a simple way to use dropout during evaluation mode?</p>
<p>I have a plan to use a pre-trained network where I reapply the training dropout, and find outliers by measuring the variance in predictions -all done in evaluation mode so that no gradients are backpropogated.</p>
<p>Many thanks in advance!</p>

SimonW: <p>Assuming that you are using the dropout modules.</p>
<pre><code class="lang-auto">model.eval()
for m in model.modules():
  if m.__class__.__name__.startswith('Dropout'):
    m.train()
</code></pre>

spacemeerkat: <p>Thanks for the fast reply! So I assume this makes an exception for ‘Dropout’ and sets it active using train() but leaves batchnorm etc. switched off in eval mode?</p>

SimonW: <p>yep, that is the idea!</p>

spacemeerkat: <p>One quick question: will this method zero different nodes every time you present a new image to the network like it would in training?</p>

SimonW: <p>if you don’t reset the seed, yes</p>

spacemeerkat: <p>That’s great, I was just checking because I was unsure if it would use a particular dropout selection and then stick to that selection until the model was re-instantiated with a line like “m.eval()” before presenting a new image</p>

LMA: <p>Hi Simon, Thanks for suggesting this approach! I tried it, and it gives predictions with certain randomness for in-class inputs. But the randomness is almost non-existent for out-of-class inputs. However, if I put the model in the training mode <strong>without back propagation</strong> (so that the weights are not updated) using <code>model.train()</code>, I observe significantly more randomness in the predictions for both in-class and out-of-class inputs, but the accuracy for the in-class inputs becomes very low (dropped from 95% to 63%). To me, the two approaches should give the same results. Any thoughts on the different behavior? My model also has batch norm. Thanks.</p>
Combining multiple boxes

ahmed: <p>I have trained a network for detection using bounding boxes. During evaluation I remove boxes using IoU technique and thresholding. However, I am getting multiple boxes for a single object (as seen in picture below). The bounding boxes are<br>
<code>tensor([[448.00, 219.25, 476.33, 287.26], [453.84, 223.89, 473.85, 279.77], [450.89, 218.27, 467.31, 276.45], [448.76, 219.92, 484.23, 283.18]])</code></p>
<p>Is there any way i can combine or remove the extra boxes?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/58adb8299eea790caaf4a40a105524c5208a0b31.png" data-download-href="https://discuss.pytorch.org/uploads/default/58adb8299eea790caaf4a40a105524c5208a0b31" title="test"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/58adb8299eea790caaf4a40a105524c5208a0b31_2_500x500.png" alt="test" data-base62-sha1="cEucZbuY8PdltiiOuclMvLM35Cx" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/58adb8299eea790caaf4a40a105524c5208a0b31_2_500x500.png, https://discuss.pytorch.org/uploads/default/original/2X/5/58adb8299eea790caaf4a40a105524c5208a0b31.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/5/58adb8299eea790caaf4a40a105524c5208a0b31.png 2x" data-dominant-color="B8B2AD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">test</span><span class="informations">720×720 214 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Hyperparameter fix_length for Field not working? (torch text)

Jeffrey_Tsang: <p>Hey, I am fairly new to deep learning and am now doing a NLP project for the task of text summarization.</p>
<p>For some reason when I am loading in the dataset with the Field where I’ve specified the fixed length, the original length seems unchanged for the data points.</p>
<pre><code class="lang-auto">SRC = Field(tokenize = tokenize_en, 
            init_token = '&lt;sos&gt;', 
            eos_token = '&lt;eos&gt;', 
            fix_length = 400,
            lower = True)

TRG = Field(tokenize = tokenize_en, 
            init_token = '&lt;sos&gt;', 
            eos_token = '&lt;eos&gt;', 
            fix_length = 100,
            lower = True)

fields = {'doc': ('doc', SRC), 'summaries': ('summaries', TRG)}

train_data,valid_data,test_data = TabularDataset.splits(path='./', train='train.json', validation='val.json', test='test.json', format='json', fields=fields)
</code></pre>
<p>If i were to run the code beneath for example, the length still exceeds 400.</p>
<p><code>print(len(vars(train_data.examples[40])['doc']))</code></p>
<p>Can someone point out to me what I am doing wrong or perhaps suggest another solution? Thanks for your time!</p>

XinDongol: <p>I have the same problem</p>
More complicated inlace calculations - readability of the code

LeanderK: <p>I am currently working on a module for a research paper where I need to perform a few calculations, like solving quadratic equations. Because the module is used a lot throughout the network and my memory is under pressure, I want to perform the calculations inplace. But the code is turning out to be quite ugly and borderline incomprehensible. Is there any way to improve the readability so that it’s not a massive chain of <code>.mul_</code>, <code>.add_</code> and parentheses everywhere?</p>
<p>Also the autograd-errors are quite hard to debug if something is wrong in the inplace-calcuation.</p>
<p>Can something like the JIT-compiler help in this case? If yes, I can’t jit only part of the code, can I? I imagine the analysis for the compiler to be quite easy and well-understood.</p>
Performance (Time) difference between different Pytorch Versions

Gullal: <p>Hi,<br>
I am porting some of my old code (Pytorch 0.4.0) to latest pytorch version, although there is hardly any syntactic difference. But, i have observed that some architectures like an <strong>autoencoder trains much slower</strong> on the latest pytorch, while using a classification network like taking off-the-shelf resnet from torchvision models takes the same amount of time for training.</p>
<p>I have tried using the <code>torch.backends.cudnn.benchmark=True</code> solution, but that doesn’t help.<br>
I have also tried reinstalling conda and the environments, but it makes no difference.<br>
My machine’s configuration is <code>Ubuntu 18.04, Nvidia GTX 1080 Ti</code></p>
<p>For a sample run, I took this simple convolutional autoencoder from [here] and trained it on mnist.(<a href="https://github.com/L1aoXingyu/pytorch-beginner/tree/master/08-AutoEncoder" rel="nofollow noopener">https://github.com/L1aoXingyu/pytorch-beginner/tree/master/08-AutoEncoder</a>)</p>
<p>With <code>pytorch 0.4.0, torchvision 0.2.1, cudatoolkit 9.0, cudnn 7.6.0</code>, each epoch takes ~4 seconds.<br>
With <code>pytorch 1.1.0, torchvision 0.3.0 and cudatoolkit 10.0, cudnn 7.5.1</code>, each epoch takes ~7 seconds.</p>
<p>Let me know if someone has observed a similar issue and any help in rectifying is appreciated.</p>

justusschock: <p>I don’t know if that’s the issue, but your 0.4.0 version comes with cudnn 7.6.0, which should be faster than the cudnn version of your pytorch 1.1.0 according to <a href="https://developer.nvidia.com/cudnn" rel="nofollow noopener">this</a> link. If you use GPUs, the operations are often dispatched to cudnn ops.</p>

Gullal: <p>Thats probably the issue I guess.<br>
But thats what is installed with the respective versions of pytorch and cuda. It seems like I will have to install pytorch from source.</p>
Keras to Pytorch model

Kris_Kulivnyk: <p>Hi dear community,</p>
<p>I’ve started with PyTorch recently, and I’m a bit confused about the basic problems. So, my issue is - what will be the Pytorch analogue for this the most simple Keras model:</p>
<pre><code class="lang-auto">model = tf.keras.models.Sequential([
      tf.keras.layers.Dense(
          10, activation=tf.nn.softmax, kernel_initializer='zeros', input_shape=(784,))])
</code></pre>
<p>I’ll appreciate any help and advise</p>

gost-sniper: <p>I think this might help you <a href="https://discuss.pytorch.org/t/pytorch-equivalent-of-keras/29412/2" class="inline-onebox">Pytorch equivalent of Keras</a></p>

justusschock: <p>So, if you want a 1:1 translation you’d probably end up with</p>
<pre><code class="lang-auto">model = torch.nn.Sequential(
    torch.nn.Linear(
        784, # number inputs
        10 # number outputs
    ),
    torch.nn.Softmax(dim=-1)
 )
</code></pre>
<p>And then call a function, initializing the weights with <code>model.apply(init_func)</code>, where <code>init_func</code> is a function which will be called on every submodule of <code>model</code>.</p>
What does model.parameters() include

Alpha: <p>when use:<br>
<code>optimizer = torch.optim.SGD( model.parameters() )</code></p>
<p>What does the <code>model.parameters()</code> include?<br>
e.g. what I know are as follows:</p>
<ol>
<li>network weight, bias</li>
<li>BN  beta, gamma</li>
<li>nn.Parameter</li>
</ol>
<p>What anything else?</p>
<p>Thank you ?</p>
Upsampling/ convTranspose2d getting the original tensor size

gost-sniper: <p>I’m trying to build a convolutional auto encoder but at the end of the neural net I don’t get the original size of the tensor <code>initial size : torch.Size([2, 3, 360, 640])</code>  I’m using <code>nn.ConvTranspose2d(in_channels=2x, out_channels=x, kernel_size=2, stride=2)</code> multiple times giving the these results :</p>
<pre><code class="lang-auto">initial size :  torch.Size([2, 3, 360, 640])

just before 1st convTranspose2d  torch.Size([2, 1024, 22, 39])

just after 1st convTranspose2d  torch.Size([2, 512, 44, 78])

just after 2nd convTranspose2d  torch.Size([2, 256, 88, 156])

just after 3rd convTranspose2d  torch.Size([2, 128, 176, 312])

just after 4th convTranspose2d  torch.Size([2, 64, 352, 624])

end of the NN  torch.Size([2, 3, 354, 626])

</code></pre>
<p>Is there any how I can restore my original <code>torch.Size([2, 3, 360, 640])</code> ?</p>