akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Store the latest input data into a layer

username: amin_sabet
cooked: <p>In my code, I need to store the latest input data into a layer (Here is _AdaptiveAvgPoolNd ) and do the average pooling operation on the subtraction of new input data and latest input data, then updating the latest value. I was wondering what is the best way to define the latest_inpu? With the variable, I get following error during the training</p>
<pre><code class="lang-auto">class _AdaptiveAvgPoolNd(nn.Module):
    __constants__ = ['output_size']

    def __init__(self, batch,channel,width,height,output_size):
        super(_AdaptiveAvgPoolNd, self).__init__()
        self.output_size = output_size
        self.latest_input = Variable(torch.zeros(batch,channel,width,height), requires_grad=True)

    def extra_repr(self):
        return 'output_size={}'.format(self.output_size)


class AdaptiveAvgPool2d_modified(_AdaptiveAvgPoolNd):
   
    def forward(self, input):

        output = F.adaptive_avg_pool2d((input - self.latest_input), self.output_size)
        self.latest_input = input
            
        return output
</code></pre>
<pre><code class="lang-auto">Variable._execution_engine.run_backward(
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.
</code></pre>

username: Shisho_Sama
cooked: <p>I guess your issue is your latest_input interfers with the current graph as the error suggest so, just do <code>self.latest_input.clone().detach()</code> so your only providing the data needed.</p>
How to support k40m gpu

username: Hartbook
cooked: <p>Hello</p>
<p>I use libtorch (compiled by myself) with several gpu without problems, but when I try to use it on a nvidia K40m i get this error <code>43134 Illegal instruction</code></p>
<p>How can I add support for this card when compiling ?<br>
I’m using CUDA 10.1 and I compile libtorch with this command</p>
<pre><code class="lang-auto">MAX_JOBS=4 USE_CUDA=1 USE_CUDNN=1 CFLAGS="-Ofast" python ../tools/build_libtorch.py
</code></pre>

username: ptrblck
cooked: <p>Are you getting this error only if you execute CUDA code or also CPU operations?<br>
Could you try to get a stack trace via:</p>
<pre><code class="lang-auto">$ gdb --args python my_script.py
...
Reading symbols from python...done.
(gdb) run
...
(gdb) backtrace
...
</code></pre>
<p>and post it here?</p>
<p>Also, could you post the versions of PyTorch, CUDA, NVIDIA driver?</p>
PyTorch print image name - AttributeError: 'Tensor' object has no attribute 'files'

username: alameer
cooked: <p>Sorry, if this question is very basic. I’m trying to print the image names at each iterations to a csv file whilst training the model. but I am getting the following error message:</p>
<p>filename=output.files AttributeError: ‘Tensor’ object has no attribute ‘files’</p>
<pre><code class="lang-auto"># main training loop
global_step = 0
best_test_error = 10000
#filename=CellsDataset(args.data_dir,transform=ToTensor(),return_filenames=True)
for epoch in range(50):
    print("Epoch %d" % epoch)
    model.train()
    for images, paths in tqdm(loader_train):
        images = images.to(device)
        targets = torch.tensor([metadata['count'][os.path.split(path)[-1]] for path in paths]) # B
        targets = targets.float().to(device)
        filename=CellsDataset(args.data_dir,transform=ToTensor(),return_filenames=True)
        #print(filename.files)

        output = model(images) # B x 1 x 9 x 9 (analogous to a heatmap)
        #print(output.filename)
        preds = output.sum(dim=[1,2,3]) # predicted cell counts (vector of length B)
        filename=output.files
        print(preds)
        with open('preds_base_model.csv','a') as fd:
            fd.write( ','.join(map(str, preds.detach().tolist())) + '\n')
            fd.write( ','.join(map(str, targets.detach().tolist())) + '\n')
            fd.write(( ','.join(map(str, filename.detach().tolist())) + '\n')</code></pre>

username: alpha_omega_alpha
cooked: <p>You started with <code>filename=CellsDataset(args.data_dir,transform=ToTensor(),return_filenames=True)</code>, which I assume returns the names of the files you’re after, but then you try to assigned <code>filename</code> to be <code>output.files</code>, which fails because the <code>output</code> doesn’t have <code>.files</code> attribute.<br>
Maybe try commenting this out and keep the first <code>filename</code> variable?</p>

username: alameer
cooked: <p>If i try to to do:</p>
<pre><code class="lang-auto">print(filename)

</code></pre>
<p>I get: &lt;dataset2.CellsDataset object at 0x7f8a8315a190&gt;</p>
How to insert information text to image frame?

username: ramen-ctrl
cooked: <p>Hi, I am working on a 3-class image detector. I am currently using yolov4 on pytorch. I am able to detect several classes in a single image. I want to improve it by counting the number of images per class and displaying on the top left hand of the image. For example, if the image has detected 2 broccolis and 1 carrot, the image should have “broccoli: 2, carrot: 1” text on the top left hand of the image in addition to the bounding boxes. What command do I have to input to have text on the top left corner of the image? [note: I am new to pytorch so I’m not familiar with the commands]</p>

username: ptrblck
cooked: <p>You won’t be able to use PyTorch directly to add text to an image, but should instead use any image processing library such as <code>PIL</code>.<br>
<a href="https://stackoverflow.com/a/16377244">This post</a> describes how text can be added to a <code>PIL.Image</code>.</p>
Pytorch with cuda 9.0 with latest supported version

username: braindotai
cooked: <p>Can anyone plz provide me the link for the latest pytorch version compatible with cuda 9.0</p>
<p>And also the steps to install it.</p>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/pensive.png?v=9" title=":pensive:" class="emoji only-emoji" alt=":pensive:"></p>

username: chetan_patil
cooked: <p>I dont think newer versions of Pytorch has support for cuda 9.<br>
The only version which supports cuda-9 is 0.4</p>

username: ptrblck
cooked: <p>PyTorch <code>1.1.0</code> supported CUDA9.0 and the conda wheel can be downloaded <a href="https://anaconda.org/pytorch/pytorch/1.1.0/download/linux-64/pytorch-1.1.0-py3.7_cuda9.0.176_cudnn7.5.1_0.tar.bz2">here</a>.</p>
<p>Why do you need this particular CUDA version? Would you be able to update to 9.2?</p>

username: braindotai
cooked: <p>I am using an ssh remote server… and it had cuda 9.0 only.</p>
<p>Anyways I’ve upgraded to 10.2 <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
No attribute cmake_prefix_path

username: as754770178
cooked: <p>I refer to <code>https://pytorch.org/cppdocs/installing.html</code>, exec <code>cmake -DCMAKE_PREFIX_PATH=</code>python -c 'import torch;print(torch.utils.cmake_prefix_path)'<code>.  But report error </code>has no attribtue cmake_prefix_path<code>. </code></p>

username: ptrblck
cooked: <p>Which PyTorch version are you using?<br>
If an older one, could you update to the latest stable release (or nightly) and retry the command?</p>

username: as754770178
cooked: <pre><code class="lang-auto">$ py15
Python 3.7.5 (default, May 26 2020, 08:53:41)
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.utils.cmake_prefix_path
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
AttributeError: module 'torch.utils' has no attribute 'cmake_prefix_path'
&gt;&gt;&gt; torch.__version__
'1.5.0'
</code></pre>
<p>and</p>
<pre><code class="lang-auto"># py15 is the python that install torch1.5
$ py15  -c 'import torch;print(torch.utils.cmake_prefix_path)'
Traceback (most recent call last):
  File "&lt;string&gt;", line 1, in &lt;module&gt;
AttributeError: module 'torch.utils' has no attribute 'cmake_prefix_path'

</code></pre>

username: ptrblck
cooked: <p><a href="https://github.com/pytorch/pytorch/pull/38559">This PR</a> added the functionality and was merged on May 19th, which is after the <code>1.5.0</code> release (April 21st), so you would need to update to the latest stable version (<code>1.5.1</code>) or the nightly binary.</p>
Confusion about 'master_port'

username: Meraki
cooked: <p>Hi, there. Recently I used multiple cpu cores for training. On my own PC, macbook 2017 (1 cpu 4 cores), I just set os.environ[‘MASTER_PORT’] as one single value and multiprocesses could run on the same server. However, when I migrated codes to the cluster in order to use more cores, I need to give a different value to os.environ[‘MASTER_PORT’]  for each process. If not, the permission denied as below.</p>
<pre><code class="lang-auto">store = TCPStore(master_addr, master_port, world_size, start_daemon, timeout)
RuntimeError: Permission denied
</code></pre>
<p>I don’t know much about the reason here, could someone explain it?</p>

username: mrshenli
cooked: <p>Hey <a class="mention" href="/u/meraki">@Meraki</a> the <code>MASTER_PORT</code> needs to be set to the same value for all processes, otherwise, they cannot conduct rendezvous correctly. This error might be caused by other configurations. It might be helpful to print the value of the following environment variables on all processes right before <code>init_process_group</code> is called: “MASTER_ADDR”, “MASTER_PORT”, “RANK”, “WORLD_SIZE”</p>

username: Meraki
cooked: <p>Yep, you are right. I just figure out the reason is that on Linux, you need root permissions to open a port below 1024. That’s why the permison denied.</p>
<p>Thanks for your help.<a class="mention" href="/u/mrshenli">@mrshenli</a></p>
Global explanations in captum?

username: ccarmel
cooked: <p>are there any global explanations algorithms embedded in captum?<br>
is there any plan to add some anytime soon?<br>
which ones?</p>

username: Narine
cooked: <p>Hi <a class="mention" href="/u/ccarmel">@ccarmel</a>, We are currently working onTCAV - Testing with Concept Activation Vectors.<br>
Besides that we can always aggregate sample level attributions to get a global view, but this of course it tied to the data.<br>
Do you have any global explanation algorithms for Neural Networks in mind ?</p>
Extracting weights from one class from softmax layer

username: Hemantr05
cooked: <p>I have trained a neural network for binary image classification and want to retrieve the weights of one of the classes for training it with another network.<br>
Is it possible? If so, how can I do so?</p>

username: ptrblck
cooked: <p>A <code>softmax</code> layer doesn’t use any parameters, but just applies an operation to normalize the logits to probabilities.<br>
I’m not sure I understand the question correctly.<br>
Would you like to somehow extract all weights, which are responsible to predict a specific class from all layers?</p>

username: Hemantr05
cooked: <p>I would like to extract the weights learned for a particular class to use them for further use</p>

username: ptrblck
cooked: <p>Would you like to get the weigths for a particular class from the last linear layer?<br>
If so, you could use:</p>
<pre><code class="lang-python">in_features = 32
nb_classes = 10
lin = nn.Linear(in_features, nb_classes)
# print weight and bias for class0
print(lin.weight[0], lin.bias[0])
</code></pre>
<p>There wouldn’t be an easy way to extract “all parameters” for a specific class, since the preceding layers would be completely used (in case of linear layers) for all classes.</p>
Loss increase when load pth

username: as754770178
cooked: <p>I train my model and loss decrease to 2.0+. But when I load the pth and training continue, the loss increase to 3.0 - 4.0+, I need retrain the model that loss decreate to 2.0+. But I reload pth again, the effect is same.</p>

username: ptrblck
cooked: <p>How are you testing the accuracy after loading?<br>
If you are using the validation set, did you make sure to call <code>model.eval()</code> before?<br>
Are you also restoring the <code>optimizer</code> for the training after reloading?</p>

username: as754770178
cooked: <p>I fix I train dataloader not use shuffle.</p>
AttributeError: module 'warpctc_pytorch' has no attribute 'gpu_ctc'

username: LXC
cooked: <p>When I run the CRNN in pytorch with gpu,I got the problem like that.Did anyone how to fix it please?</p>

username: Ktakuya332C
cooked: <p>If the warp-ctc is <a href="https://github.com/SeanNaren/warp-ctc" rel="nofollow noopener">warp-ctc of SeanNaren</a>, then this <a href="https://github.com/SeanNaren/deepspeech.pytorch/issues/112" rel="nofollow noopener">thread</a> could be useful.</p>

username: Fortune_Seeker
cooked: <p>Hello, Have you solved this issue?</p>
Pytorch xla vs Pytorch Lightning: TPU pod

username: kendreaditya
cooked: <p>I am trying to utilize Colab’s multi-TPU feature. I was wondering which would be better to use, Pytorch’s <a href="https://github.com/pytorch/xla" rel="nofollow noopener">xla</a>, or <a href="https://github.com/PyTorchLightning/pytorch-lightning" rel="nofollow noopener">Pytorch Lightning</a>. What are the pros and cons?</p>

username: ailzhang
cooked: <p>Hi <a class="mention" href="/u/kendreaditya">@kendreaditya</a>, Pytorch lightnight’s TPU support is built on top of pytorch xla. So they’re essentially the same in terms of functionality with slightly different frontend.<br>
Thus this question is more about whether you are more familiar with pytorch native frontend or the lightning frontend.<br>
(btw if you have questions w.r.t pytorch/xla, we currently support all questions/bug reports/feature requests through github issues in pytorch/xla.</p>

username: kendreaditya
cooked: <p>Thank you for the classification! By front-end do you mean like PyTorch’s structure, and style? <a class="mention" href="/u/ailzhang">@ailzhang</a></p>

username: ailzhang
cooked: <p><a class="mention" href="/u/kendreaditya">@kendreaditya</a> Yea it’s more like which way you are comfortable writing the code.</p>
How to convert a 32-bit operation to a 4-bit or 8-bit operation on cpu?

username: learningsteady0J0
cooked: <p>To the best of my knowledge, the existing quantization method is operating on 32-bit.<br>
In order to quantize weight of CNN as well as reduce memory footprint and then port the quantized model into the mobile device, how to convert a 32-bit operation to a 4-bit or 8-bit operation on cpu?</p>

username: Vasiliy_Kuznetsov
cooked: <p>PyTorch quantization supports int8 (but not int4), with fast kernels for CPU on mobile via QNNPACK.  <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html</a> has some information to get started, and you would want to set the backend to <code>qnnpack</code> to target mobile CPUs.</p>
Implementing Transformer Decoder for Machine Translation

username: Rafael_R
cooked: <p>Hi,</p>
<p>I am not understanding how to use the transformer decoder layer provided in PyTorch 1.2 for autoregressive decoding and beam search.</p>
<p>In LSTM, I don’t have to worry about masking, but in transformer, since all the target is taken just at once, I really need to make sure the masking is correct. Clearly the masking in the below code is wrong, but I do not get any shape errors, code just runs but…</p>
<p>The below code just leads to perfect perplexity in the case of a transformer decoder.</p>
<pre><code class="lang-auto">        mask = torch.zeros(max_length)

        for t in range(1, max_length):
            mask[t-1] = 1.0
            # Call Decoder
            output = self.decoder(input_tensor, src_encodings, mask)
            # Get only the first output
            output = output[t, :, :]
            # Do Softmax
            output_dist = F.softmax(output.squeeze(0), dim=1)
            # Get mle score
            mle_scores += self.criterion(output.squeeze(0), input_tensor[t]) * (input_lengths &gt; t).float()
</code></pre>
<p>What should I change here, how do do the correct maskingt? Please note that the decoder is just the transformer decoder layer (<a href="https://pytorch.org/docs/stable/_modules/torch/nn/modules/transformer.html" rel="nofollow noopener">https://pytorch.org/docs/stable/_modules/torch/nn/modules/transformer.html</a>) + linear softmax over vocabulary.</p>
<p>Any help would be greatly appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: zhangguanheng66
cooked: <p>You can pass the mask as tgt_mask in <code>nn.TransformerDecoder</code>.</p>

username: Rafael_R
cooked: <p>thanks, i am already passing the mask (the transformer decoder is just many layers of the decoder i am using: <a href="https://pytorch.org/docs/stable/_modules/torch/nn/modules/transformer.html" rel="nofollow noopener">https://pytorch.org/docs/stable/_modules/torch/nn/modules/transformer.html</a>)</p>
<p>i am not getting how to construct the mask?</p>
<p>should i prepare a mask for each pass of the decoding?</p>

username: zhangguanheng66
cooked: <p>No, the decoder layers shares the mask.</p>

username: Rafael_R
cooked: <p>thanks, yes, but i am passing the wrong mask to a single decoder layer, how should i create masks for autoregressive decoding…</p>

username: tomb88
cooked: <p>Any update on this issue? If so, can you share an example?</p>

username: itay1itzhak
cooked: <p>You can use the built-in function:<br>
tgt_mask = torch.nn.Transformer().generate_square_subsequent_mask(length)<br>
Where length is the length of the target (tgt)</p>
Dataset from Kaggle not having the expected structure

username: desmond13
cooked: <p>Hi, I am new to pytorch and I am trying to use a dataset on Kaggle to train a neural network.</p>
<p>The dataset is a classic one (dogs vs cats) and can be found here: <a href="https://www.kaggle.com/c/dogs-vs-cats" rel="nofollow noopener">https://www.kaggle.com/c/dogs-vs-cats</a></p>
<p>However, when I download it and I do</p>
<pre><code class="lang-auto">data_dir = 'cat_dog_data/train'

transform = transforms.Compose([transforms.Resize(255),
                                transforms.CenterCrop(224),
                                transforms.ToTensor()])
dataset = datasets.ImageFolder(data_dir, transform=transform)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=32, shuffle=True)
</code></pre>
<p>I get the following error:</p>
<pre><code class="lang-auto">RuntimeError: Found 0 files in subfolders of: PATHTOTHEFOLDER/intro-to-pytorch/cat_dog_data/train
Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif,.tiff,.webp
</code></pre>
<p>It seems that if I remove the <code>/train</code> from the <code>data_dir</code> I don’t have the error. However, I think that this would be wrong since the real structure I should have is:</p>
<p><code>cat_dog_data/train/dogs</code><br>
<code>cat_dog_data/train/cats</code></p>
<p>Instead, I have something like this</p>
<pre><code class="lang-auto">cat_dog_data/train/cat.13.jpg
cat_dog_data/train/cat.11.jpg
...
cat_dog_data/train/dog.1.jpg
cat_dog_data/train/dog.21.jpg
...
</code></pre>
<p>Should I manually create the folders <code>dogs</code> <code>cats</code> and remove the <code>dog.</code> and <code>cat.</code> from the <code>.jpg</code> filename? This seems weird but I don’t understand how to proceed otherwise.</p>
<p>Thanks for your help.</p>

username: desmond13
cooked: <p>It was as I thought, the dataset that I download was not correctly structured.</p>
<p>I found out with <a href="https://github.com/udacity/deep-learning-v2-pytorch/issues/139#issuecomment-480480117" rel="nofollow noopener">this issue</a> that also gives a link to a correctly structured dataset.</p>
<p>I hope this will help someone else in my situation.</p>
Conv2d and Linear layer for 1x1 image

username: bernardomig
cooked: <p>Hi. Today I was trying to convert a model weight to my implementation. I found a problem converting a Squeeze-Excitation Block that is the following:</p>
<pre><code class="lang-auto">class SEBlock(nn.Module):

    def __init__(self, in_channels, out_channels, reduction_ratio=4):
        super(SEBlock, self).__init__()

        red_channels = round_by(in_channels / reduction_ratio, 8)
        self.conv1 = nn.Conv2d(in_channels, red_channels, 1)
        self.activation = nn.ReLU(inplace=True)
        self.conv2 = nn.Conv2d(red_channels, out_channels, 1)

    def forward(self, input):
        x = F.adaptive_avg_pool2d(input, 1)
        x = self.conv1(x)
        x = self.activation(x)
        x = self.conv2(x)
        return input * hard_sigmoid(x)
</code></pre>
<p>However, in the original model, the pointwise convolutions are <code>nn.Linear</code> layers. I think that they could be easily converted as such (because the input is a 1x1 image):</p>
<pre><code class="lang-python">linear = nn.Linear(16, 8)
conv = nn.Conv2d(16, 8, 1)

conv.weight.copy_(linear.weight.reshape(8, 16, 1, 1));
conv.bias.copy_(linear.bias);
</code></pre>
<p>Unfortunelly, this does not reproduce the same result!</p>
<pre><code class="lang-python">x = torch.randn((3, 16, 1, 1))

torch.norm(conv(x) - linear(x.view(3, 16)))
&gt;&gt;&gt; tensor(16.5712)
</code></pre>
<p>What am I missing here?</p>

username: Nikronic
cooked: <p>Hi, the issue is the way you are calculating norm. As outputs do not have same shape, torch broadcast values so you get wrong norm.</p>
<aside class="quote no-group" data-username="bernardomig" data-post="1" data-topic="87309">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bernardomig/48/26159_2.png" class="avatar"> bernardomig:</div>
<blockquote>
<p><code>torch.norm(conv(x) - linear(x.view(3, 16)))</code></p>
</blockquote>
</aside>
<pre><code class="lang-auto">linear = nn.Linear(16, 8)
conv = nn.Conv2d(16, 8, 1)

conv.weight = nn.Parameter(linear.weight.reshape(8, 16, 1, 1))
conv.bias = nn.Parameter(linear.bias)

batch=5
x = torch.randn(batch, 16, 1, 1)
co = conv(x)
lo = linear(x.view(batch, 1, 16))
co = co.view(lo.shape)
torch.norm(co - lo)  # small value around 2.0e-07
</code></pre>
<p>Bests</p>

username: bernardomig
cooked: <p>Yeah you are absolutely correct <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:">. Gotta check the broadcasting rules of pytorch again.</p>
Using differentiable relaxation of quantization for backward pass

username: Maytide
cooked: <p>I am trying to run a custom module which performs quantization:</p>
<pre><code class="lang-auto">class Quantizer(nn.Module):
    """
    Scalar Quantizer module
    Source: https://github.com/mitscha/dplc
    """
    def __init__(self, centers=[-1.0, 1.0], sigma=1.0):
        super(Quantizer, self).__init__()
        self.centers = centers
        self.sigma = sigma

    def forward(self, x):
        centers = x.data.new(self.centers)
        xsize = list(x.size())

        # Compute differentiable soft quantized version
        x = x.view(*(xsize + [1]))
        level_var = Variable(centers, requires_grad=False)
        dist = torch.pow(x-level_var, 2)
        output = torch.sum(level_var * nn.functional.softmax(-self.sigma*dist, dim=-1), dim=-1)

        # Compute hard quantization (invisible to autograd)
        _, symbols = torch.min(dist.data, dim=-1, keepdim=True)
        for _ in range(len(xsize)): centers.unsqueeze_(0) # in-place error
        centers = centers.expand(*(xsize + [len(self.centers)]))

        quant = centers.gather(-1, symbols.long()).squeeze_(dim=-1)

        # Replace activations in soft variable with hard quantized version
        output.data = quant

        return output

if __name__ == '__main__':
    quantizer = Quantizer(centers=[-2, -1, 1, 2])
    z = quantizer(torch.Tensor([[1,2.5]]))
</code></pre>
<p>This code is supposed to compute a hard nearest-neighbor quantization but use a differentiable relaxation on the backward pass. The idea is to replace <code>output.data</code> with the hard computation and make it so that autograd only sees the soft forward pass (which will be unused). It yields the following error:</p>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2]] is at version 3; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>Now I have seen suggestions to fix this such as changing the inplace operation <code>centers.unsqueeze_(0)</code> to <code>centers.unsqueeze(0)</code>, but this is suspicious to me as the whole hard quantization block should be invisible to autograd. Does replacing a Variable’s data field actually work or should I be computing a soft relaxation for backprop some other way?</p>
<p>I am using <code>torch==1.2.0</code>, but the module was adapted from code using 0.4.1.</p>

username: albanD
cooked: <p>Hi,</p>
<p>In general, you should not use <code>.data</code> as it is very unlikely to do what you want.<br>
If you want to specify the backward corresponding to a given function, you can create a custom autograd.Function as described in the doc here: <a href="https://pytorch.org/docs/stable/notes/extending.html#extending-torch-autograd">https://pytorch.org/docs/stable/notes/extending.html#extending-torch-autograd</a></p>
How to debug reinforcement learning

username: skerlet_flandorle
cooked: <p>How to debug reinforcement learning</p>
<p>I am implementing reinforcement learning myself, but it does not work well<br>
I use pytorch The environment is Pendulum-v0</p>
<p>Even if you print plt the variable, you do not know whether it is the correct value<br>
It’s difficult to separate each function,<br>
I don’t know if the hyperparameters are bad or the implementation is wrong</p>
<p>Is there any good debugging method?</p>
Error when using DDP on multiply gpus

username: kaimss
cooked: <p>-1</p>
<p>When I trained by 4 GPUs like this:<br>
<code>python -m torch.distributed.launch --nproc_per_node=4 train_net.py</code> .<br>
There would be a error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/home_ex/tianhongtao/SW/anaconda3/envs/Hisense/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home_ex/tianhongtao/SW/anaconda3/envs/Hisense/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home_ex/tianhongtao/SW/anaconda3/envs/Hisense/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in &lt;module&gt;
    main()
  File "/home_ex/tianhongtao/SW/anaconda3/envs/Hisense/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/home_ex/tianhongtao/SW/anaconda3/envs/Hisense/bin/python', '-u', 'Run.py', '--local_rank=3']' returned non-zero exit status 1.
</code></pre>
<p>Could anyone tell me what happend?</p>

username: mrshenli
cooked: <p>Can you share a minimum repro of <code>train_net.py</code>, especially how you call <code>init_process_group</code> and <code>DistributedDataParallel</code> ctor?</p>
<p>BTW, could you please add a “distributed” tag for future <code>torch.distributed</code>-related posts? So that the PT distributed team can get back to you promptly.</p>

username: kaimss
cooked: <p>Thank you very much, I finally find the error occurs just because I don’t have permission to modify the file. Thanks again!</p>
Mobile GPU support on Android

username: saeed_masoomi
cooked: <p>I know that currently there isn’t any gpu support for android platforms(am I right?) but is there any plan that we can estimate that such support when will be in the release. And if you love pytorch(like me) and use pytorch for training, how you deploy your application on mobile devices with gpu support?(like torch-&gt;tflite or torch-&gt;onnx-&gt;coreML)</p>
IndexError when indexing Tensor (Iterating through DataLoader)

username: nxrmrz
cooked: <p>Hi, very new to Pytorch and this forum generally. I hope i’m posting in the right thread, and/or using the right terminology. Please feel free to correct me if i’m wrong (very willing to learn)!</p>
<p>I’m trying to iterate through a DataLoader i’ve created for my validation dataset (batch size 16, sampler is Sequential Sampler). The original dataset I pass to the sampler isof length 22. The problem is, it’s raising an <strong>IndexError</strong></p>
<p>Printing the indices out, it seems the sample in index 1 is causing the error (see below)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/7/1739e19780dd18153f25a19aeb34a8b7d98fb259.png" data-download-href="https://discuss.pytorch.org/uploads/default/1739e19780dd18153f25a19aeb34a8b7d98fb259" title="Screen Shot 2020-06-26 at 1.03.33 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/7/1739e19780dd18153f25a19aeb34a8b7d98fb259_2_690x393.png" alt="Screen Shot 2020-06-26 at 1.03.33 AM" data-base62-sha1="3jsZujHblX1M9rgyaUHSQ7JIpTj" width="690" height="393" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/7/1739e19780dd18153f25a19aeb34a8b7d98fb259_2_690x393.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/7/1739e19780dd18153f25a19aeb34a8b7d98fb259_2_1035x589.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/7/1739e19780dd18153f25a19aeb34a8b7d98fb259.png 2x" data-dominant-color="3A3B3C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-06-26 at 1.03.33 AM</span><span class="informations">1086×620 69.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Any hints on how to fix, or insight as to what the error might be? Thanks!</p>

username: ptrblck
cooked: <p>If your <code>Dataset</code> has the length 22, then the last index should be 21, which would also explain the <code>IndexError</code>.</p>
<p>Could you check the length of the <code>Dataset</code> via <code>print(len(dataset))</code>? I assume it would return a value which is larger than 22.<br>
In that case, check the <code>Dataset.__len__</code>  method and make sure you are returning the right value for the length. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
<p>Let me know, if this helps.</p>

username: nxrmrz
cooked: <p>Thank you so much for your help! It turns out the issue wasn’t that, but my loading of a different-sized data-set (instead of the intended one) into DataLoader. Very rookie mistake, but taught me to be more careful next time!</p>
Numerically unstable Cholesky?

username: cagatayyildiz
cooked: <p>Hi,</p>
<p>I’m trying to sample from a simple Gaussian process (without a diagonal jitter term). It seems that torch’s and numpy’s multivariate Gaussian implementations are slightly different:</p>
<pre><code class="lang-auto">import numpy as np, torch
dx = 0.1
N = 10
x = torch.arange(N).view([-1,1]) * dx
mean = torch.zeros(N)
dist = torch.cdist(x, x)
K = torch.exp(-0.5 * dist**2)
s = torch.distributions.MultivariateNormal(mean,K).sample() # error: cholesky_cpu: U(8,8) is zero, singular U.
s = np.random.multivariate_normal(mean.numpy(), K.numpy()) # executes
</code></pre>
<p>Both numpy and torch arrays are float32.</p>
<p>I’m using anaconda3 2019.07 on my Mac 10.12.6.<br>
Python 3.7.3<br>
Numpy 1.16.4<br>
Torch 1.4.0</p>
Complex valued tensors

username: a_d
cooked: <p>Is there any way we could have complex-valued tensors in PyTorch… similar to dtype.complex in tensorflow…<br>
If nit is there anyway I could implement it?</p>

username: Nikronic
cooked: <p>Hi,</p>
<p>Actually the work is in progress but there are some work arounds by third party libraries as you can find in this post:</p>
<aside class="quote" data-post="2" data-topic="60920">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/pytorch-with-complex-tensors/60920/2">Pytorch with complex tensors</a> 
  </div>
  <blockquote>
    Hi, 
You can check this issue to see the internal work on complex tensors: <a href="https://github.com/pytorch/pytorch/issues/755" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/issues/755</a> 
Also third party libraries might give some partial support for complex tensors. A quick search points to <a href="https://github.com/williamFalcon/pytorch-complex-tensor" rel="noopener nofollow ugc">https://github.com/williamFalcon/pytorch-complex-tensor</a> for example.
  </blockquote>
</aside>

<p>Bests</p>

username: a_d
cooked: <p>Will check this out, thanks a lot.</p>

username: albanD
cooked: <p>Hi,</p>
<p>We are actually adding this right now.<br>
Complex dtypes already exist (complex64 and complex128). Basic ops are working fine.<br>
We have first draft of the autograd support with a limited set of functions right now.<br>
Note that this will be in the upcoming 1.6 release.</p>
<p>Was there some specific features you were looking for?</p>

username: a_d
cooked: <p>Thank you for replying <a class="mention" href="/u/alband">@albanD</a><br>
Are they avaliable in the nightly builds ?<br>
I was just looking to implement the paper deep complex networks, nothing specific though…<br>
from there I thought would be cool to have complex backprop, gradients, optim etc etc…</p>

username: albanD
cooked: <p>Yes everything that will be in 1.6 is already in the nightly build.</p>
Detecting when Pytorch will run out of memory

username: adi8862
cooked: <p>I’m currently trying to detect in advance if a particular model + gpu + PyTorch version combination will run into an out-of-memory issue without actually running the model on the device.</p>
<p>torch.cuda.max_memory_allocated() is working well to verify if I can correctly predict the minimum memory requirements of the model tensors on the GPU.</p>
<p>However I was wondering if there was any way to deterministically predict the overhead imposed by PyTorch + CUDA?</p>

username: ptrblck
cooked: <p>I don’t think there is a reliable way to calculate the real GPU memory usage without running the code.<br>
Depending on your hardware, CUDA version, PyTorch version etc. the CUDA context would need a different memory footprint. While the difference might be small, this would already add some version dependency. Also, if you are using cudnn (and maybe other libs), the memory usage could be non-deterministic for different cudnn versions (e.g. due to updated heuristics).<br>
Especially if you are using <code>torch.backends.cudnn.benchmark =True</code> to profile different algorithms and select the fastest one.<br>
For each new input shape, this benchmarking mode will be executed and might discard algorithms, which would need too much memory and would this not fit in the available workspace.</p>
<p>However, if you precompute the CUDA context and use the deterministic cudnn mode, there would still be memory fragmentation, so that the theoretically calculated memory requirement might still be lower than the actual one.</p>

username: adi8862
cooked: <p>Thanks a lot for confirming this. The data did seem to suggest some amount of non-deterministic behaviour.</p>
Why torch.take is tremendously slower than torch.index_select with two reshapes?

username: s1ddok
cooked: <p>I have a huge model where I have 2D look-up-table inside. The model itself is a huge gan so that LUT performance is basically neglected.</p>
<p>previously, I had it implemented like this:</p>
<pre><code class="lang-python">flat_input = input.reshape(-1)
output = torch.index_select(lut, 0, flat_input).reshape(input.shape)
</code></pre>
<p>I thought that it is too many ops and introduce a cleaner version:</p>
<pre><code class="lang-python">output = torch.take(lut, input)
</code></pre>
<p>And it made my model 30% slower, while before this change, that lut op was basically neglectable in terms of performance impact.</p>
<p>Can somebody explain the nature of such a slowdown?</p>

username: ptrblck
cooked: <p>Could you post the shapes of both tensors, so that we could profile it?<br>
Using these shapes:</p>
<pre><code class="lang-python">x = torch.randn(100, 100, 100)
idx = torch.randint(0, x.nelement(), (1000,))
</code></pre>
<p>yield approx. the same timing for CPU and CUDA runs in the latest nightly binary.</p>

username: s1ddok
cooked: <p>Thanks for the reply!</p>
<p>The shapes grow progressively from 10, 1024, 32, 32 to 10, 1024, 512, 512. And lut is of size 38.</p>
<p>I train with CUDA and I think that backprop is to blame here, CPU inference is comparable in my experience as well.</p>
Memory required for calculating nuclear norm

username: JayHan
cooked: <p>Hi!<br>
I’m dealing with the nuclear norm, torch.norm(…, p=“nuc”).<br>
I know that I’m using very large matrix, but why the required memory doesn’t change where the first dimension size changes??<br>
The simple code result image is as below.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/4/f4f3d6518b666ba970bc1c8d43af4fc65362e298.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f4f3d6518b666ba970bc1c8d43af4fc65362e298" title="SharedScreenshot"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f4f3d6518b666ba970bc1c8d43af4fc65362e298_2_690x282.jpeg" alt="SharedScreenshot" data-base62-sha1="yWWV8pilERlMChQyxMsAc2K61pm" width="690" height="282" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f4f3d6518b666ba970bc1c8d43af4fc65362e298_2_690x282.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/f/4/f4f3d6518b666ba970bc1c8d43af4fc65362e298.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/4/f4f3d6518b666ba970bc1c8d43af4fc65362e298.jpeg 2x" data-dominant-color="1B1B1B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">SharedScreenshot</span><span class="informations">981×402 119 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also, while I’m using tensor at GPU using CUDA, I think that calculating the nuclear norm is only available on CPU, is it right???<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/2/22c79f268677736e30feda366653163ba6c492ff.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/22c79f268677736e30feda366653163ba6c492ff" title="d"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22c79f268677736e30feda366653163ba6c492ff_2_690x103.jpeg" alt="d" data-base62-sha1="4XFV89rna7IHxcdOSARrXjUgILd" width="690" height="103" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22c79f268677736e30feda366653163ba6c492ff_2_690x103.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/2/2/22c79f268677736e30feda366653163ba6c492ff.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/2/22c79f268677736e30feda366653163ba6c492ff.jpeg 2x" data-dominant-color="191919"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">d</span><span class="informations">976×147 40.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Found param roberta.embeddings.word_embeddings.weight with type torch.FloatTensor, expected torch.cuda.FloatTensor

username: Nawel
cooked: <p>Hi, I am just trying apex for the first time and running into the following error when training camembert using fast-bert:</p>
<p>Found param roberta.embeddings.word_embeddings.weight with type torch.FloatTensor, expected torch.cuda.FloatTensor.</p>
<p>I also get this indication, but i still don’t know how to deal with it:</p>
<p>When using amp.initialize, you need to provide a model with parameters<br>
located on a CUDA device before passing it no matter what optimization level<br>
you chose. Use model.to(‘cuda’) to use the default device.</p>
<p>PS: I don’t have an Nvidia gpu so i didn’t install cuda</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Nawel" data-post="1" data-topic="88046">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/ecd19e/48.png" class="avatar"> Nawel:</div>
<blockquote>
<p>PS: I don’t have an Nvidia gpu so i didn’t install cuda</p>
</blockquote>
</aside>
<p>Unfortunately, you won’t be able to use mixed-precision training without a GPU.<br>
The error message points towards an expected <code>CUDATensor</code>, while a <code>CPUTensor</code> was found.</p>
<p>That being said, once you get access to a GPU, we recommend to try out the native mixed-precision support from the current master branch or the nightly binaries as described <a href="https://pytorch.org/docs/master/amp.html">here</a>. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: Nawel
cooked: <p>This is my first experience, how do I get access to a GPU ? <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>
<p>Do you think install cuda on an none  nvidia provided gpu machine would fix it ?<br>
Thank you</p>

username: ptrblck
cooked: <p>No, you would need to have a GPU installed in your machine in order to run CUDA code.</p>
<p>If you currently don’t have a GPU device, you could use Google Colab or any other cloud service, which provides GPUs.<br>
If I’m not mistaken, you can use a GPU in Colab for a specific time frame (12h?) until the runtime disconnects.</p>
[onnx export] How to give a name to intermediate output tensors?

username: andrew.yang
cooked: <p>I used the method of “torch.onnx.export” to convert into onnx format like the below codes<br>
intermediate tensor name was given by the sequence number.<br>
Is there a way to give a name string instead of a auto generated sequence?</p>
<pre><code class="lang-auto">class ExModel(nn.Module):
    def __init__(self):
        super(ExModel, self).__init__()
        out_planes = 6
        self.conv = nn.Sequential(
            nn.Conv2d(3, out_planes, 3, 1, 1, groups=1, bias=False),
            nn.BatchNorm2d(out_planes),
            nn.ReLU(inplace=True)
        )

    def forward(self, x):
        return self.conv(x)

def convert_onnx_named(m):
    input_names = ["input_img", "in_1", "in_2"]
    output_names = ["o1"]
    dummy_input = torch.randn(1, 3, 224, 224)
    torch.onnx.export(m, dummy_input, "exmodel.onnx", verbose=True,
                      input_names=input_names, output_names=output_names)

m = ExModel()
convert_onnx_named(m)
</code></pre>
<p>the output of converting into onnx.</p>
<pre><code class="lang-auto">graph(%input_img : Float(1, 3, 224, 224),
      %in_1 : Float(6, 3, 3, 3),
      %in_2 : Float(6),
      %conv.1.bias : Float(6),
      %conv.1.running_mean : Float(6),
      %conv.1.running_var : Float(6),
      %conv.1.num_batches_tracked : Long()):
  %7 : Float(1, 6, 224, 224) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%input_img, %in_1), scope: ExModel/Sequential[conv]/Conv2d[0] # /root/.pyenv/versions/3.7.1/lib/python3.7/site-packages/torch/nn/modules/conv.py:340:0
  %8 : Float(1, 6, 224, 224) = onnx::BatchNormalization[epsilon=1e-05, momentum=0.9](%7, %in_2, %conv.1.bias, %conv.1.running_mean, %conv.1.running_var), scope: ExModel/Sequential[conv]/BatchNorm2d[1] # /root/.pyenv/versions/3.7.1/lib/python3.7/site-packages/torch/nn/functional.py:1656:0
  %o1 : Float(1, 6, 224, 224) = onnx::Relu(%8), scope: ExModel/Sequential[conv]/ReLU[2] # /root/.pyenv/versions/3.7.1/lib/python3.7/site-packages/torch/nn/functional.py:911:0
  return (%o1)
</code></pre>
<p>I want to give a name ‘in_1’ for ‘%7’ and ‘in_2’ for ‘%8’.</p>

username: Liming
cooked: <p>Hi <a class="mention" href="/u/andrew.yang">@andrew.yang</a> , how did you solve it? Thanks.</p>
Help with double headed NN

username: bfeeny
cooked: <p>I have images I wish to predict on, as well as some array data.  So I have a Net I am using that looks like this:</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self, arch, n_meta_features: int):
        super(Net, self).__init__()
        self.arch = arch
 
        self.arch._fc = nn.Linear(in_features=1280, out_features=500, bias=True) # B1


        self.meta = nn.Sequential(nn.Linear(n_meta_features, 500),
                                  nn.BatchNorm1d(500),
                                  nn.ReLU(),
                                  nn.Dropout(p=0.2),
                                  nn.Linear(500, 250),  # FC layer output will have 250 features
                                  nn.BatchNorm1d(250),
                                  nn.ReLU(),
                                  nn.Dropout(p=0.2))

        self.final = nn.Linear(500 + 250, 1)
        
    def forward(self, inputs):
        x, meta = inputs
        cnn_features = self.arch(x)
        meta_features = self.meta(meta)
        features = torch.cat((cnn_features, meta_features), dim=1)
        output = self.final(features)
        return output
</code></pre>
<p>So you can see I send the images through my CNN (EffiicientNet), and then my metadata through a series of Linear models.  They are combined at the end with torch.cat() and then sent through a final Linear model.  The result is binary.  There is no sigmoid on the result because I am using BCEWithLogitsLoss.  I do a sigmoid when I get my prediction back from the model.  Here is basic train sequence:</p>
<pre><code class="lang-auto">for x, y in train_loader:
            x[0] = torch.tensor(x[0], device=device, dtype=torch.float32)
            x[1] = torch.tensor(x[1], device=device, dtype=torch.float32)
            y = torch.tensor(y, device=device, dtype=torch.float32)
            optim.zero_grad()
            z = model(x)
            loss = criterion(z, y.unsqueeze(1))
            loss.backward()
            optim.step()
            pred = torch.round(torch.sigmoid(z))  # round off sigmoid to obtain predictions
            correct += (pred.cpu() == y.cpu().unsqueeze(1)).sum().item()  # tracking number of correctly predicted samples
            epoch_loss += loss.item()
</code></pre>
<p>I am looking for feedback if this looks sound or not?  Perhaps there are better ways to handle these two types of data?  I definitely have some tweaking to do on the models I am using, right now all my probabilities are heavily skewed toward 0.</p>

username: ptrblck
cooked: <p>Your model architecture as well as the training look alright.</p>
<p>Note that <code>torch.round(torch.tensor(0.5))</code> would round to <code>0.</code>, since it’s sticking to the numpy rounding implementation, if I’m not mistaken. So you might want to apply a threshold, if you want to round <code>0.5</code> to <code>1.0</code>.</p>

username: bfeeny
cooked: <p>Thank you, I did not know that.  This is a model being scored with ROC AUC, and so accuracy isn’t really important, just a bit informative.</p>