akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Torchscript does not take over the weights

DSL: <p>Hello together,<br>
I have successfully converted my model to “script”. Only when I test my model now, the weights do not match, so my program does not work. Does anyone have any idea?</p>
<pre><code class="lang-auto">import argparse

import os
from os import path
from unifuse import UniFuse
import torch


def load_model(model, args):
    if args.v:
        print("Load Model")
    model_dict = model.state_dict()
    pretrained_dict = torch.load(args.model_path)
    pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
    model_dict.update(pretrained_dict)
    return model

def convert(args):
    model = UniFuse(18, args.height, args.width, True, args.max_depth, "cee", True)
    model = load_model(model, args)
    if args.cuda:
        model.cuda()
        if args.v:
            print("Set model GPU")
    else:
        model.cpu()
        if args.v:
            print("Set model CPU")

    my_script_module = torch.jit.script(model)
    save(my_script_module, args)
    if args.v:
        print(my_script_module.code)

def save(script, args):
    name = "wrapped.pt"
    file = os.path.join(args.output_path, name)
    torch.jit.save(script, file)

def main():
    parser = argparse.ArgumentParser(description="Convert the trained model with torchscript into a binary format")
    # path to the data
    parser.add_argument("--model_path", default="/model.pth", type=str, help="Path to the trained model")
    parser.add_argument("--output_path", default=".", type=str, help="Output path where it will be stored")

    # model params
    parser.add_argument("--num_layers", default=18, type=int, help="Number on layers will be needed")
    parser.add_argument("--height", default=512, type=int, help="Define the height of the input and output Image")
    parser.add_argument("--width", default=1024, type=int, help="Define the width of the input and output")
    parser.add_argument("--max_depth", default=10.0, type=float, help="Define the max depth")
    parser.add_argument("--cuda", default=True, action="store_true", help="if set, use cuda")

    # show the logs
    parser.add_argument("-v", default=True, action="store_true", help="if set, shows the logs")

    args = parser.parse_args()

    if not path.exists(args.model_path):
        exit("Model path didnt exists!")
    if not path.exists(args.output_path):
        exit("Output path didnt exists!")

    convert(args)

if __name__ == "__main__":
    main()
</code></pre>

ptrblck: <p>How are you comparing the models after storing them?<br>
Based on your code snippet it seems you are filtering out some keys from the <code>state_dict</code> in <code>load_model</code>? If so, did you checked that this doesn’t create any mismatch in the comparison?</p>

DSL: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a>,</p>
<p>Thx for your answer, but the comparison was right. I just forgot the command <code> model.load_state_dict(model_dict)</code> and after this all worked fine.<br>
Thx for your interest and help.</p>
Pack_padded_sequence and tensorboard

Leo2: <p>I’m having an issue with tensorboard summarywriter and nn.pack_padded_sequence. In the pack_padded_sequence docs it says the ‘lengths’ can be either a tensor(which MUST be on the cpu) or a list object.</p>
<p>Since i’m using a GPU , I use pack_padded_input in my model like so</p>
<blockquote>
<p>packed_input = torch.nn.utils.rnn.pack_padded_sequence(embedding_batched_sentences, seq_lengths_clamped.cpu().numpy(), batch_first=True,enforce_sorted=False)</p>
</blockquote>
<p>When I run my model without any use of summarywriter it works just fine as expected. No errors at all.</p>
<p>However when I try to add_graph of my model to summarywriter. It checks my forward method and i get the following error</p>
<blockquote>
<p>→ 249 _VF._pack_padded_sequence(input, lengths, batch_first)<br>
250 return _packed_sequence_init(data, batch_sizes, sorted_indices, None)<br>
251</p>
<p>RuntimeError: ‘lengths’ argument should be a 1D CPU int64 tensor, but got 0D cpu Long tensor</p>
</blockquote>
<p>this error output occurs further down from</p>
<blockquote>
<p>→ 286 trace = torch.jit.trace(model, args)</p>
</blockquote>
<p>Anyone know what;s going on here?</p>
Remove "device='cuda:0' " from tensor

cbd: <p>In below code, when tensor is move to GPU and if i find max value then output is " tensor(8, device=‘cuda:0’)". How should i get only value (8 not 'cuda:0) in assign variable?</p>
<p>import torch<br>
x=torch.arange(0,9)<br>
print("cpu ",x)<br>
x=x.to(0)<br>
print("gpu ",x)<br>
y=torch.max(x)<br>
print("max is ",y)</p>

Sayed_Nadim: <p>use <code>y.item()</code> instead of <code>y</code>. For example,</p>
<pre><code class="lang-auto">import torch

x = torch.arange(0, 9)
print("cpu ", x)
x = x.to(0)
print("gpu ", x)
y = torch.max(x)
print("max is ", y.item())
</code></pre>
<p>will give you,</p>
<pre><code class="lang-auto">cpu  tensor([0, 1, 2, 3, 4, 5, 6, 7, 8])
gpu  tensor([0, 1, 2, 3, 4, 5, 6, 7, 8], device='cuda:0')
max is  8
</code></pre>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji only-emoji" alt=":slight_smile:"></p>
Does a docker container need to use the exact same libcuda version of the host server?

RicardoFunke: <p>I’m trying to run an application that uses torch and tensorflow inside a docker container but I’m getting this error message:</p>
<p>“kernel version 450.66.0 does not match DSO version 450.142.0 – cannot find working devices in this configuration”</p>
<p>The cuda-compat-11-0 version inside the container is 450.142.0 but the host driver version is 450.66.0.</p>
<p>As I can’t find cuda-compat-11-0 version 450.66.0, I tried other versions around it but I got the same error.</p>
<p>I’m not able to change the Host driver, so my question is:</p>
<p><strong>Do I really have to install the exact same version on the docker container in order to make it work, or am I doing something wrong?</strong></p>
<p>Please help me understand this issue. Thanks in advance!</p>

J_Johnson: <p>Are you using the Nvidia Container Toolkit? If not, I think this may resolve the issue for you. Please let me know how it works out.</p>
<p><a href="https://link.medium.com/WxBZuBTpgkb" class="inline-onebox" rel="noopener nofollow ugc">How to Properly Use the GPU within a Docker Container | by Jacob Solawetz | Towards Data Science</a></p>
Learning rate modulation and gradient descent

Martin_Barry: <p>Hello !</p>
<p>I was trying to implement a model these last few days and I stumbled across problem that I can not solve:<br>
Here is the newtork (simplified)<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/4/d4fa4eaeb600aa09efb196b3583257d97a2557d9.png" alt="Screen Shot 2021-10-07 at 17.00.15" data-base62-sha1="uo5vijZxjyWGos25sNkUSKTmuIV" width="420" height="417"></p>
<p>The blue part has its <code>requires_grad</code> set to False, and I manually update the weight as:</p>
<p><code>self.pred.weight.data  = self.pred.weight.data + 0.1 * modulation * deltaW[0,0].T</code><br>
and<br>
<code>modulation = torch.mean(out_modulation)</code>.</p>
<p>To make it clear I run a simulation of N steps and every time step I update the weights manually of the online layer.<br>
After all these steps I want to minimize the overall error by doing a backprop (autograd) over the red area. where the error is the sum of the prediction error in blue area (i.e. the error does not explicitly depend on the modulation population).</p>
<p><strong>The problem is</strong> since the error does not depend explicilty of the red area, pytorch can not estimate a gradient. Is there a way to make the graph understand that the manual updates is dependent of the modulation population ?</p>
<p>full forward code:</p>
<p>`    def forward(self, previous, observation,hidden_modulation):</p>
<pre><code>    prediction = self.pred(previous)  #linear unit
    prediction_error = observation - prediction
    
    out_modulation, hidden_modulation = self.surprise(torch.abs(prediction_error), hidden_modulation) #LSTM
    modulation = torch.mean(torch.sigmoid(out_modulation))
    deltaW = (
            torch.einsum(
                "bij,bik-&gt;bijk",
                previous,
                prediction_error))
    self.pred.weight.data = self.pred.weight.data + modulation * deltaW[0,0].T #online update
     self.pred.weight.data[self.pred.weight.data &lt; 0] = 0
    return prediction_error, hidden_pred, hidden_surprise,torch.relu(prediction),modulation`
</code></pre>

soulitzer: <p>What happens when you don’t use <code>.data</code> here?</p>

Martin_Barry: <p>thanks for you reply.  without data I can not assign a tensor</p>
<p><code>cannot assign 'torch.FloatTensor' as parameter 'weight' (torch.nn.Parameter or None expected)</code></p>
Large linear increase in memory when predicting with torchvision.detection model in for loop

Oliver: <p>Hi all,</p>
<p>I have the following code</p>
<pre><code class="lang-auto">model = detection.fasterrcnn_resnet50_fpn(pretrained=True, progress=True,pretrained_backbone=True).to(DEVICE)
for i in tqdm(range(train.shape[0])):
    image = cv2.imread(train_img_paths[i])
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    image = image.transpose((2, 0, 1))
    image = image / 255.0
    image = np.expand_dims(image, axis=0)
    image = torch.FloatTensor(image)
    image = image.to(DEVICE)
    predictions = model(image)[0]
</code></pre>
<p>the images aren’t very big, between 200 and 800 pixels.</p>
<p>after about 30 images I reach 16GB of memory.<br>
Is there a way to avoid that?</p>
<p>thank you for any tips and help, apologies if this is silly code!</p>

Oliver: <p>Someone on stackoverflow provided me with a very simple solution:</p>
<p>wrap everything in<br>
<code>with torch.no_grad():</code></p>
Concatenate ":" to array values

cbd: <p>I have the tensor <strong>x=[ [0,1],[2,3] ]</strong>. I want to concatenate <strong>“:”</strong> with tensor x[0] and x[1] such that output will be <strong>x=[ [:,0,1],[:,2,3] ]</strong>. I want to use x as indexing.</p>

ptrblck: <p>I don’t quite understand the use case. Could you post an example input tensor and how the desired indexing would work to create the desired output tensor?</p>

cbd: <p>I liked your post <a href="https://discuss.pytorch.org/t/extract-batchwise-multiple-coordinates-with-pytorch-functions-similar-to-tf-gather-nd/28097/5" class="inline-onebox">Extract batchwise multiple coordinates with pytorch functions (similar to tf.gather_nd) - #5 by ptrblck</a>.</p>
<p>Thanks.</p>
ERROR: could not find a version that satisfies the requirement torch==1.9.1+cu111

ZeeCur: <p>Hi I am having a problem when I type</p>
<p>Pip install -requirement.txt</p>
<p>ERROR: could not find a version that satisfies the requirement torch==1.9.1+cu111(from versions 0.1.2, 0.1.2post1, 0.1.2post2<br>
ERROR: No matching distribution found for torch==1.9.1+cu111</p>
<p>I use python 3.10</p>

ptrblck: <p>It seems your <code>pip</code> environment can only find old versions, so you might need to update it.</p>
Is there any difference between AdaptiveAvgPool and mean()?

jh_shim: <pre><code class="lang-auto">x = torch.rand(2, 3, 128, 128)
F.adaptive_avg_pool2d(x, output_size=1).flatten(start_dim=1)
</code></pre>
<pre><code class="lang-auto">x = torch.rand(2, 3, 128, 128)
x.mean(dim=(2, 3))
</code></pre>
<p>Is there any difference between two above?</p>
<p>I think both are same and the second one is more simple and understandable.<br>
But I assume there is any reason why most people are using first one.</p>

ptrblck: <p>There is no difference in the posted code snippets and the former one will also use the latter approach internally for an <code>output_size</code> of 1.<br>
The former allows you to use other output sizes and is thus more flexible. Also, you could use it as an <code>nn.Module</code> without creating a custom module for the <code>mean</code> operation.</p>
Conv2d: Sharing filters among input channels

Ruokai: <p>Hi! My question is about sharing filters in nn.Conv2d/nn.f.conv2d.</p>
<p>Say I have a 3 channel input shape: (1,3,4,4). And I have a (1,2,4,4) filter. Here the filter size is (batch_size, number of filters, H, W). What I hope to achieve is that let the first filter to convlve with each of the three channels and get three (1,1) results. Then I repeat this with the second filter, get three (1,1) results again. In the end, I am hoping to get some results like (1,2*3,1,1).</p>
<p>Are there some pytorch built-in functions that can achieve this? Or I need to use for loops on both input channels and weights to achieve this.</p>
<p>The reason why I want to do this is I am trying to manually backpropagate the gradients through conv2d. I saw some articles prove that the dL/dW is the convolution between X and dL/dH from previous layer. If there are better ways to manually achieve backpropagation on convolution layer, please let me know!</p>
<p>Thank you very much! And this is my first time making a post, please forgive me for potential format mistakes!</p>

ptrblck: <aside class="quote no-group" data-username="Ruokai" data-post="1" data-topic="133828">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ruokai/48/42864_2.png" class="avatar"> Ruokai:</div>
<blockquote>
<p>Here the filter size is (batch_size, number of filters, H, W).</p>
</blockquote>
</aside>
<p>The filter shape does not contain the batch size, as layers are created independently from their batch size. <code>nn.Conv2d</code> filters use <code>[out_channels, in_channels, height, width]</code> instead.</p>
<p>If I understand your use case correctly, you would like to use a grouped convolution as seen here:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(1, 3, 4, 4)
conv = nn.Conv2d(3, 6, 4, groups=3)
out = conv(x)
print(out.shape)
&gt; torch.Size([1, 6, 1, 1])
</code></pre>
Can I use torchvision detection models on batches of images with varying sizes?

Oliver: <p>Hi all,</p>
<p>I would like to run one of the detection models on a batch from a dataset.<br>
I am fairly new to torch (though I like it so far and am very comfortable with tensorflow), I tried to use one of the faster-rcnn images to predict on a batch of images.</p>
<p>when calling model(dataloader) I get the following error:</p>
<pre><code class="lang-auto">RuntimeError: stack expects each tensor to be equal size, but got [3, 960, 720] at entry 0 and [3, 960, 1021] at entry 2
</code></pre>
<p>Is the model complaining or the dataloader? It seems like it’s the dataloader?<br>
Can it not work with varying image sizes?</p>
<p>thank you for hints and help!</p>

Preetham_R_Patlolla: <p>CNN expects uniform image sizes. If you are feeding your model with varying input images, it will result in feature vectors of varying sizes. You can address this either by resizing or padding the image.</p>

Oliver: <p>Hi Preetham,<br>
thanks, the detection models work with any sizes. I actually found a solution for the above, it was indeed the dataloader and using a custom_collate function worked.<br>
Though forks (i.e. parallel execution) doesn’t with my custom collate_fn, which is a bit of a shame.</p>
Gradients for DQL

Baboon: <p>Hi, I am trying to make a simple AI to learn to <em>walk down halls</em>. What is to its left right above and below are its inputs. and the output actions are move left right up or down. The reward System is as follows.</p>
<ul>
<li>-1 if tries to walk into a wall</li>
<li>-1 if walks over a space already walked over</li>
<li>+1if walks over a space not yet walked over</li>
</ul>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/0/e00e23c53f3ae72122947fc104d4957a5ba98370.png" alt="Hallway Walk" data-base62-sha1="vY5flOcv6o7LmUTsz0fQ1LTrlle" width="330" height="464"></p>
<p>I am trying to use the following loss function based of a TRPO algorithm to train the neural network:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/b/bbc093473713e6a529f8e040df8494544a5036e8.png" alt="image" data-base62-sha1="qMVSpJEMuTcdqCK3zz6ZOhurmru" width="348" height="36"></p>
<p>I am able to calculate the loss number. However I am not able to run loss.backwards with out the following error:</p>
<p><em>element 0 of tensors does not require grad and does not have a grad_fn</em></p>
<p>Below is the code I am using:</p>
<pre><code>    model.opt.zero_grad()
    qEval = model.forward(stateBatch)  # Gets Gradients
    qEval = torch.stack([torch.tensor([a.max()]) for a in qEval])

    qValsNext = model.forward(nextStateBatch)
    qValsNext = torch.stack([torch.tensor([a.max()]) for a in qValsNext])

    loss = (rewardBatch + qValsNext - qEval)**2
    loss.backward()
    model.opt.step()
</code></pre>
<p>Thank you for the help!</p>
Gap between train and valid loss

prog_asker: <p>my dataset training is 30.000  and for testing 5000. I got a gap between training and valid</p>
<p>how can I solve this problem, please ? or it is not a problem ?</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/a/ca855953d2b7d61411c4a0bf8cf22965b8f2892e.png" alt="1" data-base62-sha1="sTA9aawY4WEu6fp9in3kLBbLeQ6" width="640" height="480"></p>
Is it possible to get the evaluated value of f(x) from torch.autograd.functional.jacobian(f,x)

SakuraTea: <p>Since the function value f(x) is evaluated during the process of getting the jacobian matrix, is there a way to access it?</p>
<p>Thanks for the help.</p>

soulitzer: <p>Hmm, don’t think it is possible right now with the current API, but I think it is a valid request especially if you are setting vectorized=True. Feel to free to file an issue for this.</p>
Strange behavior of functional.interpolate during training

James_Lee: <p>Hi. This is the line of code that leads to strange behavior.</p>
<pre><code class="lang-auto">        labels1 = functional.interpolate(labels, size=24, mode='bilinear')

        print("### labels.long().min()", labels.long().min())
        print("### labels.min()", labels.min())

        print("### labels1.long().min()", labels1.long().min())
        print("### labels1.min()", labels1.min())
</code></pre>
<p>I got,</p>
<pre><code class="lang-auto">### labels.long().min() tensor(-1, device='cuda:2')
### labels.min() tensor(-1., device='cuda:2')
### labels1.long().min() tensor(0, device='cuda:2')
### labels1.min() tensor(0., device='cuda:2')
</code></pre>
<p>What’s wrong with functional.interpolate here? Why did the labels.long().min() differ from labels1.min()? Thanks.</p>

ptrblck: <p>Double post from <a href="https://discuss.pytorch.org/t/assertion-t-0-t-n-classes-failed-error/133794/6">here</a>.</p>
Extra class predictors for mask-rcnn

gpldecha: <p>Hello community, this is my first time posting here <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>The <a href="https://pytorch.org/vision/stable/_modules/torchvision/models/detection/mask_rcnn.html" rel="noopener nofollow ugc">mask-rcnn</a>  implementation outputs a masks, bounding boxes and class labels. For my use-case I would like to predict multiple attributes besides a class label (three to be specifc). For instance if I was detecting bikes I would like to further predict (sport bike, etc…).  I can foresee a three approaches:</p>
<ul>
<li>Keep single class/attribute prediction per bbox, this would involve creating classes from all the possible combinations of the attributes (bike_sport_red, bike_leisure_blue, etc…) which I don’t find very appealing (lack of scalability)</li>
<li>Modify/reimplement <a href="https://pytorch.org/vision/stable/_modules/torchvision/models/detection/mask_rcnn.html" rel="noopener nofollow ugc">mask-rcnn</a> to accomodate for multiple categories per bounding box. I am not sure how much effort this would entail, but I am wary of tinkering with it.</li>
<li>Create my own Module which takes as member variable mask-rcnn module and connect the feature vector layers to my own classifiers.</li>
</ul>
<p>Talking out loud the last option seems the best approach. I would appreciate any feedback or guidance.<br>
Also if anyone knows of a paper or implementation which does this I would greatly appreciate knowing about them.</p>
Preprocess input for pretrained networks using batchnorm

blrm: <p>Hi all,<br>
I am trying to preprocess input images in the model using batch_norm instead of normalizing the input tensor using mean/std values. I have the following code in python which works as intended:</p>
<pre>class PreProcess(torch.nn.Module):
    def __init__(self,mean,var):
        super(PreProcess, self).__init__()
        self.mean = torch.FloatTensor(mean)
        self.var = torch.FloatTensor(var)

    def forward(self, x):
        """
        In the forward function we accept a Tensor of input data and we must return
        a Tensor of output data. We can use Modules defined in the constructor as
        well as arbitrary operators on Tensors.
        """
        x = x.permute(0, 3, 1, 2)

        x = torch.nn.functional.batch_norm(x,
                                           self.mean,
                                           self.var,
                                           weight=None,
                                           bias=None,
                                           training=False,
                                           momentum=0.1,
                                           eps=1e-05)

        return x

def load_model(model_path, mean, std):
    model = torch.jit.load(model_path)
    var = [x*x for x in std]
    ppmodel = torch.nn.Sequential(PreProcess(mean, var), model)
    ppmodel.eval()
    return ppmodel
</pre>
<p>I want to do the same using libtorch in C++, so I wrote the following C++ code:</p>
<pre>struct PreProcessImpl : torch::nn::Module {
  torch::Tensor forward(torch::Tensor x) {
    x = at::permute(x, {0, 3, 1, 2});

    namespace F = torch::nn::functional;

    x = F::batch_norm(x,
                torch::ones({3}),
                torch::ones({3}),
                F::BatchNormFuncOptions().momentum(0.1).eps(1e-05).training(false));

    return x;
  }
};

TORCH_MODULE(PreProcess);
PreProcess pp;
auto ppmodule = torch::nn::Sequential(pp, torch::jit::load(modelFile));
</pre>
<p>However, it won’t compile because of this error:</p>
<pre>torch/include/torch/csrc/api/include/torch/nn/modules/container/sequential.h:250:8:
note: candidate function not viable: no known conversion from 'torch::jit::Module'
to 'torch::nn::AnyModule' for 1st argument
  void push_back(AnyModule any_module) {
       ^
</pre>
<p>Any ideas on how to do this in C++ using libtorch?</p>
Visualing the attention heatmaps from the beginner Tutorial?

Tupac: <p><a href="https://pytorch.org/tutorials/beginner/translation_transformer.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/translation_transformer.html</a></p>
<p>I am currently reviewing this tutorial, but I see that is it missing some things (as compared to the Tensorflow example: <a href="https://www.tensorflow.org/text/tutorials/transformer#create_the_transformer" class="inline-onebox" rel="noopener nofollow ugc">Modelo de transformador para compreensão de linguagem  |  Text</a></p>
<p>Basically, I would like to visualise the attention heatmaps at the encoder and decoder layers, but I am not sure which is the output for that? Anyway, I was hoping that someone had an implementation of it already, so that I could check it out.</p>
<p>And how could I use the test iters, for testing the model? Thank you!</p>
Pytorch multi class Classification where image belongs to multiple class

Veer_Maruthesh: <p>Hey, I would love to know how to implement a model which takes in a 3,224,224 chest X-ray image that can belong to multiple classes and classify the image correctly. Any help would be greatly appreciated.</p>

KFrank: <p>Hi Veer!</p>
<aside class="quote no-group" data-username="Veer_Maruthesh" data-post="1" data-topic="133795" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/veer_maruthesh/48/29898_2.png" class="avatar"> Veer_Maruthesh:</div>
<blockquote>
<p>image that can belong to multiple classes</p>
</blockquote>
</aside>
<p>You are asking about what is called  <em>multi-label,</em> multi-class<br>
classification problem.</p>
<p>Let me illustrate the typical use case with an example:</p>
<p>Let’s say that you have three classes: “pneumonia,” “artery blockage,”<br>
and “tumor,” and that a single image can display all, some, or none<br>
of these features – that is, it can belong to multiple classes at the<br>
same time.</p>
<p>The input to your model would typically be a batch of images, that is,<br>
a tensor of shape<br>
<code>[nBatch, nChannel = 3, height = 244, width = 244]</code>.  Your<br>
<code>target</code> – your ground-truth labels – would be a batch of multi-label<br>
labels of shape <code>[nBatch, nClass = 3]</code>, and each value in the <code>target</code><br>
tensor would (in the case of “hard” labels) be <code>0.0</code> if the image were not<br>
in the corresponding class and <code>1.0</code> if it were in the corresponding class.<br>
(In the “soft”-label case, these values would be numbers ranging from<br>
<code>0.0</code> to <code>1.0</code> that would represent the probability that the image were in<br>
the corresponding class.)  For a given image all, some, or none of the<br>
values along the <code>nClass</code> dimension could be <code>1.0</code>.  (In the soft-label<br>
case there would be no requirement that, for a given image, the values<br>
along the <code>nClass</code> dimension sum to <code>1.0</code>.)</p>
<p>The last layer of your model would typically be a <code>Linear</code> with<br>
<code>out_features = nClass</code>, you would interpret the output values as<br>
raw-score <em>logits</em> for the image being in each of the classes, and you<br>
would use <code>BCEWithLogitsLoss</code> as your loss criterion.</p>
<p>(For this kind of image classification problem, you would most likely<br>
use a <em>convolutional-neural-network</em> (CNN) architecture where the<br>
first several layers would be convolutions, only switching over to<br>
<em>fully-connected</em> <code>Linear</code> layers at the end.)</p>
<p>Best.</p>
<p>K. Frank</p>

Veer_Maruthesh: <p>Thank you so much. Very much appreciated.</p>
Optimise for loop in model forward

shashimalcse: <p>I am solving a problem using a deep learning model that generates a mask during the forward pass. Can someone help me to optimize these for loops?</p>
<pre><code class="lang-auto">            mask = torch.zeros(image.shape[0],1,224,224).cuda()
            for batch in range(image.shape[0]):
                for i in range(224):
                    for k in range(224):
                        arr = torch.tensor([k,i],dtype=torch.float32).cuda() - head_point[batch,:]
                        mask[batch,:,i,k] = torch.dot(arr,xy[batch,:])/(torch.norm(arr,p=2)*torch.norm(xy[batch,:],p=2))
</code></pre>
<p>This is the equation<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/a/1a9d78f8de8e288d0500ed276386d56306511d4f.png" data-download-href="https://discuss.pytorch.org/uploads/default/1a9d78f8de8e288d0500ed276386d56306511d4f" title="Screenshot 2021-10-10 at 07.20.53"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/a/1a9d78f8de8e288d0500ed276386d56306511d4f_2_690x119.png" alt="Screenshot 2021-10-10 at 07.20.53" data-base62-sha1="3NrNs0so2i88hBaFjCJz98pj7TF" width="690" height="119" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/a/1a9d78f8de8e288d0500ed276386d56306511d4f_2_690x119.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/a/1a9d78f8de8e288d0500ed276386d56306511d4f_2_1035x178.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/a/1a9d78f8de8e288d0500ed276386d56306511d4f_2_1380x238.png 2x" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2021-10-10 at 07.20.53</span><span class="informations">1582×274 16.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Speech-to-Text (STT) - Hidden layer representations of pretrained model

bilalhsp: <p>Hi, I need to use some pre-trained speech to text model like DeepSpeech and I want to access the hidden layer representations so preferably the model should be subclass of nn.Module. I tried to use Deep Speech but they have all the instruction and use case examples using CLI. I simply want to load model using python script and be able to access the hidden layers as I do inference.<br>
Transformers from Huggingface provides good interface for my problem but they only have models based on transformer architecture. I need some RNN or LSTM based model. Can someone share example code that allows me to load some pretrained model and do inference? (the model should subclass nn.Module).</p>
Machine learning text segmentation

julkhami: <p>I would like to segment a text and I am trying to approach ways of dealing with plaintext files where a paragraph is composed of lines of text with newline characters between each line. I believe standard rule-based segmentation regular expression functions would naturally think the end of a line must be the end of a segment; whereas if I first joined the text file on newlines it might join two lines which are separate but don’t have punctuation between them (like a title and a subtitle). Then the rule-based segmentation wouldn’t realise those are to be kept separate.</p>
<p>I think I need an algorithm which somehow knows if a line is a segment even if it’s not punctuated - it could be a title or a line of code, for example - and can join all lines that are part of a continuous paragraph, then segment them by sentences.</p>
<p>I am open to different strategies for this but the simplest that come to mind are:</p>
<ul>
<li>
<p>some very general model which has been trained on well-segmented documents and I can just give it the file without trying to consciously prepare the file in some way; or</p>
</li>
<li>
<p>a model which understands if a line of text is a continuation of the sentence in the previous line or if they are discursively/sententially separate.</p>
</li>
</ul>
<p>In either case, is it likely I could train such a model myself? I’d need to find a large source of relevant data somehow, wouldn’t I? Maybe if I can find some kind of original source code for the dumped plaintext of various documents, I could train the model to effectively back-generate the source code of plaintext. In the source code the text would be more structured, with markup tags.</p>
<p>In which case I guess it makes more sense to try to find a pre-trained model.</p>
<p>Does anyone know of any pre-trained model that has the kind of knowledge I describe here?</p>
<p>Or could an unsupervised algorithm potentially separate headers and lines of code from paragraphs of text? Maybe an unsupervised algorithm is quite good at identifying the structural components of a document: header, paragraph, etc?</p>
<p>Thanks very much to anyone who can help me figure out a way to approach this.</p>
Image gradient wrt to a class score

Monty_Hall: <p>I’m playing with image gradients and have seen they want to compute an image gradient with respect the raw class score.  I’ve seen, “ypred=model(batch_img) loss = ypred[range(len(labels)), labels].sum() loss.backward()”  and I’ve seen this “ypred=model(batch_img) loss = ypred[range(len(labels)), labels] loss.backward(torch.ones(labels.shape[0]))”  I know the first implementation is correct.  But I just want to make sure the second implementation is equivalent.  It seems like it’d be correct too.  I get the same image gradients - but I could be lucky.</p>

soulitzer: <p>Yes they are equivalent because, if no gradient is passed in to .backward when the output is a scalar, it is implicitly <code>Tensor(1.)</code>, and backprop through sum will into the same shape as the final output before you reduced it, which recovers <code>torch.ones(labels.shape[0]))</code>. Doing .sum() is slightly more efficient though because expand won’t materialize the entire ones tensor, it just creates a view. Or even better you could just do <code>loss.backward(tensor(1.).expand_as(labels.shape[0]))</code>.</p>
Apply the same function(like torch.topk) to a list of tensors

pyjhzwh: <p>Hi all,<br>
I am trying to get the topk for multi-dimensional tensors over the last dimension. The k value is tensor rather than a int. I was wondering if there is<br>
For example</p>
<pre><code class="lang-auto">a = torch.rand(2,3,4) # size(2,3,4)
b = torch.randint(0,4,(2,3))  #size(2,3)

# What I want to do, but slow
for i in  range(2):
  for j in range(3):
    indices = torch.topk(a[i,j,:], b[i,j])
</code></pre>
<p>Is there an efficient way to do this instead of for loop?</p>
<p>Thank you!</p>
How is broadcast implemented?

sunshine1: <p>We all know that broadcast is one of the most important tools in numpy and pytorch. In general, an array is broadcast automatically to align another array.  How is this done?</p>

googlebot: <p>essentially, with binary ops you have two read pointers, that would move at the same time in case of identical shapes. when one of the tensors is smaller, its read pointer is moved less often. more specifically, as you can only do the broadcasting when some dimension size is 1, one pointer is fixed while another iterates over a dimension. then they both move to a next dimension (next memory blocks) according to tensor strides, this is done from right to left.</p>
SummaryWriter error

Leo2: <p>I’m having an issue with tensorboard summarywriter and nn.pack_padded_sequence. In the pack_padded_sequence docs it says the ‘lengths’ can be either a tensor(which MUST be on the cpu) or a list object.</p>
<p>Since i’m using a GPU , I use pack_padded_input in my model like so</p>
<blockquote>
<p>packed_input = torch.nn.utils.rnn.pack_padded_sequence(embedding_batched_sentences, seq_lengths_clamped.cpu().numpy(), batch_first=True,enforce_sorted=False)</p>
</blockquote>
<p>When I run my model without any use of summarywriter it works just fine as expected. No errors at all.</p>
<p>However when I try to add_graph of my model to summarywriter. It checks my forward method and i get the following error</p>
<blockquote>
<p> → 249         _VF._pack_padded_sequence(input, lengths, batch_first)<br>
250     return _packed_sequence_init(data, batch_sizes, sorted_indices, None)<br>
251</p>
<p>RuntimeError: ‘lengths’ argument should be a 1D CPU int64 tensor, but got 0D cpu Long tensor</p>
</blockquote>
<p>this error output occurs further down from</p>
<blockquote>
<p> → 286             trace = torch.jit.trace(model, args)</p>
</blockquote>
<p>Anyone know what;s going on here?</p>
Minimum variance of binary matrix

tungdop2: <p>I have a matrix of 0 and 1, example:</p>
<pre><code class="lang-auto">X = 
[[1, 1, 0, 0],
[1, 0, 1, 1],
[0, 0, 1, 1],
[1, 1, 1, 1],
</code></pre>
<p>In each row, choose only an ‘1’ and leave orthers become 0, to get the same number of ‘1’ in each colunm. Basically, minimum variance after get sum by columns. Example: From X above, the answer is:</p>
<pre><code class="lang-auto">Y = 
[[1, 0, 0, 0],
[0, 0, 0, 1],
[0, 0, 1, 0],
[0, 1, 0, 0],
</code></pre>
<p>Each column in Y have a ‘1’. How can i do that? Sorry for my bad english.</p>
Why validation loss gets 'lost" for a while when validating on the same data set

apytorch: <p>I have a model that I’ve been slowly hyperparameter tuning with a large dataset (thousands of subjects). And seems to work just fine learning to generalize on unseen data.</p>
<p>But, for fun, today I tried training on just 1 or 10 subjects and setting the validation set to be the same as the training set (same 1 or 10 subjects)—to see if it could memorize the data. Obviously, the loss should be the same. However, that’s not what I see. The training and validation loss are identical before training. Then, for the first few epochs they both decrease, but diverge. Then, around 10 epochs in, the validation loss will reverse and quickly shoot up to its <strong>maximum</strong> value. And sometime around the first or second time the learning rate is halved (~100 epochs in), the validation loss will begin to decrease again, eventually matching the training loss after a few hundred more epochs.</p>
<p>I can’t for the life of me figure out why this is occurring. I use the same loop for train/validation (with the optimizer.zero_grad(), loss.backward(), and optimizer.step() behind a conditional).</p>
<p>Any pointers?</p>

apytorch: <p>I suspect this is due to either BatchNorm or Dropout. Although, I suspect BatchNorm is more likely. Maybe it takes a very long time to compute the running stats, or maybe the layers are changing so quickly.</p>
The difference between pytorch and paddle

sunshine1: <p>There is a framework called paddle contributed by baidu, which is  very similar to pytorch. I want to know if the paddle is adopted by the researcher around the world?</p>

ptrblck: <p>I think you might get a biased opinion when asking in the PyTorch forum <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=10" title=":wink:" class="emoji" alt=":wink:"> so I would recommend to search for statistics you are interested in (e.g. how many research papers are using which framework, how many job listings can be found etc.). Based on the stats you are using I’m sure you could get different results.</p>
Incorrect version spec in GetStarted section torch==1.7.0

Cbhihe: <p>Hi all,<br>
Post directed toward <strong>Admin</strong> about typo in the GetStarted <a href="https://pytorch.org/get-started/previous-versions/" rel="noopener nofollow ugc">section</a>, and more specifically in the how-to <em>wheel</em> install of the legacy version of <em>torch==1.7.0</em> on CPU only nodes.</p>
<p>It reads:</p>
<pre><code class="lang-auto"># CPU only
pip install torch==1.7.0+cpu torchvision==0.8.0+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
</code></pre>
<p>whereas it should be:</p>
<pre><code class="lang-auto"># CPU only
pip install torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
</code></pre>
<p>as there is no <em>torchvision==0.8.0+cpu</em> available at least in my <em>pyenv</em> Python 3.7.0 virtual environment. (I realize this may in fact be related and specific to that virtual environment. Unsure.)</p>

ptrblck: <p>Thanks for reporting the issue.<br>
The <code>torchvision==0.8.0</code> package is available as seen <a href="https://download.pytorch.org/whl/torch_stable.html">here</a>, but you are right that the additional <code>+cpu</code> argument is missing in the wheel name.</p>
<p>CC <a class="mention" href="/u/seemethere">@seemethere</a></p>
Batch masked select implementation

Zhaoyi-Yan: <p>Suppose I get some labels from a super-pixel algorithm, then I want to calculate the mean of SP region. Here comes the problem: how to perform batched_masked_select below?</p>
<pre><code class="lang-python">import torch

a = torch.tensor([[0,0,1,1,1],[0,0,1,1,2],[3,3,3,3,2]])
c = 2
feat = torch.randn(c, *a.size())

num_sr = a.max().long()

output = []
# How to batch this op?
for i in range(num_sr+1):
    output.append(torch.masked_select(feat, a==i).view(c, -1).mean(-1, keepdim=True)) # (2*1), mean of each SP.

output = torch.stack(output)
</code></pre>

Zhaoyi-Yan: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Can you help me with this one, it is a bottleneck of my model.</p>

Daniel_Wang: <p><a class="mention" href="/u/zhaoyi-yan">@Zhaoyi-Yan</a>  <a class="mention" href="/u/ptrblck">@ptrblck</a>  Is there any solution ？Thank you!</p>

Daniel_Wang: <p>I want to get the masked elements of each row in 2D tensor.  But the current method is slower :</p>
<pre><code class="lang-auto">x = torch.tensor([[1., 2., 2., 2., 3.],
        [1., 2., 4., 3., 2.]])

ones = torch.tensor([[1., 1., 1., 1., 1.],
        [1., 1., 1., 1., 1.]])

masks = torch.tensor([[ True, False, False, False,  True],
        [ True, False,  True,  True, False]])

for i in range(x.size(0)):
    mask = masks[i]
    s = torch.masked_select(x[i], mask)
    ones[i][:s.size(0)] = s

print(f'ones:{ones}')

#output: 
tensor([[1., 3., 1., 1., 1.],
        [1., 4., 3., 1., 1.]])


</code></pre>
<p>How to let the for  loop to batch?</p>
<p><a class="mention" href="/u/melike">@melike</a> <a class="mention" href="/u/zhaoyi-yan">@Zhaoyi-Yan</a> <a class="mention" href="/u/ptrblck">@ptrblck</a>   Is there any solution ？Thank you!</p>

melike: <p><a class="mention" href="/u/daniel_wang">@Daniel_Wang</a> Does <code>x</code> always have values equal to or greater than <code>1.</code> ?</p>

Daniel_Wang: <p><a class="mention" href="/u/melike">@melike</a>  yes, in this case i want to remove ‘2’ in x, ‘1’ is the mask ID.</p>
<pre><code class="lang-auto">x = torch.tensor([[1., 2., 2., 2., 3.],
        [1., 2., 4., 3., 2.]])

#convert to:

tensor([[1., 3., 1., 1., 1.],
        [1., 4., 3., 1., 1.]])
</code></pre>
Official way to borrow tensor options

dashesy: <p>I currently have this idiom:</p>
<pre data-code-wrap="c++"><code class="lang-c++">auto ious = at::zeros({batches, num_anchor, height, width, max_gt}, xy.options());
</code></pre>
<p>Is it the correct way to basically build a tensor that has same options as <code>xy</code> but different shape?</p>
<p>Another example working in Pytorch 1.4:</p>
<pre data-code-wrap="c++"><code class="lang-c++">at::Tensor mask =
      at::empty({dets_num * col_blocks}, dets.options().dtype(at::kLong));
</code></pre>
<p>One reason  ask this is because in newest Pytorch I get this error:</p>
<blockquote>
<p>ImportError: /opt/python/lib/python3.7/site-packages/mtorch/smtl_cpu.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at6tensorEN3c108ArrayRefIiEERKNS0_13TensorOptionsE</p>
</blockquote>
Effects of increasing Batch_Size on Accuracy of Model

Bugjub: <p>Hi,<br>
I have more of a conceptual question. When training a NN model (especially through DDP) on single-node, multiple-GPUs, there is an urge to utilize the maximum GPU memory. This is usually done by increasing the training Batch_Size of input samples (e.g., from 32 to 64, 128, 256, 512 or even 1024).</p>
<p>So I have three questions:</p>
<ul>
<li>
<p>Is there any change in terms of validation accuracy (and loss) when we do the training on more # of GPUs?</p>
</li>
<li>
<p>If I increase the Batch_size to utilize the GPU memory, is there any effect on validation accuracy and loss?</p>
</li>
<li>
<p>Do we need to tune the other hyperparameters (like learning rate, weight decay, epochs, etc.) when changing the Batch_size in the above scenario?</p>
</li>
</ul>
<p>Thanks.</p>

H-Huang: <p>Response to your questions:</p>
<ol>
<li>Yes potentially, DDP keeps loss local to each model and averages gradients, which might produce different result compared to having a global loss and back-propagating from there.</li>
<li>Also potentially yes, batch_size is another hyperparameter that can be tuned.</li>
<li>As a general note it is good to re-tune all hyperparameters when switching to DDP. These hyperparameters can all affect the accuracy of the model when compared to single GPU. However difference in accuracy should not be very drastic (this is subjective), if that is the case then it requires additional investigation.</li>
</ol>
Loading mp3 file using torchload

kranti: <p>Hi, I noticed there is a difference in the values from mp3 file when loaded using <code>torchaudio.load</code> vs <code>librosa.load</code>.  Also, the shapes of the tensors are different. I am loading an mp3 file with 44.1kHz sampling frequency of 1 sec. duration and I am getting the following output.</p>
<pre><code class="lang-auto">librosa_audio, sr_librosa = librosa.load(os.path.join(root, path), sr=44100)
torch_audio, sr_torch = torchaudio.load(os.path.join(root, path))
print(librosa_audio.shape, sr_librosa)
print(torch_audio.shape, sr_torch)
# (44100,) 44100        
# torch.Size([1, 46040]) 44100 
</code></pre>
<p>I am loading a one second audio and I expect the shape to be 44100 for both the case. Can someone please explain what is happening here?</p>
<p>Thanks.</p>

ptrblck: <p>I think you might be hitting <a href="https://github.com/pytorch/audio/issues/1500">this issue</a> so feel free to comment on this issue with your use case and description of the difference.</p>

kranti: <p>Thanks. Yes, I am indeed hitting the same issues. It seems there is some issue with mp3 loading.</p>
What is "torch._thnn.type2backend"?

szandala: <p>I am trying to revive an old software and found this import: <code>torch._thnn</code> with only one method: type2backend.<br>
It appears to be old PyTorch module, but I cannot find how to replace it.<br>
Any idea how to rewrite this to modern PyTorch?</p>

ptrblck: <p>I think this method was used to call backend methods through the Python API. You should now be able to use the exposed methods directly.</p>

josh-bone: <p>Do you mean that “type2backend[type(input)]” would be equivalent to just using “input” now?</p>