akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Parse feature vector (Tensor)

username: duddal
cooked: <p>Hello,</p>
<p>I am using CenterNet to train my model with non-square images of size (512, 1536). I am using this repository (<a href="https://github.com/yjh0410/CenterNet-plus/blob/main/models/centernet_plus.py" class="inline-onebox" rel="noopener nofollow ugc">CenterNet-plus/centernet_plus.py at main · yjh0410/CenterNet-plus · GitHub</a>).</p>
<p>For this repository, however, (512, 512) images were used, and hence the step is done on Line 255 is valid, i.e. dividing by input_size which is 512.</p>
<p>My question is:<br>
Because I have a non-square image I have a non-square feature map,<code> [1, 128, 384, 2]</code> (This is the output shape of the variable <code>txty_pred</code> (line 238) inside the aforementioned script). I would like to divide these values by the image size. Therefore, I want to divide the values inside “128” channel by 512 and values inside “384” channel by 1536. Does this make sense at all? If so, how can this be done?</p>
<p>Thank You.</p>
Is this efficient?

username: fudu
cooked: <p>New to pytorch.</p>
<p>I have got a 1d array called A from numpy, which i’m turning into a torch tensor using torch.from_numpy(A).unsqueeze(0). Is the unsqueeze part slow?</p>

username: ptrblck
cooked: <p>No, it shouldn’t be slow as it should manipulate the metadata of the tensor alone if I’m not mistaken.</p>

username: fudu
cooked: <p>thanks. id read that unsqueeze doesn’t modify in place so it can be expensive but i didn’t know about htis particular situation where its a 1d array.</p>
AttributeError: 'numpy.ndarray' object has no attribute 'mode'

username: Mai_Magdy
cooked: <p>class Stack(object):</p>
<pre><code>def __init__(self, roll=False):

    self.roll = roll

def __call__(self, img_group):

    if img_group[0].mode == 'L':

        return np.concatenate([np.expand_dims(x, 2) for x in img_group], axis=2)

    elif img_group[0].mode == 'RGB':

        if self.roll:

            return np.array([np.array(x)[:, :, ::-1] for x in img_group])

        else:

            return np.array([np.array(x) for x in img_group])</code></pre>

username: ptrblck
cooked: <p>I guess your code expects <code>img_group</code> to be a <code>list</code> (or another container) containing <code>PIL.Image</code>s, which provide the <code>mode</code> attribute.</p>

username: Mai_Magdy
cooked: <p>should i make loop on img_group list to convert all images on it to PIL.images like this ?</p>
<p>for i in range (len(img_group)):</p>
<pre><code>        img=Image.fromarray(np.uint8(img_group[i]))</code></pre>

username: ptrblck
cooked: <p>I don’t think you would necessarily need to transform the inputs first to <code>PIL.Image</code>s, as the method seems to return numpy arrays.<br>
Since you are already passing numpy arrays to this method, you would need to check the channel dimension (instead of the <code>mode</code>) and re-write the logic to work on numpy arrays directly.</p>
How to use a self argument in a function define outside the class. AttributeError: 'function' object has no attribute 'data'

username: danielmanu93
cooked: <p>Hi everyone, I have a class “MolecularDataset” which has a self.data argument at line 35 to line 39 as shown in the first pic. I want to access this “self.data” argument in a function “data_noniid”  defined outside the class as in the second pic but I have tried both “MolecularDataset.generate.data” and “MolecularDataset.data” but I get an error "AttributeError: ‘function’ object has no attribute ‘data’ ". How can I use the self.data argument in the “data_noniid”.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/6/26111e2b474f0274c64e0902b2ba5a68b23a1e48.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/26111e2b474f0274c64e0902b2ba5a68b23a1e48" title="2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/26111e2b474f0274c64e0902b2ba5a68b23a1e48_2_690x167.jpeg" alt="2" data-base62-sha1="5qKOL4pXOBpwLYba105BsBUIoc0" width="690" height="167" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/26111e2b474f0274c64e0902b2ba5a68b23a1e48_2_690x167.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/2/6/26111e2b474f0274c64e0902b2ba5a68b23a1e48.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/6/26111e2b474f0274c64e0902b2ba5a68b23a1e48.jpeg 2x" data-dominant-color="EFF1ED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">2</span><span class="informations">760×185 28.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Based on the pictures, the <code>self.data</code> attribute will be created inside the <code>generate</code> method, so you would need to call it first:</p>
<pre><code class="lang-python">dataset = MolecularDataset()
dataset.generate(args)
print(dataset.data)
</code></pre>
<p>Also, note that line34 has a condition to create <code>self.data</code>, which is then used in line37+ without a guard, so it could easily fail.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which would make debugging easier and would allow the search to index your post.</p>

username: danielmanu93
cooked: <p>Ok I got what you meant by the args. You meant “filename, add_h, filters, size, validation, test"</p>

username: ptrblck
cooked: <p>Yes, exactly. I’ve used <code>args</code> as a placeholder for the actual input arguments for <code>generate</code>.</p>
How to create None dimmension in Pytorch

username: Rituraj_19_Dutta_100
cooked: <p>I wanted to create a <code>None</code> dimension in a Pytorch tensor just like we do in Keras something like <code>(None,224,224,3)</code>. Is it possible in Pytorch?</p>

username: ptrblck
cooked: <p>I don’t think it’s possible to add this “symbolic” None dimension to a PyTorch tensor (at least not in the standard eager tensors), as it wouldn’t be needed. If I’m not mistaken, Keras (and/or TF) uses it to specify a dimension with a variable size. If so, you can just use a different (batch) size in plain PyTorch tensors.</p>
Cuda11.2 not working in rtx2060

username: bes1
cooked: <p>I want to run gpu for yolov5 on laptop rtx2060<br>
cuda11.2<br>
nvidia-smi 462.31       Driver Version: 462.31       CUDA Version: 11.2</p>
<p>what should I do<br>
please tell me pip installation line</p>
<p>Thank you</p>

username: ptrblck
cooked: <p>You could install the latest PyTorch binaries with either CUDA10.2 or CUDA11.1 as described <a href="https://pytorch.org/get-started/locally/">here</a> (you can also find the install commands for <code>pip</code> and <code>conda</code> there by selecting your setup).</p>
SARSA / Q-Learning / Double Q Learning

username: Arthur_Zakirov
cooked: <p>Hello everybody,</p>
<p>I am trying to make SARSA / Q-Learning / Double Q Learning work, but none of them works. Maybe someone can find a mistake in the setup of my problem?</p>
<p>So my world is a simple Markov Model with 2 states and 2 actions.<br>
I modelled x(s,a) as a concatenation of one hot vectors for both, because my states.<br>
I set P[s, a, s’] and R[s, a] so that the ideal policy would be pi(s0) = a0 and pi(s1) = a1. However every algorithm I tried SARSA / Q-Learning / Double Q Learning, it converges to a policy, but not to the optimal one. The interesting thing is that the policy it converges to always has the same action for every state.<br>
I tried every possible value for eps, gamma, learning rate and different periods how often to update the target-weights.</p>
<p>My last hope is, maybe I didn’t understand the formula correctly?<br>
This is my procecudure:</p>
<ol start="0">
<li>set requires_grad=False on all parameters of Q_target</li>
<li>sample a, observe (s,a,r,s’)</li>
<li>calculate Y and  Q= max Q(s,a) and a_max = argmax Q(s,a)</li>
<li>update agent policy    (1-eps) → a_max,        (eps) → other action</li>
<li>loss = (Y - Q)**2</li>
<li>loss.backward(), optimizer.step(), optimizer.grad_zero()</li>
</ol>
<p>after couple timesteps copy weights from Q to Q_target<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2de679cb4e1fa23a2a6c656945e9f89ddd1825c.png" data-download-href="https://discuss.pytorch.org/uploads/default/b2de679cb4e1fa23a2a6c656945e9f89ddd1825c" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2de679cb4e1fa23a2a6c656945e9f89ddd1825c.png" alt="image" data-base62-sha1="pwlumm5FhIyKpEJjavQHFSEmjlG" width="690" height="441" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">810×518 17.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Arthur_Zakirov
cooked: <p>So I found one error, which is that my network takes concat[s, a] as an argument and outputs a scalar Q. I’ve literally followed the stanford theory lecture, why can’t people indicate the implementation in their damn math formulas?<br>
So this brings me to the question, how do we extract the right Q value in regards to keeping the gradient flow without problems?<br>
Just “Q[a]” , or something like “torch.dot(Q, one_hot(a))”?</p>
<p>Same goes for this part of double DQN:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/5/55035ecc417b30444db40c3901ee5a0223fb6476.png" alt="image" data-base62-sha1="c83NXCeAn6YOzUwIcOnzXOfToVg" width="622" height="63"><br>
Should we propagate the gradient of the Q network, which is nested inside the Q_target network?</p>
Why is DataLoader slower when pin_memory = True?

username: Mayur28
cooked: <p>Hi!</p>
<p>I have been trying to optimize the way that I am looking my image data, and I have considered using pin_memory =True, however, when I profiled my code (using torch.utils.bottleneck), I noticed something unusual.</p>
<p>(All experiments I conducted were for 1 epoch using an 8GB image dataset (LRS2))<br>
When pin_memory =True, most of the time is spent on “method ‘acquire’ of ‘_thread.lock’ objects”. This achieved a cumtime of 1934 seconds</p>
<p>When pin_memory = False, a cumtime of 1127 seconds is achieved.<br>
I find this quite unusual as all my models are on the GPU and when reading severl forum posts, it seems that pin_memory=True when working on the GPU is the way to go.</p>
<p>Any insight into this matter would be highly appreciated.</p>

username: ptrblck
cooked: <p>Your system might suffer, if you lock too much memory for the data (and thus other processes won’t be able to use this memory and their performance might decrease). For this reason it’s also not used by default.</p>

username: Mayur28
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="126620">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Your system might suffer, if you lock too much memory for the data (and thus other processes won’t be able to use this memory and their performance might decrease). For this reason it’s also not used by default.</p>
</blockquote>
</aside>
<p>Great! Thank you for the clarification.</p>
Why do I get a different image at the same index?

username: Simplicity
cooked: <p>I have the following code portion:</p>
<pre><code class="lang-auto">images = []
image_labels = []

for i, data in enumerate(train_loader,0):
        inputs, labels = data
        inputs, labels = inputs.to(device), labels.to(device)
        inputs, labels = inputs.float(), labels.float()
        images.append(inputs)
        image_labels.append(labels)

image = images[7]
image = image[0,...].permute([1,2,0])
image = image.numpy()
image = (image * 255).astype(np.uint8)
img = Image.fromarray(image,'RGB')
img.show()
</code></pre>
<p>As you can see, I’m trying to display the image at index 7. However, every time I run the code I get a different image displayed although using the same index, why is that?</p>
<p>The image displayed also is like degraded and has less quality than the original one.</p>
<p>Any thoughts on that?</p>
<p>Thanks.</p>

username: Epoching
cooked: <p>Did you make your train_loader with <code>shuffle=True</code>?</p>
<p>Also I ran into problems where fetching all files wasn’t necessarily in the same order every time (i.e. if I have a folder of 10k files, sometimes the order that they are retrieved are different). To avoid this, you can sort your retrieved files (assuming you’re working with files)</p>

username: Simplicity
cooked: <p>Thanks for your kind reply. Yes, the issue was the <code>shuffle=True</code>.</p>

username: Epoching
cooked: <p>Nice! Don’t forget to mark as solution in case others have same problem <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
AttributeError: 'NestedTensor' object has no attribute 'shape'

username: seyeeet
cooked: <p>I have a variable called <code>samples</code> and when I do <code>type(samples)</code> it gives me <code>&lt;class 'util.misc.NestedTensor'&gt;</code>.<br>
when I print it, it looks like a normal tensor, but when I do <code>samples.shape</code> it gives an error that <code>*** AttributeError: 'NestedTensor' object has no attribute 'shape'</code></p>
<p>I guess my question is what is <code>NestedTensor</code> n pytorch and how I can get the shape of it?</p>

username: ptrblck
cooked: <p><a href="https://github.com/pytorch/nestedtensor">NestedTensor</a> is a package to work with tensors of different sizes and lengths. It should be released soon, so I guess you might be using the nightly binary?</p>
<p>Anyway, I’m not sure if <code>.shape</code> should return this error or some <code>struct</code> containing different lengths.<br>
CC <a class="mention" href="/u/cpuhrsch">@cpuhrsch</a> who is the main author of this package.</p>

username: seyeeet
cooked: <p>I was actually using <a href="https://github.com/facebookresearch/detr/blob/bc3c7e324067b05f07f0e20328614e40581e21bf/engine.py" rel="noopener nofollow ugc">DERT</a> and I see their samples in that format, so I was confused about what is that</p>

username: ptrblck
cooked: <p>Ah OK, in that case it seems to be a custom implementation from <a href="https://github.com/facebookresearch/detr/blob/bc3c7e324067b05f07f0e20328614e40581e21bf/util/misc.py#L303-L323">here</a>, which just doesn’t support the <code>shape</code> attribute, but I also haven’t looked at this repository deeply.</p>

username: seyeeet
cooked: <p>got it! thanks!!<br>
if pytorch is working on NestedTensor as well I guess it would be good to consider supporting .shape  if it does not already have</p>

username: goldfishJY
cooked: <p>Hi i think nested tensors were implemented in DETR. To get the shape, use: Nested_Tensor.tensors.shape</p>
<p>Here’s the link to the DETR hands-on where I found it; its in the attention visualisation section. Cheers!</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/detr_attention.ipynb#scrollTo=hYVZjfGhYTEa">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20210715-060125-RC01_384970452" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/detr_attention.ipynb#scrollTo=hYVZjfGhYTEa" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/detr_attention.ipynb#scrollTo=hYVZjfGhYTEa" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
SCAN for clustering in NLP

username: sans_sense
cooked: <p>For text clustering have found technique introduced in SCAN <a href="https://arxiv.org/abs/2005.12320" rel="noopener nofollow ugc">paper</a> for image classification very useful. I have used BERT for pretext and SCANloss for classification. Have wrote findings in my <a href="https://arxiv.org/abs/2107.06426" rel="noopener nofollow ugc">paper</a>. Has anyone else experimented with SCAN for text clustering. Unlike k-means we get balanced clusters and a confidence value instead of distance. Distance within a cluster turns a bit subjective, while confidence learned by the system incorporates it already. Everything is implemented in pytorch, this post is a bit general, sorry if it is the wrong forum.</p>
Most efficient way to mask tensor with other tensor to replace values with 0s

username: Trigaten
cooked: <p>I have a (3, h, w) rgb tensor and a (1, h, w) tensor. The latter tensor is sparse except for some &gt; 0 values. I would like to set all the values in the rgb tensor to 0 except for the values whose indices correspond to (h, w) indices which contain non-zero values in the latter tensor.<br>
I would like this to occur along the channel dimension so the result is a (3, h, w) rgb tensor where each channel becomes sparse except for the indices corresponding to the (1, h, w) tensor. How can I do this most efficiently?</p>
<p>Thank you.</p>

username: googlebot
cooked: <p>Huh, does the second tensor contain values (as opposed to int indexes)? If so, the first tensor is not used, and second.expand(3,-1,-1) (+ .contiguous() if needed) would broadcast across channels. If I misunderstood, you likely need where() or scatter() ops to merge values.</p>

username: Trigaten
cooked: <p>Yes, the second tensor contains values rather than int indexes. I basically want to make the first tensor have 0s in the same places the second tensor has 0s. I am not sure why the first tensor wouldnt be used. Here is an example:<br>
rgb tensor:<br>
tensor([<br>
[<br>
[ 23,  54],<br>
[  1,  90]],<br>
[<br>
[ 22, 190],<br>
[ 30,  10]<br>
]])</p>
<p>Other tensor:<br>
tensor([<br>
[<br>
[12,  0],<br>
[ 0, 12]<br>
]])</p>
<p>Resultant tensor:<br>
tensor([<br>
[<br>
[ 23,  0],<br>
[  0,  90]],<br>
[<br>
[ 22, 0],<br>
[ 0,  10]<br>
]])</p>

username: googlebot
cooked: <p>a * b.bool() is enough for that (cast to float manually if your torch version is very old)</p>
Understanding loss function

username: musgan
cooked: <p>Hi sir, can you help me. Until now, I just use the function of framework about loss function. But i not understand the function “log” in equation. Why we need “log” from equation of loss function ?</p>

username: Azerus
cooked: <p>Hi,</p>
<p>Please be specific, which loss function are you talking about? What equation ? What are you training ? Nothing is obvious without context.</p>
<p>My guess is that you are working on a classification task. So when you run your model in forward pass for one input data sample, your model will output a prediction for each class. For each output, the prediction represent the probability that your data sample belong to the class (in [0, 1]). But in the other hand, the ground-true target value will be in {0, 1} for each class (1 if it belongs to the class, 0 if not). In this context, it make sense to measure the loss of network on a log scale, as <a href="https://en.wikipedia.org/wiki/Log_probability" rel="noopener nofollow ugc">Wikipedia explains</a>:</p>
<blockquote>
<p>Since the probabilities of <a href="https://en.wikipedia.org/wiki/Independence_(probability_theory)" rel="noopener nofollow ugc">independent</a> <a href="https://en.wikipedia.org/wiki/Event_(probability_theory)" rel="noopener nofollow ugc">events</a> multiply, and logarithms convert multiplication to addition, log probabilities of independent events add. Log probabilities are thus practical for computations, and have an intuitive interpretation in terms of <a href="https://en.wikipedia.org/wiki/Information_theory" rel="noopener nofollow ugc">information theory</a>: the negative of the average log probability is the <a href="https://en.wikipedia.org/wiki/Information_entropy" rel="noopener nofollow ugc">information entropy</a> of an event. Similarly, <a href="https://en.wikipedia.org/wiki/Likelihood" rel="noopener nofollow ugc">likelihoods</a> are often transformed to the log scale, and the corresponding <a href="https://en.wikipedia.org/wiki/Log-likelihood" rel="noopener nofollow ugc">log-likelihood</a> can be interpreted as the degree to which an event supports a <a href="https://en.wikipedia.org/wiki/Statistical_model" rel="noopener nofollow ugc">statistical model</a>.</p>
</blockquote>
<p>Hope it help,<br>
Thomas</p>
What does this code do in pytorch?

username: sandeep1
cooked: <pre><code class="lang-auto">class GradientReverse(torch.autograd.Function):

  lambd = 1.0

  @staticmethod
  def forward(ctx, inp):
    out = inp.clone()
    return out

  @staticmethod
  def backward(ctx, grad_out):
    return grad_out.neg() * GradientReverse.lambd


def reverse_gradient(x, lambd):
  GradientReverse.lambd = lambd
  return GradientReverse.apply(x)
</code></pre>
<p>Any more article blog or information to understand the working of the above section of code will be helpful.<br>
As I investigated, I found that this code is of gradient reversal layer.</p>
Why does scaling the learning rate and the loss for a same factor produce a different result?

username: Mafa_can
cooked: <p>Hello,</p>
<p>I have a question about the vanilla SGD optmizer. I noticed that if I scale for a same factor the loss and the learning rate like for example:</p>
<pre><code class="lang-auto">    optimizer = torch.optim.SGD(lenet_model.parameters(), lr = learning_rate / alpha)
    criterion_train = nn.CrossEntropyLoss()
    .......
    loss = alpha * criterion_train(train_labels_head, train_labels)
</code></pre>
<p>the result of the training is different (of course I am comparing the same training with all the seeds set to the same value).</p>
<ol>
<li>In theory, scaling the loss and the learning rate should not produce any differences since they will cancel each other, is it correct?</li>
<li>Is it because of the limited number precision?</li>
</ol>
Np.gradient equivalent

username: AkashGanesan
cooked: <p>Hi,</p>
<p>Is there a np.gradient equivalent in Pytorch?</p>

username: albanD
cooked: <p>Hi,</p>
<p>I don’t think there is as we don’t usually deal with sampled function (if that’s what I understand correctly from the numpy doc).<br>
That being said, if you don’t plan on doing backprop through this op, you can simply do: <code>torch.from_numpy(np.gradient(your_f.numpy()))</code>.<br>
If you need to be able to get gradient through it, then you’ll need to check how numpy implements this function and reproduce it with pytorch’s ops.</p>

username: AkashGanesan
cooked: <p>Thought as much.  I can do that but I assume this is going to bring the tensor from the GPU to the CPU and then will have to put it back.  That doesn’t seem very efficient.  I’ll see if I can implement the same thing.  I want the velocity from each sample but was kinda lazy since this accomplishes almost the same (well, not really np.diff) and preserves the shape for my purposes.</p>
<p>I’ll see if I can do something with it.</p>

username: Hmrishav_Bandyopadhy
cooked: <p>Update : There is now one <a href="https://pytorch.org/docs/stable/generated/torch.gradient.html" rel="noopener nofollow ugc">here</a></p>
What does PyTorch do when calling tensor.half()

username: Beinan_Wang
cooked: <p>I want to understand how pytorch does fp16 inference. Say I have a pretrained fp32 model and I run fp16 inference by calling model.half(). Then when I run y = model(x), does pytorch simply calculate in fp16 format or are there some hidden optimizations applied to make sure there is not significant drop in accuracy?</p>

username: albanD
cooked: <p>If you only do <code>model.half()</code> and then forward. Pytorch will only convert all the model weights to half precision and then forward with that.<br>
If you want something smarter (that keeps single precision buffers for some ops for stability), you can check out <a href="https://github.com/NVIDIA/apex/tree/master/apex/amp" rel="nofollow noopener">nvidia’s amp package</a>.</p>

username: Beinan_Wang
cooked: <p>Thank you for the clarification.</p>

username: jiacheng1gujiaxin
cooked: <p>Can nvidia-apex only be used for inference? I want to deploy my model with mix precision, but I find that when I run it, my memory is not reduced.</p>
<p>futhermore, when i run fp16 inference by calling model.half(), the memory is not reduced either</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="jiacheng1gujiaxin" data-post="4" data-topic="58597">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/f17d59/48.png" class="avatar"> jiacheng1gujiaxin:</div>
<blockquote>
<p>Can nvidia-apex only be used for inference?</p>
</blockquote>
</aside>
<p>No, you can use <code>apex</code> for training and inference.</p>
<aside class="quote no-group" data-username="jiacheng1gujiaxin" data-post="4" data-topic="58597">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/f17d59/48.png" class="avatar"> jiacheng1gujiaxin:</div>
<blockquote>
<p>I want to deploy my model with mix precision, but I find that when I run it, my memory is not reduced.</p>
<p>futhermore, when i run fp16 inference by calling model.half(), the memory is not reduced either</p>
</blockquote>
</aside>
<p>If you are checking the used memory via <code>nvidia-smi</code>, note that you might see the cached memory as well. <code>torch.cuda.memory_allocated()</code> might give you a better number.</p>

username: jiacheng1gujiaxin
cooked: <p>May i ask you how can i deploy my model with fp16 would get significant performance promote.<br>
is tensorrt helpful?</p>

username: ptrblck
cooked: <p>Depending on your model, you might just call <code>model.half()</code> on it.<br>
However, IIRC you could export the vanilla FP32 model and pass it to TensorRT, which should use the specified precision.</p>

username: Rohith_Rajesh
cooked: <p>Hi!<br>
Does that mean the activations will also be computed in FP16?</p>
LSTM batch size vs sequence length

username: Aureate
cooked: <p>I am new to PyTorch and am trying to do some time series prediction for speech. The dimension of each datapoint is 384. What I am confused about is whether the memory of the LSTM is separate for each sequence in the batch or whether the batch is basically treated as one long sequence. In other words, in a simplified example, suppose that the input to our LSTM is (batch size, sequence length, input size). Now suppose we want our LSTM to retain memory of 50 datapoints. Is there a difference between having our input shape as (1,50,384) or (50,1,384). Does the latter mean that each datapoint is basically treated independently of the others?</p>
<p>Could someone please help with this. Thanks in advance!</p>

username: googlebot
cooked: <aside class="quote no-group" data-username="Aureate" data-post="1" data-topic="126987">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/f07891/48.png" class="avatar"> Aureate:</div>
<blockquote>
<p>Is there a difference between having our input shape as (1,50,384) or (50,1,384).</p>
</blockquote>
</aside>
<p>Yes</p>
<aside class="quote no-group" data-username="Aureate" data-post="1" data-topic="126987">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/f07891/48.png" class="avatar"> Aureate:</div>
<blockquote>
<p>Does the latter mean that each datapoint is basically treated independently of the others?</p>
</blockquote>
</aside>
<p>Yes. Basically, transition is the same, but it is applied to different vectors (accumulated context + step input concatenated). What you refer to as “memory” is a transformation matrix, applied multiple times, roughly speaking.</p>
Will dataset object be replicated on each worker process for map-style dataset?

username: wusuowei60
cooked: <p>In the <a href="https://pytorch.org/docs/stable/data.html#iterable-style-datasets" rel="noopener nofollow ugc">torch.utils.data — PyTorch 1.9.0 documentation</a> about <code>utils.data</code>, it is clearly pointed out that each worker process gets a replica of the <code>dataset</code> object for <strong>iterable-style</strong> datasets, but it’s not clear whether that is also the case for <strong>map-style</strong> dataset.</p>
<p>So will dataset object be replicated on each worker process for map-style dataset😕?</p>
Does UnsafeViewBackward cause backprop issues?

username: Dylan-Kerler
cooked: <p>In my net I am getting a tensor like this after doing a <code>my_tensor.flatten()</code>:</p>
<p><code>grad_fn=&lt;UnsafeViewBackward&gt;</code></p>
<p>Does this effect how the gradient is propagated or will everything work just fine?</p>

username: ptrblck
cooked: <p>It will work fine and should not create any issues regarding backpropagation or gradient calculation.</p>
How to normalize losses of different scale

username: hankdikeman
cooked: <p>I have two losses, one is normal MSELoss and another is a custom loss function that I have made for regularization.</p>
<p>The problem that I have is that these losses are not necessarily on the same numerical scale, so I have to figure out how to weight them every time (and divide/multiply one by a constant so they are the same scale). I would much prefer if I could set the relative weight of each loss once and not worry about it again, such that loss1 constitutes 99% of my loss value and loss2 constitutes 1% (so some sort of normalized weighting)</p>
<p>I had previously added the two different loss functions together like this:</p>
<pre><code class="lang-auto">batch_loss = reconstruction_loss + monotonic_loss
</code></pre>
<p>But instead I want to normalize the losses so I can choose how much they contribute to parameter updates, this is what I was thinking.</p>
<pre><code class="lang-auto">batch_loss = CombineLosses([reconstruction_loss, monotonic_loss], [0.99,0.01])

def CombineLosses(losses, weights):
    combined_loss = torch.Tensor([0])
    for loss, wt in zip(losses, weights):                                                             
        combined_loss = combined_loss + ((loss * wt) / loss.item())                                   

    return combined_loss
</code></pre>
<p>The idea is that then I can weight them and if my regularization function sums to 10E8 and MSELoss to 0.16 I can still incorporate both. However, I might have a lack of understanding about how autograd works so I have a couple questions:</p>
<ol>
<li>Will scaling my two loss functions actually limit each loss’s contribution proportionally to it’s weight? If not, what is the preferred way to combine losses of different scales in PyTorch?</li>
<li>As an alternative, could it be possible to alternate between training on one loss function and training on the other? i.e. switching off epochs</li>
</ol>
<p>Thanks in advance</p>

username: eqy
cooked: <p>Scaling should make a difference.<br>
e.g., consider the trivial scenario where L = (x-y)^2 vs. L= 5(x-y)^2.<br>
dL/dx = 2(x-y) in the first and dL/dx = 10(x-y) in the second.</p>
<p>There is nothing wrong with the second approach, and there are many variations such as freezing part of the model trained on another method and finetuning a different part with a different objective. Of course, experimentation is probably the way to confirm what works and what doesn’t work well.</p>
<p>The tool of weighting losses can seem crude, but last I checked even huge models with countless objectives and sub-tasks (e.g., Tesla Autopilot) often balance the objectives in this way, often with hand-tuned weighting.</p>

username: hankdikeman
cooked: <p>Thank you, I appreciate the help. I was somewhat worried it would make training difficult if one objective had a gradient which was larger by a factor of 10 or 100 million, glad there is a method to weight them and it’s not completely off-the-wall to do so!</p>
WeightedSampler by number of labels

username: Ilya_Kotlov
cooked: <p>Hi, what way would you suggest to create a version of Sampler which will allow me to control the distribution of samples in a batch by a number of their labels, talking about multi-label data ?</p>
<p>For example, in each batch the distribution will be 3 samples with 4 labels, 7 samples with 2 labels and 10 with one label.</p>
<p>Moreover in my case I would prefer to have, lets say with batch_size=20, each time a full batch of samples with same number of labels, but looping over all the batches over the whole variety of amounts of labels.</p>
<p>The easiest way i think about is to create, if I have 1-5 labels, 5 dataloaders, and using random.random() toss a number and according to different amount of labels prior distribution (lets say in a dataset of 10000 samples, where there are 2000 for each amount of labels between 1 and 5) I will firstly choose which dataloader to load from and then load the batch with the corresponding number of labels I tossed.</p>
<p>Any other creative suggestions ? Any built-in torchy way to do it ?</p>
Autocast with indexing incorrect types

username: tuero
cooked: <p>I’m looking if there is a better or easier way to do the following. I’m creating an empty tensor, and only want to fill in the indices with the output of a network if the mask condition is true.</p>
<pre><code class="lang-auto">...
with torch.cuda.amp.autocast():
    next_state_values = torch.zeros(len(non_final_mask), device=self.device)
    next_state_values[non_final_mask] = target_net(non_final_next_states).max(1)[0].detach()
</code></pre>
<p>However, this will throw the error</p>
<pre><code class="lang-auto">RuntimeError: Index put requires the source and destination dtypes match, got Float for the destination and Half for the source.
</code></pre>
<p>I can do the following as a workaround, but this feels a bit hacky. Is there a better way to create the initial empty tensor while using amp?</p>
<pre><code class="lang-auto">...
with torch.cuda.amp.autocast():
    next_state_values = torch.zeros(len(non_final_mask), device=self.device)
    net_out = target_net(non_final_next_states).max(1)[0].detach()
    next_state_values = next_state_values.to(net_out.dtype)
    next_state_values[non_final_mask] = net_out
</code></pre>
Subset Dataloader

username: jhp
cooked: <p>Hello, I’d like to ask  about data loader.<br>
The data I’m using is very big, so I’d like to experiment with subsets and compare with multiple baselines for comparing with my model quickly. Would it be possible to experiment with subsets  simply by reducing the length in the loader (i.e__len__: return {original_data_length}//10) for applying identical subset datasets to the multiple training models?</p>
<p>All maybe… it would be appreciated if you tell me the tips for comparing models with huge datasets quickly for not being waste of my time.</p>

username: ptrblck
cooked: <p>Yes, you can reduce the <code>len</code> of the <code>Dataset</code>, which would then create samples using <code>index</code> values in the <code>__getitem__</code> method in <code>[0, len(dataset)-1]</code>.</p>
Error while loading state_dict

username: Surya1212
cooked: <p>When I am trying to load a model :</p>
<h1>Load the trained model from file</h1>
<p>trained_model = Net()  <span class="hashtag">#Net</span> is myModelClass</p>
<p>m = torch.load(’./modell.pth’, map_location=lambda storage, loc: storage)</p>
<p>trained_model.load_state_dict(m)</p>
<p>trained_model.load_state_dict(m)<br>
File “/home/surya/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 751, in load_state_dict<br>
<strong>state_dict = state_dict.copy()</strong><br>
File “/home/surya/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 539, in <strong>getattr</strong><br>
type(self).<strong>name</strong>, name))<br>
AttributeError: ‘Net’ object has no attribute ‘copy’</p>
<p>Next I have used <strong>copy.copy(state_dict)</strong>. Then I got a new error.</p>
<p>trained_model.load_state_dict(m)<br>
File “/home/surya/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 763, in load_state_dict<br>
load(self)<br>
File “/home/surya/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 758, in load<br>
state_dict, prefix, local_metadata, True, missing_keys, unexpected_keys, error_msgs)<br>
File “/home/surya/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 720, in _load_from_state_dict<br>
for key in state_dict.keys():<br>
File “/home/surya/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 539, in <strong>getattr</strong><br>
type(self).<strong>name</strong>, name))<br>
AttributeError: ‘Net’ object has no attribute ‘keys’</p>

username: alex.veuthey
cooked: <p>My guess is, you used <code>torch.save(model)</code> (saves the entire model with architecture and weights and probably other parameters) instead of <code>torch.save(model.state_dict())</code> (saves only the state dict, i.e. the weights).</p>
<p>If you need to stick to the first approach, then you must load it as follows: <code>model = torch.load(PATH)</code>.</p>
<p><a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html" rel="nofollow noopener">More info here</a>.</p>

username: Jayant_Parashar
cooked: <p>What if we saved a torch.save(model) but now need to load the state dict because torch.load(modelpath) code breaks as warned in official documentation ?</p>

username: alex.veuthey
cooked: <p>Do newer versions of PyTorch block you from loading a model directly? In that case, load the model directly from an older version of PyTorch, and then save its <code>state_dict</code>, which you should be able to load with a more recent version.</p>

username: Jayant_Parashar
cooked: <p>Thanks, I will double check the pytorch version. It most likely is the same though. The problem seems to be something else. It gave a size mismatch error.</p>

username: ByShui
cooked: <p>thank you very much, i just ignore this problem after i modify my save way</p>
How to parse an annotation file that has multiple objects in an image

username: Simeon
cooked: <p>I have CSV annotated data and corresponding images, each image contains more than one object.  The annotation has different columns such as image names column, bounding box coordinates in xmin, ymin, xmax, ymax format(top left and bottom right respectively), and class names such as a person, car, cat, etc. I tried to use a customer dataset preprocessing from different sources and data loader, but I have got an error saying that the length of input in the image and the annotation are not the same. This is due to the image has more than one object so that the name is mentioned more than once in the image name column for each bounding box coordinates. I would appreciate it if anyone can give me  a reference to read to solve the problem or gave a skeleton sample code. My final goal is to build an object detection model to classify objects based on the category, aeroplane, car, cat etc. All images are in the same folder, not based on the category.<br>
Thanks,<br>
Simeon</p>
Why does my model stops updating when i modify one of its layers

username: MRRP
cooked: <p>Hi<br>
I was curious about why my model stops updating when i change one of its layers by the code below</p>
<blockquote>
<p>model1.x_layer.weight = nn.Parameter(model2.x_layer.weight.detach().clone())</p>
</blockquote>
<p>is that because the parameter reference in my optimizer?<br>
if that’s the case what do you suggest to do to be able to change a layer and still be able to train the model?</p>

username: AlphaBetaGamma96
cooked: <p>If you do swap out weights of a trained model you’d need to update the parameter list within the optimizer.</p>
<p>I’m not 100% about the use of the detach function perhaps you’d be better just copying it directly across with <code>model.x_layer.weight = model.x_layer.weight.copy()</code>?</p>

username: MRRP
cooked: <p>Hi Thanks for replying<br>
Well when i modify my model1 weights with the code below</p>
<blockquote>
<p>model1.x_layer.weight.data = model2.x_layer.weight.detach().clone()</p>
</blockquote>
<p>my model keeps updating without any problem. i mean i don’t understand exactly if i should update my optimizer or it is not needed.<br>
This is a federated learning scenario in which i update my models a few epochs and then i want to aggregate them and feed the aggregated model back to them.</p>
Can I use the derivative of part network to the input in the .forward method

username: ZhenyuWei
cooked: <p>Dear all</p>
<p>I am trying to use NN to solve an ODE. The idea is that the trial solution can be written as an expression of NN output. And the loss function can be defined as the square of ODE when substituting the trial solution. Here I met the question that the NN should be derivated by the input during the substitution. And I am wondering how to realize this goal in the definition of my model.</p>
<p>Thank u in advance</p>
Any means of promoting Windows-specific issue?

username: dorozhkin-stc
cooked: <p>Currently we suffer from a severe crash in our application which loads and unloads PyTorch libraries on demand [1]. Basically the issue boils down to some form of use-after-free in the destructors of the static objects scattered though many of torch_* DLLs while portion of those DLLs gets unloaded by the system. I’ve managed to avoid faulty accesses by adding RAII cleanups downstream and provided possible fix as an illustration of the issue in the form of a PR [2].</p>
<p>Besides purely technical details which are gory but comprehensible, it’s been a while it has been reported. The issue itself seemingly draws little or no attention from upstream developers though. I wonder if I did something wrong. Is the report unclear, incomplete or invalid, or maybe GH issues is just an unsuitable place for discussing such kind of problems? In that case may I kindly ask to be directed to a proper place or a person?</p>
<p>1 <a href="https://github.com/pytorch/pytorch/issues/61111" class="inline-onebox" rel="noopener nofollow ugc">[Windows] Crash when unloading torch_cpu.dll using LOCAL SYSTEM account · Issue #61111 · pytorch/pytorch · GitHub</a><br>
2 <a href="https://github.com/pytorch/pytorch/pull/61290" class="inline-onebox" rel="noopener nofollow ugc">Proper deregistration of static types and objects by dorozhkin-stc · Pull Request #61290 · pytorch/pytorch · GitHub</a></p>
Data set normalization in torchvision_tutorial.html

username: Laurent_Guichard
cooked: <p>Hello,<br>
I have been implementing this tutorial :<br>
<a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a><br>
The input images are not normalized to 0 mean and 1 variance.<br>
Is this “normal” ? perhaps an oversight ?<br>
Best regards</p>

username: tom
cooked: <p>It’s a good question to ask.<br>
The answer is that one has to consider whether inputs are expected to be normalized by the model - a very natural alternative is to consider normalization part of the model.</p>
<p>In this case we can refer to the <a href="https://pytorch.org/vision/stable/models.html#torchvision.models.detection.fasterrcnn_resnet50_fpn" rel="noopener nofollow ugc">TorchVision documentation</a>:</p>
<blockquote>
<p>The input to the model is expected to be a list of tensors, each of shape <code>[C, H, W]</code>, one for each image, and should be in <code>0-1</code> range. Different images can have different sizes.</p>
</blockquote>
<p>But again, you are absolutely right that this is something to check (and maybe it would be worth mentioning in the tutorial, too).</p>
<p>Best regards</p>
<p>Thomas</p>
Complex valued layers

username: Peter_Featherstone
cooked: <p>What is the timeline on supporting complex-valued layers like convolution, batch-normalisation and some appropriate activation functions like modrelu?</p>
<p>There are libraries that simulate these using real tensors but they are extremely memory hungry and therefore impractical to use.</p>
<p>So far, the closest we are to layer support is matmult?<br>
Is it a few months, a couple years, … ??</p>
<p>Cheers</p>

username: Peter_Featherstone
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Do you have any insights?</p>
Torch.histc with Variables

username: WERush
cooked: <p>Hi, all<br>
I want to calculate the histogram with Variables and find the torch.histc function. Unfortunately, this function does not support the Variables. Any alternatives?</p>
<p>Thanks</p>

username: richard
cooked: <p>If you don’t need to backprop through the output of the histogram, you can call torch.histc on the tensor data of the variable. ie if <code>x</code> is your variable, you can get its tensor data via <code>x.data</code>.</p>

username: WERush
cooked: <p>Honestly, I want to use the histogram as the loss function, hence I need to call the histc on the variable</p>

username: malbergo
cooked: <p>Did you ever figure out how to do this? When I try to backpropagate through torch.histc tensor I am told “the derivative for “histc” is not implemented”</p>

username: EchozL
cooked: <p>Same question. It seems like torch.histc() doesn’t have derivative function, and is not supposed to be a part of loss, or any part of computational graph. Maybe one could write his own histc() and implement the derivative, but in my opinion, it is hard with select operations inside of histc().</p>
Exporting pytorch model to ONNX that accepts dictionary

username: mrpatekful
cooked: <p>I am trying to pass a mapping to a module and export it to onnx, which I attempt in the following code:</p>
<pre><code class="lang-python">import torch
import numpy
import io
import typing
import onnx
import onnxruntime


class Something(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.layer = torch.nn.Linear(10, 10)

    def forward(self, batch: typing.Dict[str, torch.Tensor]):
        y = self.layer(batch["x"])
        return {"y": y}


def main():
    something = Something()
    with io.BytesIO() as onnx_bytes:
        torch.onnx.export(something, ({"x": torch.ones(10, 10)}, {}), onnx_bytes)

        onnx_model = onnx.load_model_from_string(onnx_bytes.getvalue())

    model_proto = onnx_model.SerializeToString()
    session = onnxruntime.InferenceSession(model_proto)
    inputs = {session.get_inputs()[0].name: {"x": numpy.ones((10, 10))}}
    print(session.run(None, inputs)
</code></pre>
<p>However I get an exception:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/media/main.py", line 53, in &lt;module&gt;
    main()
  File "/media/main.py", line 32, in main
    print(session.run(None, inputs))
  File "/media/venv/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 188, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: &lt;class 'TypeError'&gt;: only size-1 arrays can be converted to Python scalars

</code></pre>
<p>Is the desired behaviour achivable somehow?</p>