akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Loading weights doesn't generate the image

username: Merwa_Mehmood
cooked: <p>#<span class="hashtag-raw">#checkpoints</span> are loaded successfully and all keys match successfully</p>
<p>print(“=&gt; Loading checkpoint”)</p>
<p>checkpoint = torch.load(‘/content/High/generator_C.pth’, map_location=“cuda”)</p>
<p>gen.load_state_dict(checkpoint[‘state_dict’])</p>
<h2><a name="now-feeding-noise-to-the-generator-t-get-the-desired-image-1" class="anchor" href="#now-feeding-noise-to-the-generator-t-get-the-desired-image-1"></a>now feeding noise to the generator t get the desired image</h2>
<p>alpha = 1.0</p>
<p>for i in range(10):</p>
<p>noise =torch.randn((1, Z_DIM, 1, 1))</p>
<p>img_size=128</p>
<p>num_steps = int(log2(img_size / 4))</p>
<p>img = gen(noise, alpha, steps=num_steps)</p>
<p>save_image(img*0.5+0.5, f"/content/img_{i}.png")</p>
<p>##<span class="hashtag-raw">#but</span> the output image is’<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/1/f13e0e2b9b2e6a5af294dee5c3613ab4a2c58c7e.png" alt="image" data-base62-sha1="yq81rN0mpcP9tRugHOFrh6fdKKq" width="128" height="128"></p>

username: ptrblck
cooked: <p>I assume you’ve saved some output images during your training (or the validation step in the training script) and are now comparing them to this new output in another script?<br>
If so, did you make sure to use the same input data distribution (i.e. was the <code>noise</code> tensor also created via <code>torch.randn</code> before)?</p>

username: Merwa_Mehmood
cooked: <p>yes noise tensor was created via torch.rand.<br>
I am getting one more error while genenrating fake images:<br>
ERROR:<br>
Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same<br>
what does this mean?<br>
Is this the reason i am not getting fake images?</p>

username: ptrblck
cooked: <p>The error points towards a device mismatch and it seems you haven’t pushed the parameters of the model to the device, so you might want to add <code>gen.to('cuda')</code>.</p>
<aside class="quote no-group" data-username="Merwa_Mehmood" data-post="3" data-topic="122036">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/merwa_mehmood/48/35422_2.png" class="avatar"> Merwa_Mehmood:</div>
<blockquote>
<p>Is this the reason i am not getting fake images?</p>
</blockquote>
</aside>
<p>If this error was also raised before, the <code>gen</code> model shouldn’t have worked and I’m unsure where the initial output image is coming from.</p>

username: Merwa_Mehmood
cooked: <p>So do I need to push gen.cuda() before training?</p>

username: ptrblck
cooked: <p>Based on the error message this would be my guess.<br>
Did you try it and was it working?</p>

username: Merwa_Mehmood
cooked: <p>Thanks solved .the input should be pushed to the cuda().before training</p>
Gpu trained parameters get different when loading on cpu

username: newman_chiang
cooked: <p>Hi,</p>
<p>I’m using torch 1.8.1+cu101.<br>
I train model on gpu and save it using torch.save(…) and load it back on cpu using torch.load(…, map_location=‘cpu’).<br>
But the prediction result on cpu is totally different from that on gpu.<br>
I then check the model parameters loaded and the parameters are different on cpu and on gpu.<br>
Why is it?</p>
<p>I’ve already seen</p><aside class="quote" data-post="1" data-topic="349">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ja-keoung_koo/48/12628_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/on-a-cpu-device-how-to-load-checkpoint-saved-on-gpu-device/349">On a cpu device, how to load checkpoint saved on gpu device</a> 
  </div>
  <blockquote>
    I trained my network on a gpu device and saved checkpoint by torch.save 
Loading this checkpoint on my cpu device gives an error: 
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled```
  </blockquote>
</aside>
<p>
but it still doesn’t work.</p>
<p>Thanks</p>

username: ptrblck
cooked: <p>Could you post the model definition as well as a minimal code snippet to reproduce the issue of the non-matching parameters, please?</p>
How to decompose deep learning trainable parameter into two parts for Pytorch?

username: jb4382
cooked: <p>I got this problem when I read the paper ‘Federated Semi-Supervised Learning with Inter-Client Consistency &amp; Disjoint Learning’. I am quite wondering about the disjoint learning. The author said ‘For a given model, it should decompose into two parts where model = <em>a</em> + <em>b</em>(each model’s parameters equal to the sum of the same location parameter <em>a</em> +  the same location parameter <em>b</em>). When we train the model on dataset A, we only update <em>a</em> and hold <em>b</em> constant. When we train the model on dataset B, we only update <em>b</em> and hold <em>a</em> constant.’ How can we do this with pytorch?</p>
<p>Thanks so much for your help!!!</p>
How to minimize gradient wrt inputs norm?

username: mipt98
cooked: <p>Please help. I have a simple net which takes several inputs, and outputs a single value. I can compute gradient of the output wrt to all the inputs by autograd. Now I want to minimize the norm of this gradient wrt to the model parameters… Is it possible? I remember it was possible to do with Tensorflow 1.x, but not so sure about Pytorch. The sample code would be something like</p>
<pre><code>    x.requires_grad_()
    y = Model(x)
    y.backward()
    loss = x.grad.square().sum()
    loss.backward()
</code></pre>
<p>Thanks!</p>
AttributeError: 'Model' object has no attribute 'parameters'

username: Khabbab_Zakaria
cooked: <p>I am using a modified Resnet18, with my own pooling function at the end of the Resnet.</p>
<p>Here is my code:</p>
<pre><code class="lang-auto">resnet = resnet18().cuda() #a modified resnet

class Model():
    def __init__(self, model, pool):
        self.model = model
        self.pool= pool #my own pool class which has trainable layers

    def forward(self, sample):
        output = self.model(sample)
        output = self.pool(output)
        output = F.normalize(output, p=2, dim=1)
        return output
</code></pre>
<p>Now, obviously I need to train not only the resnet part, but also the pool part.</p>
<p>But, when I check:</p>
<pre><code class="lang-auto">model = Model(model=resnet, pool= pool)
print(list(model.parameters()))
</code></pre>
<p>It gives:</p>
<pre><code class="lang-auto">AttributeError: 'Model' object has no attribute 'parameters'
</code></pre>
<p>Can anyone help?</p>

username: ptrblck
cooked: <p>You would have to derive your custom <code>Model</code> from <code>nn.Module</code> as:</p>
<pre><code class="lang-python">class Model(nn.Module):
    def __init__(self, model, pool):
        super().__init__()
        ...
</code></pre>
<p>to make sure all <code>nn.Module</code> methods and attributes are available.</p>

username: Khabbab_Zakaria
cooked: <p>Thanks! Yes, I understand it now</p>
How to load ImageNet

username: jS5t3r
cooked: <p>When I load the ImageNet I get the wrong classes… Either class 0 or class 1.  But ImageNet should have 1000 classes. What am I doing wrong?</p>
<p>When I change to shuffle to true then I can see a lot different classes…</p>
<pre><code class="lang-auto">import os, sys, pdb
import argparse
import torch
import torch.nn as nn
import torchvision.datasets as datasets
import torch.utils.data as data
import torchvision.transforms as transforms
import torchvision.models as models

if __name__ == '__main__'
    model = models.resnet18(pretrained=True)
    model.cuda()
    model.eval()

    # load data
    transform_list = [transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor()]
    transform_chain = transforms.Compose(transform_list)
    
    data_dir = 'path/to/ImageNet'

    item = datasets.ImageFolder(data_dir + '/val', transform=transform_chain)

    test_loader = data.DataLoader(item, batch_size=100, shuffle=False, num_workers=4)

    inputs, classes = next(iter(test_loader))
    print("classes: ", classes)
    pdb.set_trace()
</code></pre>

username: eqy
cooked: <p>The validation set for ImageNet has 50,000 images or 50 per each of the 1,000 classes. If you don’t shuffle the data then the expectation indeed is that you only see two classes for a batch size of 100.</p>
Extracting values of middle layers that hasnot been flagged as output

username: Xia_Zhengmi
cooked: <p>I’ve trained a code like the one in bellow for 1000 epochs but I forgot to flag the desired variable as the output of my model, is there any way to extract during the test time? I want to visualize it but don’t know what I need to do (unfortunately the name of all the inputs and outputs are the same, I don’t know how I should find it!!)</p>
<pre><code>class Net(nn.Module):
   def __init__(self):
      super(Net, self).__init__()
      self.conv1 = nn.Conv2d(...)
      self.conv2 = nn.Conv2d(....)
      self.conv3 = nn.Conv2d(....)

    def forward(self, x1):
      x1 = self.conv1(x1)
      x1 = self.conv2(x1)  &lt;-------------------- the variable that I want to visualize
      x1 = self.conv3(x1)
      x1
      return x1</code></pre>

username: eqy
cooked: <p>Can you try returning two outputs instead?</p>
<p>e.g.,</p>
<pre><code class="lang-auto">    def forward(self, x1):
      x1 = self.conv1(x1)
      saved = self.conv2(x1)
      x1 = self.conv3(saved)
      x1
      return x1, saved</code></pre>

username: Xia_Zhengmi
cooked: <p>No I can not train it again, it’s now a pretrained network that I want to visualize outputs of it’s hidden layers.</p>

username: eqy
cooked: <p>What happens if you define a second model with the additional outputs and just load the weights of the first model? There aren’t any additional parameters introduced here.</p>

username: Xia_Zhengmi
cooked: <p>Thank you so much for your response,</p>
How to do masked mean while reserving the first dimension

username: ziqi
cooked: <p>I have a tensor of size BxCxHxW. In the tensor there are some valid values which is larger than 0 and invalid values which equals to 0. Now I want to compute <strong>the mean of valid values in dimension CxHxW</strong>. So the result size is Bx1, the ith element equals the mean of valid values in the ith CxHxW tensor.</p>
<p>For example if I ignore C, I have a 2x1x3 tensor [[[0,1,2]], [[5,7,0]]], which means B=2, H=1, W=3. The desired result is a 2x1 tensor [1.5, 6], in which 1.5 is the average of 1 and 2, 6 is the average of 5 and 7. If I directly use tensor.mean(-1).mean(-1) it will count zeros into the result, which is not what I want.</p>

username: Sunshine352
cooked: <p>a=torch.tensor([[[0,1,2]], [[5,7,0]]]).float()<br>
mask=(a&gt;0).float()<br>
mask_mean = a.sum(-1) / mask.sum(-1)<br>
<strong>output:</strong><br>
mask_mean=tensor([[1.5000], [6.0000]])<br>
mask_mean.shape=torch.Size([2, 1])</p>

username: vmirly1
cooked: <p>One simple solution is as follows:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; a = torch.randn(10)
&gt;&gt;&gt; a
tensor([-0.7635,  0.1626, -0.2671,  0.6118,  0.0304,  1.3959, -1.0044, -0.7236,
        -1.1573, -0.4817]
&gt;&gt;&gt; aw = a &gt; 0
&gt;&gt;&gt; aw
tensor([0, 1, 0, 1, 1, 1, 0, 0, 0, 0], dtype=torch.uint8)
&gt;&gt;&gt; torch.sum(a * aw.float()) / torch.sum(aw)
tensor(0.5502)
</code></pre>

username: TaZi-loh
cooked: <p>if it may happen, that you mask out everything (e.g. if you have multiple batches and some batchentries are completly masked out), you might want to do it like this:</p>
<pre><code class="lang-auto">batchsize = 32
datasize = 2
a = torch.randn(batchsize, datasize)
mask = (a &gt; 0).float()
mask_sum = torch.sum(mask, dim=1)
mask_sum_modified = torch.clamp(mask_sum, min=1.0)
torch.sum(a * mask, dim=1) / mask_sum_modified
</code></pre>
I dont understand the Mask-RCNN training output

username: Bernd
cooked: <p>Hello<br>
I just switched from TensorFlow to PyTorch, so I know my neural networks, but not so much PyTorch.</p>
<p>When I do a training with Mask R-CNN (using ResNet-50) I get this kind of output:</p>
<p>Epoch: [8]  [410/454]  eta: 0:00:29  lr: 0.005000  loss: 0.4226 (0.4910)  loss_classifier: 0.1969 (0.2415)  loss_box_reg: 0.1210 (0.1449)  loss_mask: 0.0751 (0.0860)  loss_objectness: 0.0022 (0.0045)  loss_rpn_box_reg: 0.0064 (0.0141)  time: 0.6515  data: 0.4290  max mem: 1635</p>
<p>I know these are different losses, so they should go down, ok.<br>
But what kind of losses?<br>
And why are there 2 numbers for each loss?</p>
<p>And what is meant with “data”?</p>
<p>Can someone explain these values to me?<br>
I tried to find some documentation about this but failed (maybe someone should document that!?).</p>
<p>Someone flagged this question so it got hidden.<br>
Please tell me what is wrong with it and I revise my question.</p>
<p>Regards,<br>
Bernd</p>
Get torch.nn.modules attributes

username: Adriano_Santos
cooked: <p>Hi, guys.</p>
<p>So… I’m building a script to convert weights and configuration files from a model. And to do that, I’d like to access the attributes list from the modules. For example:</p>
<blockquote>
<p>for name, m in model.named_modules():<br>
t = type(m)<br>
if t is torch.nn.modules.conv.Conv2d:<br>
print(m.in_channels)<br>
print(m.out_channels)<br>
print(m.kernel_size)<br>
print(m.stride)<br>
print(m.dilation)</p>
</blockquote>
<p>In that case, above, I defined the attributes to the Conv2d manually. But if the next module is models.common.Conv?</p>
<p>I’d like to know if is it possible to have access to this list?</p>
<p>Tks!</p>

username: ptrblck
cooked: <p><code>vars</code> or <code>dir</code> might be useful in this case:</p>
<pre><code class="lang-python">conv = nn.Conv2d(3, 6, 3, 1, 1)
attrs = vars(conv)
print(', '.join("{}: {}".format(item[0], item[1]) for item in attrs.items()))
print(dir(conv))
</code></pre>

username: Adriano_Santos
cooked: <p>Tks! It works for me.</p>
TFIDF for learning common words in audit failures

username: Parkz
cooked: <p>I have a dataset where there are reported audit failures for each system listed next to a brief description of the failure. The problem is that the words used to describe the problem are different based on what someone types in, and I want to be able to aggregate them based on common words. For example a Redhat failure might read “Login password does not meet requirements” and a Windows Server might read “Login password is needed and there isn’t one”. They both have to do with login password, and if I aggregate them based on the password I’ll get valuable data for my organization. I need an algorithm that finds the valuable words and then I can aggregate based on those words. I’m trying tfidf but it is aggregating based on os names more than other words. Any suggestions for someone new to NLP to solve this problem?</p>
Tensor pickling inconsistent between runs

username: hlibbabii
cooked: <p>I’m trying to unpickle a pytorch tensor, but pickling it back yields different results across runs:</p>
<pre><code>&gt;&gt;&gt; import pickle

&gt;&gt;&gt; tensor1 = pickle.load(f) # I cannot reproduce the issue with some minimal manually-created tensor, only with this specific file
&gt;&gt;&gt; tensor2 = pickle.load(f)
&gt;&gt;&gt; pickled_tensor1 = pickle.dumps(tensor1) 
&gt;&gt;&gt; pickled_tensor2 = pickle.dumps(tensor2)
&gt;&gt;&gt; pickled_tensor1 == pickled_tensor2
False
</code></pre>
<p>Below are the values of <code>pickled_tensor1</code> and <code>pickled_tensor2</code> respectively:</p>
<pre><code class="lang-auto">b'\x80\x04\x95\x98\x01\x00\x00\x00\x00\x00\x00\x8c\x0ctorch._utils\x94\x8c\x12_rebuild_tensor_v2\x94\x93\x94(\x8c\rtorch.storage\x94\x8c\x10_load_from_bytes\x94\x93\x94B\r\x01\x00\x00\x80\x02\x8a\nl\xfc\x9cF\xf9 j\xa8P\x19.\x80\x02M\xe9\x03.\x80\x02}q\x00(X\x10\x00\x00\x00protocol_versionq\x01M\xe9\x03X\r\x00\x00\x00little_endianq\x02\x88X\n\x00\x00\x00type_sizesq\x03}q\x04(X\x05\x00\x00\x00shortq\x05K\x02X\x03\x00\x00\x00intq\x06K\x04X\x04\x00\x00\x00longq\x07K\x04uu.\x80\x02(X\x07\x00\x00\x00storageq\x00ctorch\nFloatStorage\nq\x01X\x0f\x00\x00\x00140382183041680q\x02X\x03\x00\x00\x00cpuq\x03K\x04Ntq\x04Q.\x80\x02]q\x00X\x0f\x00\x00\x00140382183041680q\x01a.\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00?\x94\x85\x94R\x94K\x00K\x02K\x02\x86\x94K\x02K\x01\x86\x94\x89\x8c\x0bcollections\x94\x8c\x0bOrderedDict\x94\x93\x94)R\x94t\x94R\x94.'
b'\x80\x04\x95\x98\x01\x00\x00\x00\x00\x00\x00\x8c\x0ctorch._utils\x94\x8c\x12_rebuild_tensor_v2\x94\x93\x94(\x8c\rtorch.storage\x94\x8c\x10_load_from_bytes\x94\x93\x94B\r\x01\x00\x00\x80\x02\x8a\nl\xfc\x9cF\xf9 j\xa8P\x19.\x80\x02M\xe9\x03.\x80\x02}q\x00(X\x10\x00\x00\x00protocol_versionq\x01M\xe9\x03X\r\x00\x00\x00little_endianq\x02\x88X\n\x00\x00\x00type_sizesq\x03}q\x04(X\x05\x00\x00\x00shortq\x05K\x02X\x03\x00\x00\x00intq\x06K\x04X\x04\x00\x00\x00longq\x07K\x04uu.\x80\x02(X\x07\x00\x00\x00storageq\x00ctorch\nFloatStorage\nq\x01X\x0f\x00\x00\x00140382172016592q\x02X\x03\x00\x00\x00cpuq\x03K\x04Ntq\x04Q.\x80\x02]q\x00X\x0f\x00\x00\x00140382172016592q\x01a.\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00?\x94\x85\x94R\x94K\x00K\x02K\x02\x86\x94K\x02K\x01\x86\x94\x89\x8c\x0bcollections\x94\x8c\x0bOrderedDict\x94\x93\x94)R\x94t\x94R\x94.'
</code></pre>
<p>My question is why is it happening and how can I prevent this?</p>
<p>I am using Python 3.8; pytorch 1.7.0</p>
<p>Cheers, Hlib.</p>
How to implement a function whose input and output vectors have the same gradient variance?

username: mrakgr
cooked: <p>I want to implement a PyTorch <code>Function</code> that takes in another function and a tensor as arguments, applies it, and on the backward pass matches the gradient variances between the input and the output. Something like the following code.</p>
<pre><code class="lang-auto">import torch
from torch.autograd import Function

# Is meant to be arbitrary.
def f(x): 
    return x*2

class VarianceMatch(Function):
    @staticmethod
    def forward(ctx, x):
        y = f(x)
        y_grad_fn = y.grad_fn # Results in None
        ctx.save_for_backward(x,y,y_grad_fn)
        return y

    @staticmethod
    def backward(ctx, grad_y):
        x,y,y_grad_fn = ctx.saved_tensors
        y_grad_fn(ctx,grad_y)
        x.grad *= torch.sqrt(torch.square(grad_y).sum(1) / torch.square(x.grad).sum(1))
        return None

i = torch.scalar_tensor(2,requires_grad=True)
x = VarianceMatch.apply(i)
x.backward(torch.scalar_tensor(1))
print(i.grad)
</code></pre>
<p>The trouble is that during the forward pass, the backwards function does not get instantiated so I cannot extract and save it for the backwards pass. Would it be possible to enable this somehow? Also what would be the ideal way to implement something like this?</p>
<p>This question is related to the <a href="https://stackoverflow.com/questions/67708650/how-to-reuse-the-backward-pass-primitives" rel="noopener nofollow ugc">SO one</a> I asked recently</p>

username: mrakgr
cooked: <pre><code class="lang-auto">import torch
from torch.autograd import Function

class VarianceMatch(Function):
    @staticmethod
    def forward(ctx, x):
        y : torch.Tensor = x * 2
        y.requires_grad_()
        y.retain_grad() # Does not work.
        def h(grad): # y.grad is None so this gives me an exception.
            return grad * torch.sqrt(torch.square(y.grad).sum(1) / torch.square(grad).sum(1))
        x.register_hook(h)
        return y

    @staticmethod
    def backward(ctx, grad_y):
        return grad_y * 2

i = torch.scalar_tensor(2,requires_grad=True)
x = VarianceMatch.apply(i)
x.backward(torch.scalar_tensor(1))
print(i.grad)
</code></pre>
<p>I am trying to hack it using hooks, but that is not working for me either since the output <code>y</code> does not have the gradient. At this point I am out of ideas.</p>
How to convert torch.norm to cosine distance

username: JPKikori
cooked: <p>I want to change norm distance to cosine distance, help me convert this function to cosine distance</p>
<pre><code class="lang-auto">def feat_prototype_distance(self, feat):
        N, C, H, W = feat.shape
        feat_proto_distance = -torch.ones((N, self.class_numbers, H, W)).to(feat.device)
        for i in range(self.class_numbers):
            feat_proto_distance[:, i, :, :] = torch.norm(self.objective_vectors[i].reshape(-1,1,1).expand(-1, H, W) - feat, 2, dim=1)
        return feat_proto_distance
</code></pre>
<p>This is original function using norm distance with shapes:<br>
self.objective_vectors[i].reshape(-1,1,1).expand(-1, H, W): torch.Size([256, 128, 224])<br>
feat: torch.Size([8, 256, 128, 224])  with 8 is batch_size</p>

username: sh0416
cooked: <pre><code class="lang-auto">def get_cosine_distance(x: Tensor, y: Tensor):
  """
  :param x: [B1, D]
  :param y: [B2, D]
  :return: [B1, B2]
  """
  x_norm = torch.norm(x, dim=1, keepdim=True)
  y_norm = torch.norm(y, dim=1, keepdim=True)
  return torch.mm(x, y.transpose(0, 1)) / x_norm / y_norm.transpose(0, 1)

get_cosine_distance(objective_vectors[i].reshape..., feat)
</code></pre>
<p>I think replacing the <code>torch.norm</code> part in your code with <code>get_cosine_distance</code> would resolve your issue.<br>
Is it what you want?</p>

username: JPKikori
cooked: <p>I have tried but no success. Because feat_proto_distance[:, i, :, :] is 4 dimensional but it returns 2 dimensional</p>

username: JPKikori
cooked: <p>Help me! <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"> :(((((((((((((((((((((((</p>
Outer product for batched and unbatched data

username: Pablo
cooked: <p>Hello everyone.</p>
<p>In my code I’ll need to perform the outer product for both unbatched and batched data (tensors with dimension T1(Ndim) and T2(B, Ndim) respectively).</p>
<p>For now I was using:</p>
<pre><code class="lang-auto">prod = T1[:, None] * T1 # Unbatched
prod = T2[:, :, None] * T2[:, None, :] # Batched
</code></pre>
<p>But, is there a way to have an unified expression for both computations?</p>
<p>Thanks!</p>

username: Pablo
cooked: <p>Well I found the solution. This works for both cases:</p>
<pre><code class="lang-auto">prod = T[..., None, :] * T[..., None]
</code></pre>
<p>where <code>T</code> can be either <code>T1</code> or <code>T2</code>.</p>
How to properly clean up the GPU memory occupied by PyTorch when using Flask?

username: wowdeeply
cooked: <p>Here is the code:</p>
<pre><code class="lang-auto">app = Flask('test_api')
@app.route('/recognition', methods=['post','get'])
def recognition():
    imgid =request.values.get("ImgId")
    addtime = request.values.get("DateTime")
    img = request.values.get("Image")
    img = img.replace(' ', '+')
    torch.cuda.empty_cache()
    gc.collect()

    img_b64decode = base64.b64decode(img) # base64解码
    #num_img = np.fromstring(img_b64decode, np.uint8) # 转换np序列
    num_img = np.array(Image.open(BytesIO(img_b64decode)))
    predict = test_net()

if __name__ == '__main__':
    ...
    app.run(port=6321, debug='true', host="0.0.0.0", threaded=True)
</code></pre>
<p>After running a few times, the GPU memory will be full.<br>
I don’t know if it is related to Flask, I would like to ask you for advice.</p>
<p>Python: 3.6.8<br>
Flask: 2.0.0<br>
PyTorch: 1.5.1+cu101<br>
GPU: RTX 2070</p>

username: ptrblck
cooked: <p>Make sure you are not storing unnecessary data, such as the computation graphs by appending the output, loss, etc. of the model in e.g. a <code>list</code>. If you only care about the inference results, you could also wrap the model in a <code>with torch.no_grad()</code> block, if that isn’t already used. Besides that I cannot see any obvious issues in the posted code, but it also doesn’t show a lot.</p>

username: wowdeeply
cooked: <p>Thank you for your reply.<br>
I tried with <code>torch.no_grad()</code>. The result is that GPU memory usage will fluctuate up and down, and the program can run normally at first. But after running for a period of time, the GPU memory will be full in the later stage.<br>
Moreover, “GPU memory is full” is not always full, but still “fluctuates up and down”, that is to say, it can work occasionally.</p>

username: ptrblck
cooked: <p>Since you are emptying the cache it’s expected that the memory usage fluctuates, otherwise PyTorch would reuse the cached memory.<br>
In case you are running out of memory after a while without seeing an increased memory usage in each iteration, I would guess that some inputs might be larger than others (could this be the case)?</p>
How is weighted NLL loss calculated

username: ashwani-bhat
cooked: <p>Can anyone give me the formula for weighted NLL. I want to know where weights are getting multiplied. I have read this <a href="https://pytorch.org/docs/stable/generated/torch.nn.NLLLoss.html" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.NLLLoss.html</a> but I am not able to understand what <code>$x_{n,y_n}$</code> is basically. I want to write the formula for weight NLL for C classes. Is the following correct (consider I have correct labels and incorrect labels):</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/b/4b5890b588a5c35f9bd70b808abd7ca128ac66ed.png" alt="image" data-base62-sha1="aKxyKyJ0uuSZPKWdubEfhf6hT5z" width="417" height="79"></p>
<p>​</p>

username: ptrblck
cooked: <p><code>x</code> corresponds to the logits given as the model output.<br>
<a href="https://discuss.pytorch.org/t/does-weightedrandomsampler-increase-interations/51514/16">Here</a> is a comparison between a manual approach and the weighted criterion.</p>

username: ashwani-bhat
cooked: <p>Hi. Thanks for the answer. Can you please tell me whether the formula I have written above for reduction==‘mean’ is correct or not?</p>

username: ptrblck
cooked: <p>I don’t know how <code>y</code> and <code>y'</code> are defined, e.g. is <code>y</code> a one-hot encoded target tensor acting as a mask (so alternatively also indexing would work using the class indices directly)?<br>
If so, I guess the class index in <code>w</code> would be missing but you could also compare it to the used formula in <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss"><code>nn.CrossEntropyLoss</code></a> to make sure it’s equivalent.</p>
How to include correlation among parameter vectors in the cost function?

username: rakib
cooked: <p>I have the following model:</p>
<pre><code>class Model(nn.Module):
    def __init__(self, dim_in, lambda_=.3):
        super(FeatExtractorGR, self).__init__()

       '''The linear transform layer'''
        self.lt = nn.Linear(in_features = dim_in, out_features = 10, bias=True)
    
    '''The encoder'''
    self.feature_extractor = \
        nn.Sequential(
            #nn.Linear(in_features = 10, out_features = 30, bias=True),
            nn.BatchNorm1d(10),
            nn.ReLU(),
            nn.Linear(in_features = 10, out_features = 20, bias=True),
            nn.BatchNorm1d(20),
            nn.ReLU(),
            nn.Linear(in_features = 20, out_features = 10, bias=True),
            nn.BatchNorm1d(10),
            nn.ReLU()
        )

def forward(self, x):
    transformed = self.lt(x)
    return self.feature_extractor(transformed)
</code></pre>
<p>I want to force the weight vectors of the linear transformation layer to be uncorrelated. I tried to include the dot products among the vectors in the cost function (as a proxy for correlations among them):</p>
<pre><code>        params=list(model.lt.parameters())[0]
        dotprod=torch.tensordot(params, params, dims=([1],[1])).abs().fill_diagonal_(0).sum()/2

        loss = other_losses + dotprod * weight
</code></pre>
<p>But this is not working, even with really high <code>weight</code>. The weight vectors from the <code>lt</code> layer are still highly correlated. I have also tried to remove <code>other_losses</code>, but no effect. What am I doing wrong?</p>

username: Marie154
cooked: <p>A special feature of biological models is that they usually contain a largeIn this paper, “parameter correlations” means a group of parameters in Based on Eq. (​1), MathML is expressed as vectors of the first order partial derivative functions (1) to a data set (j) by minimizing the following cost function.</p>

username: Marie154
cooked: <aside class="quote no-group quote-modified" data-username="Marie154" data-post="2" data-topic="122180" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/439d5e/48.png" class="avatar"> Marie154:</div>
<blockquote>
<p>A special feature of biological models is that they usually contain a largeIn this paper, “parameter correlations” means a group of parameters in Based on Eq. (​1), MathML is <a href="https://www.prepaidcardstatus.bid/" rel="noopener nofollow ugc">prepaidcardstatus</a> expressed as vectors of the first order partial derivative functions (1) to a data set (j) by minimizing the following cost function.</p>
</blockquote>
</aside>
<p>Thanks for the solution</p>
How to `dot` weights to batch data?

username: jef
cooked: <p>I have batch data and want to <code>dot</code> to the data. <code>W</code> is trainable parameters.<br>
How to <code>dot</code> between batch data and weights?</p>
<pre><code class="lang-auto">hid_dim = 32
data = torch.randn(10, 2, 3, hid_dim)
data = tdata.view(10, 2*3, hid_dim)
W = torch.randn(hid_dim) # assume trainable parameters via nn.Parameter
result = torch.bmm(data, W).squeeze() # error, want (N, 6)
result = result.view(10, 2, 3) # 
</code></pre>
<h2>Update</h2>
<p>This may look good.</p>
<pre><code class="lang-auto">hid_dim = 32
data = torch.randn(10, 2, 3, hid_dim)
data = tdata.view(10, 2*3, hid_dim)
W = torch.randn(hid_dim, 1) # assume trainable parameters via nn.Parameter
W = W.unsqueeze(0).expand(10, hid_dim, 1)
result = torch.bmm(data, W).squeeze() # error, want (N, 6)
result = result.view(10, 2, 3)
</code></pre>

username: SimonW
cooked: <p>First of all, all things should be wrapped into Variable for <code>W</code> to be trainable. Secondly, <code>W</code> has shape <code>(32,)</code>, which is not multiply-able with tensors of shape <code>(2 3)</code>. So I assume that <code>W</code> is of shape <code>(3, 2)</code>.</p>
<p>Then, you can use <code>torch.bmm(data, W.unsqueeze(0).expand(10, 3, 2))</code>. Probably you don’t need <code>unsqueeze</code> but I don’t have access to pytorch right now so you can check your self. <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: jef
cooked: <p><a class="mention" href="/u/simonw">@SimonW</a><br>
Thank you. I updated my post. How about my code in case of using 1 row <code>W</code>.</p>

username: SimonW
cooked: <p>It still doesn’t make sense bmm <code>(10, 2, 3)</code> and <code>(10, 32, 1)</code>. What exactly should be multiplied with each matrix?</p>

username: jef
cooked: <p><code>bmm of (10, 2*3, hid_dim) and (10, hid_dim, 1)</code>, not  (10, 2, 3) and (10, 32, 1).   Sorry for the confusing.</p>

username: SimonW
cooked: <p>Oh I see, sorry I missed the <code>hid_dim</code>. Yeah, I think your code should work. Are you still seeing errors?</p>

username: jef
cooked: <p>No, thank you for your help!</p>

username: Ravi_Raja
cooked: <p>I have tensors of size <strong>A: 32 x 4 x 1</strong> and <strong>B: 4 x 2</strong>.  I want my output tensor to be of shape <strong>32 x 2 x 1</strong></p>
<p>Can you explain, how can I multiply A and B?</p>

username: ptrblck
cooked: <p>This should work:</p>
<pre><code class="lang-python">a = torch.randn(32, 4, 1)
b = torch.randn(4, 2)

c = torch.matmul(b.unsqueeze(0).permute(0, 2, 1), a)
print(c.shape)
&gt; torch.Size([32, 2, 1])
</code></pre>

username: Ravi_Raja
cooked: <p>Thanks  a lot man! It worked.</p>
[Pytorch Geometric] Convert dense adjacency matrix to sparse matrix

username: ChenYizhu97
cooked: <p>Hi, i want to convert a batched dense edge adjacency matrix of size (B,N,N) to a batched sparse edge adjacency matrix of size (2, M), in which B denotes the batch size, N denotes the maximum number of nodes each graph and M denotes the number of edges in one batch.</p>
<p>I could only find one function for this purpose in the package torch_geometric.utils named dense_to_sparse. However, the source code shows that this function does not work for batched data. My solution is to iterate over batch, apply this function on each graph, and aggregate the results to get the batched result. I wonder if this iteration would impact the calculation performance and is there any better way to achieve the same purpose.</p>
<p>The other question is that when i convert a batched sparse edge adjacency matrix of size (2,M) with edge attributes of size (M, F), in which F denotes the dimension of features for each edge, to a batched dense edge adjacency matrix using the function to_dense_adj provided by package torch_geometric.utils, It results a tensor of size (B, N, N, F), but I couldn’t find the function for converting such a tensor back. Is there any convenient way to convert such a tensor back to sparse edge adjacency matrix of size (2,M) and edge attributes of size (M, F)?</p>

username: ChenYizhu97
cooked: <p>For anyone who have the same problem, here is the solution.<br>
<a href="https://github.com/rusty1s/pytorch_geometric/discussions/2628" rel="noopener nofollow ugc">torch_geometric discussion on github</a></p>