akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Calculation of Convolutional Kernel in Pytorch

username: PARUL_JOSHI
cooked: <p>I have an input <strong>k_norm</strong> of shape(N X D) say 32,250<br>
I want to design a convolutional kernel name <strong>Wc</strong> in such a way that the output of :<br>
<strong>Wc * k_norm</strong> would give a result shape of (N X 1)</p>
<p>So,<br>
Wc = torch.nn.Conv2d(in_channels = ?, out_channels = ?, kernel_size=(?))</p>
<p>The formula that I’m using is :<br>
kc_hat = Wc * k_norm</p>
<p>I want kc_hat shape as N X1</p>

username: KFrank
cooked: <p>Hi Parul!</p>
<aside class="quote no-group" data-username="PARUL_JOSHI" data-post="1" data-topic="190110" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/parul_joshi/48/61770_2.png" class="avatar"> PARUL_JOSHI:</div>
<blockquote>
<p>I want to design a convolutional kernel name <strong>Wc</strong> in such a way that the output of :<br>
<strong>Wc * k_norm</strong> would give a result shape of (N X 1)</p>
</blockquote>
</aside>
<p>Could you explain your use case in a little more detail, maybe illustrating it<br>
with some sample code (even if the sample code doesn’t do what you want)?<br>
What goal are you trying to achieve with your specified “result shape?”</p>
<p>A couple of comments:  Pytorch’s <code>Conv2d</code> take a <em>four-dimensional</em> tensor as<br>
input, of shape <code>[nBatch, channels, height, width]</code>.  (It’s permissible for<br>
the <code>nBatch</code> and <code>channels</code> dimensions to have length one, but they still have<br>
to be there.)  Also, you apply such a convolutions, <code>Wc = Conv2d ( stuff )</code><br>
to its input using function-call notation:  <code>kc_hat = Wc (k_norm)</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

username: J_Johnson
cooked: <p>When you take in consideration stride and dilation, there are hundreds of ways you could do this. But, for the sake of simplicity, let’s just assume those are 1 and 1 respectively. In that case, you’d need a kernel size of (1, D):</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

N = 32
D = 250
batch_size = 1
channels = 1

kernel = nn.Conv2d(channels, channels, (1, D), bias = False)

x = torch.rand((batch_size, channels, N, D))

print(kernel(x).size())
</code></pre>
Indexing tensors - the correct method?

username: adv010
cooked: <p>Hi, given 2 tensors, I wanna keep only those elements in tensor1 that are in tensor 2.  I am using the below snippet (output from debugger in italics)</p>
<pre><code class="lang-auto">    def in1d(ar1, ar2):
        mask = ar2.new_zeros((max(len(ar1), len(ar2))), dtype=torch.bool)

*tensor([False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False, False,*
*        False, False, False, False, False, False, False, False, False],*
*       device='cuda:0')*

        mask[ar2.unique()] = True
mask
**&lt;torch.Tensor object at 0x7fc12d7d1c20&gt;*

        return mask[ar1]
*CUDA Device side assert trigerred*
</code></pre>
<p>I noticed that as soon as  I try assigning values as so -  mask[ar2.unique()] = True, the mask tensor doesn’t get printed and instead only shows up as an object. Why?</p>
<p>How should tensors be indexed and masked? Is there a torch-ic way for this?</p>

username: adv010
cooked: <p>Aha, my bad, turns out I was indexing out of bounds. Out of bounds indexing of a cuDA tensor gives cUDA assert error</p>
Sparse DDP w/ gloo broken in 2.1.0?

username: daturkel
cooked: <p>I’ve made a github issue <a href="https://github.com/pytorch/pytorch/issues/111422" rel="noopener nofollow ugc">here</a>, but it’s possible that this forum is the better place to discuss it.</p>
<p>The TL;DR is that we have a model with sparse embeddings training with DDP via pytorch lightning and ray. NCCL doesn’t support allreduce on sparse vectors (yet), so we use the gloo backend. This was working fine until we recently attempted to update to torch 2.1.0, now we get an error:</p>
<p><code>RuntimeError: Backend gloodoes not support allreduce</code></p>
<p>This comes from <a href="https://github.com/pytorch/pytorch/pull/104256/files#diff-47cd01a9a1521afa3b8e11cc2c1c4e07bd3452a95f1947d3a200e15119f3c0c1R89-R96" rel="noopener nofollow ugc">this change</a>. I’m not well-versed enough in torch internals or GPU programming to understand the finer points of <a href="https://github.com/pytorch/pytorch/pull/104256" rel="noopener nofollow ugc">the PR</a>, but it seems odd that it should uniformly disable allreduce_sparse with this <code>TORCH_CHECK(False</code> condition that springs this runtime error.</p>
<p>Am I misunderstanding something, or is this a bug?</p>

username: H-Huang
cooked: <p>Just saw this forum post, definitely a bug and thank you for surfacing it! Will follow up through github</p>
Creating 2D Interpolation of Samples randomly generated from VAE

username: Srinjoy_Mukherjee
cooked: <p>Hey Everyone,</p>
<p>I am trying to generate samples from VAE. I want to achieve similar to the below image(taken from paper), where one image smoothly transitions from one to another.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/e/0e03c753de2053b6bc62f540c125a7659d9b356d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/0e03c753de2053b6bc62f540c125a7659d9b356d" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/e/0e03c753de2053b6bc62f540c125a7659d9b356d.jpeg" alt="image" data-base62-sha1="1ZYMnhmdnUyALGmOYCvNG8N87mJ" width="490" height="500" data-dominant-color="D9D9D9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">512×522 75.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I know how to interpolate between two images: Get the latent vectors z1 and z2 for both image  and interpolate between them i.e. new_z = (1-p)<em>z1 + p</em>z2 and decode them using VAE.decoder.</p>
<p>I am stuck in 2d interpolation.</p>
<p>Thanks</p>
Loading pretrained vgg16 and replacing classifier

username: Jesus
cooked: <p>Hello everyone.</p>
<p>I load a pretrained vgg16 and written some python script to replace the classifier layer and serialize it in a pt file.</p>
<p>"import torch<br>
import torchvision.models as models<br>
import torchvision.transforms as transforms<br>
from PIL import Image<br>
import torch.nn as nn</p>
<p>class Identity(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Identity, self).<strong>init</strong>()</p>
<pre><code>def forward(self, x):
    return x
</code></pre>
<p>model = models.vgg16(weights=“VGG16_Weights.IMAGENET1K_V1”)<br>
model.classifier = Identity()<br>
model.eval()<br>
torch.jit.save(torch.jit.script(model), “vgg16_features.pt”)"</p>
<p>The pretrained vgg16 file weighs around 500mo, but the file obtained after applying the script weighs 58mo. I have the impression that the weights of the pre-trained vgg16 were not set during serialization. Am I wrong?</p>

username: ptrblck
cooked: <p>This would be expected, since the <code>.classifier</code> module contains the majority of the trainable parameters:</p>
<pre><code class="lang-python">model = models.vgg16()

nb_params = 0
for param in model.classifier.parameters():
    nb_params += param.nelement()

print("{} parameters".format(nb_params))
print("{:.3f} MB".format(nb_params * 4 / 1024**2))
# 471.660 MB
</code></pre>

username: Jesus
cooked: <p>I see. Thank you very much sir</p>
Negative dice loss and IoU score for segmentation

username: ronnief1
cooked: <p>I wrote a forum <a href="https://discuss.pytorch.org/t/segmentation-runtimeerror-gather-expected-dtype-int64-for-index/190112/1">post</a> yesterday where I was running a Unet for retinal image segmentation with a resnet50 encoder backbone pretrained on imagenet.</p>
<p>Now that I have it training finally, I am getting a negative IoU score and a really large dice loss even though they are supposed to both be between 0 and 1. My input images are three channels but they look like grayscale images. I think maybe applying the same preprocessing from the resnet50 on imagenet (which are color images) may not be appropriate for gray images. When I look at an example of a training image after preprocessing, it has lost so much information.</p>
<p>This is my image preprocessing:</p>
<pre><code class="lang-auto">data_transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Resize((224, 224)),
    transforms.Normalize(mean=[0.485, 0.456, 0.406],
              std=[0.229, 0.224, 0.225])])
</code></pre>
<p>and my losses:</p>
<pre><code class="lang-auto">loss = smp.utils.losses.DiceLoss()
metrics = [
    utils.metrics.IoU(threshold=0.5),
]
</code></pre>
<p>My masks are size [1,224,224] with values ranging from 0 to 3 for each class and datatype torch.int64.</p>
<p>Edit: also, after preprocessing, a sample training image had a max pixel value of 1.6264 and a min pixel value of -1.9092. If I’m not mistaken, those values should be between 0 and 1, right?<br>
Thanks</p>

username: ronnief1
cooked: <p>Ok I fixed it. I realized in my mask_transform:</p>
<pre><code class="lang-auto">mask_transform = transforms.Compose([
    transforms.PILToTensor(),
    transforms.ConvertImageDtype(torch.int64),
])
</code></pre>
<p>that ConvertImageDtype was transforming the values of the mask from 0,1,2,3 to massive 10+ digit long numbers. I just commented that line out and now I’m getting a dice loss and iou score between 0 and 1.</p>
Is there a way to find out how many times of cuda kernels called/launched?

username: Bob_Liao
cooked: <p>For example, there are five kinds of cuda kernel in a model (conv, add, etc), and when It runs in PyTorch, these kernels will be executed/launched 40 times. So hou could I get the 40 in PyTorch? Using PyTorchProfiler?</p>

username: ptrblck
cooked: <p>Yes, you could use the built-in PyTorch profiler or e.g. Nsight Systems to see summaries of kernel launches as well as the actual timelines.</p>

username: Bob_Liao
cooked: <p>The calls of <code>cudaLaunchKernel</code> in torch profiler means the launched times of kernel?</p>
AttributeError: 'Test' object has no attribute 'maxpool'

username: sams
cooked: <p>Hi,</p>
<p>I’m new to PyTorch, and while running the example code available at this link:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://pastebin.com/pqvx0BxQ">
  <header class="source">
      <img src="https://pastebin.com/favicon.ico" class="site-icon" width="16" height="16">

      <a href="https://pastebin.com/pqvx0BxQ" target="_blank" rel="noopener nofollow ugc">Pastebin</a>
  </header>

  <article class="onebox-body">
    <img width="" height="" src="https://pastebin.com/i/facebook.png" class="thumbnail">

<h3><a href="https://pastebin.com/pqvx0BxQ" target="_blank" rel="noopener nofollow ugc">example codes - Pastebin.com</a></h3>

  <p>Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I encountered the following error:<br>
./venv9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1614, in <strong>getattr</strong><br>
raise AttributeError(“‘{}’ object has no attribute ‘{}’”.format(<br>
AttributeError: ‘Test’ object has no attribute ‘maxpool’</p>
<p>Here is my local environment information:<br>
OS: ubuntu 22.04<br>
Python: 3.9.18<br>
torch: 2.0.1+cu118</p>
<p>I’m unsure why I’m getting this error, if someone can provide me with any hints or guidance, it would be much appreciated.</p>
<p>Thanks,<br>
Sam</p>

username: ptrblck
cooked: <p>Could you post a minimal and executable code snippet here as I cannot reach the pastebin snippet?</p>

username: jonathanjsmith
cooked: <p>Make sure the <code>__init__</code> method for your <code>Test</code> class is spelled correctly, and also when you call the superclass constructor.</p>
<p>Replacing both  <code>__int__</code> with <code>__init__</code> should work.</p>

username: sams
cooked: <p>Thank you all so much! I just realized the typo; after replacing <code>__int__</code> with <code>__init__</code>, it now works perfectly.</p>
Can any experts direct me , to install pytorch?

username: Teddy_Salas
cooked: <p>MSI Katana GF66 Core i7 11th Gen 11800H-(16 GB/512 GB SSD/Windows 10|Home/ 6 GB Graphics/NVIDIA GeForce RTX 3060/144Hz)Katana GF66 11UE-861IN this my laptop configuration , how to install pytorch in this laptop? i tried it earlier by using command from pytorch website, but i am seeing an error(attaching the screenshot), can any one help?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/1/61c4f0ad222b112f5928d6af5b5ff138fd3c36de.png" data-download-href="https://discuss.pytorch.org/uploads/default/61c4f0ad222b112f5928d6af5b5ff138fd3c36de" title="Screenshot 2023-10-15 133844"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/1/61c4f0ad222b112f5928d6af5b5ff138fd3c36de_2_690x388.png" alt="Screenshot 2023-10-15 133844" data-base62-sha1="dWUfVGkuZqZ9TuBzCmQrRS76Bxc" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/1/61c4f0ad222b112f5928d6af5b5ff138fd3c36de_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/1/61c4f0ad222b112f5928d6af5b5ff138fd3c36de_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/1/61c4f0ad222b112f5928d6af5b5ff138fd3c36de_2_1380x776.png 2x" data-dominant-color="191919"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-10-15 133844</span><span class="informations">1920×1080 153 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
LLAMA : Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

username: O_Hwang_Kwon
cooked: <p>Hi. I built an sentiment classification model using LLAMA. (just adding an output layer that output 3 logits) Since llama is a huge model, I quantized it using the code snippet below.</p>
<pre><code class="lang-auto">def create_quantization_config():
            #"""
            #Quantization Config Generator
            #This function will create a confuguration for 4-bit quantization.
            #"""
            bnb_config=BitsAndBytesConfig(
                load_in_8bit=True, 
                bnb_4bit_use_double_quant=True,
                bnb_4bit_quant_type='fp4',
                bnb_4bit_compute_dtype=torch.bfloat16
                )
            return bnb_config

</code></pre>
<p>With the quantized model, I trained my model with my trainer class as below.</p>
<pre><code class="lang-auto">    def _run_batch(self,src:list,tgt:Tensor)-&gt;float:
        # Running each batch
        self.optimizer.zero_grad() 
        src[0]=src[0].to(self.gpu_id)
        src[1]=src[1].to(self.gpu_id)
        tgt=tgt.to(self.gpu_id)
        self.model=self.model.to(self.gpu_id)
        out=self.model(src[0].to(self.gpu_id),src[1].to(self.gpu_id)).to(self.gpu_id)
        loss=self.criterion(out,tgt)
        loss.backward()
        self.optimizer.step()

        self.train_acc.update(out,tgt)
        self.val_acc.update(out,tgt)
        return loss.item()
</code></pre>
<p>However, when I try to fine tune llama on 2 GPUs, I am getting</p>
<pre><code class="lang-auto">in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)
</code></pre>
<p>This is interesting because when I did the exact same thing with BERT and GPT2, there was no problem. I even trained those model with 4 GPUs(single node), but no issues. It may have something to do with the quantization I think, but I am not sure. If anybody have any idea?</p>
<p>Thanks in advance.</p>
Loading state_dict after code refactoring gives import error

username: Alon_Hazan
cooked: <p>Using the standard code to save a model:</p>
<pre><code class="lang-auto">checkpoint = {
            'model': model,
            'state_dict': model.state_dict(),
            'optimizer': optimizer,
            'scheduler': scheduler
        }
torch.save(checkpoint, path_out)
</code></pre>
<p>Since refactoring my code, the architecture Class is in a different name so i get an error trying to load the model: <code>ModuleNotFoundError: No module named '.....'</code> which makes sense, it isn’t there <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
<p>My current solution is to create a fake module with an empty Class using the old module name, and once loaded, I just load the weights: <code>model.load_state_dict(state_dict=state_dict)</code><br>
But this solution is not elegant to say the least.</p>
<p>using load with <code>weights_only</code> returns similar error.</p>
<p>Is there a way to “force” load the file? or perhaps load only the state_dict without trying to load the “model” part?</p>

username: ptrblck
cooked: <p>You are not using the standard and recommended approach since you are storing the model directly besides its <code>state_dict</code>. Storing <code>model</code> will add the requirement to restore the same file location, so remove <code>'model': model</code> and just store the <code>state_dict</code> instead to avoid such issues.</p>
How to specify CUDA device number in C++?

username: dambo
cooked: <p>Hello,<br>
I have four different libtorch services and also four GPU’s available. In PyTorch its is quite easy to state the GPU device, how do I do it in C++ so that each service is assigned to a different GPU?</p>
<p>Thank,</p>

username: ptrblck
cooked: <p>The <code>.to()</code> operation should also work in C++ as described <a href="https://discuss.pytorch.org/t/moving-model-to-cuda-in-c/26786/16">here</a>.</p>

username: dambo
cooked: <p>Thanks, for reference:<br>
Change:</p>
<pre><code class="lang-auto">torch::jit::script::Module module = torch::jit::load(s_model_name, torch::kCUDA);
</code></pre>
<p>To:</p>
<pre><code class="lang-auto">torch::jit::script::Module module = torch::jit::load(s_model_name, torch::kCUDA);
const std::string device_string = "cuda:2";
module.to(device_string);
</code></pre>
<p>And:</p>
<pre><code class="lang-auto">input_tensor = input_tensor.to(at::kCUDA);
</code></pre>
<p>To:</p>
<pre><code class="lang-auto">input_tensor = input_tensor.to(device_string);
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/b/b6e9210c18cd80a2a567e8db2e8630b366884978.png" data-download-href="https://discuss.pytorch.org/uploads/default/b6e9210c18cd80a2a567e8db2e8630b366884978" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/b/b6e9210c18cd80a2a567e8db2e8630b366884978.png" alt="image" data-base62-sha1="q66nkRx50Ll600z17sLytwjmTws" width="690" height="232" data-dominant-color="1E201E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1185×400 36.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: SeptOct_Li
cooked: <p>nice! very simple, thank you very much!</p>

username: mamsds
cooked: <p>Confirm that this works!</p>

username: MaxC2
cooked: <p>Maybe too late, but after reading the correction, I suggest to not use raw string to specify your device number, but instead use a little helper like:</p>
<pre><code class="lang-auto">auto get_cuda_device(const int number) -&gt; std::string
{
     assert(number &gt;= 0 &amp;&amp; number &lt;= torch::cuda::device_count());
     return "cuda:" + std::to_string(number);
}
</code></pre>
<p>just replace:</p>
<pre><code class="lang-auto">const std::string device_string = "cuda:2";
</code></pre>
<p>by</p>
<pre><code class="lang-auto">auto device_string = get_cuda_device(2):
</code></pre>
<p>This way of doing the code prevents:</p>
<ol>
<li>Problems with hardware changes</li>
<li>Typo problems when using several gpu numbers.</li>
</ol>
How to hold a nn:Module in class member

username: MaxC2
cooked: <p>In C++, a <code>std::shared_ptr</code> devired from a base class can hold a subclass, but in the pimpl instanciation way of libtorch, I don’t see how to do the same. For example:</p>
<p>In the constructor of a theorical encoder class, some (libtorch) modules member can be from different type:</p>
<pre><code class="lang-auto">
		if (freq)
			conv = register_module("conv", 
				   torch::nn::Conv2d(
				   torch::nn::Conv2dOptions(chin, chout, { kernel_size, 1 })
				  .stride({ stride, 1 })
				  .padding({ padding, 0 })));
		else
			conv = register_module("conv", 
				   torch::nn::Conv1d(
				   torch::nn::Conv1dOptions(chin, chout, kernel_size)
				  .stride(stride)
				  .padding(padding)));
</code></pre>
<p>or</p>
<pre><code class="lang-auto">        if (norm)
			norm1 = register_module("norm1",
					torch::nn::GroupNorm(
					torch::nn::GroupNormOptions(norm_groups, chout)));
		else
			norm1 = register_module("norm1",
					torch::nn::Identity());
</code></pre>
<p>In a first instance, I wrote class member like that:</p>
<pre><code class="lang-auto">torch::nn::Module conv;
torch::nn::Module norm1;
</code></pre>
<p>Do not work, probably because the pimpl-like instanciation method (macro for module holder). Sorry, i’m really beginner with the libtorch api so I can be wrong.</p>
<p>The other idea is to use <code>std::variant</code> but that introduce a complexity and broke the direct access to the process by the <code>operator()</code>.</p>
<p>Is there a simple way to hold variant like module in a class member?</p>

username: MaxC2
cooked: <p>Self-Reply:</p>
<p>The solution is pretty trivial, just use <code>torch::nn::AnyModule</code> as class member.</p>
Why dataset needs to be moved to loader?

username: osar
cooked: <p>Hello,<br>
I am wondering why the dataset needs to be moved to the loader? Indeed, all examples for creating a loader are like this:</p>
<pre><code class="lang-auto">auto loader = torch::data::make_data_loader&lt;torch::data::samplers::RandomSampler&gt;(std::move(dataset), batch_size);
</code></pre>
<p>If we don’t move the dataset to the loader, like this:</p>
<pre><code class="lang-auto">auto loader = torch::data::make_data_loader&lt;torch::data::samplers::RandomSampler&gt;(dataset, batch_size);
</code></pre>
<p>everything seems to work fine, and we moreover still have access to the dataset, for example to request its size with dataset.size().value(). If the dataset is moved to the loader, the size request returns 0.<br>
So, can anyone explain me why moving the dataset to the loader is mandatory, and what problems I might expect if I don’t do it?<br>
Thank you very much for your help.</p>

username: osar
cooked: <p>Answering to myself…<br>
Not moving the dataset means that the loader makes a <strong>copy</strong> of the dataset, which can lead to memory overflow for huge datasets. Moving the dataset means that the memory allocated for it is passed to the loader, ensuring that we do not use twice the memory needed. But then, the pointer to the dataset is lost in the main program, which implies that its size was stored in a variable (if needed) before the loader is constructed. But in the normal flow of learning, there should normally be no need to still have a reference to the dataset after loader creation.<br>
Hope I am right, and hope it helps.</p>

username: MaxC2
cooked: <p>The underlaying of the data loader constructor is:</p>
<pre><code class="lang-cpp">DataLoaderBase(
      DataLoaderOptions options,
      std::unique_ptr&lt;Dataset&gt; main_thread_dataset = nullptr)
      : options_(std::move(options)),
        main_thread_dataset_(std::move(main_thread_dataset)),
        sequencer_(new_sequencer()) {}
</code></pre>
<p>and the <code>torch::data::make_data_loader&lt;torch::data::samplers::RandomSamper&gt;</code> is a generic wrapper which points towards another <code>torch::data::make_data_loader</code> for stateless.</p>
<pre><code class="lang-auto">template &lt;typename Dataset, typename Sampler&gt;
torch::disable_if_t&lt;
    Dataset::is_stateful,
    std::unique_ptr&lt;StatelessDataLoader&lt;Dataset, Sampler&gt;&gt;&gt;
make_data_loader(Dataset dataset, Sampler sampler, DataLoaderOptions options) {
  return torch::make_unique&lt;StatelessDataLoader&lt;Dataset, Sampler&gt;&gt;(
      std::move(dataset), std::move(sampler), std::move(options));
}
</code></pre>
<p>the <code>StatelessDataLoader</code> base class is <code>DataLoaderBase</code>.</p>
<p>As you can see the <code>Dataset</code> is hold by a <code>std::unique_ptr</code>.</p>
<p>So, the <code>std::move</code> or not <code>std::move</code> is not really a problem or a choice. It depends on the permissiveness of the compiler, some compilers make the move explicit, others consider it implicit.</p>
<p>Yours seems to handle this implicitly.</p>
Augmentation with mmpose 0.29

username: Sylvain_Ard
cooked: <p>Hello I am using mmpose 0.29 and want to do an augmentation in my training images database with rotating of 90, 180, 270 degrees and flip horizontal or vertical please !<br>
Thank you<br>
Best regards</p>

username: ptrblck
cooked: <p>Neither of your posts describes what you have tried and where you are stuck. This one also seems to be quite similar to <a href="https://discuss.pytorch.org/t/augmentation-with-detectron2/190233">this topic</a>.<br>
As discussed before, we would expect more information from your side as well as some implementations from your side as you cannot expect users to provide a working solution directly.</p>
Augmentation with detectron2

username: Sylvain_Ard
cooked: <p>Hello I want to do an augmentation comprising rotating 90, 180, 270 degrees, flip vertical and horizontal to my training database.<br>
I use detectron2 with configs/COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml<br>
Please help me !<br>
Best regards</p>
Understanding Mixed Distribution likelihood

username: pumplerod
cooked: <p>In trying to better my understanding of mixed distributions, I’m confused why a sampling of values entirely within a subset of the mixed distribution would result in a relatively low likelihood.</p>
<p>My example…<br>
I create two separate distributions (<code>A_dist</code>,<code> B_dist</code>) as well as a mixed distribution ( <code>mix_dist</code>), where <code>mix_dist</code> results in a sampling from both A and B.</p>
<pre><code class="lang-auto">A_means = torch.tensor( [1.75, 1.75])
A_stdevs = torch.tensor( [0.3, 0.3])
B_means = torch.tensor( [-0.5, -0.25])
B_stdevs = torch.tensor( [0.25, 0.25])
AB_means = torch.vstack( [ A_means, B_means])
AB_stdevs = torch.vstack( [ A_stdevs, B_stdevs])

A_dist = torch.distributions.Normal( A_means, A_stdevs)
B_dist = torch.distributions.Normal( B_means, B_stdevs)
AB_dist = torch.distributions.Independent( torch.distributions.Normal( AB_means, AB_stdevs), 1)
mix_weight = torch.distributions.Categorical( torch.tensor( [1.0, 1.0]))
mix_dist = torch.distributions.MixtureSameFamily( mix_weight, AB_dist)
</code></pre>
<p>To verify the distributions are as I expect, I’ve created a scatter plot…</p>
<pre><code class="lang-auto">A_samp = A_dist.sample( (500,))
B_samp = B_dist.sample( (500,))
mix_samp = mix_dist.sample( (500,))
plt.scatter( A_samp[:,0], A_samp[:, 1], alpha=0.3, label='A')
plt.scatter( B_samp[:,0], B_samp[:, 1], alpha=0.3, label='B')
plt.scatter( mix_samp[:, 0], mix_samp[:, 1], alpha=0.3, label='mix')
plt.legend()
</code></pre>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/f/efaf9b649e8eed8bf5ed432c3d8cced1d7db05f7.png" alt="mixed_gauss_dist" data-base62-sha1="ycmlNXXdsDeLF93RmfdfK4BtL5t" width="377" height="245"><br>
To determine likelihood I am taking <code>torch.exp( DIST.log_prog( SAMPLE))</code></p>
<pre><code class="lang-auto">print( f"Likelihood of A samples within A dist  : {torch.exp( A_dist.log_prob( A_dist.sample((5000,))).mean( )):0.4f}")
print( f"Likelihood of B samples within A dist  : {torch.exp( A_dist.log_prob( B_dist.sample((5000,))).mean( )):0.4f}")
print( f"Likelihood of A samples within mix dist: {torch.exp( mix_g.log_prob( A_dist.sample((5000,))).mean( )):0.4f}")
print( f"Likelihood of B samples within mix dist: {torch.exp( mix_g.log_prob( B_dist.sample((5000,))).mean( )):0.4f}")
</code></pre>
<p>which shows…</p>
<pre><code class="lang-auto">Likelihood of A samples within A dist  : 0.8089
Likelihood of B samples within A dist  : 0.0000
Likelihood of A samples within mix dist: 0.3318
Likelihood of B samples within mix dist: 0.4642
</code></pre>
<p>It make sense to me that A samples would have a high likelihood of falling within the A distribution.  And that B samples would have low/no likelihood of falling within the A distribution.  However, since samples from both A and B fall entirely within the mix distribution, shouldn’t they result in a higher likelihood of falling within the mix distribution?</p>
Split CNN Feature Map

username: gourav_kumar
cooked: <p>Hi,<br>
I have the Input image size:  torch.Size([2, 3, 128, 128]),  After 2nd Conv size is  [2, 32, 128, 128] (nn.Conv2d(ch_in, ch_out, kernel_size=3, stride=1, padding=1, bias=False) ) I want to divide this feature map into 4 feature block of [2,32,32,32],  and pass to different attention Model.  After combining it should be same as initial size  [2, 32, 128, 128].</p>
<p>My question is, How to split the Feature map into 4 parts and add it back?</p>
<p>Thank you.</p>

username: ptrblck
cooked: <p>You could use <code>F.fold/.unfold</code> as seen e.g. <a href="https://discuss.pytorch.org/t/fold-unfold-x-x/17437/3">here</a>.</p>
Cuda:12.2 pytorch cuda:12.1 is this working?

username: Wayne_127
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/2/c2a67696841520e6d33ec8f0a1df5a577886b89c.png" data-download-href="https://discuss.pytorch.org/uploads/default/c2a67696841520e6d33ec8f0a1df5a577886b89c" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/2/c2a67696841520e6d33ec8f0a1df5a577886b89c.png" alt="image" data-base62-sha1="rLXh3uoKQK0VUNkkOQIqQbHeryk" width="690" height="271" data-dominant-color="1A1A1A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1260×495 15.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>PyTorch binaries ship with their own CUDA dependencies and you would only need to install a supported NVIDIA driver.</p>

username: HELLORPG
cooked: <p>And also, the ‘nvidia-smi’ only shows the latest CUDA version that you can use, but not the exact version you actually installed.</p>
Where should I shuffle?

username: cinjon
cooked: <p>I have a DataPipe that look like the following. I’d like to shuffle it, but I am unsure when to add the shuffle in for optimal performance. Right now, I am doing it after the <code>list_files_by_fsspec</code> (as shown) because that’s been reasonable wrt timing. However, this is definitely suboptimal because it means only the tars are shuffled and not the examples after the <code>ExpandDataSource(dp)</code> call, which will make it so multiple examples from the same wav are added to the dataset.</p>
<p>Is there a better way to do this that optimizes both of these?</p>
<pre><code class="lang-auto">def groupby(row):
    return os.path.basename(row[0]).split(".")[0]

def process(row):
    if row[0][0].endswith('json'):
        stream_json_wrapper = row[0][1]
        stream_wav_wrapper = row[1][1]
    else:
        stream_json_wrapper = row[1][1]
        stream_wav_wrapper = row[0][1]
    labels = stream_json_wrapper.read()
    labels = json.loads(labels.decode('utf-8'))
    wav = io.BytesIO(stream_wav_wrapper.read())
    wav, _ = torchaudio.load(wav)
    return wav, labels


dp = IterableWrapper(["s3://&lt;bucket&gt;/train"])
dp = dp.list_files_by_fsspec()
dp = dp.shuffle()
dp = dp.open_files_by_fsspec(mode="rb")
dp = dp.load_from_tar("r")
dp = dp.groupby(groupby, group_size=2, guaranteed_group_size=2)
dp = dp.map(process))
dp = ExpandDataSource(dp)
</code></pre>
Why is my DataLoader so slow when using DataPipe?

username: cinjon
cooked: <p>I have a DataPipe and DataLoader that look like the following.</p>
<p>When I include iterate through it using the datapipe <code>dp</code>, it’s reasonably fast, even if I add on a <code>batch(16)</code>. However, when I iterate through it using the DataLoader, it never produces anything. What’s going on? I would like to use DataLoader so that I can take advantage of having multiple workers.</p>
<pre><code class="lang-auto">def groupby(row):
    return os.path.basename(row[0]).split(".")[0]

def process(row):
    if row[0][0].endswith('json'):
        stream_json_wrapper = row[0][1]
        stream_wav_wrapper = row[1][1]
    else:
        stream_json_wrapper = row[1][1]
        stream_wav_wrapper = row[0][1]
    labels = stream_json_wrapper.read()
    labels = json.loads(labels.decode('utf-8'))
    wav = io.BytesIO(stream_wav_wrapper.read())
    wav, _ = torchaudio.load(wav)
    return wav, labels


dp = IterableWrapper(["s3://&lt;bucket&gt;/val"])
dp = dp.list_files_by_fsspec()
dp = dp.shuffle()
dp = dp.open_files_by_fsspec(mode="rb")
dp = dp.load_from_tar("r")
dp = dp.groupby(groupby, group_size=2, guaranteed_group_size=2)
dp = dp.map(process))

dl = DataLoader(dataset=dp, batch_size=16, num_workers=4)
</code></pre>
Training fcn_resnet50 from scratch on PASCAL VOC

username: c_youuuuu
cooked: <p>Hi everyone!</p>
<p>Thanks for dropping by, and apologies if this is a dumb post but this is my first big project in Deep Learning and Computer Vision. I’m trying to training the fcn_resnet50 on the PASCAL VOC 2012 SegmentationClass dataset. But I’ve been constantly getting a low mIoU score. I’ve also tried to visualize the masks generated by the model but they seem to reflect the poor performance.</p>
<pre><code class="lang-auto">import torch
import os
import csv
from torch import optim
from dataloader import CustomVOCDataset
from torch.utils.data import DataLoader
from torchvision.models.segmentation import fcn_resnet50
from eval import model_eval

train = CustomVOCDataset("VOCtrainval_11-May-2012", "train", True)
val = CustomVOCDataset("VOCtrainval_11-May-2012", "val", False)

train_loader = DataLoader(train, batch_size=16, shuffle=True, num_workers=12)
val_loader = DataLoader(val, batch_size=16, shuffle=True, num_workers=12)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

wsss_model = fcn_resnet50(weights=None, weights_backbone=None, num_classes=21)
wsss_model.to(device)

optimizer = optim.Adam(wsss_model.parameters(), 1e-4)
num_epochs = 200
criterion = torch.nn.CrossEntropyLoss(ignore_index=255).to(device)

batches = len(train_loader)

best_miou = 0.0

for epoch in range(num_epochs):
    wsss_model.train()
    total_loss = 0.0

    for batch_idx, (data, target) in enumerate(train_loader):
        if (batch_idx + 1) % 30 == 0:
            print(f"Now on batch - {batch_idx+1} / {batches}")
        data, target = data.to(device), target.to(device)

        optimizer.zero_grad()
        output = wsss_model(data)
        loss = criterion(output["out"], target)
        loss.backward()
        optimizer.step()

        total_loss += loss.item()

    loss_epoch = total_loss / batches

    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {loss_epoch}")

    # if (epoch + 1) % 3 == 0:
    mean_iou = model_eval(wsss_model, val_loader, 21, device)
    print("Mean IoU on evaluation data: {}".format(mean_iou))
    with open("model_train_log.csv", "a") as file:
        writer = csv.writer(file)
        writer.writerow([epoch + 1, loss_epoch, mean_iou.item()])
        if mean_iou &gt; best_miou:
            best_miou = mean_iou
            torch.save(
                wsss_model.state_dict(),
                "model_weights/voc_model_epoch_{}_weights.pth".format(epoch + 1),
            )
</code></pre>
<p>I have the training script here and for some reason, it always seems to plateau around 18% mIoU on the validation set after training for around 140 epochs. Is this behavior expected? Or have I royally screwed up somewhere?</p>
<p>Thanks for all your help!</p>
Getting an error “UnpicklingError: invalid load key, 'v'.” in Pytorch model deploying in Streamlit

username: Anubhav1107
cooked: <p>I am trying to deploy my wheat detection model of Pytorch in Streamlight. But, I am getting an error as</p>
<pre><code class="lang-auto">UnpicklingError: invalid load key, 'v'.
Traceback:
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)
File "/app/streamlit/streamlit.py", line 84, in &lt;module&gt;
    model.load_state_dict(torch.load(weights_file, map_location=device))
File "/home/appuser/.local/lib/python3.7/site-packages/torch/serialization.py", line 529, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/appuser/.local/lib/python3.7/site-packages/torch/serialization.py", line 692, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
</code></pre>
<p>My Github repo is in <a href="https://github.com/Anubhav1107/streamlit" class="inline-onebox" rel="noopener nofollow ugc">GitHub - Anubhav1107/streamlit</a></p>
<p>Please tell if you want something, Thanks for helping !</p>

username: ptrblck
cooked: <p>I’m not sure how you are trying to load the <code>state_dict</code>, but did you make sure that it was properly downloaded before trying to open it?</p>

username: Anubhav1107
cooked: <p>hey <a class="mention" href="/u/ptrblck">@ptrblck</a>, thanks for replying.<br>
Indeed the problem is related to improper download of the torch file, When I try to access the .pth file(168 MB) through git lfs, it just gives a log file of 12 kb, that is why the error occurs.</p>
<p>So, I was thinking of what is another way to download a weights file, either drive or something else</p>

username: ptrblck
cooked: <p>I’m unsure if this question is related to PyTorch or to e.g. Google Colab / Github.<br>
Are you generally having trouble downloading these files?</p>

username: thepurpleowl
cooked: <p>For future visitors- this might be happening because transformer model sharing makes use of git lfs system. So the pytorch_model.bin, that got downloaded is just a pointer.</p>
<p>As <a class="mention" href="/u/anubhav1107">@Anubhav1107</a> pointed out, you can download the file instead of pointer, and you should not get this error.</p>

username: excode
cooked: <p>Thanks, this worked like a charm!</p>

username: Giacomo_Bocchese2
cooked: <p>Thanks, i’m having the same issue.<br>
and how can i do to download the entire file and not pointer?</p>
<p>i’m working on colab and this is my code for download<br>
!wget <a href="https://huggingface.co/BoccheseGiacomo/longformer-mail/raw/main/longformer-mail.pth" rel="noopener nofollow ugc">https://huggingface.co/BoccheseGiacomo/longformer-mail/raw/main/longformer-mail.pth</a></p>
Segmentation fault when running detect.py during Yolov5 inference on raspberrypi 4

username: nanakwamekankam
cooked: <p>For context, I’m running 64-bit bullseye Raspberry OS on a Raspberry 4 in headless mode via realVNC connection. I have created a virtual environment where I have cloned the yolov5 repository and installed the requirements. After I run the detect.py on the images, I get a segmentation fault error that I am puzzled about. Can someone help?</p>
<p>" detect: weights=[‘yolov5s.pt’], source=/home/pi/yolov5/data/images, data=data/coco128.yaml, imgsz=[416, 416], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=True, save_txt=False, save_csv=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False, dnn=False, vid_stride=1<br>
YOLOv5 <img src="https://discuss.pytorch.org/images/emoji/apple/rocket.png?v=12" title=":rocket:" class="emoji" alt=":rocket:" loading="lazy" width="20" height="20"> v7.0-227-ge4df1ec Python-3.9.2 torch-2.1.0 CPU</p>
<p>Fusing layers…<br>
Segmentation fault"</p>
<p>That’s the error message.</p>
Random initialization of neural networks in PyTorch: how is the seed managed?

username: AlanTuring
cooked: <p>Dear PyTorch community,<br>
in PyTorch, can you tell me if and how i can know if the weights of a neural net are re-initialized with a random seed at each run, i.e., if I run multiple times my .py script, are the generated weights different (i.e., different realizations of a R.V. of the same distribution (uniform I guess))  i.e. not using the same seed?</p>
<p>The question may seem a bit strange but in Matlab, the default behavior is that the same seed is used, resulting in the same sequence of random numbers (a very bad choice in my opinion, since if I wanted the same seq. for reproducibility I would set it manually… but that’s how it is). I am not sure about PyTorch so I would need confirmation because I want to run my neural net multiple times (same architecture) but see the effect of different random initializations?</p>
<p>Thanks.</p>

username: ptrblck
cooked: <p>You can simply <code>print</code> any weight (or e.g. its <code>.double().abs().sum()</code>) and will see that different seeds are used in each run unless you are explicitly seeding the code.</p>

username: AlanTuring
cooked: <p>indeed i see what you mean</p>

username: ptrblck
cooked: <p>Also, alternatively just check the initial seed at the beginning of your script via <code>print(torch.initial_seed())</code>, which should also differ.</p>
Torchaudio Compatibility

username: JohnHerry
cooked: <p>I want to install pytorch==1.13.1 into the python==3.11 env. my cuda==11.7<br>
there is a torch-1.13.1+cu117-cp311-cp311.whl<br>
but no torchaudio-0.13.1+cu117-cp311-cp311,  there is only  cp310 version latest. can I install the torchaudio tagged as cp310 with the torch tagged with cp311 here?</p>

username: ptrblck
cooked: <p>No, since you need to install the wheel built for your Python version.<br>
When <code>torch.1.13.1</code> was released the Python 3.11 build might have been experimental and other libraries, such as <code>torchaudio</code> might not have been built with 3.11 back then.</p>
CPU and CUDA float32 calculations are different

username: huntzhan
cooked: <p>Can you help explain?</p>
<pre><code class="lang-python">import torch

(10000 ** (torch.tensor(2.0, dtype=torch.float32, device='cpu') / 128)).tolist()
1.1547819375991821

(10000 ** (torch.tensor(2.0, dtype=torch.float32, device='cuda:0') / 128)).tolist()
1.1547820568084717
</code></pre>
<p>Simplified version:</p>
<pre><code class="lang-python">(10000 ** torch.tensor(0.015625, dtype=torch.float32, device='cpu')).tolist()
1.1547819375991821
(10000 ** torch.tensor(0.015625, dtype=torch.float32, device='cuda:0')).tolist()
1.1547820568084717
</code></pre>

username: ptrblck
cooked: <p>This is expected since different architectures could use different algorithms/implementations.<br>
Both are correct and show a similar error to a wider <code>dtype</code> reference:</p>
<pre><code class="lang-python">torch.set_printoptions(precision=15)


x_cpu = (10000 ** torch.tensor(0.015625, dtype=torch.float32, device='cpu'))
print(x_cpu)
# tensor(1.154781937599182)

x_gpu = (10000 ** torch.tensor(0.015625, dtype=torch.float32, device='cuda:0'))
print(x_gpu)
# tensor(1.154782056808472, device='cuda:0')

x_ref = (10000 ** torch.tensor(0.015625, dtype=torch.float64, device='cpu'))
print(x_ref)
# tensor(1.154781984689458, dtype=torch.float64)

err_cpu = (x_ref - x_cpu.double())
print(err_cpu)
# tensor(4.709027612292971e-08, dtype=torch.float64)

err_gpu = (x_ref - x_gpu.cpu().double())
print(err_gpu)
# tensor(-7.211901342785154e-08, dtype=torch.float64)
</code></pre>
Weight clipping of the classifer layer

username: papoo13
cooked: <p>I am plotting the norm of classes in a Resnet18 classifier. so if I have this matrice as the last layer [100, 512], where 100 is the classifier size and 512 feature extractors size, the norm of each class will be computed from the corresponding 512 connections. I want to make sure that the norm value represents the logit correctly. This means, that if weight is negative, for example, -3.5, it will contribute to the norm being large but logit not (because we have a Relu as the activation before the classification layer). So, I would like to do weight clipping so the norm of a class represents the logit of that class better. I tried clipping by mapping all the negative values to 0 and keeping all the positive ones. This ruined the training. I was wondering if anyone has some experience in this regard here and can guide me. I am confused how to choose the range for clipping.</p>
Triplet data loader for few-shot learning

username: S_M
cooked: <p>Hello everyone, how can I implement a triplet data loader to map K-way N-shot classification for triplet network training?<br>
I know one thing that anchor is selected from query set while positive and negative are selected from support set. the question is how?<br>
Is there any sample code in PyTorch?</p>
<p>Any help would be really appreciated.</p>
Understanding the Role of resize_out Function in PyTorch Operations

username: linlang1837
cooked: <p>I have a question regarding the usage of the <code>resize_out</code> function in PyTorch operations. Specifically, when I specify an <code>out</code> tensor for an operation like <code>torch.max(x, out=y)</code>, it seems that the <code>resize_out</code> function is called during the execution of the <code>max</code> operation to check the device and dtype of the output tensor (<code>y</code>). However, in the case of operations like <code>torch.mul(x, 3, out=y)</code>, it appears that the <code>resize_out</code> function is not invoked.</p>
<p>I would like to understand under what circumstances the <code>resize_out</code> function is called and when it is not. Are there specific criteria or conditions that determine whether an operation will utilize <code>resize_out</code> to check the device and dtype of the specified output tensor?</p>