akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Load the same batch with different transformation in Dataloader

username: smani
cooked: <p>I need to load a batch of 16 images and process it (Step 1), and then I need to apply a different transformation to each of these images and then process it (Step 2). I need to iterate Step 2 few times. How to do it in PyTorch?</p>

username: ptrblck
cooked: <p>I think you could apply your workflow as described. Load the images and apply the transformations in a loop for each image using the desired configuration.</p>

username: smani
cooked: <p>Currently I am trying like the following:</p>
<pre><code class="lang-auto">def mytransform(self, x):
     for i in x.shape[0]:
          x[i,:,:,:] = self.transform(x[i,:,:,:])
     return x

def train(self):
    for (x,y) in loader:
         xt = self.mytransform(x)
         process(xt)
         ...
</code></pre>
<p>but is there any efficient way to do it?</p>

username: mMagmer
cooked: <aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/LeeDoYup/FixMatch-pytorch">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/LeeDoYup/FixMatch-pytorch" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/af4d86fa64d6dbca29d352d70b2c99c6e9ecf84c6cc462f78274945b45562662/LeeDoYup/FixMatch-pytorch" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/LeeDoYup/FixMatch-pytorch" target="_blank" rel="noopener nofollow ugc">GitHub - LeeDoYup/FixMatch-pytorch: Unofficial Pytorch code for  "FixMatch:...</a></h3>

  <p>Unofficial Pytorch code for  "FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence" in NeurIPS'20. This repo contains reproduced checkpoints. - GitHub - LeeD...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
fixmatch uses strong and weak augmentation on same batch .<br>
it might help you to find a solution.</p>
Tensors with no content but size

username: Kore_ana
cooked: <p><code>torch.zeros(0,3,3)</code>, when printed, yields, <code>tensor([], size=(0, 3, 3))</code>, as opposed to <code>torch.zeros(3,3)</code>, which yields <code>tensor([[0., 0., 0.],         [0., 0., 0.],         [0., 0., 0.]]) </code>]</p>
<p>Can anyone please explain the differences between the two? What is the first tensor? It doesn’t seem to have any values (zeros) inside it, but is of size (0,3,3)… I looked up the documentations on <code>torch.zeros()</code>, but didn’t see anything related to this…</p>
<p>Thank you in advance for your help!</p>

username: tom
cooked: <p>The first has literally 0 = <code>0*3*3</code> elements. The second has 9 = <code>3*3</code> elements.</p>
<p>You didn’t ask, but as background: Zero-sized tensors are not useful per se (and the earliest PyTorch versions didn’t support them), but in many applications it turned out that it is good to have them to more gracefully deal with corner cases.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Kore_ana
cooked: <p>thank you for the answer! Could you please explain what “having 0=033 elements” mean?</p>

username: tom
cooked: <p>It means “Thomas needs to use backticks more”. <img src="https://discuss.pytorch.org/images/emoji/apple/stuck_out_tongue.png?v=10" title=":stuck_out_tongue:" class="emoji" alt=":stuck_out_tongue:"> I edited the post above to show the <code>*</code>.</p>

username: Kore_ana
cooked: <p>Ahh I see <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"> ! Could you please explain, then, the <code>size=(0,3,3)</code> part of the output? When I print out <code>print(torch.zeros(0,3,3).shape)</code> it yields <code>torch.Size([0, 3, 3])</code>. Does this mean that the tensor has no elements (0=<code>0*3*3</code>) , but has an attribute <code>Size</code> that is <code>[0,3,3]</code>?? Shouldn’t the <code>.shape</code> yield zero as the tensor has no elements?</p>

username: tom
cooked: <p>All tensors have some metadata, size is one of them. Also, a 0x3x3 tensor is different to a 0x3 one, but you could not tell the difference if it didn’t print it.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Kore_ana
cooked: <p>I see! Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"> it was really helpful!</p>
Mechanism of represneting PCM16 by float32

username: Felix_astw
cooked: <p>Hello everyone, I am the novice to audio coding. I simply use “torchaudio” to load WAV files whose format is PCM16 with sample rate=16kHz and bit depth=16bits. In addition, datatype of loaded tensor is torch.float32.</p>
<p>My questions are:<br>
(1) What is the mechanism of representing PCM16bits by float32?<br>
(2) Since original data has been quantized in terms of 16bits already,  finer representation(&gt;16bits) cannot specify more details yet consume more hardware memories. Thus, what are necessity and benefit of using float32 to represent PCM16 by “torchaudio”?</p>
<p>Sincerely thanks for your reply!</p>

username: JamesWright
cooked: <p>Hey <a class="mention" href="/u/felix_astw">@Felix_astw</a> , welcome to the forum!</p>
<p>In audio file containing 16 bit PCM samples, the values are restricted to the range <code>[-32768, 32767]</code>. The conversion from from 16bit to to float is done by dividing each sample value by <code>32768</code>. This results in signal represented in floating point values in the range <code>[-1.0, 1.0]</code>.</p>
<p>Using float is the canonical way of representing audio samples. There are many advantages, for example: getting full 32-bit precision, normalisation - regardless of whether data in the file is 8, 16, 24 or 32 bit wide, simple representation of clipping and easy application of operations (e.g. mixing signals is as easy as adding sample values together). At the end, it is also fast to scale/convert to the target bit depth - it’s just matter of multiplying by target range.</p>
How do I index a tensor with a tensor (not quite index_select)?

username: genjiii
cooked: <p>Let’s say I have a tensor a = torch.tensor(([0,1,2],[3,4,5])). I have another tensor, b = torch.tensor([0,1]), that I want to use to index a, such that we obtain the 0th position of the first row, 1st position of the second row (resulting tensor is [0, 3]). How can this be done? It’s not the same as index_select, which finds the 0th and 1st positions for every row. Thanks!</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="genjiii" data-post="1" data-topic="141561">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/genjiii/48/45738_2.png" class="avatar"> genjiii:</div>
<blockquote>
<p>uch that we obtain the 0th position of the first row, 1st position of the second row (resulting tensor is [0, 3]).</p>
</blockquote>
</aside>
<p>Wouldn’t the result be <code>[0, 4]</code> in this case?<br>
If so, this should work:</p>
<pre data-code-wrap="python"><code class="lang-python">a = torch.tensor(([0,1,2],[3,4,5]))
idx = torch.tensor([0, 1])
a[torch.arange(a.size(0)), idx]
# &gt; tensor([0, 4])
</code></pre>

username: genjiii
cooked: <p>Yes, actually [0, 4]  would be correct! Thank you so much for the fast reply <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
How to load finetuned torch hub models

username: Angry_potato
cooked: <p>Hello all,<br>
I am currently working on a kaggle competition for which I have to finetune an efficientnet model.</p>
<p>I have a finetuned efficientnet model which loaded from torch hub and saved it using <code>torch.save</code>. But when I use <code>torch.load</code> to load the model and weights in a different notebook there is a <code>ModuleNotFoundError</code> saying <code>'Pytorch' Module not found.</code></p>
<p>Kindly suggest a good way to save and load torch hub model which does not require active internet connection as internet connection is disabled in the submission notebook on Kaggle</p>

username: ptrblck
cooked: <p>Store the trained <code>state_dict</code> of the model, create the model object in the other notebook, and load the <code>state_dict</code> afterwards. Based on the error message it seems as if PyTorch is not installed in the other environment so I’m unsure if your issue is really related to loading the model only.</p>

username: Angry_potato
cooked: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a>,<br>
I do understand that but I would like to know how to create model object for a pretrained model.</p>
<p>For example</p>
<pre><code class="lang-auto">model=models.vgg16()
</code></pre>
<p>It would be really helpful if you can tell me how to build model object for models built in this manner.</p>
<p>Thank you for the reply.</p>

username: ptrblck
cooked: <p>I’m not sure I understand the question correctly.<br>
You could initialize a model with random parameters via your code snippet or generally as:</p>
<pre><code class="lang-python">model = MyModel()
</code></pre>
<p>where <code>MyModel</code> is the model class. Afterwards, you could load the <code>state_dict</code> via <code>model.load_state_dict(state_dict)</code>.<br>
The difference between a pretrained and a randomly initialized model are the parameters and buffers which are stored in the <code>state_dict</code>.</p>

username: Angry_potato
cooked: <p>Oh I understand it now. Thanks for the clarification</p>
Pytorch GDB can not get detail info

username: wukon007
cooked: <p>I use gdb to debug pytorch, but cann’t step in func and get detail func body.  And backtrace is discontinuous.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/9/1991e8b72de1e042d76b189cba3f349b58cbb327.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/1991e8b72de1e042d76b189cba3f349b58cbb327" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/1991e8b72de1e042d76b189cba3f349b58cbb327_2_690x273.jpeg" alt="image" data-base62-sha1="3EcxOHFP6BYVRosyWCSLFb7wapx" width="690" height="273" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/1991e8b72de1e042d76b189cba3f349b58cbb327_2_690x273.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/1991e8b72de1e042d76b189cba3f349b58cbb327_2_1035x409.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/1991e8b72de1e042d76b189cba3f349b58cbb327_2_1380x546.jpeg 2x" data-dominant-color="3B1730"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1812×718 168 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>To get more information in <code>gdb</code> rebuild PyTorch with debug symbols via <code>DEBUG=1 python setup.py install</code>.</p>

username: wukon007
cooked: <p>I did use DEBUG=1 python setup.py develop <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: wukon007
cooked: <p>As show above, I found missing debug symbols. When I set breakpoint in the func body like <code>runProfilingOptimizations</code> function, it’s called in <code>ProfilingGraphExecutorImpl::getOptimizedPlanFor</code> function, the debug would hit breakpoint,  when I key in <code>si</code> in gdb, it would also print <code>ProfilingGraphExecutorImpl::runProfilingOptimizations</code>,  but when I key in <code>s</code>, it will jump out of the func and print <code>ProfilingGraphExecutorImpl::getOptimizedPlanFor</code>. <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: wukon007
cooked: <p>It seems the breakpoint is the deepest call, cannot step in.</p>

username: ptrblck
cooked: <p>I don’t know why a debug build is not working as I haven’t seen this behavior before (i.e. they work in my setup, but I also haven’t stepped into the mentioned methods).</p>

username: wukon007
cooked: <p>Solved.  <code>python setup.py clean</code> first <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=10" title=":smiley:" class="emoji" alt=":smiley:"></p>
Multi-label models trained on single-label data

username: FaultyBagnose
cooked: <p>I wonder if it is a good idea to train a multi-label model on a single-labelled data set (using e.g. BCELoss). Is this likely to generalize well to images with multiple of the categories present? Even more important, possibly: would such a model be likely to respond correctly if no category is present?</p>
<p>Has anyone tried this?</p>

username: FaultyBagnose
cooked: <p>Just found this article myself:</p><aside class="onebox pdf" data-onebox-src="https://openaccess.thecvf.com/content/CVPR2021/papers/Cole_Multi-Label_Learning_From_Single_Positive_Labels_CVPR_2021_paper.pdf">
  <header class="source">

      <a href="https://openaccess.thecvf.com/content/CVPR2021/papers/Cole_Multi-Label_Learning_From_Single_Positive_Labels_CVPR_2021_paper.pdf" target="_blank" rel="noopener nofollow ugc">openaccess.thecvf.com</a>
  </header>

  <article class="onebox-body">
    <a href="https://openaccess.thecvf.com/content/CVPR2021/papers/Cole_Multi-Label_Learning_From_Single_Positive_Labels_CVPR_2021_paper.pdf" target="_blank" rel="noopener nofollow ugc"><span class="pdf-onebox-logo"></span></a>

<h3><a href="https://openaccess.thecvf.com/content/CVPR2021/papers/Cole_Multi-Label_Learning_From_Single_Positive_Labels_CVPR_2021_paper.pdf" target="_blank" rel="noopener nofollow ugc">Cole_Multi-Label_Learning_From_Single_Positive_Labels_CVPR_2021_paper.pdf</a></h3>

  <p class="filesize">506.20 KB</p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
It claims that learning from single labels can work even when the single labeled training data actually had multiple classes represented in each image. (My plan was to use training images with just a single class present in each image.)<br>
F</p>
Substitute values in a vector

username: blade
cooked: <p>I have a tensor of N unique target labels, randomly selected from [0,R], where N&lt;R. I would like to transform the labels to [0,N]. Is there a function available for this target transform? e.g. input vector: [12, 6, 4, 5, 3, 12, 4] → transformed vector : [4, 3, 1, 2, 0, 4, 1]</p>

username: ptrblck
cooked: <p><code>unique</code> with <code>return_inverse=True</code> would yield the desired result:</p>
<pre><code class="lang-python">x = torch.tensor([12, 6, 4, 5, 3, 12, 4])
x.unique(return_inverse=True)
# &gt; (tensor([ 3,  4,  5,  6, 12]), tensor([4, 3, 1, 2, 0, 4, 1]))
</code></pre>
Pascal VOC Detection input shape

username: Gabriel_Dornelles
cooked: <p>I’m trying to use the torchvision pascal voc package</p>
<pre><code class="lang-auto">from torchvision.datasets import VOCDetection
</code></pre>
<p>My problem is that I’m having issues when resizing the images with transforms.</p>
<pre><code class="lang-auto">data_transforms = transforms.Compose([
        transforms.Resize(size=(256,300)),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])

dataset = VOCDetection(root="./", download=True, transform=data_transforms)
train_size = int(len(dataset) * 0.9)
val_size = int(len(dataset) - train_size)
train, val = random_split(dataset, [train_size, val_size])

train_loader = DataLoader(train, batch_size=32, num_workers=4)
val_loader = DataLoader(val, batch_size=32, num_workers=4)
</code></pre>
<p>I’m passing the size as a tuple, still I’m getting this when I run:</p>
<pre><code class="lang-auto">RuntimeError: each element in list of batch should be of equal size
</code></pre>
<p>If I pass something like  <code> transforms.Resize(size=(256))</code> I get a reasonble error</p>
<pre><code class="lang-auto">RuntimeError: stack expects each tensor to be equal size, but got [3, 256, 341] at entry 0 and [3, 341, 256] at entry 1
</code></pre>
<p>which makes sense based on transforms.resize documentation. I’m confused because even passing it as a tuple, still I get the error.</p>
<p>I never worked with  this detection datasets , so if there’s anything I should know I appreciate the help.</p>

username: Gabriel_Dornelles
cooked: <p>is it right to use batch sizes larger than 1? or is just standard like segmentation to use 1?</p>

username: ptrblck
cooked: <p>I would guess that the error is raised while trying to stack the target images as you are only resizing the input images.<br>
Use <code>target_transform</code> to transform the target images too or use <code>transforms</code> and pass transformations to the dataset, which accept an image and target.</p>
How to convert and extracting multiple images from tensor?

username: mohandu_anil
cooked: <ol>
<li>image tensor shape = (561, 561, 3)</li>
<li>convert/transform tensor shape as (3, 1, 561, 561)</li>
<li>Now get three images from tensor.</li>
</ol>
<p>NOTE: If tensor is (561, 561, 4) then converting shape should be (6, 1, 561, 561) and we should get 6 images.</p>
<p>Thanks in advance.</p>

username: ptrblck
cooked: <p>If you want to split the channels of the first image, you could use:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(561, 561, 3)
xs = x.split(1, dim=2)
</code></pre>
<p>To permute the dimensions, use:</p>
<pre data-code-wrap="python"><code class="lang-python">x_perm = x.permute(2, 0, 1)
x_perm = x_perm.unsqueeze(1)
print(x_perm.shape)
# &gt; torch.Size([3, 1, 561, 561])
</code></pre>
<aside class="quote no-group" data-username="mohandu_anil" data-post="1" data-topic="141533">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mohandu_anil/48/45729_2.png" class="avatar"> mohandu_anil:</div>
<blockquote>
<p>If tensor is (561, 561, 4) then converting shape should be (6, 1, 561, 561) and we should get 6 images.</p>
</blockquote>
</aside>
<p>I don’t understand how 4 channels could yield 6 “images” so could you explain this use case a bit more?</p>
Efficient way to create block diagonal with different block size

username: Shuqi_Liu
cooked: <p>I want to create a mask matrix through torch.block_diag() method. However, each diagonal block has a different shape.</p>
<pre><code class="lang-auto">torch.block_diag(torch.ones((2,2)), torch.ones((3,3)), torch.ones((2,2)))
</code></pre>
<p>Please can I know if there is an efficient way to achieve this?</p>

username: ptrblck
cooked: <p>Your code works fine so I’m unsure I understand the issue correctly. Are you seeing any errors?</p>
Effect of np.random.seed(epoch) inside DistributedSampler.set_epoch()?

username: RylanSchaeffer
cooked: <p>As a follow up question to <a href="https://discuss.pytorch.org/t/dataloader-behaves-the-same-at-every-epoch-with-np-random/19227" class="inline-onebox">DataLoader behaves the same at every epoch with np.random</a>, if I call <code>np.random.seed(epoch)</code> inside my distributed sampler’s <code>set_epoch(epoch)</code> method, what will happen?</p>
<pre><code class="lang-auto">    def set_epoch(self, epoch):
        np.random.seed(epoch)
</code></pre>
<p>Will that screw up the randomness of my samples?</p>

username: RylanSchaeffer
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> would you happen to know?</p>

username: ptrblck
cooked: <p>The <code>set_epoch</code> method should be called in the <code>DistributedSampler</code> in each epoch before creating the <code>DataLoader</code> to avoid creating the same random indices, as internally the indices will be randomly shuffled using the base seed and the current epoch as seen <a href="https://github.com/pytorch/pytorch/blob/3cc34a4502898d274f1b4cc6625e9aef0670961b/torch/utils/data/distributed.py#L94-L99">here</a>.<br>
If you are re-seeding the numpy seed to each epoch, all following random operations in numpy would respect this seed. Since you’ve linked to the <code>DataLoader</code> thread, I assume you are already seeding numpy (and other libraries) in the <code>worker_init_fn</code> in the <code>DataLoader</code>?</p>
Create an instance of AdamParamState

username: afshin67
cooked: <p>I need to create an instance of <code>AdamParamState</code>. I looked through the <code>adam.cpp</code> code as an example, and accordingly copied the following code from there. But, with the provided headers, it still does not recognize <code>AdamParamState</code>.</p>
<p>I appreciate any help or comment on this matter.</p>
<pre><code class="lang-auto">#include &lt;torch/optim/adam.h&gt;
#include &lt;torch/csrc/autograd/variable.h&gt;
#include &lt;torch/nn/module.h&gt;
#include &lt;torch/serialize/archive.h&gt;
#include &lt;torch/utils.h&gt;

#include &lt;ATen/ATen.h&gt;

void get_state(torch::optim::Optimizer *optimizer){
    for (auto&amp; group : optimizer-&gt;param_groups()) {
        for (auto &amp;p : group.params()) {
            if (!p.grad().defined()) {
                continue;
            }

            auto grad = p.grad();
            TORCH_CHECK(!grad.is_sparse(),
                        "Adam does not support sparse gradients"/*, please consider SparseAdam instead*/);
            ska::flat_hash_map&lt;std::string, std::unique_ptr&lt;torch::optim::OptimizerParamState&gt;&gt;&amp; state_ = optimizer-&gt;state();
            auto param_state = state_.find(c10::guts::to_string(p.unsafeGetTensorImpl()));
            auto tmp_ = p.dim();
            int tmp_0;
            int tmp_1;
            if (tmp_ &gt; 0)
                tmp_0  = p.size(0);
            if (tmp_ &gt; 1)
                tmp_1  = p.size(1);

            std::cout &lt;&lt; tmp_ &lt;&lt; tmp_0 &lt;&lt; tmp_1 &lt;&lt; std::endl;
//                auto&amp; options = static_cast&lt;AdamOptions&amp;&gt;(group.options());
            auto&amp; state = static_cast&lt;AdamParamState&amp;&gt;(*state_[c10::guts::to_string(p.unsafeGetTensorImpl())]);
        }
    }

}
</code></pre>

username: afshin67
cooked: <p>I found that this works:<br>
<code>auto&amp; state = static_cast&lt;torch::optim::AdamParamState&amp;&gt;(*state_[c10::guts::to_string(p.unsafeGetTensorImpl())]);</code><br>
very simple and juicy!</p>
How to find the code of some kind of grad_fn?

username: Haozhi
cooked: <p>If I want to see the detail of <code>grad_fn</code> of some kind of operation like <code>SliceBackward</code>, where can I find it?</p>

username: ptrblck
cooked: <p><code>slice_backward</code> should be defined <a href="https://github.com/pytorch/pytorch/blob/cfc55196611e7637826d791cdccae92e1abb62b0/aten/src/ATen/native/TensorShape.cpp#L1303-L1307">here</a>. Generally you could search for the function name in the repository to find its definition. I don’t know if there is an easier way (<code>grep -r</code> might also come in handy).</p>

username: Haozhi
cooked: <p>Got it. Thanks for your kind reply.</p>
Error Handling with the Torchscript serialized code

username: anjali-chadha
cooked: <p>Hello folks -</p>
<p>I want some advice on error handling with torchscript code</p>
<ol>
<li>Sample Code snippet which validates the user input in forward() and throws ValueError for invalid inputs. Serialize this code using Torchscript</li>
</ol>
<pre><code class="lang-auto">import torch
from torch import nn

class Foo(nn.Module):
  def forward(self, user_input_length: int):
       if user_input_length &gt; 512:
          raise ValueError(f" Invalid user input. Max length allowed 512")
foo_ts = torch.jit.script(Foo())  
torch.jit.save(foo_ts, "foo_ts.pt")
</code></pre>
<ol start="2">
<li>Sharing the torchscript file with the client who loads it in their environment.</li>
</ol>
<pre><code class="lang-auto">foo_ts = torch.jit.load("foo_ts.pt")
foo_ts(198) # Valid Input
foo_ts(513) # Invalid Input - Let the user know
</code></pre>
<ol start="3">
<li>RuntimeError seen by the client when they pass an invalid user input</li>
</ol>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "ts_exception.py", line 11, in &lt;module&gt;
    foo_ts(513)
  File "/home/anjch/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
torch.jit.Error: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
  File "ts_exception.py", line 8, in forward
    def forward(self, user_input_length: int):
         if user_input_length &gt; 512:
            raise ValueError(f" Invalid user input. Max length allowed 512")
            ~~~~~~~~~~~~~~~~~~ &lt;--- HERE
RuntimeError:  Invalid user input. Max length allowed 512
</code></pre>
<p>In the current behavior, even if the actual code throws <code>ValueError</code>,  client side always receives a <code>RuntimeError</code>. Due to this behavior, the client will not be able to programmatically distinguish between an actual invalid input error v/s other Runtime error (unless they parse the string error  message)</p>
<p>Are there any suggestions on how to handle it gracefully where the client can distinguish between input errors vs other errors?</p>
Model idle time on GPU

username: tominator
cooked: <p>Initial situation: I have written an API to serve my object detection models. The model is loaded onto the GPU. Subsequently, the inferences are executed. However, if I have a break in between (let’s say 5 minutes), the first inference after the break then takes significantly longer.</p>
<p>My question: Are there certain parameters to prevent this? What is happening with the model on the GPU?</p>

username: ptrblck
cooked: <p>Your GPU might go into an idle state and the wake up might take some time. You could enable the persistence demon as described in the <a href="https://docs.nvidia.com/deploy/driver-persistence/index.html#background">docs</a>:</p>
<blockquote>
<p>This approach would prevent the kernel module from fully unloading software and hardware state when no user software was using the GPU.</p>
</blockquote>
Pytorch nightly build install with conda stops working

username: cathyzhyi
cooked: <p>I have been using the command from <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a> to install the nightly version <code>conda install pytorch torchvision torchaudio -c pytorch-nightly</code>.</p>
<p>However, conda claims “# All requested packages already installed.” while the torch version is still showing the following version which are 2 month old.</p>
<pre><code class="lang-auto">pytorch                   1.11.0.dev20211112     py3.9_cpu_0    pytorch-nightly
torchaudio                0.11.0.dev20211112        py39_cpu    pytorch-nightly
torchvision               0.12.0.dev20211112        py39_cpu    pytorch-nightly
</code></pre>
<p>Does anyone know why the cond install command doesn’t work anymore?</p>

username: cathyzhyi
cooked: <p>oh, seems I should be using <code>conda update</code> rather than <code>conda install</code> for upgrading.</p>
I can’t build for LibTorch on aarch64. Cannot find libcublas.so

username: Alio
cooked: <p>I’m trying to install LibTorch on a Jetson AGX Xavier.<br>
I do a recursive git clone of this pytorch repo and run 'python3 tools/build_libtorch.py.<br>
The build fails with the following error</p>
<p>FAILED: lib/libtorch_global_deps.so<br>
: &amp;&amp; /usr/bin/cc -fPIC -fopenmp -DNDEBUG -O3 -DNDEBUG -DNDEBUG -Wl,–no-as-needed -rdynamic -shared -Wl,-soname,libtorch_global_deps.so -o lib/libtorch_global_deps.so caffe2/CMakeFiles/torch_global_deps.dir/__/torch/csrc/empty.c.o -Wl,-rpath,/usr/lib/aarch64-linux-gnu/openmpi/lib:/usr/local/cuda/lib64:::::::: /usr/lib/aarch64-linux-gnu/openmpi/lib/libmpi_cxx.so /usr/lib/aarch64-linux-gnu/openmpi/lib/libmpi.so /usr/local/cuda/lib64/libcurand.so /usr/local/cuda/lib64/libcufft.so -lCUDA_cublas_LIBRARY-NOTFOUND /usr/lib/aarch64-linux-gnu/libcudnn.so /usr/local/cuda/lib64/libcudart.so /usr/local/cuda/lib64/libnvToolsExt.so &amp;&amp; :<br>
/usr/bin/ld: cannot find -lCUDA_cublas_LIBRARY-NOTFOUND<br>
collect2: error: ld returned 1 exit status<br>
[2357/3927] Building CXX object c10/test/CMakeFiles/c10_intrusive_ptr_test.dir/util/intrusive_ptr_test.cpp.o<br>
ninja: build stopped: subcommand failed.</p>
<h3>
<a name="versions-1" class="anchor" href="#versions-1"></a>Versions</h3>
<p>JetPack 4.6<br>
Nvidia Jetson AGX Xavier<br>
The current pytorch repo available on GitHub.</p>

username: ptrblck
cooked: <p><code>libcublas.so</code> won’t be built, but ships with the CUDA toolkit. Make sure this library was installed in the CUDA toolkit and set the CUDA path if needed. The install log should show which CUDA version was detected and where it’s located.</p>

username: Alio
cooked: <p>I ran ‘python3 tools/build_libtorch.py’ with python3.8 instead of python3.6 and it successfully installed LibTorch on my AGX Xavier.</p>
Running conditional variational auto encoder on non image data

username: sparshgarg23
cooked: <p>I have been working on conditional variational auto encoders ,and I would like to use it to generate new data samples.</p>
<p>My existing data is in a json file,which has 218 parameters that vary between -1 and 1.</p>
<p>Let’s call this X</p>
<p>Apart from this the conditional data is a numpy array of dimension 33x3.(which is c),these are basically contain pose landmark information obtained from Mediapipe.</p>
<p>Now as per my understanding, the decoder will calculate P(X|z,c) ,and from what I have seen,usually c/label is a one-hot vector</p>
<p>So,now the question is ,how can I use a conditional Variational auto encoder to generate a new X conditioned on c.Also is it possible to use a CVAE for my case,if not then what other generative models can I use?</p>
<p>I know that in input the input to the encoder will be a tensor of dimension 218,but I am unsure of how to use the condition variable .</p>
Tf.image.total_variation pytorch implementation

username: arya47
cooked: <p>Does pytorch have a implementation of TV??</p>
<p>Thanks</p>

username: ptrblck
cooked: <p>I don’t think there is a built-in <code>total_variation</code> method in <code>torchvision</code>, but based on the <a href="https://github.com/tensorflow/tensorflow/blob/v2.7.0/tensorflow/python/ops/image_ops_impl.py#L3213-L3282">TF implementatio</a> you could use try to use the same approach as it looks quite simple.</p>
ValueError: Layer weight shape not compatible

username: samm
cooked: <p>excuse me I got this error<br>
<code>ValueError: Layer weight shape (30522, 768) not compatible with provided weight shape torch.Size([1, 15, 768])</code><br>
while using bert as embedding layer as<br>
<code> embed = Embedding(30522, 768, weights=[embedding_matrix], input_length=max_length , trainable=False)(inputs2)</code></p>
<p>embedding_matrix from</p>
<pre><code class="lang-auto"> input_ids =torch.tensor(tokenizer.encode(sentence)).unsqueeze(0)
 outputs = BERT(input_ids)
 embedding_matrix = outputs[0]
</code></pre>

username: ptrblck
cooked: <p>The internal <code>weight</code> matrix of an <code>nn.Embedding</code> layer will have the shape <code>[num_embeddings, embedding_dim]</code>, which is <code>[30522, 768]</code> in your case. The error is raised since you are trying to pass a 3D tensor in the shape <code>[1, 15, 768]</code> to this layer as its <code>weight</code>. Based on your code it seems you want to use the output activation of a model as the embedding weight, so maybe try to remove the batch dimension and change <code>num_embeddings</code> to 15.</p>

username: ptrblck
cooked: <p>I don’t know how BERT can be used as an embedding layer, but in your previous code snippet you were trying to assign the model output as the <code>weight</code> parameter to an embedding layer, which sounds strange at least.<br>
An embedding layer can be seen as a lookup table, which accepts indices (i.e. sparse inputs) and maps it to feature tensors (i.e. dense outputs).<br>
Your current code doesn’t seem to be using PyTorch so I don’t know if it could work or not.</p>

username: ptrblck
cooked: <p>I’m not familiar with your use case and don’t know how the model outputs can be used as an embedding as asked in my previous post.</p>
Tensor To Pil Image

username: julianslz
cooked: <p>Hello all,</p>
<p>Could you please help me with a doubt? I have a 2D NumPy array that represents an image in black and white (shape: 1x224x224). After converting the NumPy array to a PyTorch tensor, I want to apply transformations to the image (using ToPILImage). Nevertheless, I got a noisy image as a result.</p>
<p>The NumPy array in matplotlib looks like this:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/6/268ce28074ccad9b46fc5f4b77b603bb80554c69.jpeg" alt="image" data-base62-sha1="5v1ZgikV0he9WDlBw3MzBB4NDwR" width="640" height="480"></p>
<p>But the ToPILImage() figures looks noisy:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae404fb31d78553351dca8b1d8a018c61f77b342.jpeg" alt="image" data-base62-sha1="oRuRHPlgyLhgnoFhwQ2FeiwkWxY" width="640" height="480"></p>
<p>My question is:</p>
<ol>
<li>How can I convert the PyTorch tensor (in black and white) to the PIL object, so it looks more like the first matplotlib image? I am using transforms.ToPILImage() at the moment.</li>
</ol>
<p>Thank you so much,</p>

username: ptrblck
cooked: <p>Could you check the <code>dtype</code> and shape of the numpy array before transforming it to a <code>PIL.Image</code> and compare it to the result of the transformation? Based on the output the transformation might cause overflows e.g. if the <code>dtype</code> is also changed.<br>
If you get stuck, please post an executable code snippet to reproduce this issue so that we could help in debugging.</p>
Making a circulant matrix from a given vector in a memory efficient way

username: AK47
cooked: <p>I’d like to make a circulant matrix from a given vector of dimension N in a way such that the operation and the resulting matrix consume only O(N) memory cost. Higher dimensional generalization of this operation is needed for a certain variant of Transformer I’m trying to investigate. A naive approach is to apply ‘expand’ to the original vector and then to apply ‘view’ and ‘slice’ alternately. However, this doesn’t work, as ‘view’ demands the input to be contiguous. Is there any viable way you can think of?</p>

username: SimonW
cooked: <p>it’s not really possible with the strided arrays (<a href="https://en.wikipedia.org/wiki/Stride_of_an_array">https://en.wikipedia.org/wiki/Stride_of_an_array</a>). pytorch, numpy and many other scientific computing framework adopts this approach.</p>

username: AK47
cooked: <p>Thank you so much for your reply. I guess I’ll give up my current attempt and look for another direction.</p>

username: SimonW
cooked: <p>what’s the exact use case? i might be able to help.</p>

username: AK47
cooked: <p>Finally, I’ve noticed that my approach of bidirectional language model was unnecessarily complicated, which required the use of circulant matrix for managing the cache of Transformer. I’m sorry.</p>

username: SimonW
cooked: <p>No worries! Good luck with your project!</p>

username: htd
cooked: <p>A bit late but here is a generic function for pytorch tensors, getting the circulant matrix for one dimension. It’s based on <code>unfold</code> operation.</p>
<pre><code class="lang-python">def circulant(tensor, dim):
    """get a circulant version of the tensor along the {dim} dimension.
    
    The additional axis is appended as the last dimension.
    E.g. tensor=[0,1,2], dim=0 --&gt; [[0,1,2],[2,0,1],[1,2,0]]"""
    S = tensor.shape[dim]
    tmp = torch.cat([tensor.flip((dim,)), torch.narrow(tensor.flip((dim,)), dim=dim, start=0, length=S-1)], dim=dim)
    return tmp.unfold(dim, S, 1).flip((-1,))
</code></pre>
Continue backward from tensor.grad

username: tree33
cooked: <p>Hi, I’m using another third-party code where the authors define a input x as nn.Parameter and computes the grad himself (without using loss.backward() for sparsity). So the code looks like</p>
<pre><code class="lang-auto">x = nn.Parameter(..., requires_grad=True)
# then come custom ops
# not loss.backward() here
sparse_grad()
optim = optim.RMSprop([x])
optim.step()
optim.zero_grad()
</code></pre>
<p>I checked x after the sparse_grad function, x.grad indeed gave me the correct output. And this should be correct as we are replacing torch’s backward with my own backward func. Now I want to continue the computation from the gradients of x to some other input tensor. Is it possible in torch?</p>
<p>Many thanks in advance!</p>

username: ptrblck
cooked: <p>You could call <code>backward()</code> on intermediate tensors, but since <code>x</code> is created as an <code>nn.Parameter</code> I don’t know which “other input tensor” would be connected to it. Could you describe your use case a bit more?</p>

username: tree33
cooked: <p>Hi, <a class="mention" href="/u/ptrblck">@ptrblck</a> here x is a 2d tensor, say 1000*1 dim. I think we can only backward from a 1x1 tensor right?</p>
<p>My use case looks somewhat like</p>
<pre><code class="lang-auto">input = cv2.imread()
fc = nn.Linear()
x = fc(input)
</code></pre>
<p>Then the package calculates x’s grad for me.  But I wish to optimize the parameters of the fc layer as well. I can currently think of something like</p>
<pre><code class="lang-auto">loss = x*x.grad
loss = loss.sum()
loss.backward()
</code></pre>
<p>But I’m not sure this is the correct way.</p>

username: ptrblck
cooked: <p>The <code>tensor.backward</code>() operation can be called without passing an explicit gradient to it as an argument if the <code>tensor</code> is a scalar tensor. If the tensor has more than a single element you would have to pass the <code>gradient</code> to <code>backward</code>. You could use <code>loss = x.sum()</code> to calculate the gradient in <code>fc</code> if you would like to use <code>x.sum()</code> as the loss.</p>

username: tree33
cooked: <p>Many thanks! <a class="mention" href="/u/ptrblck">@ptrblck</a> . Never thought we can write backwards this way.<br>
Just want to make sure.</p>
<pre><code class="lang-auto">x = nn.Parameter(...)
y = x**2
</code></pre>
<p>And I manage to calculate y’s grad with another package and want to update x.<br>
So I can write as, correct?</p>
<pre><code class="lang-auto">y.backward(y.grad)
</code></pre>

username: ptrblck
cooked: <p>Yes, this should work as seen here:</p>
<pre><code class="lang-python">x = nn.Parameter(torch.randn(2, 2))
y = x**2

y.backward(torch.ones_like(y) * 2)
print(x.grad)
print(2*x*2)
</code></pre>
Help! adding data of the target subject to training performs worse

username: vpeterson
cooked: <p>Hi Pytorch community,</p>
<p>I have a held-one-patient-out experiment. I have designed a CNN+RNN net. During training, since I have an imbalanced dataset I used the sampler within the DataLoader.<br>
Now I am adding some labeled samples (e.g 10) to my training set from the target subject with the objective of improving the performance of my task. Although I would expect this to be beneficial for my model, in some situations I see the performance drops down. So my question is, how can I be sure the DataLoader is using also the added target samples? I would expect adding samples for the target subject would be at least as good as not adding them, but never worse. Any help would be more than appreciated.</p>
<p>I train my model from scratch. I stop the training at 20 epochs. I use BatchNorm with the PyTorch momentum param equal to 0.01</p>
<p>This is a sniped of my code.</p>
<pre><code class="lang-auto">import torch
torch.manual_seed(0) # the same seed, to ensure the same weight init

  
train_df = pd.DataFrame()
train_df = pd.concat([train_df, train_df_aux, seeds]) 
# train_df_aux is the training data without any labelled samples from the target subject
# seeds are the labelled samples from the target subject, this is also an umbalaced subset, since I get the labelled samples untill X positive class are found.
train_df.reset_index(drop=True, inplace=True)
  
   
# DATA LOADERS
train_data = torch.utils.data.ConcatDataset([train_data_ori, train_data_trf1])
# train_data_trf1 is the train data with augmentation strategies.
            
sampler = torch.utils.data.sampler.WeightedRandomSampler(weights, len(weights))
#weights is constructed using cutomized fuctions to have during training balanced batched. 

# During training
kwargs = {'num_workers': hparams["num_workers"], 'pin_memory': True} if use_cuda else {}
train_loader = DataLoader(train_data, batch_size=hparams["batch_size"],
                          sampler=sampler,
                              **kwargs)
</code></pre>
<p>Thank you in advance!</p>

username: nivek
cooked: <p>If you just iterate through DataLoader and print out each sample, do you see the label samples that you added?</p>

username: vpeterson
cooked: <p>Hi nivek! thank you for your reply. Yes, I have checked, and I see the added samples!</p>
Shouldn't we project the gradients when the weight matrix is constrained?

username: Tomarchelone
cooked: <p>Using torch.nn.utils.parametrizations one can enforce some constraint on weight matrix: we can make the matrix orthogonal or make it have spectral norm of 1. This is achieved by recomputing the weights in forward pass.</p>
<p>However, the gradients of the weights are not constrained in backward pass. This leads to a situation when the gradient of the weight matrix can have a component pointing outside the desired set of parameters (for example orthogonal matrices). This component is redundant, but can be very large. This seems potentially harmful, as for example when using gradient clipping this component will be considered when computing gradient norm.</p>
<p>Can we project the gradients so they are always pointing inside the desired set? Should we care to try to reimplement these parametrizations, so they can support gradient projection?</p>
Which compiler is used for the pre-built libtorch packages?

username: allinall
cooked: <p>Dear all,</p>
<p>we use the pre-built binarys for Windows, given through the links from the pytorch homepage such as</p>
<ul>
<li><a href="https://download.pytorch.org/libtorch/cu101/libtorch-win-shared-with-deps-1.4.0.zip" rel="noopener nofollow ugc">https://download.pytorch.org/libtorch/cu101/libtorch-win-shared-with-deps-1.4.0.zip</a></li>
</ul>
<p>We are interested in which compiler was used to build these packages. Does anyone know which compiler was used?</p>
<p>Thanks</p>
OpenMP library under macos

username: paulus
cooked: <p>I was looking into adding simple parallelizations to my model, and and while failing to utilize &gt;1 thread, encountered something I do not understand. Maybe this is as it should be, but for me it looks slightly inconsistent.</p>
<p>System info:</p>
<ul>
<li>macOS 10.15.5</li>
<li>pytorch installed with <code>conda install pytorch torchvision -c pytorch</code> resulting to torch version 1.5.1. (as per the instructions)</li>
</ul>
<p>Strangeness:</p>
<ul>
<li>at::get_num_thread() returns 1</li>
<li>from torch.__config__parallel_info():</li>
</ul>
<pre><code class="lang-auto">OpenMP not found
ATen parallel backend: OpenMP
</code></pre>
<ul>
<li>in the build flags from <code>torch.__config__.show()</code>: <code>USE_OPENMP=OFF</code>
</li>
</ul>
<p>Still the <a href="https://pytorch.org/docs/stable/notes/cpu_threading_torchscript_inference.html" rel="nofollow noopener">documentation</a> says “Pre-built PyTorch releases are compiled with OpenMP support.”</p>
<p>The libiomp5.dylib is present in the library directory.</p>
<p>The inconsistency for me: ATen should use OpenMP as the parallel backend and the dynamic library is included in the distribution, but OpenMP support is disabled in the build. The same applies when downloading only libtorch for C++ API. I’ve understood that OpenMP support on mac hasn’t been there officially for too long, so is this still just work-in-progress?</p>

username: Vanjoy
cooked: <p>Got the same issue with <code>1.10.1</code>.</p>
Join in EcoSystem

username: XIXIXIX
cooked: <p>How to add my project to the ecosystem，what conditions need to be met before filling out the table (<a href="https://pytorch.org/ecosystem/join" class="inline-onebox" rel="noopener nofollow ugc">Join | PyTorch</a>)</p>

username: ptrblck
cooked: <p>As described in the linked page, if you’ve answered “yes” to both questions you should fill out the form to send the application. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=10" title=":wink:" class="emoji" alt=":wink:"></p>

username: XIXIXIX
cooked: <p>Whether the project is to be uploaded to GitHub, our project was only uploaded to PYPI before. If it does not meet the requirements of the ecosystem, can you point out the deficiencies so that it can be improved again. <img src="https://discuss.pytorch.org/images/emoji/apple/smile_cat.png?v=10" title=":smile_cat:" class="emoji" alt=":smile_cat:"></p>
RGB to Lab conversion

username: Arash
cooked: <p>Is there an easy way to convert from RGB to Lab color space and vice versa in Pytorch? I want to do it in a differentiable way within the model.</p>
<p>Thanks</p>

username: lucasb-eyer
cooked: <p>I’m not aware of an easy shortcut, but the conversion is a composition of differentiable operations, so you can just implement the forward computation and backprop through it.</p>
<p>You can copy my implementation in <a href="https://github.com/lucasb-eyer/go-colorful" rel="nofollow noopener">go-colorful</a> which includes this conversion <a href="https://github.com/lucasb-eyer/go-colorful/blob/master/colors.go#L537" rel="nofollow noopener">here</a>, it does the conversion by passing through XYZ space, follow the code for equations.</p>
<p>The only non-trivial part in the chain is a couple data-dependent switches, you could either use <a href="https://github.com/lucasb-eyer/go-colorful/blob/master/colors.go#L537" rel="nofollow noopener">this trick</a> or <a href="http://pytorch.org/docs/master/tensors.html#torch.Tensor.masked_scatter_" rel="nofollow noopener">masked_scatter_</a> for that, or you might get away with just ignoring them because they apply in the very low-luminosity range only.</p>

username: Arash
cooked: <p>Thanks! I think there is a hard conditioning in RGB to LAB conversion (if value&gt;C: …) that makes it non-differentiable. Is there a workaround for that?</p>

username: lucasb-eyer
cooked: <p>That should not be a problem, the function is simply not smooth at one specific value, but we can still take the (sub-)derivative and do (sub-)gradient descent. It is essentially the same as ReLU, LeakyReLU, etc.</p>

username: Oktai15
cooked: <p><a class="mention" href="/u/lucasb-eyer">@lucasb-eyer</a> as fas as I understood, I need to convert RGB to XYZ and after that I can convert my image to LAB, right? It’s still differentiable?</p>

username: edgarriba
cooked: <p>you can convert RGB to XYZ using <a><code>kornia.rgb_to_xyz</code></a></p>
<p>pending to implement to Lab. Contributions are welcomed.</p>

username: Mirsadeghi
cooked: <p>Look at <a href="https://github.com/smartcameras/EdgeFool/blob/master/Train/rgb_lab_formulation_pytorch.py" rel="noopener nofollow ugc">this link</a> for PyTorch implementation of lab2rgb and rgb2lab.</p>