akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Find the L2 Nearest Neighbor in a queue?

username: cane95
cooked: <p>Hi <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"> , I’m having a hard time figuring out how to compute the following efficiently. What I would like to do is, for each element of x, compute the L2 difference with all the other elements in the queue. What I came up with is something like:</p>
<pre><code class="lang-auto">inp_size = 10
queue_size = 50
dim = 64

x = torch.randn([inp_size, dim])
queue = torch.randn([queue_size, dim])

score = (x.repeat_interleave(queue_size, 0) - queue.repeat(inp_size, 1)).pow(2).sum(1)
nn_idx = score.view(inp_size, queue_size).min(1)[1]
</code></pre>
<p>any advice?</p>
Differentiable round or early indexing method

username: laoguru
cooked: <p>I have a binary output LSTM method that I weigh based on an output between 0 and 1, numbers above 0.5 being “right” and below being wrong.</p>
<p>I’m trying to get the prediction to heavily favor early predictions and therefore need the index of the earliest occurrence of a value greater than 0.5.</p>
<p>I’m not sure how to approach this as round is non differentiable and I dont clearly see a way to do this without rounding/doing an argsort. Any suggestions?</p>
Saving and Loading Profile

username: eduardo4jesus
cooked: <p><code>torch.profile</code> can be <code>export_chrome_trace</code> as shown <a href="https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html#using-tracing-functionality" rel="noopener nofollow ugc">here</a>. However, I am wondering if there is a way to import somehow a profile object.</p>
<p>I tried to import it with <code>torch.save</code>, however I didn’t have success, as shown below.</p>
<pre><code class="lang-auto">    482     pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
    483     pickler.persistent_id = persistent_id
--&gt; 484     pickler.dump(obj)
    485     data_value = data_buf.getvalue()
    486     zip_file.write_record('data.pkl', data_value, len(data_value))

AttributeError: Can't pickle local object 'schedule.&lt;locals&gt;.schedule_fn'
</code></pre>
<p>Is there a way to save and load a profile object?</p>
Content of tuple

username: saran
cooked: <p>any one can help how i can check the content of tuple tensor<br>
when we print its shape or size its says<br>
" Tuple object has no attribute  shape "<br>
or<br>
" Tuple object has no attribute  size"<br>
although with len function we can check the length but its not showing any detail about the exact content of the tuple<br>
for example i am printing the length of tuple  named “sample”</p>
<p>print(" sample len",len(sample))</p>
<p>its shows the length which is two and  has one element on its 2nd dimension but no idea about the shape of first dimension<br>
shown as</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/8/a8790e30e1168240708e1c755d7ce74306d13700.png" alt="tupple has no atrb size" data-base62-sha1="o2nzEHFQE1jomuO5l6hrIBzQc1y" width="615" height="179"></p>
<p>any help regrading it</p>

username: st-vincent1
cooked: <p>Hi saran,</p>
<p>Try this:</p>
<pre><code class="lang-auto">for i in sample:
 print(f'shape of sample[{i}]:', sample[i].shape)
</code></pre>
Vocab in torchtext 0.10.0: no max_size?

username: st-vincent1
cooked: <p>So torchtext 0.10.0 saw Vocab undergo significant changes. New Vocab class is:<br>
<code>torchtext.vocab.Vocab(vocab)</code><br>
where <code>vocab</code> is<br>
<code>torchtext.vocab.vocab(ordered_dict: Dict, min_freq: int = 1) → torchtext.vocab.Vocab</code></p>
<p>In previous versions (up until 0.9) the Vocab class was taking the following arguments upon initialisation:<br>
<code>counter, max_size=None, min_freq=1, specials=('&lt;unk&gt;', '&lt;pad&gt;'), vectors=None, unk_init=None, vectors_cache=None, specials_first=True</code></p>
<p>While I can see most of the functionality repeated in the new class as methods, I could not figure out how to constrain the <code>max_size</code> in the new class. There is no parameter or method which does this. Am I missing something? Should/will this be implemented in future updates?</p>
Executer un code

username: Achrouf
cooked: <p>python demo.py -i /C:/Users/ETTAQI/Documents/STAGE/imagecode/img2.png -o /C:/Users/ETTAQI/Documents/STAGE/imagecode -c /C:/Users/ETTAQI/Documents/STAGE/code2406/PathoNet-master/configs/demo.json<br>
File “”, line 1<br>
python demo.py -i /C:/Users/ETTAQI/Documents/STAGE/imagecode/img2.png -o /C:/Users/ETTAQI/Documents/STAGE/imagecode -c /C:/Users/ETTAQI/Documents/STAGE/code2406/PathoNet-master/configs/demo.json<br>
^<br>
SyntaxError: invalid syntax</p>
<p>Hello, I would like to execute a code but I cannot correct this error<br>
thank you in advance</p>

username: ptrblck
cooked: <p>I guess the used syntax is wrong: <code>/C</code>, as Windows wouldn’t use a <code>/</code> (“root”) directory.</p>

username: Achrouf
cooked: <p>you mean rather like that C: \ Users \ ETTAQI \ Documents \ STAGE \ code2406 \ PathoNet-master \ configs ?</p>
Using with_flops argument of Profiler

username: eduardo4jesus
cooked: <p>In the <a href="https://pytorch.org/docs/stable/profiler.html" rel="noopener nofollow ugc">profiler documentation</a> we have an argument called <code>with_flops</code>. When I set that value to <code>True</code> the exported profile in the <code>json</code> format had a huge size. However, I could tell how useful is this argument. I also tried to autocomplete the profile object, but I couldn’t pick anything that seemed to be related to the number of FLOPS.</p>
<p>How to use this properly?</p>
<p>Thanks in advance.</p>
Concatenating images and passing through the model

username: asha97
cooked: <p>What is the difference between concatenating two image and passing it through resnet network ,while passing both images individually to the network. Code 1 for 1st scenario and code 2 for second.</p>
<p><strong>Code 1</strong></p>
<pre><code class="lang-auto">images=[x1,x2]
images = torch.cat([images[0], images[1]], dim=0)
bsz = labels.shape[0]
features = model(images)
f1, f2 = torch.split(features, [bsz, bsz], dim=0)
features = torch.cat([f1.unsqueeze(1), f2.unsqueeze(1)], dim=1)
</code></pre>
<p><strong>Code 2</strong></p>
<pre><code class="lang-auto">x1 and x2 images
out_1 = model(x1)
out_2 = model(x2)
out = torch.cat([out_1, out_2], dim=0)
</code></pre>
<p>Note : The output dimension from the model will be 128</p>

username: rahulvigneswaran
cooked: <p>Technically there should be no difference but it looks like in <strong>code 1</strong>, you are doing the concatenation at <code>dim=0</code>. This could cause issues,</p>
<p>Say two image dims are,</p>
<pre><code class="lang-auto">x1 -&gt; 32x32x3
x2 -&gt; 32x32x3
</code></pre>
<p>Doing <strong>code 1</strong> will provide the following dims,</p>
<pre><code class="lang-auto">images -&gt; 64x32x3
</code></pre>
<p>You need to unsqueeze it first at <code>dim=0</code>. This would make the dims as follows,</p>
<pre><code class="lang-auto">x1 -&gt; 1x32x32x3
x2 -&gt; 1x32x32x3
images -&gt; 2x32x32x3
</code></pre>
<p>This wont be an issue in <strong>code 2</strong> as the images are handled individually. Let me know if that’s the case.</p>

username: asha97
cooked: <pre><code class="lang-auto">x1 -&gt; [256, 3, 32, 32]
x2 -&gt; [256, 3, 32, 32]
images -&gt; [256, 3, 32, 32]
</code></pre>
<p>Here 256 is the batch size</p>
<p>features-&gt;[256, 2, 128]</p>

username: syedmech47
cooked: <p>I think in code 1, you are concatenating 2 batches of Images, so if there is enough RAM, than it will work fine.<br>
In second code you are running model twice, so more time for same work.</p>
<p>Here only thing to take care is whether you have the capacity to sent 256 Batch Size to model or not, if yes than you can use code1 which will run all 256 images in parallel.</p>
Understanding Variations on Executing Time on GPU

username: eduardo4jesus
cooked: <p>I have the following code snippet.</p>
<pre><code class="lang-auto">import time
import torch
import numpy as np
import matplotlib.pyplot as plt

from scripts.custom.functional.python.functional_operations import get_functional_operation

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
input = torch.randint(32, size=(128, 32, 64, 64), dtype=torch.float, device=device, requires_grad=True)
weights = torch.randint(32, size=(64, 32, 5, 5), dtype=torch.float, device=device, requires_grad=True)

n_rounds = 1024
duration = np.zeros((n_rounds))
conv2d = get_functional_operation('fft-custom')
for i in range(n_rounds):
  start = time.time()
  output = conv2d(input, weights)
  if torch.cuda.is_available(): torch.cuda.synchronize()
  duration[i] = time.time() - start
</code></pre>
<p>I got assigned the <code>Tesla V100-SXM2...</code> and using the Torch Version <code>1.9.0+cu102</code>.</p>
<p>I ran this code once and got the following plot for <code>duration</code><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba2fa7393e09592c041909ed6c3db4d7b198cac7.png" alt="download1" data-base62-sha1="qz4UaRcJyjgoMix04lV33tcGldZ" width="362" height="259"></p>
<p>Then I reset my kernel and ran it again, resulting on the following plot.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/0/b0e683f0007b8bb6440fadf27a3404e7b82f60f2.png" alt="download2" data-base62-sha1="peVUwFyJ1uDHQsH9nuW2cs74GmC" width="397" height="248"></p>
<p>As part of my research I have to measure the time of the functions I am working with, I am wonder what makes these oscillations on the execution time, even on an extremely controlled and unpractical scenario like the one above were the exact same function with the exact same parameters are evaluated over and over.</p>
<p>Would that be cause because I am using a virtual environment that does not guarantee dedicated resources?</p>
<hr>
<p>[EDIT]</p>
<p>By the way, how does the graph execution gets optimized anyways? PyTorch is purely on lazy-execution, right? I noticed that usually the first couple of interactions are slower, kind of the execution is still being optimized or it is “warming up”. So, when measuring time I usually disregard the first <code>128</code> iterations to be safe. Is there a fixed number of “warming up” iterations?</p>

username: eqy
cooked: <p>PyTorch executes eagerly by default. You might be referring to the asynchronous execution of kernel launches on GPUs, where the interpreter will get back to you right away even if the kernel hasn’t finished yet.</p>
<p>There isn’t really a fixed number of warmup iterations, but the reason that they are necessary is that in cases when benchmarking is used to select algorithms (e.g., with <code>torch.backends.cudnn.benchmark=True</code>), the first run will be slower due to the benchmarking overhead. I don’t think there will be any graph-level optimizations without jit: <a href="https://pytorch.org/docs/stable/jit.html" class="inline-onebox" rel="noopener nofollow ugc">TorchScript — PyTorch 1.9.0 documentation</a></p>
Jupyter ImportError torchtext symbol not found

username: rmb
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/9/c9670b0d94df98483fe8990bc6ca9f10f8aa7ef8.png" data-download-href="https://discuss.pytorch.org/uploads/default/c9670b0d94df98483fe8990bc6ca9f10f8aa7ef8" title="Screen Shot 2021-06-24 at 3.58.13 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/9/c9670b0d94df98483fe8990bc6ca9f10f8aa7ef8_2_690x329.png" alt="Screen Shot 2021-06-24 at 3.58.13 PM" data-base62-sha1="sJGJUEZWzjmGWGsPlpSTMXNoscM" width="690" height="329" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/9/c9670b0d94df98483fe8990bc6ca9f10f8aa7ef8_2_690x329.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/9/c9670b0d94df98483fe8990bc6ca9f10f8aa7ef8_2_1035x493.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/9/c9670b0d94df98483fe8990bc6ca9f10f8aa7ef8_2_1380x658.png 2x" data-dominant-color="F3F5F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2021-06-24 at 3.58.13 PM</span><span class="informations">2204×1052 274 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I installed torchtext with Conda and also tried pip to see if it would fix the problem, I even uninstalled it with pip twice and reinstalled it but the same error keeps popping up when I try to import torchtext.</p>
<p>ImportError                  Traceback (most recent call last)<br>
 in <br>
----&gt; 1 import torchtext</p>
<p>~/opt/anaconda3/envs/SpacyEnv/lib/python3.9/site-packages/torchtext/<strong>init</strong>.py in <br>
3 from . import datasets<br>
4 from . import utils<br>
----&gt; 5 from . import vocab<br>
6 from . import legacy<br>
7</p>
<p>~/opt/anaconda3/envs/SpacyEnv/lib/python3.9/site-packages/torchtext/vocab.py in <br>
11 from typing import Dict, List, Optional, Iterable<br>
12 from collections import Counter, OrderedDict<br>
—&gt; 13 from torchtext._torchtext import (<br>
14     Vocab as VocabPybind,<br>
15 )</p>
<p>ImportError: dlopen(/Users/…/opt/anaconda3/envs/SpacyEnv/lib/python3.9/site-packages/torchtext/_torchtext.so, 2): Symbol not found: __ZN2at6detail10noopDeleteEPv<br>
Referenced from: /Users/…/opt/anaconda3/envs/SpacyEnv/lib/python3.9/site-packages/torchtext/_torchtext.so<br>
Expected in: /Users/…/opt/anaconda3/envs/SpacyEnv/lib/python3.9/site-packages/torch/lib/libtorch_cpu.dylib<br>
in /Users/…/opt/anaconda3/envs/SpacyEnv/lib/python3.9/site-packages/torchtext/_torchtext.so</p>
CTC loss and metric divergence

username: Vijay_Singh
cooked: <p>I am using a CRNN model (EffNet + BiLSTM) for an OCR task, and using CTC loss to train the model. I have observed a weird phenomenon wherein the loss per epoch keeps increasing while the model get better. I know this because I calculate the edit distance between the decoded predictions and the labels, and that’s improving.</p>
<p>Here’s how my loss function is set up:</p>
<p>criterion = nn.CTCLoss(blank=blank_label, reduction=‘sum’, zero_infinity=True)</p>
Using Subsequent Mask for Transformer Decoder Leads to NaN values

username: FeryET
cooked: <p>Hi.</p>
<p>I’m trying to implement an autoregressive transformer model similar to the paper “attention is all you need”. From what I’ve understood, in order to replicate the architecture fully, I need to give the transformer decoder 3 masks.</p>
<p>1 - Target subsequent mask: this is for casaulity.<br>
2 - Target padding indexes: just to look at non-padded indices.<br>
3 - Encoder padding indices: just to look at non-padded inputs from the encoder.</p>
<p>The snippet is here:</p>
<pre><code class="lang-auto">y = self.decoder(y, x,
                         tgt_mask=tgt_causal_mask,
                         tgt_key_padding_mask=tgt_padding_mask,
                         memory_key_padding_mask=src_padding_mask)
</code></pre>
<p>With masks being generated like this:</p>
<pre><code class="lang-auto">
    def generate_no_peek_mask(self, sz):
        mask = (torch.triu(torch.ones(sz, sz)) == 1).transpose(0, 1)
        mask = mask.float().masked_fill(mask == 0, float("-inf")).masked_fill(mask == 1, float(0.0))
        mask = mask.to(self.device)
        return mask

    def generate_padding_mask(self, seq, pad_idx):
        return (seq != pad_idx).to(self.device)

</code></pre>
<p>The problem is that using these masks leads to issues with the Softmax function because of NaN values. Without these masks, the model does not generate any NaN value.</p>
Why is Normalization causing my network to have exploding gradients in training?

username: Bled_Clement
cooked: <p>I’ve built a network (In Pytorch) that performs well for image restoration purposes. I’m using an autoencoder with a Resnet50 encoder backbone, however, I am only using a batch size of 1. I’m experimenting with some frequency domain stuff that only allows me to process one image at a time.</p>
<p>I have found that my network performs reasonably well, however, it only behaves well if I remove all batch normalization from the network. Now of course batch norm is useless for a batch size of 1 so I switched over to group norm, designed for this purpose. However, even with group norm, my gradient explodes. The training can go very well for 20 - 100 epochs and then game over. Sometimes it recovers and explodes again.</p>
<p>I should also say that in training, every new image fed in is given a wildly different amount of noise to train for random noise amounts. This has been done in other papers however and they say that batch norm helps to address the covariate shift.</p>
<p>I’m scratching my head at this one and I’m wondering if anyone has suggestions. I’ve dialed in my learning rate and clipped the max gradients but this isn’t really solving the actual issue. I can post some code but I’m not sure where to start and hoping someone could give me a theory. Any ideas as to why my network trains well without GN but badly with it? Thanks!</p>
How to use netCDF data for DCGAN?

username: ljeonjko
cooked: <p>Hello!</p>
<p>I’m using NASA’s Ocean color data, chlorophyll to be precise, and a single day of data looks like this:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/4/5484f19e3bcfdfd577cf79f7659c392e4ce7023b.png" alt="image" data-base62-sha1="c3GWaYGq7iFz0p9eaMcTlRAgWXF" width="639" height="274"></p>
<p>Basically, each pair of (lat,lon) has a single float value of chlor_a. There’s a whole bunch of daily data to be used.</p>
<p>How would I go about feeding this type of .nc data directly into a DCGAN, without reverting to using actual images (.png and .jpeg.)?</p>
<p>Thanks!</p>

username: ptrblck
cooked: <p>You could try to create an “image-like” input tensor in the shape <code>[batch_size, channels, height, width]</code>, where the <code>channels</code> could be set to 1, since you are dealing with a single float for each pixel location, and the <code>height</code> and <code>width</code> could be set to the <code>lat</code> and <code>lon</code>.</p>

username: ljeonjko
cooked: <p>Thanks! Will give it a try in  a few hours, and let you know how it works! Cheers <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

username: ljeonjko
cooked: <pre><code class="lang-auto">[batch_size, channels, height, width]
</code></pre>
<p>If I have 100 files of data, each representing a single day values, that would be</p>
<pre><code class="lang-auto">
[100 (number of daily data/satellite "screenshots"), 1, 941 ( number of lat coordinates), 1200 (number of lon coordinates]
</code></pre>
<p>right?</p>
<p>Where do I then input the actual chlor_a values? Sorry if this is a dumb question, I’m very new to all of this and have trouble finding tutorials.</p>
<p>Does that mean that the <code>height, width</code> part is just the<code> x = torch.from_numpy(x)</code> where <code>x = np.array(nc.variables['chlor_a'])</code> and where <em>nc</em> is the “xarray inputed” netCDF file? <em>x</em>  already has <code>(lat, lon)</code> dimensions.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="ljeonjko" data-post="4" data-topic="124870">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/b5ac83/48.png" class="avatar"> ljeonjko:</div>
<blockquote>
<p><em>x</em> already has <code>(lat, lon)</code> dimensions.</p>
</blockquote>
</aside>
<p>In that case I think you can just <code>unsqueeze</code> <code>dim0</code> and <code>dim1</code> as <code>x</code> should already contain the values to create the batch and channel dimension.<br>
This would create a tensor in the shape <code>[1, 1, lat, lon]</code> which would be an “image-like” tensor.</p>
Calling enumerate on dataloader gives list instead of tensor

username: avalon1511
cooked: <p>Hi, I have created a dataloader object from a subsetted dataset as:</p>
<pre><code class="lang-auto">target_index = np.random.choice(len(target_dataset), k_samp, replace= True)
target_dataset = torch.utils.data.Subset(target_dataset, target_index)
target_loader = torch.utils.data.DataLoader(target_dataset, batch_size=batch_size,
                                         shuffle=True, num_workers=workers)
</code></pre>
<p>When I enumerate over this dataloader as follows, the type for <code>data</code> is actually a list and not tensor</p>
<pre><code class="lang-auto">for i, data in enumerate(target_loader, 0):
</code></pre>
<p>I created my <code>target_dataset</code> as</p>
<pre><code class="lang-auto">target_dataset = dset.ImageFolder(root=target_dataroot,
                           transform=transforms.Compose([
                               transforms.Resize(image_size),
                               transforms.CenterCrop(image_size),
                               transforms.ToTensor(),
                               transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                           ]))
</code></pre>
<p>I double checked and the contents of my target_dataroot are in fact images.</p>
<p>Not sure why I am getting a list when iterating over the dataloader instead of tensors?</p>
<p>Tried it with a few other subsetted datasets and iterating over those give me tensors.</p>
<p>Thanks</p>

username: ptrblck
cooked: <p><code>ImageFolder</code> will return a <code>data</code> and <code>target</code> tensor. You are currently assigning both return values to <code>data</code> in the <code>DataLoader</code> loop. If you want to use both objects as tensors, you could either use two return variables via:</p>
<pre><code class="lang-python">for i, (data, target) in enumerate(target_loader, 0):
</code></pre>
<p>or unwrap them inside the loop:</p>
<pre><code class="lang-python">for i, data in enumerate(target_loader, 0):
    x, y = data
</code></pre>

username: avalon1511
cooked: <p>Thanks, that fixed it.</p>
<p>Although, I find it strange that I have another dataset set up the same way using <code>ImageFolder</code> and I am not having this issue with that dataset.</p>
<p>What causes this issue to occur?</p>

username: ptrblck
cooked: <p>This shouldn’t be the case, as it would indicate that the other <code>ImageFolder</code> approach would return a single tensor (instead of the data and target tensors).<br>
Could you post a code snippet to reproduce this behavior?</p>

username: avalon1511
cooked: <p>I think I fixed it! I had to do</p>
<pre><code class="lang-auto">images = data[0].to(device)
</code></pre>
<p>after enumerating over the dataloader! That fixed the problem if it being a list and not a tensor</p>
RuntimeError: number of dims don't match in permute

username: alicanakca
cooked: <p>Hi everyone!<br>
As i mentioned in the topic, I got an error with the permute function.</p>
<pre><code class="lang-auto">    def __getitem__(self, i: int) -&gt; Tuple[torch.Tensor, torch.Tensor]:
        """ Input-Output Generator by index
        Args:
            i (int): Order of data
        Returns:
            (Tuple[torch.Tensor, torch.Tensor]): Image and mask PyTorch tensor
        """

        _id = self.__image_ids[i]
        paths = self.__id2path(_id)

        image = self.__load_n_preprocess(paths["image_path"], False)
        mask = self.__load_n_preprocess(paths["mask_path"], True)

        image = torch.from_numpy(image)
        mask = torch.from_numpy(mask)
        image = image.permute(2, 0, 1)
        mask = mask.permute(2, 0, 1)

        return image, mask
</code></pre>
<p>ERROR</p>
<pre><code class="lang-auto">    mask = mask.permute(2, 0, 1)
RuntimeError: number of dims don't match in permute
</code></pre>
<p>I realized that It gets stuck on mask.permute after doing image.permute just fine. By the way image’s shape is torch.Size([256, 256, 3]) mask’s shape is torch.Size([256, 256]).</p>
<p>Also, this error arised when this function was called the third time.</p>

username: ssgosh
cooked: <aside class="quote no-group" data-username="alicanakca" data-post="1" data-topic="124950">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alicanakca/48/37711_2.png" class="avatar"> alicanakca:</div>
<blockquote>
<p>mask’s shape is torch.Size([256, 256]).</p>
</blockquote>
</aside>
<p>This is the issue – the mask is 2-dimensional, but you’ve provided 3 arguments to mask.permute().</p>
<p>I am guessing that you’re converting the image from h x w x c format to c x h x w. However, looks like the mask is only in an h x w format.</p>

username: alicanakca
cooked: <p>Yeah, you are right. I changed it to 2 dimension. Thanks!</p>
Beam Search Decoder for CNN LSTM model

username: Aditya_Shukla
cooked: <p>I am working on recognition of cursive handwritten text. I am using a CNN LSTM model with Connectionist Temporal Classification loss function.<br>
I need a <strong>beam search decoder</strong> or <strong>greedy decoder</strong> for decoding the output of the network (logits). Please recommend a library or module.</p>
<p>Thank you in advance.</p>
<p>Regards<br>
Aditya Shukla</p>

username: Sanbot-kk
cooked: <p>DId you find a greedy decodeR?</p>
A tiny difference between outputs of PyTorch and LibTorch

username: Albert-Zhao-2020
cooked: <p>Hello,</p>
<p>there are different outputs between my Python code and my C++ code. I just simply loaded the ResNet101 model, and loaded a static input with torch.ones following these instructions: <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" class="inline-onebox" rel="noopener nofollow ugc">Loading a TorchScript Model in C++ — PyTorch Tutorials 1.9.0+cu102 documentation</a>.<br>
Using<br>
Pytorch 1.7.1<br>
Libtorch 1.7.1<br>
centos 7.9</p>
<p>Python Code</p>
<p>import torch<br>
import torchvision</p>
<p>torch.manual_seed(0)</p>
<p>model = torchvision.models.resnet101()<br>
in_data = torch.ones(1, 3, 224, 224)</p>
<p>model.eval()<br>
output = model(in_data)</p>
<p>print(output[0, :5].detach().numpy())</p>
<p>traced_script_module = torch.jit.script(model)<br>
traced_script_module.save(“traced_resnet_model.pt”)</p>
<p>C++ code<br>
<span class="hashtag">#include</span> &lt;torch/script.h&gt; // One-stop header.<br>
<span class="hashtag">#include</span> <br>
<span class="hashtag">#include</span> <br>
<span class="hashtag">#include</span> </p>
<p>int main(int argc, const char* argv[]) {<br>
if (argc != 2) {<br>
std::cerr &lt;&lt; “usage: example \n”;<br>
return -1;<br>
}</p>
<p>torch::jit::script::Module module;<br>
try {<br>
// Deserialize the ScriptModule from a file using torch::jit::load().<br>
module = torch::jit::load(argv[1]);<br>
}<br>
catch (const c10::Error&amp; e) {<br>
std::cerr &lt;&lt; “error loading the model\n”;<br>
return -1;<br>
}<br>
std::cout &lt;&lt; “ok\n”;</p>
<p>// Create a vector of inputs.<br>
std::vector&lt;torch::jit::IValue&gt; inputs;<br>
inputs.push_back(torch::ones({1, 3, 224, 224}));</p>
<p>// Execute the model and turn its output into a tensor.<br>
at::Tensor output = module.forward(inputs).toTensor();</p>
<p>std::cout &lt;&lt; output.slice(1, 0, 5) &lt;&lt; ‘\n’;<br>
}</p>
<p>and the python output is [-9897.268  -1432.8728 -2543.7222 11986.332   3412.1372], but the c++ output is [-9897.2686 -1432.8741 -2543.7183 11986.331 3412.1426]. Some values precision is less than 6 decimal places.<br>
Can someone explain that?<br>
@ <a href="https://discuss.pytorch.org/u/ptrblck">ptrblck</a></p>

username: pascal.soveaux
cooked: <p>Hello,</p>
<p>Differences when printed maybe. Look at the real values or use the same output precision.</p>
<p>Pascal</p>
Distributed training queue

username: mag1
cooked: <p>Hi,<br>
I am wondering if anyone came across a solution for this or maybe there is some work in progress.</p>
<p>Basically in most cases a Machine Learning engineer is running an instance of iPython notebook and running experiments from this instance. This includes training new models</p>
<p>What would be cool is to be able to send the model, training loop together with the data (just tensors) to a remote powerful GPU cluster, that maintains a queue of jobs. This jobs can then run uninterrupted with results being reported through web interface, the ability to download the models for local evaluation would also be great.</p>
Iterable Dataset Crashing Runtime

username: tantheta01
cooked: <p>I have implemented a very simple Iterable Dataset in PyTorch :</p>
<pre><code class="lang-auto">class Maps(data.IterableDataset):
    def __init__(self):
        super().__init__()
        
    def yield_items(self):
        
        for i in range(150):
          file = np.random.rand(160, 256, 512)
          for j in range(150):
              img = file[j:j+10, :, :256]
              label = file[j:j+10, :, 256:]
              img = torch.tensor(img/np.max(img))
              label = torch.tensor(label / np.max(label))
              yield img.float(), label.float()
            
    def cycle_data(self):
        return cycle(self.seeItems())

    def __iter__(self):
        return self.cycle_place()

train_dataset = Maps()
data_loader = data.DataLoader(train_dataset)

for x,y in data_loader:
  continue

</code></pre>
<p>I am running this code on google colab and it causes the RAM to fill up real quick and the runtime crashes. Does someone know what this would be due to?<br>
Edit : If I remove the loop on j it works fine. But in my actual use case that loop is inevitable.</p>
DataLoader for stream data (ReplayBuffer in Reinforcement Learning)

username: longvt
cooked: <p>Dear Pytorch community,<br>
Is there any way to apply DataLoader to get mini-batch on online stream data (like ReplayBuffer in Reinforcement Learning)<br>
As I know, DataLoader initializes the whole data before train and it very inefficiently when re-initializing data every time data changes (ReplayBuffer increases or changes)</p>
<p>Thank you so much for your help!</p>

username: Etienne_Perot
cooked: <p><a href="https://github.com/etienne87/pytorch-stream-dataloader" class="inline-onebox" rel="noopener nofollow ugc">GitHub - etienne87/pytorch-stream-dataloader</a> ,  can you give this a try?</p>
at::Tensor at::Tensor::index(const c10::List<c10::optional<at::Tensor>> &) const": Cannot convert parameter 1 from "at::Tensor" to "c10: :ArrayRef<at::indexing::TensorIndex>

username: zheng_wang
cooked: <p>When I upgraded the libtorch version to 1.9,<br>
error</p>
<pre><code class="lang-auto">at::Tensor at::Tensor::index(const c10::List&lt;c10::optional&lt;at::Tensor&gt;&gt; &amp;) const": Cannot convert parameter 1 from "at::Tensor" to "c10: :ArrayRef&lt;at::indexing::TensorIndex&gt;
</code></pre>
<p>occurred when compiling code.</p>
<pre><code class="lang-auto">conf_noobj = pred_conf.index(noobj_mask).mean();
</code></pre>
<p>Please help me what is the reason for this?</p>

username: zheng_wang
cooked: <p>I have solved this problem used index({})<br>
<code>conf_noobj = pred_conf.index({noobj_mask}).mean();</code></p>
Autograd of precomputed distance of a vector quantization codebook

username: EBGU
cooked: <p>Hi!<br>
I am new to deep learning and pytorch, and I am very confused about achieving an autograd process that I needed.<br>
There is a block in my network that calculates cosine distance on 1e10 pairs of vectors, which can be extremely slow. Therefore, I am trying to add a vector quantization module to my network and it only contains a codebook of 512 vectors. In this way, I only need to calculate 512*512 pairs of cosine distance and store them in a new codebook. For all vectors, they are first quantitated, and then the pair-wise distance can be looked up in the codebook.<br>
My question is how should I set autograd function of the lookup process, to pass the correct gradient function of cosine distance to the lookup table.</p>
<p>Thank you a lot!</p>
Freezing only part of the model

username: saar-eliad
cooked: <p>I want to change the following</p><aside class="onebox githubblob">
  <header class="source">

      <a href="https://github.com/mlcommons/inference/blob/master/speech_recognition/rnnt/pytorch_SUT.py#L73-L83" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/mlcommons/inference/blob/master/speech_recognition/rnnt/pytorch_SUT.py#L73-L83" target="_blank" rel="noopener nofollow ugc">mlcommons/inference/blob/master/speech_recognition/rnnt/pytorch_SUT.py#L73-L83</a></h4>



  <pre class="onebox">    <code class="lang-py">
      <ol class="start lines" start="73" style="counter-reset: li-counter 72 ;">
          <li>model.eval()</li>
          <li>model.encoder = torch.jit.script(model.encoder)</li>
          <li>model.encoder = torch.jit._recursive.wrap_cpp_module(</li>
          <li>    torch._C._freeze_module(model.encoder._c))</li>
          <li>model.prediction = torch.jit.script(model.prediction)</li>
          <li>model.prediction = torch.jit._recursive.wrap_cpp_module(</li>
          <li>    torch._C._freeze_module(model.prediction._c))</li>
          <li>model.joint = torch.jit.script(model.joint)</li>
          <li>model.joint = torch.jit._recursive.wrap_cpp_module(</li>
          <li>    torch._C._freeze_module(model.joint._c))</li>
          <li>model = torch.jit.script(model)</li>
      </ol>
    </code>
  </pre>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
(RNN-T from mlcommons benchmark)</p>
<p>to use the new <code>torch.jit.freeze</code> API, but it throws errors.<br>
Can this be done?</p>
How to implement the multi-layer softmax function?

username: Sayak_Paul
cooked: <p>The line of works done on weaky supervision tasks by FAIR, almost all of them use multi-label softmax to produce the prediction outputs. From the text of <a href="https://arxiv.org/pdf/1805.00932v1.pdf" rel="noopener nofollow ugc">the paper by Mahajan et al.</a>:</p>
<blockquote>
<p>Our model computes probabilities over all hashtags in the vocabulary using a softmax activation and is trained to minimize the cross-entropy between the predicted softmax distribution and the target distribution of each image. The target is a vector with k non-zero entries each set to 1/k corresponding to the k ≥ 1 hashtags for the image.</p>
</blockquote>
<p>I am looking for ways to implement this as a function.</p>
Zeroing batch of images according to given coordinates

username: Hasan_Sait_ARSLAN1
cooked: <p>Dear Pytorch enthusiasts;</p>
<p>I have an interesting problem to solve.</p>
<p>I am searching for an efficient way of cropping images by given coordinates.</p>
<p>For example;</p>
<p>Let’s say there is a <strong>coords</strong> tensor with <strong>(N, 4)</strong> size, and <strong>images</strong> tensors with <strong>(N, 640, 640)</strong> size.</p>
<p>By using given coords and their corresponding images, how can I keep the crop regions, and zero out the remaining? Is there any possible batch operation for that?</p>
<p>Thanks,</p>
Custom Loss not Converging

username: syedmech47
cooked: <p>I am trying to train a Network to predict Writer Independent Signature Verification.<br>
Here by, writer independent I mean that we train the model on some signature datasets and create a vector embedding at last layer, and the model will be able to create the signatures of different users at different points in the higher dimension space, so during inference we can find the distance between the inference image and the vector embedding of the stored images.</p>
<p>So for this we can use triplet loss while training, and after training create embedding of the users and store.</p>
<p>But in my application, we will get an image, and also the user id of the image, so we can get the stored image for that user id and have to match that the incoming image and the reference image in our database is similar or not.</p>
<p>For that I create a Custom Loss function, but it’s not converging.</p>
<pre><code class="lang-auto">class Loss(nn.Module):
    def __init__(self, p_margin, n_margin, swap=True):
        super(Loss, self).__init__()
        self.p_margin = p_margin
        self.n_margin = n_margin
        self.swap = swap

    def forward(self, anchor, positive, negative):
        pos_dis = torch.norm(anchor-positive, 2, 1)
        neg_dis = torch.norm(anchor-negative, 2, 1)
        pn_dis = torch.norm(positive-negative, 2, 1)

        if self.swap:
            neg_dis = torch.maximum(pn_dis, neg_dis)

        p_loss = pos_dis-self.p_margin
        p_loss = torch.maximum(p_loss, torch.zeros_like(p_loss))

        n_loss = self.n_margin-neg_dis
        n_loss = torch.maximum(n_loss, torch.zeros_like(n_loss))
        
        return p_loss+n_loss
</code></pre>
<p>Here I am taking an anchor image, positive image which is of same class, and a negative image. And the goal of the function is to create the Norm distance between same classes(anchor and positive) to less than p_margin and the distance between different classes(anchor and negative) to be greater than n_margin.</p>
<p>But my loss isn’t converging at all even after 270 epochs</p>
<p>Any leads onto what may be wrong in the Loss function, maybe having hard value of p_margin and n_margin is not the right way.</p>
Using 'for loop' to train model multiple times produces vastly different results

username: Rohit_R
cooked: <p>In my training script, I have a function <strong>‘train’</strong> that carries out the model training for a certain number of epochs and the training proceeds successfully. The loss gradually decreases and I obtain a decent validation set accuracy.</p>
<p>Now, I wanted to train the same model 3 times. So I essentially created a <strong>‘for loop’</strong> that executes my <strong>‘train’</strong>  function 3 times. However, I observed that the first 2 runs produces decent val set accuracies. However in the third run my model completely fails to train and the training loss stays almost the same at a very high value. I do not understand why this is happening and think it might be due to PyTorch’s random seed or with cuda memory allocation.</p>
<p><strong>Note</strong> : In my <strong>‘train’</strong> function, I re-initialize all my models, optimizer and scheduler. The only thing that remains the same across the runs is my <strong>‘train_loader’</strong> instance of the Dataloader class. I am not using any manual seed, so I do expect some variation in my val results however the model is not training at all in the third run.</p>
<p>Thanks a lot !</p>

username: ptrblck
cooked: <p>Based on the description it seems that your overall training is unstable and the success rate of the training could be low. You could retrain <code>N</code> models using different seeds for each run and check how many times the model converges properly vs. a failure. To stabilize it, you could try to use other parameter initialization methods etc.</p>
<aside class="quote no-group" data-username="Rohit_R" data-post="1" data-topic="124857">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rohit_r/48/38660_2.png" class="avatar"> Rohit_R:</div>
<blockquote>
<p>and think it might be due to PyTorch’s random seed or with cuda memory allocation.</p>
</blockquote>
</aside>
<p>I don’t quite understand how the CUDA memory allocation could be related to this issue. Could you explain your concern a bit more?</p>

username: Rohit_R
cooked: <p>Thank you so much for your suggestions! I will surely try training with different initial seeds. I was thinking that maybe CUDA cached the models weights of the previous run and somehow used these weights for the next run even though I re-initialized the model. Although I think this is mostly not the case. Thanks again!</p>
Too much GPU Memory Usage

username: Rajat_Sharma1
cooked: <p>I am trying to evalutate a pytorch based model. The evalutation is working fine but when I see the gpu memory usage during forward pass it is too high and does not freed unitl the script is finished. I know initially it should increase as the computation increases during forward pass but it should decrease when the computations are done but it remains same. If I evaluate on further iteration it becomes cumulative. Should I use torch.cuda.empty_cache() after each forward pass.</p>
<pre><code class="lang-auto"># parameters loaded to model
model.eval()
with torch.no_grad():
    for idx, batch in enumerate(test_dataloader):
        # memory-usage: 1903 MiB
        noisy_depth = batch.get("noisy_depth").unsqueeze(1).to(device)
        # memory-usage: 1903 MiB
        output = model(noisy_depth)
        # memory-usage: 5192 MiB 
        break

 # memory-usage: 5192 MiB 
print("Finished")
</code></pre>

username: ptrblck
cooked: <p>PyTorch uses a caching mechanism to reuse the device memory and thus avoid the (synchronizing) <code>malloc/free</code> calls. You can check the memory usage via <code>print(torch.cuda.memory_summary())</code>, which will report how much memory is allocated, in the cache etc.<br>
Calling <code>empty_cache()</code> in each iteration will slow down the code (since you won’t be able to reuse memory) and will not lower the memory usage.</p>
<p>If you are seeing an increase in the allocated memory usage, you are most likely storing tensors in e.g. a <code>list</code>, which are still attached to the computation graph, so you would need to <code>detach()</code> them assuming you won’t want to backpropagate through them anymore.</p>