akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Polymorphism: Creating a ModuleHolder<Module> from a custom Module?

username: iiSeymour
cooked: <p>I have a module <code>MyModel</code> that I want to instantiate then call a custom method on it, plus some base methods, then return a generic <code>ModuleHolder&lt;Module&gt;</code> to it to the calling code.</p>
<p>What is the correct way to return a <code>ModuleHolder&lt;Module&gt;</code>?</p>
<pre><code class="lang-auto">ModuleHolder&lt;Module&gt; load_model(std::string w, std::string device) {

    auto state_dict = load_weights(w);                                                                                                                                                                               
    auto module = MyModel(insize, outsize);

    // call a custom method
    module-&gt;load_state_dict(state_dict);

    // call base methods
    if (device == "gpu") {
        module-&gt;to(device);
        module-&gt;to(c10::kHalf);
    }

   return module; // How to return a ModuleHolder&lt;Module&gt; not ModuleHolder&lt;MyModel&gt;?
}
</code></pre>
<p>Example model definition:</p>
<pre><code class="lang-auto">struct ConvolutionImpl : Module {

    ConvolutionImpl(int size = 1, int outsize = 1, int k = 1, int stride = 1)  {
        conv = register_module("conv", Conv1d(Conv1dOptions(size, outsize, k).stride(stride).padding(k/2)));
        activation = register_module("activation", SiLU());
    }

    torch::Tensor forward(torch::Tensor x) {
        return activation(conv(x));
    }

    Conv1d conv{nullptr};
    SiLU activation{nullptr};

};

TORCH_MODULE(Convolution);

struct MyModelImpl : Module {
    Model(int size, int outsize) {
        conv1 = register_module("conv1", Convolution(1, 4, 5, 1));
        conv2 = register_module("conv2", Convolution(1, 16, 5, 1));
        conv3 = register_module("conv3", Convolution(16, outsize, 19, 5));
    }

    void load_state_dict(std::vector&lt;torch::Tensor&gt; weights) {
        assert (weights.size() == parameters().size());
        for (size_t idx = 0; idx &lt; weights.size(); idx++) {
            parameters()[idx].data() = weights[idx].data();
        }
     }

     Convolution conv1{nullptr}, conv2{nullptr}, conv3{nullptr};

};

TORCH_MODULE(MyModel);
</code></pre>

username: iiSeymour
cooked: <p>I think using <a href="https://pytorch.org/cppdocs/api/classtorch_1_1nn_1_1_any_module.html#classtorch_1_1nn_1_1_any_module" rel="noopener nofollow ugc">AnyModule</a> is the right approach.</p>
<pre><code class="lang-auto">ModuleHolder&lt;AnyModule&gt; load_model(std::string w, std::string device) {

    auto state_dict = load_weights(w);                                                                                                                                                                               
    auto module = MyModel(insize, outsize);

    // call a custom method
    module-&gt;load_state_dict(state_dict);

    // call base methods
    if (device == "gpu") {
        module-&gt;to(device);
        module-&gt;to(c10::kHalf);
    }

    auto any = AnyModule(module);
    auto holder = ModuleHolder&lt;AnyModule&gt;(any);

    return holder; 
}

</code></pre>
Installing Pytorch 0.3.1 on Colab

username: S_P
cooked: <p>I have a code with Pytorch 0.3.1, would anyone please tell me how to install Pytorch 0.3.1 on Colab?</p>

username: ptrblck
cooked: <p>You could try to adapt <a href="https://discuss.pytorch.org/t/pytorch-0-4-0-on-google-colab/17329/2">this code</a> to <code>0.3.1</code>, although I would recommend to update your code to the latest stable release, as e.g. you might run into already fixed bugs.</p>

username: princeofpython
cooked: <p>I tried the code, it still says it isn’t a supported wheel.</p>
The speed of index assignment of two vectors is very slow

username: Dean_Mike
cooked: <p>I am a beginner of pytorch. Now I find a problem. Suppose I have two tensor <code>sampling_relation</code> and <code>relation_db</code> , they’re all in the GPU. When I execute the following code, it takes about 2 seconds.</p>
<pre><code class="lang-auto">sampling_relation[batchs, j, 0:total_length] = relation_db[item_id, 0:total_length]
</code></pre>
<p>Can anyone provide some optimization methods. I would appreciate it if you could help me! Thx!</p>
How to use pytorch.text with character level sequence tagger

username: coyote
cooked: <p>Hi everyone</p>
<p>I am trying to use the pytorch.text for my sequence tagger model. Basically, the model predicts the Part-Of-Speech tag for each token in the sentence.  My model consists of one LSTM and one Bidirectional-LSTM. First a character based LSTM creates the word embedding for the target token by processing characters and then the second LSTM takes these word embeddings as input and process them (in both directions) to predict the POS tag for the target word.</p>
<p>Here the problem is that, I need to batch both the characters and words. Is there a way to do that with pytorch.text ?</p>
<p>Let me give an example:</p>
<p>Sentence: This is an example sentence .</p>
<p>First LSTM is going to create a word embedding for each word in the sentence W1,W2,W3,W4,W5,W6<br>
For example, it produces W2 by processing the characters of the second word “is” which are  “i” and “s”.  Then, the forward LSTM (of bi-LSTM) process W1,W2 and backward LSTM (of bi-LSTM) process W6,W5,W4,W3,W2. Then I concatenate the hidden states and make a prediction.</p>
<p>I am not asking the model implementation. I am asking if pytorch.text is flexible enough to prepare the data for such a problem.</p>
<p>As far as I see, here I can not minibatch inputs for the LSTM and inputs for the bi-LSTM at the same time. I guess batching sentences with the same number of WORDS seems more feasible but I am also open to other suggestions. I would also appreciate if you show many any similar code snippets.</p>

username: pourfazl
cooked: <p>Hi,</p>
<p>(however it’s very late to be answered!!!, But I hope it would be useful for those who have the same problem)</p>
<p>I think torch.text does it only words or char chunks.<br>
so you need to do it manually on char sequences which will done just in some lines of code.</p>
<p>First: get batches at word-level using torch text iterator which gives you [N, num_words]<br>
Second: in each batch, you need to pad other words as long as the longest word in the current batch</p>
<ul>
<li>You need to split each word to its chars, torch.text also does it, just create a dataset at char-level.</li>
</ul>
<p>Then, find length of the longest word in the batch</p>
<p>For instance, let’s input sentence be: [this is an example]<br>
torch.text at word-level will give you: [4, 3, 1, 2] (values are arbitrary)</p>
<p>So, in this batch, the longest word is “example” which has 7 chars.<br>
your char vocabulary is ready, so just map each character to its corresponding index and for the rest, put  .</p>
<p>If the word “this” is transformed as {t, h, i, s} = {22, 7, 8, 21} then you should pad it three times.</p>
<p>{t, h, i, s} ---- {t, h, i, s, , , }<br>
{i, s} ---- {i, s, , , , , }</p>
<p>For batch size [N, NW] you should get character batches in size of [N, NW, NC]<br>
(NW = Words No., NC = Characters No.)</p>
Sequence Taging

username: pourfazl
cooked: <p>Hi,</p>
<p>I’m trying to modelling such a complicated type of sequence tagging using PyTorch in which corresponding tags of input sequence could change for a certain <em>word</em> in <em>another sentence</em>.<br>
I have coded rest of job, but still something is confusing for me regarding original docs of PyTorch,<br>
suppose we use a stacked Bi-LSTM, which is batch-first format.</p>
<p>The objective is to tag sequence (w1, w2, …, wn) to desired tag labels (t1, t2, …, tn) using both character-level and word-level embedding. the word-level LSTM should take concatenation of words and their chars representation as its input. The words itself are independent from each other, but in some cases, tag of the next word (t_n+1) yields to change its previous word’s tag (t_n) regarding to some other features called sequence meter. So in this case, for a certain word [w], it’s going to have different tags, which will tends to model be over-fitted.</p>
<p>My question is that, is this type of architecture capable enough to extract such “dependent” features of the input sequence or do we should adding up another features, e.g., bi-gram representation?</p>
<p>kind regards,<br>
lots of thanks</p>
How to use .pt file locally for inferencing an image

username: Haya_Hesham
cooked: <p>Hi,<br>
I am a  beginner in computer vision. I followed excatly the steps illustrated on YOLOV5 github and I used colab and finally downloaded the weights in ‘best.pt’ file. Now, I want to use this file locally for inferencing images. I saw different pieces of code and I tried a few but I got noting. How can I do that?</p>

username: ptrblck
cooked: <p>Assuming you’ve stored the <code>model.state_dict()</code> in the <code>best.pt</code> file, you would have to create the model instance in a new script, load the <code>state_dict</code>, and could perform the inference.<br>
Something like this should work:</p>
<pre><code class="lang-python">model = YOLO(...) # create new model object
model.load_state_dict(torch.load(path_to_state_dict))
model.eval()

# load single image or dataset
img = PIL.Image.open(path_to_image)
...
x = transform(img) # transform to tensor

with torch.inference_mode():
    output = model(x)
</code></pre>

username: Haya_Hesham
cooked: <p>Alright. I understand the code, but I have one issue. I realized that in the first line ‘model = YOLO()’ I must create an instance of a class that holds the structures and parameters of yolov5. I don’t understand how to do this. Do I need to download yolov5 from github or what? Please explain in more details.</p>

username: ptrblck
cooked: <p>Yes, you need the class definition from the repository to create the model object in the same way as you’ve done it during training.</p>

username: Haya_Hesham
cooked: <p>Okay. I downloaded yolov5 from github into my project folder. What is next? Excuse me but it isn’t clear for me.</p>

username: KylevdLangemheen
cooked: <p>You will have to import the yolo model (e.g. <code>import models.yolo.Model as YOLO</code>) such that you can instantiate an untrained model as per <code>model = YOLO(...)</code>. From there you can work with the code provided.</p>
Custom loss based on old model weights

username: KylevdLangemheen
cooked: <p>Hi there,</p>
<p>This question concerns something I already have working, but a lot is dependent on how the model is defined. I would like some advice on what a better implementation would be such that it works with other (fully connected / convolutional) models and layers.</p>
<p>Currently I am working on a supplementary custom loss that is based on activation and weights. These weights and activation can be taken in various ways, e.g. the weights of a Linear layer and its direct activation output, the weights of a Conv2d layer and the feature maps thereafter, or even the patches of input to a single Conv2d filter and its weights. There are about 3 - 6 combinations possible for what I’m working on, let’s call them type A, type B, etc…</p>
<p>Because this custom loss requires activation from specific (parts of) layers/blocks in the model, what would be the best way to implement this such that it best generalizes to most or any (fully connected / convolutional) Pytorch model? The implementation I am working with uses hooks that read out activation and weights from parts of the network that matches the given type. The hook that takes activation from the first ReLU after a Conv2d layer might look something like this:</p>
<pre><code class="lang-auto">    def set_activation_hooks(self):
        conv_layer_id = None
        for name, layer in self.model.named_children():
            if isinstance(layer, Conv2d):
                conv_layer_id = name
            if isinstance(layer, ReLU) and conv_layer_id is not None:
                hook = layer.register_forward_hook(self.save_outputs_hook(conv_layer_id, name))
                self.activation_hooks.append(hook)
                conv_layer_id = None
</code></pre>
<p>However, this obviously breaks down when the model is structured even the tiniest bit differently: when for example the ReLU is part of the forward pass or when it’s part of a Sequential block. Are hooks (like this) really the way to go, or are there better alternatives?</p>
<p>Thanks in advance!</p>
Libtorch compile link flags

username: hrZ
cooked: <p>Hi team,<br>
when upgrade from 1.7.0cu110 to 1.9.0cu111 or 1.11.0cu113,<br>
compile link flags(CMakefile):<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/6/367dbc92ab49a620e703c21d93937830c0f53c86.png" data-download-href="https://discuss.pytorch.org/uploads/default/367dbc92ab49a620e703c21d93937830c0f53c86" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/367dbc92ab49a620e703c21d93937830c0f53c86_2_690x38.png" alt="image" data-base62-sha1="7M3aIse8ZI7WiIcgOiXQlguT9wW" width="690" height="38" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/367dbc92ab49a620e703c21d93937830c0f53c86_2_690x38.png, https://discuss.pytorch.org/uploads/default/original/3X/3/6/367dbc92ab49a620e703c21d93937830c0f53c86.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/6/367dbc92ab49a620e703c21d93937830c0f53c86.png 2x" data-dominant-color="F2EEF4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">833×46 9.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>works for ver 1.7.0cu110 but doesn’t do the trick for the newer version,otherwise when run app it will warning entry point “?set_fw_gradAutogradMetaautogradtorchUEAAXAEBVTensorBaseat0_K_NZ” not found</p>
<pre><code>    any solution?Thanks.
</code></pre>

username: hrZ
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/1/41edd93d5cad946896845cf87eccfcc7d618442a.png" alt="image" data-base62-sha1="9peCG2c44hM4hAlyVZEigchTJzA" width="429" height="64"></p>

username: hrZ
cooked: <p>Solved.<br>
set_target_properties(AAA PROPERTIES LINK_FLAGS “-INCLUDE:?_torch_cuda_cu_linker_symbol_op_cuda@native@at@<span class="mention">@YA</span>?AVTensor@2@AEBV32@<span class="mention">@Z</span>”)</p>
<p><a href="https://github.com/pytorch/pytorch/issues/72396" rel="noopener nofollow ugc">libtorch with Cuda 11.3 not linked properly on Windows using Visual Studio 2022 · Issue #72396 · pytorch/pytorch (github.com)</a><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9f821ce6a8aad2b5ec6bb85bf2ffd630eef51bcb.png" data-download-href="https://discuss.pytorch.org/uploads/default/9f821ce6a8aad2b5ec6bb85bf2ffd630eef51bcb" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9f821ce6a8aad2b5ec6bb85bf2ffd630eef51bcb.png" alt="image" data-base62-sha1="mL4GmXxjWemjHcgR4cRfN3JTiFt" width="690" height="278" data-dominant-color="F6F7F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">929×375 22.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
AttributeError: module 'torch.nn' has no attribute 'HuberLoss'

username: trusira
cooked: <p>I’m using <code>pytorch</code>, <code>pytorch-cpu 1.6.0 cpu_py38h3369884_1  conda-forge</code> installed through anaconda and I get the following error.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.nn.HuberLoss()
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
AttributeError: module 'torch.nn' has no attribute 'HuberLoss'
</code></pre>
<p>I can see the HuberLoss implementation in the master branch on github, just wondering why this loss function is not found in my Pytorch installation.</p>
<p>Thanks,</p>

username: ptrblck
cooked: <p><code>nn.HuberLoss</code> was introduces in PyTorch <code>1.9.0</code> so you would need to update your installation.</p>

username: trusira
cooked: <p>Thanks, I updated to the latest version 1.11.0 and HuberLoss is now available.</p>
[torch.fx deepcopy Bug]

username: aktcob
cooked: <h1>
<a name="code-1" class="anchor" href="#code-1"></a>Code:</h1>
<p>import torch<br>
import torch.fx<br>
from types import MethodType<br>
from copy import deepcopy</p>
<p>class Student(torch.nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.bn = torch.nn.BatchNorm2d(3)<br>
self.gamma = torch.nn.BatchNorm2d(3)<br>
def forward(self, x):<br>
return self.bn(x)<br>
def set_score(self, score):<br>
self.score = score</p>
<p>s = Student()<br>
func = getattr(s.<strong>class</strong>, “set_score”)<br>
fx_s = torch.fx.symbolic_trace(s)<br>
setattr(fx_s, “set_score”, MethodType(func, fx_s))</p>
<h1>
<a name="fx_s_copy-deepcopyfx_s-2" class="anchor" href="#fx_s_copy-deepcopyfx_s-2"></a>fx_s_copy = deepcopy(fx_s)</h1>
<p>Bug: RecursionError: maximum recursion depth exceeded while calling a Python object</p>
Cons of using torch.gpu.synchronize()

username: whatdhack
cooked: <p>See messages about needing to use torch.gpu.synchronize()  at the end of an iteration for “correct” time .  Does not adding this statement destroy  data  transfer and compute overlapping ?  Hence,  increases  duration or reduces throughput ? That certainly would not produce the correct time , would it ?</p>

username: ptrblck
cooked: <p>A synchronization is needed if you want to profile the workload since CUDA operations are executed asynchronously.<br>
If you don’t synchronize you would profile the dispatching and kernel launch instead of the actual kernel execution time.</p>
<p>You are right that synchronizations slow down your code and should be generally avoided unless you want to profile the GPU workload.</p>

username: whatdhack
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , thanks for  confirming my suspicion. In that case, what is the most accurate  way to measure throughput performance (  images/sec,  samples/sec , etc.)  with time.time()  - per epoch , per 100 mini-batches  while not handicapping measurements  with synchronize() ?</p>

username: ptrblck
cooked: <p>Synchronize the code before starting and stopping the timers and execute the code for <code>N</code> iterations. Once done, calculate the average.</p>

username: whatdhack
cooked: <p>So something like the following:</p>
<pre><code class="lang-auto">start_timer
torch.gpu.synchronize()
# run N iters
torch.gpu.synchronize()
stop_timer
</code></pre>
<p>Also, I think when N is sufficiently large,  synchronization is not really needed .  That number likely is closer to the maximum attainable rate.</p>

username: ptrblck
cooked: <p>The start timer should be started after the first synchronization.</p>

username: whatdhack
cooked: <p>Makes sense to not include synchronize() delay of  previous  ops.  if N  is 25+ , I think we can do away with the synchronize altogether.</p>
I can not find draw_keypoints from utils.py

username: Yuxiang_Zhou
cooked: <p>My torch version is 1.10.2+cu113 and torchvision version is 0.11.3+cu113, but why do some funtions(draw_keypoints and flow_to_image) lost ?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/7/076213b2874f5c9a6935e69322a57877f8fca0d2.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/076213b2874f5c9a6935e69322a57877f8fca0d2" title="utils"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/7/076213b2874f5c9a6935e69322a57877f8fca0d2_2_690x84.jpeg" alt="utils" data-base62-sha1="13jtdbZGuiscDNLpIWnZjm8Uf9E" width="690" height="84" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/7/076213b2874f5c9a6935e69322a57877f8fca0d2_2_690x84.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/0/7/076213b2874f5c9a6935e69322a57877f8fca0d2.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/7/076213b2874f5c9a6935e69322a57877f8fca0d2.jpeg 2x" data-dominant-color="2D2E2C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">utils</span><span class="informations">878×108 4.72 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: InnovArul
cooked: <aside class="quote no-group" data-username="Yuxiang_Zhou" data-post="1" data-topic="148099">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yuxiang_zhou/48/45449_2.png" class="avatar"> Yuxiang_Zhou:</div>
<blockquote>
<p>why do some funtions(draw_keypoints and flow_to_image) lost ?</p>
</blockquote>
</aside>
<p>I see that these functions were added in torchvision version <code>0.12.0</code>.</p>
How to create a similar tensor except shape in C++?

username: yangzh
cooked: <p>This is about creating a PyTorch tensor using C++.<br>
I have an input tensor A. I want to create a new zero tensor B, which has the same properties as A except for a different shape (for example, A.shape=(3,) and B.shape=(3, 2)). Is there an elegant way to achieve this?<br>
I know if A and B have the same shape, I can use B = torch::zeros_like(A), but now they have different shapes, what can I do?</p>
How to apply knowledge distillation in a multi-label setup?

username: pratik_mazumder
cooked: <p>Since knowledge distillation seems to work on the softmax of the logits, how can we apply knowledge distillation in a multi-label setting where generally a sigmoid is used to obtain the probability of each label?</p>

username: satinder_singh
cooked: <p>Not 100% sure, but I think the following should work.<br>
loss for multi class classification task<br>
<code>alpha * loss_fn(student_model_output, one_hot_encoded_vectors) + (1 - alpha) * kl_divergence_loss(teacher_model_output, student_model_output)</code></p>
<p>Since in multi label setup each output is a probability in itself. I think the modified loss with knowledge distillation should be.<br>
If n = number of outputs</p>
<pre><code class="lang-auto">alpha * loss_fn(student_model_output, hard encoded labels) + 
(1-alpha) * 1/n * sum(kl_divergence_loss(student_output[output_idx], teacher_output[output_idx]) for output_idx in outputs) 
</code></pre>
Ways to online inference streaming ASR

username: Rachel_Zhang
cooked: <p>Hi <a class="mention" href="/u/nateanl">@nateanl</a>,</p>
<p>For online streaming ASR inference, I found two ways,</p>
<ol>
<li>
<p><a href="https://pytorch.org/audio/main/tutorials/online_asr_tutorial.html" rel="noopener nofollow ugc">Here</a> list a tutorial, I wonder whether it is only designed for CPU? (Because I find this page is only available for v0.12.0dev+cpu) And is there any plan to extend it to GPU inference?</p>
</li>
<li>
<p>v0.11.0/examples/asr/librispeech_emformer_rnnt/pipeline_demo.py provides a demo for online inference, which is a feature of v0.11.0 but is removed from branch <code>main</code>. I wonder whether it is stable and can reproduce the results as shown in the mov of <a href="https://github.com/pytorch/audio/pull/2192" class="inline-onebox" rel="noopener nofollow ugc">Revise RNN-T pipeline streaming decoding logic by hwangjeff · Pull Request #2192 · pytorch/audio · GitHub</a>?</p>
</li>
</ol>
<p>Further, is there any other ways to implement online ASR with GPU?</p>
<p>Waiting for your reply, sincerely.</p>

username: hwangjeff
cooked: <p>Hi <a class="mention" href="/u/rachel_zhang">@Rachel_Zhang</a> —</p>
<ol>
<li>It can be easily adapted to work on GPU. Moving the decoder to GPU</li>
</ol>
<pre><code class="lang-auto">decoder = bundle.get_decoder().to(device="cuda")
</code></pre>
<p>and moving the <code>features</code> and <code>length</code> tensors in the inference loop to GPU</p>
<pre><code class="lang-auto">features = features.to(device="cuda")
length = length.to(device="cuda")
</code></pre>
<p>should suffice.</p>
<ol start="2">
<li>That demo script still exists; it has just been moved to <a href="https://github.com/pytorch/audio/blob/main/examples/asr/emformer_rnnt/pipeline_demo.py" class="inline-onebox" rel="noopener nofollow ugc">audio/pipeline_demo.py at main · pytorch/audio · GitHub</a>.</li>
</ol>

username: Rachel_Zhang
cooked: <p>Thank you very much. <a class="mention" href="/u/hwangjeff">@hwangjeff</a></p>
DDP is affected by code modification

username: Jacfger
cooked: <p>I was training a model using DDP and I wished to train multiple instance with some difference in configuration. However, when I tried to checkout to another branch, the running instance suddenly report an error and stopped. The error also only existed in the newly checkout branch, indicating the process actually read the modified code. Is that supposed to happen and how exactly does this work? I don’t quite understand…</p>
<p>I saw there’s a similar question but unfortunately no one had answer.</p><aside class="quote" data-post="1" data-topic="142020">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ckx506772099/48/45901_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/with-the-pytorch-distributed-training-model-code-changes-affect-running-applications/142020">With the PyTorch Distributed training model, code changes affect running applications</a> 
  </div>
  <blockquote>
    I noticed two strange things, 
First, when I used Distributed for training, I modified the code. After a while, the application reported an error when RUNNING because of my modification. 
Second, I added print to the code, and after a while, the program that was already running will start to print accordingly
  </blockquote>
</aside>


username: ptrblck
cooked: <p>The behavior seems to depend on the OS or rather the method to fork or spawn the processes as described <a href="https://stackoverflow.com/a/60910365">here</a>.</p>
F.avg_pool2d can not get the same result as F.adaptive_avg_pool2d when I trying to feed a same input and expecting the same output shape

username: feiyuhuahuo
cooked: <p>My code in a network forward:</p>
<pre><code class="lang-auto">        print(feat.shape)
        
        atten = F.avg_pool2d(feat, feat.size()[2:])
        lll = F.adaptive_avg_pool2d(feat, (1, 1))

        print(atten.shape)
        print(lll.shape)
        print((atten-lll).max())
        print((atten-lll).min())
</code></pre>
<p>And the result:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/3/5/35a554a37ea499709c4536170d6c3028632cecaf.png" alt="image" data-base62-sha1="7EzwyGEb2rHf17cpLJyLJuph6az" width="583" height="251"><br>
The outputs have the same shape, but there’s a minor difference between them. Is this normal?</p>

username: ptrblck
cooked: <p>You are most likely running into small numerical errors due to the limited floating point precision caused by e.g. a different order of operations in these layers.</p>
Terminate called after throwing an instance of 'c10::Error' what() : CUDA error: uncorrectable NVLink error detected during the execution

username: twisti14
cooked: <p>When running faster_rcnn,I found this error,I hope someone can help me!<br>
I just runned epoch0<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/7/e78e89965dd8173419325565288a25bc830096a3.png" data-download-href="https://discuss.pytorch.org/uploads/default/e78e89965dd8173419325565288a25bc830096a3" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e78e89965dd8173419325565288a25bc830096a3_2_690x205.png" alt="image" data-base62-sha1="x2rGcOAhBxExz6XetPVyKYnIAgz" width="690" height="205" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e78e89965dd8173419325565288a25bc830096a3_2_690x205.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e78e89965dd8173419325565288a25bc830096a3_2_1035x307.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e78e89965dd8173419325565288a25bc830096a3_2_1380x410.png 2x" data-dominant-color="E5E5E5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×573 107 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Could you rerun the script with:</p>
<pre><code class="lang-auto">NCCL_DEBUG=INFO TORCH_DISTRIBUTED_DEBUG=DETAIL python script.py args
</code></pre>
<p>and post the logs here, please?</p>

username: twisti14
cooked: <p>Sorry, I can’t run these scripts because I use a high computing platform to calculate,<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/2/22bbbe27efd38eb4d31d0cc3a17df0970dc1dc8c.png" data-download-href="https://discuss.pytorch.org/uploads/default/22bbbe27efd38eb4d31d0cc3a17df0970dc1dc8c" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/2/22bbbe27efd38eb4d31d0cc3a17df0970dc1dc8c.png" alt="image" data-base62-sha1="4XgtcQ9kuBbdHEPiHkWK0ZPj7Pu" width="690" height="55" data-dominant-color="151515"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">743×60 4.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: twisti14
cooked: <p>Can I chat with you privately?</p>

username: ptrblck
cooked: <p><code>script.py</code> and <code>args</code> are only placeholders and you should replace them with your actual script file name and its arguments.</p>

username: twisti14
cooked: <p>thanks,I’ve solved it</p>

username: eric-haibin-lin
cooked: <p>How did you resolve the issue and what was the root cause?</p>
Error in Custom Loss function

username: Chukwuemeka_Ekwonu
cooked: <p>I wrote a custom loss function; Dice coefficient loss (DCWithLogitsLoss) but I dont know what am missing out as it keep giving me error of "Element 0 of tensors does not require grad and does not have a grad_fn.<br>
But when I used BCEWithLogitsLoss, I don’t get any error.</p>
<p>Here is my code:</p>
<p>class DCWithLogitsLoss(nn.Module):<br>
“”“Dice Coefficient loss function with sigmoid activation”""</p>
<pre><code>def __init__(self):                                                                         
    super().__init__()                                                                      
                                                                                            
def __call__(self, SR, GT):                                                                 
    eps = 1e-5                                                                              
    assert SR.shape == GT.shape, "Predicted and Groundtruth images must have same size!"    
                                                                                            
    #SR = torch.sigmoid(SR)                                                                 
    SR = (SR &gt; 0.5).float()                                                                 
    inter = SR * GT                                                                         
    union = torch.sum(SR ** 2) + torch.sum(GT ** 2) + eps                                   
    score = (2 * inter.float() + eps) / (union.float())                                     
                                                                                            
    return 1. - score</code></pre>

username: ptrblck
cooked: <p>The threshold operation is not differentiable, so you are detaching the result from the computation graph:</p>
<pre><code class="lang-python">SR = torch.randn(10, 10, requires_grad=True)
SR = (SR &gt; 0.5).float() 
print(SR.requires_grad)
&gt; False
</code></pre>
<p>You could take at e.g. the <code>kornia</code> <a href="https://kornia.readthedocs.io/en/latest/_modules/kornia/losses/dice.html#dice_loss">implementation of dice loss</a> and reuse it.</p>

username: Chukwuemeka_Ekwonu
cooked: <p>Thank you for the response. I will use it.</p>
Autograd failed to compute gradient

username: Chukwuemeka_Ekwonu
cooked: <p>I tried computing gradients for implementation of Physics informed deep learning but I kept getting runtime error:</p>
<p>RuntimeError: One of the differentiated Tensors does not require grad.</p>
<p>I Have tried all possible solutions that I cam across online but not successfully resolved it. Below is the code for computing my first and second derivative.</p>
<p>def fwd_gradients(Y, x):<br>
dummy = torch.ones_like(Y)<br>
G = torch.autograd.grad(Y, x, grad_outputs=dummy, create_graph=True)[0]<br>
Y_x = torch.autograd.grad(G, x, grad_outputs=torch.ones_like(G), create_graph=True)[0]</p>
<pre><code>return Y_x</code></pre>
"Unable to find a valid cuDNN algorithm to run convolution" on backward() function

username: ksmdanl
cooked: <p>I am getting an error on the backward function.</p>
<pre><code class="lang-auto">  File "/home/kusuma/.conda/envs/pyssd/lib/python3.8/site-packages/torch/autograd/__init__.py", line 154, in backward
    Variable._execution_engine.run_backward(

RuntimeError: Unable to find a valid cuDNN algorithm to run convolution
</code></pre>
<p>I looked it up what might cause it and some sources point out that the batch size caused it. I reduced the batch size from 8 to 1 and it worked.<br>
But is there any other way to avoid this bug since using batch size of 1 for the training makes less sense for me?<br>
I’m using PyTorch 1.10, running on 1080Ti.<br>
Does anyone have any insight on this matter?</p>
<pre><code class="lang-auto">&gt;&gt;&gt; print(torch.__version__) # =&gt; 1.10.1
&gt;&gt;&gt; print(torch.version.cuda) # =&gt; 11.3
&gt;&gt;&gt; print(torch.backends.cudnn.version()) # =&gt; 8200
</code></pre>

username: ptrblck
cooked: <p>Could you install the latest nightly binary as it should use cuDNN8.3.2 and might have already solved the issue. If you are still hitting the error, could you post the model definition as well as the input shapes so that we could try to reproduce it, please?</p>

username: ksmdanl
cooked: <p>Thanks for the heads up and sorry for the late response! I wasn’t be able to install the nightly binary as it’s not my personal gpu and I’m not assigned as an administrator.</p>
<p>What I tried though was debugging via <code>CUDA_LAUNCH_BLOCKING=1</code> with assumption that it’d show me something I missed since I only use single gpu.<br>
Turned out it was because of memory issue.<br>
Next assumption is I believe having my training script computing quite a few losses from multiple loss functions and having a speed-memory trade off via setting <code>torch.backends.cudnn.benchmark = False</code> might also be the case.</p>
<p>After some clean ups and disabling the inbuilt auto-tuner the training worked just fine:)<br>
But what would the latest nightly binary highlight though in my case? Would it show me a more specific error? I’m still just wondering as I have no personal gpu to try this debugging alternative.</p>

username: ptrblck
cooked: <p>It wouldn’t necessarily show a better error message unfortunately, but it would give me a signal if this issue still persists (and might indeed be a functional issue). It seems you’ve narrowed it down to the memory requirement now.</p>
Save model or save the best model?

username: Z_Huang
cooked: <p>I see it in many different PyTorch tutorials. There are two types of methods to save models.</p>
<p>For example, a model is trained using train/validation/test (k-fold cross-validation).</p>
<p>The first method is that after training/validation is completed, then save the model (no epoch accuracy and best accuracy comparison).</p>
<p>The second method is that during the validation process, save the model where the validation accuracy is the highest.</p>
<p>Which method is the right/recommended/better one?</p>

username: Andrei_Cristea
cooked: <p>Not going to pretend that I know the definitive answer here, but just two thoughts to consider:</p>
<ul>
<li>The more you think overfitting is a risk (either because you don’t have enough training data given your particular model design choice, or for any other reason) the more you should favor saving the “best” model. This is because overfitting will cause actual performance to start to drop after enough training has occurred (as the model starts to train on the particulars of your training data, and not on its desired generalizable characteristics) and you want to make sure you use a version of the model from before the falling-off in performance.</li>
<li>The less validation data you have, the more “noisy” your validation accuracy is, and therefore the less you should trust the “best” model.</li>
</ul>
<p>Just a couple of crude rules of thumb.</p>
Multi Input Multi Output (MIMO) for time series forecasting

username: zaratruta
cooked: <p>Hi.</p>
<p>I’m dealing with a problem where I have several different sources of multivariate time series, and these sources have an interaction, in a way that the dynamics of the overall system is affected by the dynamics of each source.</p>
<p>I would like to try a MIMO model (using LSTMs OR TFTs) for dealing with this scenario. My idea is to model each source as an input and consider the separate outputs of each source as the outputs of my network.</p>
<p>I think that in this way I would have a stronger supervisor signal for my model and it would be able to learn the interactions between the several sources.</p>
<p>Is there any good example of code using Pytorch that implements a MIMO model that is similar to this one?</p>
Generating Numeric data using GANs

username: HassanAli
cooked: <p>Hi,<br>
I am working on activity recognition using gait analysis. I need to generate gait data data synthetically. Gait data will be numeric like this.<br>
Acc = 0,345, .567, 0.89, …</p>
<p>Is it possible to generate this type of data using pytorch?</p>

username: mxahan
cooked: <p>I guess it should be possible. You can think of the ACC data as real input X (similar to the image). The problem seems like “Generative time-series data.” <a href="https://towardsdatascience.com/synthetic-time-series-data-a-gan-approach-869a984f2239" rel="noopener nofollow ugc">Link</a></p>
Loss grad print None while shows grad func during loss print

username: cbd
cooked: <p>I print the loss function it gives the below output.</p>
<pre><code class="lang-auto">LOSS VALUE IS  tensor(92268.0312, device='cuda:0', grad_fn=&lt;SumBackward0&gt;)
</code></pre>
<p>I print the loss.shape and loss.grad, it gives below output.</p>
<pre><code class="lang-auto">LOSS shape  torch.Size([]) and grad None
</code></pre>
<p>Why it shows nothing in shape and None in grad while i print loss it has value as well grad_fn?<br>
Is the gradients flow correctly or am i doing some mistake?</p>

username: KFrank
cooked: <p>Hi Cbd!</p>
<aside class="quote no-group" data-username="cbd" data-post="1" data-topic="148044" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/13edae/48.png" class="avatar"> cbd:</div>
<blockquote>
<pre><code class="lang-auto">LOSS shape  torch.Size([]) and grad None
</code></pre>
<p>Why it shows nothing in shape</p>
</blockquote>
</aside>
<p>This is the shape of a zero-dimensional tensor – a pure scalar.  It’s<br>
a conceptually-legitimate beast that is supported by pytorch,</p>
<aside class="quote no-group">
<blockquote>
<p>None in grad</p>
</blockquote>
</aside>
<p>Do you get <code>grad</code> for leaf variables (after calling <code>loss.backward()</code>)?</p>
<p>Although it computes them, autograd does not <em>retain</em> <code>grad</code> for non-leaf<br>
variables.</p>
<p>Best.</p>
<p>K. Frank</p>
Vectorized power function

username: francescocarzaniga
cooked: <p>I have a single tensor where all elements need to be elevated to each power contained in another tensor.</p>
<p>Let’s say I have tensor A with shape [100], and tensor B with shape [64, 5]. Tensor A is my initial tensor, and tensor B contains a 64-batch of 5 elements.</p>
<p>Tensor C should be [torch.pow(A, exp) for exp in batch for batch in B], with shape [64, 100, 5]. torch.pow does not allow this natively (it does element-by-element exponentiation if I feed it a vector as exp as described in the docs). How I do it now is with the aforementioned double for-loop, but it is not efficient. Is there a way to vectorize this?</p>

username: KFrank
cooked: <p>Hi Francesco!</p>
<aside class="quote no-group" data-username="francescocarzaniga" data-post="1" data-topic="148038" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/b3f665/48.png" class="avatar"> francescocarzaniga:</div>
<blockquote>
<p>Is there a way to vectorize this?</p>
</blockquote>
</aside>
<p>Use pytorch’s <a href="https://pytorch.org/docs/1.11/notes/broadcasting.html" rel="noopener nofollow ugc">broadcasting</a> with <code>unsqueeze()</code> to line the dimensions<br>
up appropriately:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.pow (A.unsqueeze (-1), B.unsqueeze (1))
# or
torch.pow (A.unsqueeze (0).unsqueeze (-1), B.unsqueeze (1))
</code></pre>
<p>Best.</p>
<p>K. Frank</p>
Strange torch.nn.Module Behaviour with Multiple Inheritance

username: sicroci
cooked: <p>I am trying to create a module class that inherits from two classes as follows:</p>
<pre><code class="lang-auto">from torch import nn

class Module1(nn.Module):
	def __init__(self):
		nn.Module.__init__(self)

		self.l1_loss = nn.L1Loss()


class Module2(nn.Module):
	def __init__(self):
		nn.Module.__init__(self)


class Module12(Module1, Module2):
	def __init__(self):
		Module1.__init__(self)

		if hasattr(self, 'l1_loss'):
			print('l1_loss at position 1')

		Module2.__init__(self)

		if hasattr(self, 'l1_loss'):
			print('l1_loss at position 2')
</code></pre>
<p>When I run the code, “l1_loss at position 1” is printed but not “l1_loss at position 2”. It seems that when  Module2.<strong>init</strong>(self) is called, the member variable self.l1_loss is deleted. Why? how can I create a module with multiple inheritance?</p>

username: ptrblck
cooked: <p>I think you could use:</p>
<pre><code class="lang-python">class Module12(Module1, Module2):
	def __init__(self):
		super(Module12, self).__init__()
		if hasattr(self, 'l1_loss'):
			print('l1_loss at position 1')

		if hasattr(self, 'l1_loss'):
			print('l1_loss at position 2')

m = Module12()
</code></pre>
<p>which will print both statements.</p>

username: TaZi-loh
cooked: <p>if the example is a bit more complicated and you actually <strong>have</strong> to call the init methods separately, you can try something like</p>
<pre><code class="lang-auto">class Module1(nn.Module):
	def __init__(self):
		if not hasattr(self, "__module_initialized"):
			nn.Module.__init__(self)
			self.__module_initialized = True

		self.l1_loss = nn.L1Loss()


class Module2(nn.Module):
	def __init__(self):
		if not hasattr(self, "__module_initialized"):
			nn.Module.__init__(self)
			self.__module_initialized = True
</code></pre>
<p>though I have to say, that your usecase looks a bit strange to me. I had a similar problem which came however from a diamond inheritance structure. In your case I wonder, if it might be better to let <code>Module12</code> have two members of types <code>Module1</code> and <code>Module2</code></p>
torch._C._LinAlgError after calling torch.linalg.inv from pytorch 1.11 using CUDA 11.3

username: Sureerat_Reaungamorn
cooked: <p>Hi everyone,</p>
<p>I got torch._C._LinAlgError when I called <code>torch.linalg.inv</code> on an invertible matrix representing a similarity transformation (comprising scaling and translation) from Pytorch 1.11 and CUDA 11.3.</p>
<p>Below is the code that produced the error</p>
<pre><code class="lang-auto">import torch
import numpy as np

device=torch.device('cuda')
normalization=np.asarray([[[ 0.0138,  0.0000,  0.0000, -2.6834],
 [ 0.0000,  0.0138,  0.0000, -2.2656],
 [ 0.0000,  0.0000,  0.0138, -1.2021],
 [ 0.0000,  0.0000,  0.0000,  1.0000]]])
normalization=torch.from_numpy(normalization).to(device)
torch.linalg.inv(normalization)
 
</code></pre>
<p>The full error message is</p>
<pre><code class="lang-auto">torch._C._LinAlgError: cusolver error: CUSOLVER_STATUS_EXECUTION_FAILED, when calling `cusolverDnDgetrf( handle, m, n, dA, ldda, static_cast&lt;double*&gt;(dataPtr.get()), ipiv, info)`. This error may appear if the input matrix contains NaN.
</code></pre>
<p>I did not get the error if I used Pytorch 1.9 with CUDA 11.1, but got an inverse matrix</p>
<pre><code class="lang-auto">tensor([[[ 72.4638,   0.0000,   0.0000, 194.4493],
         [  0.0000,  72.4638,   0.0000, 164.1739],
         [  0.0000,   0.0000,  72.4638,  87.1087],
         [  0.0000,   0.0000,   0.0000,   1.0000]]], device='cuda:0',
       dtype=torch.float64)
</code></pre>
<p>Are there anyone successfully call <code>torch.linalg.inv</code> using Pytorch 1.11 with CUDA 11.3? Would anyone know why I got the error and how I could fix it?</p>
<p>Thank you very much</p>

username: ptrblck
cooked: <p>Could you update to the nightly with CUDA11.5, as I cannot reproduce the issue with it?</p>

username: Sureerat_Reaungamorn
cooked: <p>Thank you very much, for a quick respond! We have to update Ubuntu 16.04.7 to 20.04.3 to be able to use CUDA 11.5, but we will do so.</p>
<p>Thanks so much again.</p>
Early Stopping for GAN

username: Najeh_Nafti
cooked: <p>How can I add early stopping technique for a GAN model ? Should be added for both generator and discriminator ?</p>

username: mxahan
cooked: <p>The early stopping technique works as a regularizer by stopping updates once some condition is matched (usually no improvement in the validation loss for some epoch).</p>
<p>You can set some conditions for your G or D and stop gradient flow (stop model updating) based on that.</p>

username: Najeh_Nafti
cooked: <p>So, it is enough to be applied for just one model (G or D)?</p>

username: mxahan
cooked: <p>Yes, it is. You have the flexibility to apply it on G or D or both. You should have two optimizers (one for G and another for D).</p>
Exponentiating a float to sparse tensor

username: batatovich
cooked: <p>Is there a way of doing torch.pow(a, A), where a is a float and A is a sparse tensor?. Given that we know that a^0 = 1, how can i do this operation faster, i.e. without computing a^Aij for each Aij = 0.<br>
Im currently using my GPU so my intention is to do this paralelly.</p>
Simultaneously indexing a 1D tensor with rows of a 2D tensor

username: batatovich
cooked: <p>I have a one-dimensional tensor X whose components are either 0 or 1. Let’s say that this tensor has n elements.<br>
On the other hand, i have a 2D list/numpy array N whose rows dont have the same lenght.<br>
For example, N = [[1,4,8], [2,5],[0]…]. In particular, this array represents the structure of a given graph G, so that row i corresponds to the neigbhors nodes of vertex i.<br>
What i want is a way of simultaneously (on my GPU) index the 1D tensor X with each row of N, so i can do operations like computing the number of neigbhors whose X value is 1, for each node i.<br>
Long story short, is there a way of doing: torch.sum(X[N[i]])  for each row i of N, and obtain a tensor like this: B = [torch.sum(X[N[0]), torch.sum(X[N[1]), torch.sum(X[N[2]]), …] of lenght n?</p>