akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Unet model problem?

Karan_Mehta: <p>I am trying to train a model by using Unet model. My images are in RGB channel with size of 1024, I am resizing it to (400,400,3). But while training i am getting this error. “The size of tensor a (400) must match the size of tensor b (11) at non-singleton dimension 2”. How to solve this?</p>

_Bustion: <p>You might have several problems. It would be easier to tell if you have provided the implementation, but I am going to name a couple of things to look out for:</p>
<ul>
<li>
<strong>Resizing</strong>. You need to make sure you did not just cut a picture, but actually resized it</li>
<li>
<strong>Dimensions (original issue)</strong>. <a href="https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html" rel="noopener nofollow ugc">conv2d</a> here are the docs, and you need to focus on exact dimensions described there. I cant tell where the exact issue is without seeing the implementation.</li>
</ul>
<p>Good luck and familiarize yourself with theory behind all of this machinery.</p>
Element-wise batch multiplication

Jeffrey_Alido: <p>I have tensors X and Y where X has size(B,N,N) and Y has size (N,N). I’d like to element-wise multiply Y to every batch of X without replicating Y to be of size (B,N,N), nor building a for loop. Any tips?</p>

thecho7: <p><code>torch.add</code> supports broadcasting.<br>
<a href="https://pytorch.org/docs/stable/notes/broadcasting.html#broadcasting-semantics" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/notes/broadcasting.html#broadcasting-semantics</a></p>
<pre><code class="lang-auto">a = torch.rand(2, 3, 3)
b = torch.rand(3, 3)
c = torch.add(a, b) # (2, 3, 3)
</code></pre>
Position Embedding in Vision Transformers

ays: <p>I’m a bit confused how the position embedding in happened to each patch in the transformer. I thought Ideally we’d want each patch to have a value of (1, 2, 3, 4…) to describe the position of the patch in the image. but from the implementation <a href="https://github.com/gupta-abhay/pytorch-vit/blob/main/vit/patch_embed.py" rel="noopener nofollow ugc">here</a> there do something like this:</p>
<pre><code class="lang-auto"> # positional embedding
            self.pos_embed = nn.Parameter(
                torch.zeros(1, num_patches, embedding_dim)
            )
</code></pre>
<p>Which is quite confusing because now we have some sort of mapping instead of just a value appended to each patch. Also, there is some sort of implicit position appended to the patch right? Assume we have a patch embedding output (1, 256, 768); corresponding to (batch, num_patches, position_embedding). since we have 256 patches, then can’t our network understand that each patch is in the position of its index value? Why do we have to explicitly define a position embedding for each patch?. Also, please kindly explain the implementation above I’m not sure I understand the mapping and why its initialised to zero</p>
Compute custom gradients of parameter in ParameterDict

Kikii: <p>Hi all,</p>
<p>Is it possible to compute custom gradients for all parameter in a ParameterDict and return them as e.g. another dict in a custom backward pass?</p>
<pre><code class="lang-auto">class AFunction(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x, weights):
        ctx.x = x
        ctx.weights = weights
        return 2*x
    @staticmethod
    def backward(ctx, grad_output):
        grad_x = ...
        # ParameterDict weights does not need a gradient, only its entries
        # Is there a way to return or even compute the gradients for the parameter in a dict?
        return grad_x, grad_weights

class A(nn.Module):
    def __init__(self):
        self.weights = torch.nn.ParameterDict('a': nn.Parameter(torch.randn(3)), 
                                              'b': nn.Parameter(torch.randn(3)))
        
</code></pre>
<p>Thank you!</p>

soulitzer: <p>Custom autograd Function does not consider Tensors in structures as participating in autograd, you’ll need to write a wrapper around your custom autograd Function to first unpack the tensors from that dict before calling into autograd Function, and pack the tensors back into the structure after. After writing a lot of boilerplate of converting dict to list and back it could look like the following:</p>
<pre><code class="lang-python">import torch

def to_dict(ks, vs):
    return {k: v for k, v in zip(ks, vs)}

def to_list(dct):
    return zip(*[(k, v) for k, v in dct.items()])

class Func(torch.autograd.Function):
    @staticmethod
    def forward(ctx, *args):
        ks, *vs = args
        ctx.save_for_backward(*vs)
        ctx.ks = ks
        inp = to_dict(ks, vs)
        return inp["a"] * inp["b"]

    @staticmethod
    def backward(ctx, grad_out):
        ks = ctx.ks
        vs = ctx.saved_tensors
        inp = to_dict(ks, vs)
        grad_dict = {
            "a": inp["b"] * grad_out,
            "b": inp["a"] * grad_out
        }
        _, gvs = to_list(grad_dict)
        return None, *gvs

def func(in_dict):
    in_keys, in_list = to_list(in_dict)
    return Func.apply(in_keys, *in_list)

inp = {
    "a": torch.tensor(1., requires_grad=True),
    "b": torch.tensor(2., requires_grad=True)
}

out = func(inp)
out.backward()
print(inp["a"].grad, inp["b"].grad)
</code></pre>
<p>Does that work for your case?</p>

Kikii: <p>Thank you <a class="mention" href="/u/soulitzer">@soulitzer</a> for your quick reply!<br>
Your idea is a very clever way to manage this. Unfortunately, I dynamically add new parameters in the forward pass and I want to take these into account in the backward pass. So I might want to calculate more gradients than I pass parameters to the forward function. So far I have solved the problem by setting the gradients using <code>weights['a'].grad = ...</code>, but I am not sure whether this works exactly as I wish.</p>
Is manually manipulating gradient of a tensor bad idea?

sodlqnf123: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/4/5419c104d38884d91e6b54f8cab203854e90873a.png" data-download-href="https://discuss.pytorch.org/uploads/default/5419c104d38884d91e6b54f8cab203854e90873a" title="picture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/5419c104d38884d91e6b54f8cab203854e90873a_2_518x500.png" alt="picture" data-base62-sha1="bZZhGaQiRWyN2Tgr7wx1PV0ioKC" width="518" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/5419c104d38884d91e6b54f8cab203854e90873a_2_518x500.png, https://discuss.pytorch.org/uploads/default/original/3X/5/4/5419c104d38884d91e6b54f8cab203854e90873a.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/4/5419c104d38884d91e6b54f8cab203854e90873a.png 2x" data-dominant-color="E8E8E8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">picture</span><span class="informations">645×622 46.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>This is a figure from <strong><a href="https://arxiv.org/pdf/2008.02268.pdf" rel="noopener nofollow ugc">Nerf in the wild</a></strong>. (You may not need to know what it is exactly)<br>
In brief, <strong>“Static”</strong> RGB and <strong>“Transient”</strong> RGB value will be compared to the ground truth value and loss for each is calculated.<br>
What I want to do is, for instance, transient embedding is updated only based on loss from static value, not the transient one. So, I write the code like below.</p>
<pre><code class="lang-auto">static_loss.backward(retain_graph=True)
static_grad = transient_embedding.grad
transient_loss.backward()
transient_embedding.grad = static_grad
optimizer.step()
</code></pre>
<p>If the code serves right, all the other parameters must be updated based on both losses, and transient embedding is updated based only on static loss. The result is not quite satisfactory, but I want to know whether the bad result comes out because the approach above is not a good idea in itself (or just wrong)</p>

soulitzer: <p>Your transient embedding is also being updated on both losses. You should update your code to:</p>
<pre><code class="lang-python">static_loss.backward(retain_graph=True)
static_grad = transient_embedding.grad.clone()   # &lt;---- added a clone
transient_loss.backward()
transient_embedding.grad = static_grad
optimizer.step()
</code></pre>
<p>The clone is necessary because otherwise static_grad still points to the same grad tensor and transient loss would add the new gradients to that tensor in-place.</p>
Feedback loop in training is not obvious

yaitskov: <p>Hi,</p>
<p>In math loss function gets just 2 arguments expected and real function values and returns the error, which is used by optimizer together with gradients to tweak NN parameters.</p>
<p>In PyTorch examples loss object doesn’t get reference to NN and has magic method <code>backward()</code>, which apparently doing optimization, but it doesn’t have arguments nor returned value.</p>
<p>Is there a global variable?</p>
<pre><code class="lang-auto">criterion = nn.NLLLoss()

def train(category_tensor, line_tensor):
    hidden = rnn.initHidden()

    rnn.zero_grad()

    for i in range(line_tensor.size()[0]):
        output, hidden = rnn(line_tensor[i], hidden)

    loss = criterion(output, category_tensor)
</code></pre>
<p><strong>Following line is</strong></p>
<pre><code class="lang-auto">    loss.backward()

    # Add parameters' gradients to their values, multiplied by learning rate
    for p in rnn.parameters():
        p.data.add_(p.grad.data, alpha=-learning_rate)

    return output, loss.item()
</code></pre>

ptrblck: <aside class="quote no-group" data-username="yaitskov" data-post="1" data-topic="175788">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yaitskov/48/58765_2.png" class="avatar"> yaitskov:</div>
<blockquote>
<p>In PyTorch examples loss object doesn’t get reference to NN and has magic method <code>backward()</code>, which apparently doing optimization, but it doesn’t have arguments nor returned value.</p>
</blockquote>
</aside>
<p>That’s not the case as <code>backward</code> calculates the gradients. The <code>optimizer.step()</code> method is updating the parameters using the previously calculated gradients or as seen in your case you are manually updating the parameters.<br>
<a href="https://pytorch.org/tutorials/beginner/nn_tutorial.html">This tutorial</a> might be helpful.</p>

yaitskov: <p>So loss calculation is optional for training?!</p>

ptrblck: <p>You would calculate the loss first and call <code>.backward()</code> on it afterwards in common use cases. However, you could also call <code>backward()</code> directly on the model output if it would fit your use case.</p>

yaitskov: <p>Everything became clear for me after watching sections Autograd and Backpropagation in <a href="https://www.youtube.com/watch?v=c36lUUr864M" rel="noopener nofollow ugc">Deep Learning with  PyTorch course</a>. Indeed PyTorch Tensors are magic (not just a wrapper around fast C array) and they track all their history of computations via <code>requires_grad</code> option, so every operand gets gradient automatically including neuron weights!</p>
Unable to install pytorch and cudatoolkit

nkkaushal: <p>I tried running this below command inside my conda environment (python version 3.10.10) on macbook (M1 Pro chip)<br>
conda install pytorch==1.7.1 torchaudio==0.7.2 cudatoolkit=10.1 -c pytorch</p>
<p>And I got this error</p>
<p>===================================================================</p>
<p>PackagesNotFoundError: The following packages are not available from current channels:</p>
<ul>
<li>cudatoolkit=10.1</li>
</ul>
<p>Current channels: (These channels r actually links I am unable to post more than 2 links because I am a new user)</p>
<ul>
<li>osx-64(<a href="https://conda.anaconda.org/pytorch/osx-64" class="inline-onebox" rel="noopener nofollow ugc">pytorch/osx-64</a>)</li>
<li>noarch</li>
<li>osx-64</li>
<li>noarch</li>
<li>osx-64</li>
<li>noarch</li>
</ul>

ptrblck: <p>I don’t think your Macbook supports CUDA and I also don’t think the MPS backend was already available in <code>torch==1.7.1</code>. Did you try to install the latest stable or nightly release instead?</p>
CUDA 11.6 deprecation help

mya2152: <p>Hi all,</p>
<p>I’ve recently been getting the error: RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf</p>
<p>in my google colab instance, I then updated pytorch and torchaudio to latest then kept getting error:</p>
<p>ImportError: cannot import name ‘fail_with_message’ from ‘torchaudio._internal.module_utils’ (/usr/local/lib/python3.9/dist-packages/torchaudio/_internal/module_utils.py)</p>
<p>There was also an error somewhere claiming: pytorch CUDA version 11.6 is the current version but CUDA was on 11.7 (I’m presuming this is the version that the Google instance GPU is on)? Either way from this message I decided to update pytorch and torchaudio to latest but then kept getting:</p>
<p>ImportError: cannot import name ‘fail_with_message’ from ‘torchaudio._internal.module_utils’ (/usr/local/lib/python3.9/dist-packages/torchaudio/_internal/module_utils.py)</p>
<p>when previously just yesterday my code was working fine.</p>
<p>This is what I’ve been using to install pytorch in my instance which was working last night:<br>
pip install --extra-index-url <a href="https://download.pytorch.org/whl/cu116" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu116</a></p>
<p>Any ideas?</p>
<p>Thanks in advance</p>

ptrblck: <p>Your install command would install an older PyTorch and <code>torchaudio</code> release as the current <code>2.0.0</code> release ships with CUDA 11.7 or 11.8.<br>
I also cannot reproduce the <code>torchaudio</code> error using the latest <code>2.0.1</code> version:</p>
<pre><code class="lang-python">from torchaudio._internal.module_utils import fail_with_message
fail_with_message
&lt;function torchaudio._internal.module_utils.fail_with_message(message)&gt;

torchaudio.__version__
'2.0.1+cu118'
</code></pre>
Is it possible to add `amax` operator in an old version(1.10.0) pytorch?

4daJKong: <p>When I use <code>torch.onnx.export</code> to export my model to an onnx, it shows error,</p>
<blockquote>
<p>Exporting the operator amax to  Exporting the operator amax to ONNX opset version 11 is not supported.</p>
</blockquote>
<p>I know when pip install a new version of torch(1.12.0) it can be solves this problem,<br>
However, due to some reason, my torch version must be fixed(detectron2 0.6 can only worked in torch 1.10), so is it possible to only add part of code in old version torch to make it work?</p>
<p>In detail, I found this part in new ver pytorch</p>
<pre><code class="lang-auto">(site_packages/torch/onnx/symbolic_opset9.py),
@_onnx_symbolic("aten::amax")
@symbolic_helper.quantized_args(True)
@symbolic_helper.parse_args("v", "is", "i")
@_beartype.beartype
def amax(g: jit_utils.GraphContext, self, dim, keepdim):
    return g.op("ReduceMax", self, axes_i=dim, keepdims_i=keepdim)
</code></pre>
<p>I was wondering if I can copy/paste corrsponding code into old version…</p>

ptrblck: <p>You can certainly try to copy/paste the code into the locally installed PyTorch source files given in the posted path but would need to make sure <em>all</em> needed code is available.<br>
However, in case you need to add C++ code, you would need to rebuild PyTorch.</p>

4daJKong: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="176219">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>o make sure <em>all</em> needed code is availa</p>
</blockquote>
</aside>
<p>Hi ptrblck, thanks for your reply again, now the problem is how and where to add/copy the new operator to old version…</p>

ptrblck: <p>Based on your previous post I assumed you have already found the missing code. To add it, you could check <code>print(torch.__path__)</code> and manipulate the corresponding files directly. However, as mentioned before, backporting a new feature into an old release might need more code changes than just adding the new method so you could also check the corresponding PR to see where manipulations were done.</p>
How to get round memory leak caused by accumulating loss in for loop?

JimW: <p>Hi I have received a model from someone else， and I am trying to train it with our flow.</p>
<p>For this model, the most convenient way is to generate one label for a single sample. So I am doing something like this:</p>
<pre><code class="lang-auto">total_loss=0 
for i in range(len(dataset)):
     pred = model(dataset[i])
     gt = labels[i]
     if i%batch_size==0:
         total_loss.backward()
         optimizer.step()
         total_loss = 0
     else：
         total_loss += CrossEntropy(pred, gt) 
</code></pre>
<p>I understand it is not the best practice to accumulate loss in this way.  My observation is :  when I used batch_size larger than 4,  it will run for several epochs and then crash with CUDA Out of Memory error.</p>
<p>Is there a way in pytorch to get around this?<br>
I found some solutions online which is to add “.item()” to total_loss,  but I think it will make this node undifferentiable.</p>

Joschka: <p>I see two issues with your code.</p>
<ol>
<li>
<p>You are never zeroing the gradients of your model in-between <a href="https://pytorch.org/docs/stable/optim.html#taking-an-optimization-step" rel="noopener nofollow ugc">optimization steps</a>. This is not necessarily related to your memory leak, but most likely still a problem.</p>
</li>
<li>
<p>I think if <code>i % batch_size</code> is equal to 0, the prediction made in that iteration and the correpsonding computation graph might never be cleared, because with your current control-flow, they are never subject to any backward pass. Computing and accumulating the loss in <em>every</em> iteration should fix this.</p>
</li>
</ol>
<p>Hope this helps:</p>
<pre><code class="lang-python">total_loss=0 
for i in range(len(dataset)):
     pred = model(dataset[i])
     gt = labels[i]
     total_loss += CrossEntropy(pred, gt)
     if i%batch_size==0:
         optimizer.zero_grad()
         total_loss.backward()
         optimizer.step()
         total_loss = 0
</code></pre>
Pthread_setaffinity_np causes performance degradation for allocating tensor

Seungsu: <p>Hi. I got some weird issue about pthread_setaffinity_np and Tensor allocation.</p>
<pre><code class="lang-auto">Hardware
  Intel(R) Xeon(R) CPU Gold 5218 x 2
  Nvidia Titan RTX

Software
  Ubuntu 20.04
  Pytorch v1.9.0
  CUDA 11.3
  CUDNN 8.2.1
</code></pre>
<p>Below is my program code snippet.</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt;
#include &lt;torch/cuda.h&gt;
#include &lt;chrono&gt;
#include &lt;iostream&gt;
#include &lt;thread&gt;
#include &lt;pthread.h&gt;
#include &lt;cuda_runtime_api.h&gt;

void create()
{
  auto options = torch::TensorOptions();

  while(true) {
    std::vector&lt;torch::jit::IValue&gt; inputs;
    long start_time = now(); // now() is my function that use chrono library and steady_clock
    auto tensor= torch::ones({1, 3, 224, 224}, torch::TensorOptions().pinned_memory(true));
    long end_time = now();
    std::cout &lt;&lt; (end_time - start_time) &lt;&lt; std::endl;
  }
}

int main(int argc, char* argv[])
{
  std::thread crt(create);

//  cpu_set_t cpuset;
//  CPU_ZERO(&amp;cpuset);
//  CPU_SET(0, &amp;cpuset);
//
//  int s = pthread_setaffinity_np(crt.native_handle(), sizeof(cpu_set_t), &amp;cpuset);

  crt.join();


  return 0;
}
</code></pre>
<p>When i run this program, the time for creating tensor is under 1ms. (almost 0.01ms)</p>
<p>But, if i remove // and make that code lines work, the time result increases extreamely. (60ms ~ 100ms and more)</p>
<p>I also use under command. That is normal.</p>
<pre><code class="lang-auto">taskset -c 0 ./program
</code></pre>
<p>Can you give me some advice for this issue?</p>
Can't Install PyTorch on MacOS

RobertPaulson: <p>I’m on MacOS (Ventura 13.1) and trying to install PyTorch on Python version 3.11.1</p>
<p>I ran the following command per the PyTorch website via the terminal</p>
<pre><code class="lang-auto">pip3 install torch torchvision 
</code></pre>
<p>but getting the following error when running</p>
<pre><code class="lang-auto">ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch
</code></pre>
<p>After much troubleshooting, it’s not clear what could be wrong here.</p>
<p>I expected the installation to take place but getting this error instead</p>

M_leo: <p>pip3 install --upgrade pip<br>
try update your python</p>
Executing two models in parallel while utilizing the torch.no_grad() function to disable gradient computation during inference

Benjamin_Kelenyi: <p>Hello everyone,</p>
<p>I’m trying to make a knowledge distillation between two losses but I encounter the following issue:</p>
<p><code>RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument </code>find_unused_parameters=True<code>to</code>torch.nn.parallel.DistributedDataParallel<code>; (2) making sure all </code>forward<code>function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's</code>forward<code>function. Please include the loss function and the structure of the return value of</code>forward<code> of your module when reporting this issue (e.g. list, dict, iterable).</code></p>
<p>I am currently running two models simultaneously, with one of the models being solely for inference purposes. However, it appears that the parameters from this inference model are being combined with the parameters of the main model, which is not my intention.</p>
<p>I tried with:</p>
<pre><code class="lang-auto">        with torch.no_grad():
            self.net = FeatureExtractor(pipeline.extractor, device, {'resize': 1280})
</code></pre>
<p>also,<br>
tried to add “find_unused_parameters=True”, but without solving the problem.</p>
<p>I’m using pytorch lightning.</p>
<p>Do you have any idea what cloud be the problem?</p>
<p>Thanks,<br>
B</p>

ptrblck: <p>You might want to wrap the inference model into <a href="https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html#torch.nn.parallel.DistributedDataParallel.no_sync"><code>torch.nn.parallel.DistributedDataParallel.no_sync</code></a> which will disable the gradient synchronization.</p>
Pipeline Parallelism training

Avyay_Casheekar: <p>I have tried to train a transformer using pipeline parallelism on one gpu. It used to work before the update to pytorch 2.0<br>
I’m unsure about why there is a loop back in installing the older version where it reinstalls pytorch 2.0 when installing torch data to a previous version</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/1i_gjiI1etC_HLklw5v9vz56pTXO68Rqc?usp=sharing">
  <header class="source">
      <img src="https://ssl.gstatic.com/colaboratory-static/common/6a85a4b6ee4a668e84fbd2202db7bf9b/img/favicon.ico" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1i_gjiI1etC_HLklw5v9vz56pTXO68Rqc?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/drive/1i_gjiI1etC_HLklw5v9vz56pTXO68Rqc?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


ptrblck: <p>I cannot reproduce the issue using <code>torch==2.0.0</code> and <code>torchtext==0.15.1</code> and also see an error in the <code>import torch</code> line of code as you are using <code>!import torch</code> which is also raising an error:</p>
<pre><code class="lang-python">/bin/bash: import: command not found
</code></pre>
How to create a class from a .pt file model, or load a model from a .pt file without the class

Daniel_Piovanelli_Ar: <p>Hello!</p>
<p>I have a .pt file for a ASR model that I didn’t create, and I want to deploy it. But I don’t have it’s parameters, or the class that I should use to deploy it.</p>
<p>Is there a way to create the class from the .pt file?</p>
<p>If not, how could I deploy the model without it’s class? Is there a pytorch framework with which I could deploy this model?</p>
<p>Thank you!</p>

ptrblck: <p>It depends what the <code>.pt</code> files contains and how it was stored.<br>
If the <code>state_dict</code> was stored in it, you would need the actual model code to recreate a model instance and to be able to load the <code>state_dict</code> afterwards. The same would apply with even more restrictions, if the entire model was saved via <code>torch.save(model, PATH)</code>, as you would also need to restore the actual source code structure.<br>
However, if a scripted model was stored via <code>torch.jit.save(scipted_model, PATH)</code> you could load it directly via <code>torch.jit.load</code>.</p>
Question about gradient

YK_Z: <p>I have implemented here the code:</p>
<p>def calc_grad_hessian(func, x):<br>
x = torch.tensor(x, dtype=torch.float, requires_grad=True)<br>
y = func(x)<br>
grad = torch.autograd.grad(y, x, create_graph=True)[0]<br>
hessian = torch.zeros(x.shape[0], x.shape[0])<br>
for i in range(x.shape[0]):<br>
hessian[:,i] = torch.autograd.grad(grad[i], x, retain_graph=True)[0]<br>
return y.detach().numpy(), grad.detach().numpy(), hessian.detach().numpy()</p>
<p>Even I have decleared that the requires_grad of x is true, it always says there is an error in<br>
grad = torch.autograd.grad(y, x, create_graph=True)[0]:<br>
<em>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</em></p>
<p>Can anyone tell me why?</p>
<p>Thank you in advance.</p>
<p>Best Regards,<br>
YK_Z</p>

soulitzer: <p>It’s impossible to tell why without looking at what <code>func</code> does. You probably want to bisect your func to see what operations you are doing that break the autograd graph. Common ways to break the graph are constructing a new tensor with <code>torch.ones_like</code> or <code>torch.tensor(...)</code> or converting to numpy and back.</p>
Request for Sample PyTorch Implementation of Variable Length Many-to-One LSTM/GRU Regression with monotonically increasing constraints on intermediate output values

WirryWoo: <p>Hello all, new member of the PyTorch community here.</p>
<p>I have been trying to find PyTorch code examples online to implement an RNN with multiple constraints specific to the problems I’m currently working on for my side project but found no success in doing so.</p>
<p>I want to build an RNN model (ideally using LSTMs or GRUs) with the following criteria:</p>
<ul>
<li>The model accepts variable length inputs with no upper limit on the length of the input.</li>
<li>The model has a many-to-one RNN framework (input is sequential data with many features, output is one value)</li>
<li>The model produces a positive output value (float) for any input received.</li>
<li>The model’s intermediate outputs in the RNN monotonically increases as more timesteps are observed in the architecture.</li>
</ul>
<p>Using a picture as reference here (<a href="https://scientistcafe.com/ids/images/rnnrollout.png" rel="noopener nofollow ugc">https://scientistcafe.com/ids/images/rnnrollout.png</a>), respectively:</p>
<ul>
<li>The model should handle varying <code>&lt;t&gt;</code> values</li>
<li>Only the true value <code>y^{&lt;t&gt;}</code> is known in model training.</li>
<li>
<code>\hat{y}^{&lt;t&gt;} &gt; 0</code> for all inputs received in the model, and <code>\hat{y}^{&lt;t&gt;}</code> is type float.</li>
<li>
<code>\hat{y}^{&lt;1&gt;} \leq \cdots \leq \hat{y}^{&lt;t-1&gt;} \leq \hat{y}^{&lt;t&gt;}</code> for all <code>&lt;t&gt;</code>
</li>
</ul>
<p>Any explanations and some reproducible code implementations using synthetic dummy data would greatly help me! Thank you!</p>
RuntimeError: Both events must be recorded before calculating elapsed time

octadion: <p>Traceback (most recent call last):<br>
File “train.py”, line 531, in <br>
main() # pylint: disable=no-value-for-parameter<br>
File “/home/octa/anaconda3/envs/venvdiffgan/lib/python3.7/site-packages/click/core.py”, line 1130, in <strong>call</strong><br>
return self.main(*args, **kwargs)<br>
File “/home/octa/anaconda3/envs/venvdiffgan/lib/python3.7/site-packages/click/core.py”, line 1055, in main<br>
rv = self.invoke(ctx)<br>
File “/home/octa/anaconda3/envs/venvdiffgan/lib/python3.7/site-packages/click/core.py”, line 1404, in invoke<br>
return ctx.invoke(self.callback, **ctx.params)<br>
File “/home/octa/anaconda3/envs/venvdiffgan/lib/python3.7/site-packages/click/core.py”, line 760, in invoke<br>
return __callback(*args, **kwargs)<br>
File “/home/octa/anaconda3/envs/venvdiffgan/lib/python3.7/site-packages/click/decorators.py”, line 26, in new_func<br>
return f(get_current_context(), *args, **kwargs)<br>
File “train.py”, line 524, in main<br>
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)<br>
File “train.py”, line 357, in subprocess_fn<br>
training_loop.training_loop(rank=rank, **args)<br>
File “/home/octa/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/training/training_loop.py”, line 437, in training_loop<br>
value = phase.start_event.elapsed_time(phase.end_event)<br>
File “/home/octa/anaconda3/envs/venvdiffgan/lib/python3.7/site-packages/torch/cuda/streams.py”, line 177, in elapsed_time<br>
return super(Event, self).elapsed_time(end_event)<br>
RuntimeError: Both events must be recorded before calculating elapsed time.</p>

ptrblck: <p>The issue seems to be specific to <code>Diffusion-GAN</code> and solved <a href="https://github.com/Zhendong-Wang/Diffusion-GAN/issues/9">here</a>.</p>
Multidimensional gather

diskhkme: <p>Hi,</p>
<p>Is there any operation available in PyTorch to do something like the one below?</p>
<pre><code class="lang-auto">out[x][y] = in[ind[0][x][y]] [ind[1][x][y]] [ind[2][x][y]]
</code></pre>
<p>I want to avoid loops here so I prepared an index tensor like this.</p>
<p>So the context is there are batches of input images(<code>in</code>).</p>
<p>In index tensor(<code>ind</code>), the channel of the target image is stored in 0 dim and where to sample pixels from that target image for corresponding output(<code>out</code>) pixel is stored in 1,2 dim.</p>

ptrblck: <p>Could you write a (slow) reference code using nested loops to see what the expected output would be, please?</p>

diskhkme: <p>Thanks for your reply. Actually, I just solved the problem by simply flattening the <code>in</code> and <code>ind</code> tensors.</p>
<p><code>in (b,H,W)</code> flattened to <code>in (b x H x W)</code><br>
<code>ind (b,H,W)</code> flattened to <code>ind (b x H x W)</code></p>
<p>Then simply by <code>in[ind]</code>, I could get <code>out (H x W)</code> and rearranged values.</p>
<p>Thank you for your interest.</p>
Getting RuntimeError: Expected all tensors to be on the same device

Zowie_Tay: <p>im currently following the guide on quantization from here<br>
<a href="https://pytorch.org/tutorials/prototype/fx_graph_mode_ptq_static.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/prototype/fx_graph_mode_ptq_static.html</a></p>
<p>i tried using “model.to(self.device)” in the calibrate function but it doesnt seem to be working, does anyone know what i did wrong?</p>
<pre><code class="lang-auto">        self.device = "cuda" if torch.cuda.is_available() else "cpu"
        print(self.device)
        self.model = Net().to(self.device)
</code></pre>
<pre><code class="lang-auto">            model_to_quantize = copy.deepcopy(self.model)
            model_to_quantize.eval()
            qconfig = get_default_qconfig("x86")
            qconfig_mapping = QConfigMapping().set_global(qconfig)
            def calibrate(model, data_loader):
                model.to(self.device)
                model.eval()
                with torch.no_grad():
                    for image, target in data_loader:
                        model(image)
            example_inputs = (next(iter(data_loader))[0])
            prepared_model = prepare_fx(model_to_quantize, qconfig_mapping, example_inputs)
            calibrate(prepared_model, data_loader_test)
            self.model = convert_fx(prepared_model)
</code></pre>
<p>but when i try to run it i get</p>
<pre><code class="lang-auto">  43 with torch.no_grad():
     44     for image, target in data_loader:
---&gt; 45         model(image)
...
   1163     )
   1164 else:
   1165     new_min, new_max = torch.aminmax(x)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument self in method wrapper_CUDA_out_histc_out)
</code></pre>

Mahi: <p>There is a mismatch in tensor types. Try to run with <code>print(x.type())</code> for the tensors in the model. All the tensors should either be in torch.FloatTensor or in torch.cuda.FloatTensor in general.</p>

jcaip: <p><a class="mention" href="/u/zowie_tay">@Zowie_Tay</a> Are both your inputs and model have to be on the same device? You can move your inputs the same way you move your model to a different device.</p>
<pre><code class="lang-python">def calibrate(model, data_loader):
    model.to(self.device)
    model.eval()
    with torch.no_grad():
        for image, target in data_loader:
            image.to(self.device)
            model(image)
</code></pre>

Zowie_Tay: <p>oh i realised the difference between this tutorial and the normal one and they used a</p>
<p><code>image = image.to(device)</code></p>
<p>before calling the model on the image thanks!</p>

Zowie_Tay: <p>turns out it was because i forgot to send the image to the gpu side</p>