akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
RuntimeError: Expected target size [6, 1, 224, 224], got [6, 3, 224, 224]

Tamim_Adnan: <p><a href="https://stackoverflow.com/questions/75141125/how-to-solve-typeerror-cross-entropy-loss-argument-input-position-1-mus" rel="noopener nofollow ugc">RuntimeError: Expected target size [6, 1, 224, 224], got [6, 3, 224, 224]</a></p>
Torch.where() not working and returns nan value when using exp() function

pnpsuM: <p>With Torch(1.13.x), I’ve been trying to implement some activation functions from scratch like mish or ELU, etc. for custom activation function.</p>
<p>However, I get nan value of loss after about 17 epochs when I train the model.</p>
<ul>
<li>dataset: official MNIST dataset from each framework</li>
<li>model architecture: simple dense network(25 layers with 500 neurons each)</li>
<li>lr: 1e-3 (I don’t want to fix this)</li>
<li>batch_size: 128</li>
<li>optimizer: Adam</li>
</ul>
<p>Torch code:</p>
<pre><code class="lang-auto">class Mish_Implementaion(nn.Module):
  def __init__(self):
    super(Mish_Implementaion, self).__init__()
    self.__name__ = 'Mish'
  
  def forward(self, x):
    return t.where(x &lt; -7, 0, t.where(x &gt; 30, x, x * t.tanh(t.log(1 + t.exp(x)))))
</code></pre>
<p>I used:</p>
<pre><code class="lang-auto">t.autograd.set_detect_anomaly(True)
</code></pre>
<p>and got this error message: <code>Function 'ExpBackward0' returned nan values in its 0th output.</code></p>
<p>I guess it’s because of exp. function getting overflow. But that’s why I used torch.where function to avoid exp() return too high of a value.</p>
<p>I want to add some trainable parameter here, so making this work would be important.</p>
<p>Any advice is really appreciated, Thanks in advance.</p>

ptrblck: <p>You might be running into <a href="https://discuss.pytorch.org/t/gradients-of-torch-where/26835/2">this behavior</a> so could you check if your use case might suffer from the same logic creating invalid gradients?</p>

pnpsuM: <p>You are right!<br>
I needed to regularize exp(x) according to the condition, since every single calculation matters in BP.<br>
So I just edited my code like following:</p>
<pre><code class="lang-auto">def forward(self, x):
    mask = (x &gt; 30)
    return mask * x + ~mask * x * t.tanh(t.log(1 + t.exp(~mask * x)))
</code></pre>
<p>and it’s working.</p>
<p>But still, from the link you mentioned, I don’t get the part that gradient of x is sum of two input gradients. Does it mean that where() function considers gradient of both tensor simultaneously regardless of condition?</p>
<p>Thank you for the help.</p>
Pix2pix inference results in the same outputs

bekhzod: <p>I am using Pix2pix model to generate new images based on my data (image). I am using <a href="https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix" rel="noopener nofollow ugc">pytorch_pix2pix</a>. I trained the model using my own data and get pretty good results. However,  I have a problem of having the same outputs the same input images even though they are trained using images with various shapes.</p>
<p>I think it is an issue to have the same seed for the inference but I could not find any seed variable to change in the test.py. If there is any tip to get various results please let me know. Thank you in advance.</p>
<p>Is there any variable (for example, seed) to get various images during inference?</p>
Understanding the calculation of memory bandwidth

Rishav_Sapahia: <p>Hi,<br>
I come across <a href="https://horace.io/brrr_intro.html" rel="noopener nofollow ugc">this</a> blog and couldn’t wrap my head around this calculation.</p>
<p><code>For simple operators, it's feasible to reason about your memory bandwidth directly. For example, an A100 has 1.5 terabytes/second of global memory bandwidth, and can perform 19.5 teraflops/second of compute. So, if you're using 32 bit floats (i.e. 4 bytes), you can load in 400 billion numbers in the same time that the GPU can perform 20 trillion operations.</code></p>
<p>How the value of 400 billion and 20 trillion operations is calculated here?</p>

ptrblck: <p>“400 billion numbers”  seems to be picked as an example and the corresponding “20 trillion operations” are then calculated by multiplying the compute of “19.5 teraflops/second” by the data transfer time.<br>
Here is a quick example:</p>
<pre><code class="lang-python">mem_bandwidth =  1.5 * 1024**4
compute = 19.5 * 1024**4
data = 400 * 4 * 1e9
transfer_time = data / mem_bandwidth
ops_in_transfer_time = transfer_time * compute
print(ops_in_transfer_time / 1e12)
# 20.8
</code></pre>
Quick Way to Append Batch Tokens to Vocab object

J_Johnson: <p>Refer to here: <a href="https://pytorch.org/text/stable/vocab.html" class="inline-onebox" rel="noopener nofollow ugc">torchtext.vocab — Torchtext 0.13.0 documentation</a></p>
<p><code>build_vocab_from_iterator()</code> is very quick at getting tokens and indices into a Vocab object when fed a list of words or iterator of word lists.</p>
<p>But what if I need to add another large list of words to that vocab object? I’ve used <code>vocab.append(possibly_new_token)</code> but that seems to only work with an iterator and is very slow for a large list of words. Is there a way to either:</p>
<p>a. Add a new list of tokens(and ignore existing), or<br>
b. Combine two vocab objects into one, while ignoring duplicates?</p>

dreidizzle: <p>Would it be possible to get all the words from a built vocabulary into a list, merge this list to the new list, and then call <code>build_vocab_from_iterator()</code> on this new list?</p>

J_Johnson: <p>I’m not seeing a vocab method to get the list out or manipulate it.</p>
<p>Currently, just making do with the smallest pre-trained GloVe embedding vector, which is still much larger than I’d like.</p>
<p>Am looking for a more memory-efficient embedding system. Bonus points if it can leverage phenomes, first, and connect that to a reduced embedding vector. I believe such a system would have a more intuitive “understanding” of language, given how written language was derived from oral language.</p>
How to prevent fx to trace into my custom defined module

ralphmao: <p>I have a CustomConv2d module which subclasses nn.Conv2d. When I run fx.symbolic_trace, it does not stop at CustomConv2d, but will rather trace into it and generate a graph with conv_weights, F.conv2d, etc.</p>
<p>How to inform fx to treat CustomConv2d as an end node?</p>

ralphmao: <p>One option may be to subclass Tracer and implement a new <code>is_leaf_module</code> method. Wondering if there is any other approaches</p>

Yuchen_Wang: <p><a href="https://pytorch.org/docs/stable/fx.html#torch.fx.wrap" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/fx.html#torch.fx.wrap</a></p>
Torch.fx.trace with real tensors, similar to torch.jit.trace

beni045: <p>I have been trying to create an fx graph from a module using torch.fx.symbolic_trace(). This often results in many errors because of the symbolic tracing with Proxy objects. In many cases, it would work much better to trace with real tensors, to “freeze” dynamic logic parts of the code and avoid type errors. This would be similar to how torch.jit.trace works.</p>
<p>Is there some way to do this? Maybe a different implementation of Proxy?</p>
Cifar100: spikes in accuracy and loss

Khawar_Islam: <p>I am training a <code>preactresnet18</code> network on the cifar100 dataset. I have adjusted all training settings but still, I can see spikes in my training loss and accuracy. I have no idea why it’s happening. I have changed the learning rate batch size but still i cannot solve it.</p>
<p><strong>Training Configuration</strong></p>
<p>Single GPU (NVIDIA 3080 <code>12 GB</code>)<br>
The batch size is <code>384</code><br>
Epochs <code>400</code><br>
Learning rate <code>0.001</code>, momentum 0.9 weight decay <code>5e-4</code></p>
<p>Results<br>
<a href="https://i.stack.imgur.com/1GCqR.png" rel="noopener nofollow ugc"><img src="https://i.stack.imgur.com/1GCqR.png" alt="enter image description here" width="690" height="283"></a></p>
Random crash with PyTorch 1.8.2

Gerald_Fauvelle: <p>Hi,</p>
<p>I have a random crash at init on Ubuntu 20.04, using PyTorch C++ 1.8.2.<br>
I don’t exactly know where it crashes, Valgrind report issue after TorchLibraryInit.</p>
<p>I could reproduce in Debug mode.</p>
<p>We were using PyTorch 1.5.1 before, and it was working well. So maybe we’re missing something for PyTorch 1.8.2 ?<br>
Is there a way to activate some logs from PyTorch to have more info on what is going on?</p>
<p>Thanks,</p>

ptrblck: <p>Could you try to update to the latest stable release (<code>1.13.1</code>) and check if you are still seeing the issue?</p>

Gerald_Fauvelle: <p>Well, that is what I did after posting my message. First impression is that it is not crashing anymore, I’ll need to run more tests though.<br>
So it could be enough, but I don’t really know why, as a developer, that doesn’t statisfy me <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"><br>
I’ll run more tests tomorrow to be sure I don’t see any issue.</p>
nn.DistributedDataParallelizm with batch_size = 1?

Anita: <p>Hi,</p>
<p>I am using a Multiple Instance Learning model, where I have the so-called bags that comprise a significant fraction of images with big sizes e.g batch_size, nr_patches, nr_channels, height, width  = [(1,  900, 3, 598, 598)]. These patches share the same label and are considered one input/image. The nr_patches is different between bags, due to memory constraints, and model selection on how the patches are aggregated -  I can only have a batch size of 1.</p>
<p>I wanted to use nn.DistributedDataParallel to parallelize computations of 1 batch to multiple GPUs. Is that possible? Does the nn.DistributedDataParallel parallelizes mini-batches across multiple GPUs even if batch_size =1. I am really hoping for a response form your side.</p>

ptrblck: <aside class="quote no-group" data-username="Anita" data-post="1" data-topic="170369">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/anita/48/56469_2.png" class="avatar"> Anita:</div>
<blockquote>
<p>I wanted to use nn.DistributedDataParallel to parallelize computations of 1 batch to multiple GPUs. Is that possible? Does the nn.DistributedDataParallel parallelizes mini-batches across multiple GPUs even if batch_size =1.</p>
</blockquote>
</aside>
<p>In the standard DDP setup (single process per GPU) each process is responsible to load and process the input batch. To avoid loading duplicated samples, the <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.distributed.DistributedSampler"><code>DistributedSampler</code></a> is used.<br>
You could set the batch size to <code>1</code> in each worker and load different “patches” instead. However, you would need to implement this logic into the <code>Dataset</code> and its <code>__getitem__</code> method as well as to the sampler, to avoid loading the same patches.<br>
In the default use case the <code>Dataset.__getitem__</code> receives the <code>index</code> from the sampler and is responsible to load and process one sample using the passed <code>index</code>. The <code>collate_fn</code> then stacks the different samples to an input batch.<br>
In your use case you might need to pass two indices to the <code>__getitem__</code> method (using a custom sampler) since now you have to check which sample to load and from this sample which patches.<br>
I might miss some details, so let me know if this approach makes sense to you.</p>
Undefined symbol: cublasLtGetStatusString, version libcublasLt.so.11

Mitchelll: <p>installed yolov6 on Ubuntu 22.04.  Cuda 11.2 + python3.10</p>
<p>run the command : python3 tools/infer.py -weights yolov6n.pt …<br>
Got the following message : undefined symbol</p>
<p>Any advice to solve this problem?</p>
<p>Traceback (most recent call last):<br>
File “/home/mitchell/envs/torchgpu/lib/python3.10/site-packages/torch/<strong>init</strong>.py”, line 172, in _load_global_deps<br>
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)<br>
File “/usr/lib/python3.10/ctypes/<strong>init</strong>.py”, line 374, in <strong>init</strong><br>
self._handle = _dlopen(self._name, mode)<br>
OSError: /home/mitchell/envs/torchgpu/lib/python3.10/site-packages/torch/lib/…/…/nvidia/cublas/lib/libcublas.so.11: undefined symbol: cublasLtGetStatusString, version libcublasLt.so.11</p>
<p>During handling of the above exception, another exception occurred:</p>
<p>Traceback (most recent call last):<br>
File “/home/mitchell/envs/YOLOv6/tools/infer.py”, line 8, in <br>
import torch<br>
File “/home/mitchell/envs/torchgpu/lib/python3.10/site-packages/torch/<strong>init</strong>.py”, line 217, in <br>
_load_global_deps()<br>
File “/home/mitchell/envs/torchgpu/lib/python3.10/site-packages/torch/<strong>init</strong>.py”, line 178, in _load_global_deps<br>
_preload_cuda_deps()<br>
File “/home/mitchell/envs/torchgpu/lib/python3.10/site-packages/torch/<strong>init</strong>.py”, line 158, in _preload_cuda_deps<br>
ctypes.CDLL(cublas_path)<br>
File “/usr/lib/python3.10/ctypes/<strong>init</strong>.py”, line 374, in <strong>init</strong><br>
self._handle = _dlopen(self._name, mode)<br>
OSError: /home/mitchell/envs/torchgpu/lib/python3.10/site-packages/nvidia/cublas/lib/libcublas.so.11: undefined symbol: cublasLtGetStatusString, version libcublasLt.so.11</p>

ptrblck: <p>Could you reinstall PyTorch as it could be related to <a href="https://github.com/pytorch/builder/pull/1227">this issue</a> which we already fixed?</p>

jamix: <p>I’ve had the same issue with PyTorch 1.13.1. What helped was removing <code>nvidia-cublas-cu11</code> and installing <code>nvidia-cublas-cu12</code> instead:</p>
<pre><code class="lang-auto">pip uninstall nvidia-cublas-cu11
pip install nvidia-cublas-cu12
</code></pre>

ptrblck: <p>The PyTorch wheels use <code>nvidia-cublas-cu11</code> as an explicit dependency since they were built with the corresponding CUDA toolkit shipping cuBLAS 11. Removing it and replacing it with a newer cuBLAS version from CUDA 12 could yield undefined behavior.<br>
Could you explain your previous setup in more detail, how you’ve installed it, etc.?</p>
RuntimeError: false INTERNAL ASSERT FAILED at "../aten/src/ATen/native/BatchLinearAlgebra.cpp":1491, please report a bug to PyTorch. torch.linalg.ldl_factor: Unknown error code: 1381

whoab: <p>torch.linalg.ldl_factor bugs out when applied to a symmetric, but non-PD matrix. I am not sure what the exact algorithm is, but I was expecting the “modified Cholesky” used in Newton optimization steps to correct the Hessian to be P.D. What does this error mean?</p>

ptrblck: <p>Could you post a minimal and executable code snippet to reproduce the issue as well as the output of <code>python -m torch.utils.collect_env</code>, please?</p>
Where's the backpropagation code?

Goldname: <p>I’m working on a project which requires me to rewrite the implementation of backpropagation, so I’d like to modify the current implementation of pytorch backprop. I took a look at the source code and found that it uses:</p>
<pre><code> Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass...
</code></pre>
<p>However I’m stuck here as I can’t seem to find where this function is defined. Anyone know where the code for backprop is?</p>
<p>Also, is this recommended? Would modifying backprop cause a lot of issues for regular gpu training (doesn’t need to work for distributed cases)?</p>

ptrblck: <p><code>run_backward</code> is defined <a href="https://github.com/pytorch/pytorch/blob/382ef1fda75dfce07c37a920e908ce96d99bf970/torch/csrc/autograd/autograd.cpp#L83">here</a> for the C++ backend and <a href="https://github.com/pytorch/pytorch/blob/301644d3cb8ce7ecb477b5bcb95a7a8a1304fa71/torch/csrc/autograd/python_engine.cpp#L168">here</a> for the Python frontend.</p>
<p>Yes, modifying internals could cause a lot of issues, but it depends on your actual use case in the end and what you are trying to achieve.</p>
Unsupervised Graph embedding

Peter_Samoaa: <p>Hi<br>
I’m trying to use Grap Representation in order to extract the unsupervised graph embedding.<br>
I’m using the following code</p>
<pre><code class="lang-auto">import networkx as nx
from karateclub.node_embedding.neighbourhood.grarep import GraRep


gr = GraRep(dimensions=90,order=13)
  gr.fit(G)
  embedding = gr.get_embedding()
</code></pre>
<p>The graph is pytorch geometric object<br>
I got the following error</p>
<pre><code class="lang-auto">AttributeError: 'GlobalStorage' object has no attribute 'number_of_nodes'
</code></pre>

ptrblck: <p>The issue might have been caused by a typo in PyG and <a href="https://github.com/pyg-team/pytorch_geometric/commit/af95df956f4473656f94d9ed04be0480534e397c">this commit</a> should fix it. Could you update to the latest release (including the fix) and check if you are still seeing the issue?</p>
Trying to get the shape of conv layer in the __init__() method

aquorio15: <p>I am trying to get the shape of the conv2 layer so that i can build a linear layer with that shape.</p>
<p>class PostConvNet(nn.Module):<br>
“”"<br>
Post Convolutional Network (mel → mel)<br>
“”"<br>
def <strong>init</strong>(self, num_hidden, num_input_layers):<br>
“”"</p>
<pre><code>    :param num_hidden: dimension of hidden 
    """
    super(PostConvNet, self).__init__()
    self.conv1 = nn.Conv1d(in_channels=hp.num_mels * hp.outputs_per_step,
                      out_channels=num_hidden,
                      kernel_size=16,
                      padding=4,
                      )

    self.conv_list = nn.Sequential(*[nn.Conv1d(in_channels=768,
                                 out_channels=768,
                                 kernel_size=16,
                                 padding=4,
                                 ) for _ in range(3)])
    
    self.conv2 = nn.Conv1d(in_channels=num_hidden,
                      out_channels=hp.num_mels * hp.outputs_per_step,
                      kernel_size=16,
                      padding=4)
    self.shape = self.conv2.shape[1]
    self.convLinear = nn.Linear(in_features=num_input_layers, out_features=self.shape)
    self.pre_batchnorm = nn.BatchNorm1d(num_hidden)
    self.batch_norm_list = clones(nn.BatchNorm1d(num_hidden), 3)
    self.dropout1 = nn.Dropout(p=0.1)
    self.dropout_list = nn.ModuleList([nn.Dropout(p=0.1) for _ in range(3)])

def forward(self, input_, mask=None):
    input_ = self.conv1(input_)
    input_1 = self.conv_list(input_)
    input_2 = self.conv2(input_1)
    input_ = self.convLinear(in_features=input_.shape(1), out_features=input_2.shape(1))(input_)
    
    return input_
</code></pre>
<p>My problem is i want to build a linear layer whose out_features are the shape of the conv2 layer</p>

ptrblck: <p>The output of a conv layer depends on the input shape and is thus not static. You could use an example input and check the shape manually or you could also use the <code>nn.LazyLinear</code> layer to allow PyTorch to set the <code>in_features</code> based on the input tensor.</p>

aquorio15: <p>Can i set the output_features in nn.LazyLinear to a value from the forward method?</p>

ptrblck: <p>You can set the <code>out_features</code> to any arbitrary value you wish and setting it to a computed value based on other operations would also work.<br>
However, I’m unsure why you want to set this value in the <code>forward</code> method as it does not depend on any shapes of incoming activation tensors.</p>
Torchvision for c++17

yoko: <p>I would like to use torchvision with C++17.</p>
<p>I’m trying these</p>
<ol>
<li>install torchvision from pip</li>
<li>export nms from torchvision with python code</li>
<li>build torchvision from source to use in C++</li>
<li>load scripted model in C++</li>
</ol>
<p>when I load the scripted model with c++17, they say there is no nms function in torchvision. When I use C++14, it works fine.</p>
<p>Do I need to build torchvision with C++17 by modifing CMAKEList.txt?<br>
Do I need to use torchvision for python built with C++17?</p>
Autograd of scatter_add_

IZugec: <p>Dear Pytorch community,</p>
<p>in one of the layers in forward pass of my model I am using tensor which requires grad in order to compute other tensor which I am later using as indices in scatter_add_</p>
<p>You can already see the problem as for this layer to make sense I absolutely <strong>need</strong> to continue this gradient, and on the other hand scatter_add_ expects indices to be integers. However, naturally, if I convert the tensor to dtype int, it breaks the gradient flow.</p>
<p>I would really appreciate any ideas/suggestions.<br>
Thanks in advance</p>

KFrank: <p>Hi IZugec!</p>
<aside class="quote no-group" data-username="IZugec" data-post="1" data-topic="170023" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/izugec/48/51049_2.png" class="avatar"> IZugec:</div>
<blockquote>
<p>I am using tensor which requires grad in order to compute other tensor which I am later using as indices</p>
</blockquote>
</aside>
<p>You simply can’t backpropagate through the computation of your indices.</p>
<p>Your indices – inherently integers – are discrete, so you can’t (usefully)<br>
take their derivatives, even with respect to a tensor that has<br>
<code>requires_grad = True</code>.</p>
<p>Let’s say you have a floating-point tensor, <code>x</code>, and you compute an index<br>
from it, <code>compute_index (x) = 17</code>.  Let’s say that as you vary <code>x</code> for a while,<br>
<code>compute_index (x)</code> remains <code>17</code>.  For these values of <code>x</code>, the gradient of<br>
the index with respect to <code>x</code> is zero – that is, the index is constant.</p>
<p>Now let’s say that you vary <code>x</code> a little more and the index jumps to <code>18</code>, after<br>
which it remains <code>18</code> for a while.  Right at the jump, the gradient is technically<br>
undefined (or perhaps <code>inf</code>, if you prefer).  Then after the jump the gradient<br>
is zero again.</p>
<p>So your gradient is zero almost everywhere and is undefined at some number<br>
of isolated points.  Although mathematically defined (for most values of <code>x</code>),<br>
such a gradient – being zero (almost) everywhere – isn’t useful for<br>
gradient-descent optimization.</p>
<p>Best.</p>
<p>K. Frank</p>

IZugec: <p>Hey KFrank,</p>
<p>I am aware of the problem with undefined derivative for discrete variable which integers are. I was hoping that this is an usual problem which maybe has a trick, but it seems I will have to think of an workaround.</p>
<p>Thank you very much for an extensive answer, I will use this opportunity to express that I really appreciate you beautiful knowledgeable people on this forum who take their time to answer us!</p>
<p>Best.</p>
RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment when deepcopying a model

Khabbab_Zakaria: <p>I am trying to deepcopy a model:</p>
<pre><code class="lang-auto">class Model(torch.nn.Module):
    def __init__(self, ann_private, rnn_public, policy):
        super(Model, self).__init__()
        self.ann_private = ann_private
        self.rnn_public = rnn_public
        self.policy = policy

    def forward(self, private_input, state):
      private_input = self.ann_private(private_input)
      processed_state = self.rnn_public(state)
      self.inference_state = torch.cat([processed_state, private_input], dim=-1)

      mu, sigma, state_values =  self.policy(self.inference_state)
      return mu, sigma, state_values
</code></pre>
<p>where the policy contains some ‘self.mu’, ‘self.sigma’ etc (which are as I suppose causes of this problem, as before I was just calling them mu, sigma and deepcopy was working, but now I explicitly need self for them).<br>
Any solution?</p>

dragonHyeon: <p>Using detach() might help solving your problem. I also faced a simillar issue with you. In my case I tried to deepcopy a object which contains model and other properties including the model’s output. This model’s output occurs an error since it is tensor which made by model, not by user explicitly. So I used detach() on model’s output to make it a tensor which doesn’t requires grad. This worked for me solving this problem. Hope this help solving your problem.</p>
<p>Good luck.</p>
Exact python implementation of Linear function class

richardbaxter: <p>For the purposes of fundamental research I need to modify the nn.Linear function class (both forward and backward functions). I require an exact version of the LinearFunction class in python, that does not suffer from miscellaneous errors. The example provided of a custom Linear class throws a broadcasting error when compared to the default C++ implementation (nn.functional.linear - torch._C._nn.linear);</p>
<p><a href="https://pytorch.org/docs/master/notes/extending.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/master/notes/extending.html</a></p>
<pre><code class="lang-auto">class LinearFunction(Function):
	...
	output = input.mm(weight.t()) -&gt; "RuntimeError: self must be a matrix"
</code></pre>

ptrblck: <p>You could check the backend implementation for <code>linear</code> <a href="https://github.com/pytorch/pytorch/blob/745fe35df520646350409cca5a61cf20d40b7c00/aten/src/ATen/native/Linear.cpp#L41-L96">here</a> and use similar logic in your custom code.</p>

richardbaxter: <p>Thanks I had a look at Linear.cpp but was not confident converting it. Here is a version of the extending LinearFunction example function that supports 3D matrix multiplication (replaces instances of <code>mm</code> with <code>matmul</code>, <code>t</code> with <code>swapaxes</code>);</p>
<pre><code class="lang-auto">class LinearFunction(Function):
	@staticmethod
	# ctx is the first argument to forward
	def forward(ctx, input, weight, bias):
		# The forward pass can use ctx.
		ctx.save_for_backward(input, weight, bias)
		output = input.matmul(weight.t())
		if bias is not None:
			output += bias.unsqueeze(0).expand_as(output)
		return output

	@staticmethod
	def backward(ctx, grad_output):
		input, weight, bias = ctx.saved_tensors
		grad_input = grad_weight = grad_bias = None

		if ctx.needs_input_grad[0]:
			grad_input = grad_output.matmul(weight)
		if ctx.needs_input_grad[1]:
			grad_weight = grad_output.swapaxes(-1, -2).matmul(input)
		if bias is not None and ctx.needs_input_grad[2]:
			grad_bias = grad_output.sum(0)

		return grad_input, grad_weight, grad_bias	
</code></pre>

richardbaxter: <p>Note <code>nn.linear</code> apparently supports 1-dimensional input, in which case the following check may be required;</p>
<pre><code class="lang-auto">if(len(grad_output.shape) &gt; 1):
	grad_outputT = grad_output.transpose(-1, -2)
else:
	grad_outputT = grad_output
grad_weight = grad_outputT.matmul(input)
</code></pre>
I decreased the learning rate by a factor of 10 and my model reaches 90% + accuracy, is this normal?

Ioannis_D: <p>Hello,<br>
I am using transfer learnig on vgg11 and train my model on dataset A and testing on dataset B and C to perform binary classification.<br>
Initially, the learnign rate was 0.001 and I used a scheduler every 2 epochs the accuracy doesnt decrease the accuracy would drop to 1/10th of the current learnign rate.</p>
<p>When I used this method, the accuracy of the model was stryggling to go voer 50% thought the learning rate was decreasing.</p>
<p>Then I initiated the learning rate at 0.0001 and the model accuracy after the first epoch is 90% and it keeps increasing.</p>
<p>Some more infromation about my training loop:</p>
<ol>
<li>My batch sie is 16<br>
2)I update the weights every batch size</li>
<li>At the end of each epoch I evaluate the model by testing it on the other datasets and the mean accuracy is 85% ± 2% and mean loss varies from 0.3 to 1.1</li>
</ol>
<p>Thank you</p>

ptrblck: <p>Yes, the described behavior could be normal if your learning rate was too high for an already pretrained model. Note that the original training routine might have already reduced the learning rate and increasing it to <code>1e-3</code> (still in the original training script) might have shown the same effect of catapulting the parameters out of the trained state.</p>
How to specify a built-in target when creating new node

Yuchen_Wang: <p>I want to insert a new node into the graph which slices a tensor. By checking the existing nodes, I find that this should be a node with <code>call_function</code> op and <code>&lt;built-in function getitem&gt;</code> target. My question is how to specify the target parameter as built-in get item function when creating the new node?</p>
<p>new_node = traced.graph.call_function(???, args=(slice(…)))</p>

jfix: <pre><code class="lang-python">import operator
# ...
new_node = traced.graph.call_function(operator.getitem, args=(slice(…)))
</code></pre>
nn.MultiheadAttention permutation equivalence

dichencd: <p>Hi, when I used nn.MultiheadAttention with permutated input order, I found that the output are not permutation equivalent. I would really appreciate your comments and help!</p>
<p>below are my test script:</p>
<pre><code class="lang-auto">import torch
from torch import nn
embed_dim = 128
num_heads = 4
multihead_attn = nn.MultiheadAttention(embed_dim, num_heads, batch_first = False)
multihead_attn.out_proj.weight.data = multihead_attn.out_proj.weight.data * 100
multihead_attn.out_proj.bias.data = torch.rand(embed_dim) * 10
x = torch.arange(100)[...,None].repeat(1,128).float()

perturbed_x = x.clone()
perturbed_idx = torch.randperm(100)
perturbed_x = x[perturbed_idx]

print(torch.max(perturbed_x - x[perturbed_idx]), torch.min(perturbed_x - x[perturbed_idx]))

x_in = x.unsqueeze(1)
perturbed_x_in = perturbed_x.unsqueeze(1)
out = multihead_attn(x_in,x_in,x_in)[0]
out1 = multihead_attn(perturbed_x_in,perturbed_x_in,perturbed_x_in)[0]

print(torch.max(out[permute_index] - out1), torch.min(out[permute_index] - out1))

</code></pre>
Error: XNNPACK Linear not available

ermilovim: <p>Hi all. I faced one problem… It will be great if someone could help me…</p>
<p>I just trained one nn like this:</p>
<pre><code class="lang-auto">traced_script_module = torch.jit.trace(model, example)
traced_script_module_optimized = optimize_for_mobile(traced_script_module)
traced_script_module_optimized._save_for_lite_interpreter("app/src/main/assets/model.ptl")```


Than, I'm loading it (Andoid, java):

        
</code></pre>
<p>Module module = LiteModuleLoader.load(assetFilePath(“model.ptl”), null, Device.CPU);<br>
module.forward(IValue.from(t));</p>
<pre><code class="lang-auto">
My build_gradle:

</code></pre>
<pre><code>implementation 'org.pytorch:pytorch_android_lite:1.12.1'
implementation 'org.pytorch:pytorch_android_torchvision_lite:1.12.1'
</code></pre>
<pre><code class="lang-auto">
pytorch version in python 1.12.1

After all, while debugging I see, than on module.forward I get this error:

</code></pre>
<p>E/AndroidRuntime: FATAL EXCEPTION: main<br>
Process: com.example.mlapplication, PID: 11326<br>
com.facebook.jni.CppException: XNNPACK Linear not available! Reason: The provided (weight, bias, output_min, output_max) parameters are either invalid individually or their combination is not supported by XNNPACK.</p>
<pre><code>  Debug info for handle(s): debug_handles:{-1}, was not found.
  
Exception raised from create at ../aten/src/ATen/native/xnnpack/Linear.cpp:84 (most recent call first):
(no backtrace available)
    at org.pytorch.LiteNativePeer.forward(Native Method)
    at org.pytorch.Module.forward(Module.java:52)
    at com.example.mlapplication.PyTorch_Model.Predict(PyTorch_Model.java:109)
    at com.example.mlapplication.GetSensors$AccelerometerListener.onSensorChanged(GetSensors.java:58)
    at android.hardware.SystemSensorManager$SensorEventQueue.dispatchSensorEvent(SystemSensorManager.java:886)
    at android.os.MessageQueue.nativePollOnce(Native Method)
    at android.os.MessageQueue.next(MessageQueue.java:335)
    at android.os.Looper.loopOnce(Looper.java:161)
    at android.os.Looper.loop(Looper.java:288)
    at android.app.ActivityThread.main(ActivityThread.java:7876)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1019)
</code></pre>
<pre><code class="lang-auto"></code></pre>
Difference b/w F.grad.conv2d_input and cudnn_convolution_backward

rinkujadhav2013: <p>I’m facing a memory leak (or morel likely increased objects) in my custom autograd function’s backward pass when I use a wrapper around <code>cudnn_convolution_backward</code>. If I switch to <code>F.grad.conv2d_input</code>, the CPU mem increase with each epoch is gone but I can’t use it because it’s much slower than using a wrapper around the cudnn function.<br>
So, the question is, what is <code>at::cudnn_convolution_backward_input()</code> doing that could be leading to more increased memory usage?<br>
I’m keeping the output of the layer and the grad for input on CPU and transferring it to GPU for the fwd and bwd passes.<br>
Also, the value of <code>len(gc.get_objects())</code> remains same so either some pytorch objects are not tracked by gc(?) or an object is growing(?)</p>
List index out of range when running pytorch Dataloader

aquorio15: <p>Hi everyone,</p>
<p>I have made a custom dataloader which will take files from two directory and put them in a dict. This is my code for the same.</p>
<p>##<span class="hashtag">#DataLoder</span>####</p>
<p>import numpy as np<br>
import pandas as pd<br>
from torch.utils.data import Dataset, DataLoader<br>
import os<br>
import hyperparams as hp<br>
import librosa<br>
from utils import get_spectrograms<br>
from tqdm import tqdm<br>
import glob<br>
class PrepareDataset(Dataset):</p>
<pre><code>def __init__(self, csv_file, source_dir, target_dir):
    # self.landmarks_frame = pd.read_csv(csv_file, sep='|', header=None)
    self.source_dir = source_dir
    self.target_dir = target_dir
    self.data = pd.read_csv(csv_file, sep=',', header=None)

def load_wav(self, filename):
    return librosa.load(filename, sr=hp.sample_rate)

def __len__(self):
    return len([f for f in os.listdir(self.source_dir)if os.path.isfile(os.path.join(self.source_dir, f))])

def __getitem__(self, idx):

    source_file = glob.glob(self.source_dir + '/*.wav')
    target_file = glob.glob(self.target_dir + '/*.wav')
    source_wav_name = os.path.join(self.source_dir, source_file[idx]) 
    mel_source, mag_source = get_spectrograms(source_wav_name)
    
    target_wav_name = os.path.join(self.target_dir, target_file[idx]) 
    mel_target, mag_target = get_spectrograms(target_wav_name)
    
    np.save(source_wav_name[:-4] + '.pt', mel_source)
    np.save(source_wav_name[:-4] + '.mag', mag_source)
    np.save(target_wav_name[:-4] + '.pt', mel_target)
    np.save(target_wav_name[:-4] + '.mag', mag_target)

    sample = {'source_mel':mel_source, 'source_mag': mag_source, 'target_mel':mel_target, 'target_mag':mag_target}

    return sample
</code></pre>
<p>root_dir = ‘/content/en-hi’<br>
dataset = PrepareDataset(os.path.join(root_dir,‘metadata.csv’), os.path.join(root_dir,‘Source’), os.path.join(root_dir,‘Target’))<br>
dataloader = DataLoader(dataset, batch_size=1, drop_last=False, num_workers=0)<br>
pbar = tqdm(dataloader)<br>
for d in pbar:<br>
pass</p>
<p>The issue is when i am trying to unpack the dataloader using enumerate i am getting a ‘List index out of range error’. I was successful in finding what the issue is, but i dont know how to solve it. Apparantly the idx in <strong>getitem</strong>() method is runnning even after reaching the end of the list.</p>
<p>Is there any way in which i can stop the idx loop after it reached the end of the list.</p>
<p>Thank you</p>

srishti-git1110: <p>Hi,<br>
This might be directly related to the <code>__len__</code> method.</p>
<p>Many implementations of the <code>Sampler</code> classes that are used by the DataLoader for sampling indices directly use the len method.</p>
<p>Make sure the method returns the correct length of your dataset, and also that the indices of your Map style dataset match with the indices the Sampler is using. I guess the latter is already taken care of in your case.</p>

aquorio15: <p>Hi thanks for the reply, I was able to solve the problem</p>
LSTM for multi-length time series

user36: <p>Hi!</p>
<p>Sorry for the question’s length - I believe it’s always better to be longer than unclear…<br>
I have tried a lot, ang chat-GPT has been my good friend, but I still feel what I’m trying to do should have quite a simple solution, and I’m sure I’m not the first to try something of this kind.</p>
<p>I am trying to classify time series.  I have data from a csv, where each sample is a two-row tensor of two different measurements.  The rows are all 1500 items long, meaning each item of the data set (after transposing it) should be of the shape of [1500,2].  Thing is, the series are not actually that long - it is just the maximum series length, so the vast majority of the series are padded with zeros.  I want to use an LSTM model to classify those series.</p>
<p>Thing is, I saw the simple solution is using the  <code>packed_data = nn.utils.rnn.pack_padded_sequence</code> in the forward function.  But after trying it, I found it weird that the  <code>packed_data.data</code> object is a tensor with all the batch items concatenated, and that the LSTM layer is applied to all at once, which means the result of one affects the other (is that really the case?).  After asking chat-GPT it told me that when the lengths are very different in size (which they are in my case), it is better to use dynamic unrolling.  But that seemed very weird, cause you need to create your custom dataloader (Because if <code>get_item</code> produces different sized items the regular dataloader won’t work), or put the items in one at a time.</p>
<p>So, down to the question (and another one after that):</p>
<ol>
<li>How do I train an <code>nn.LSTM</code> model that trains only on the relevant data?  My data set already has an option of returning the actual length of every item before padding.  I just want it to get an item (<code>X</code>) and it’s length (<code>L</code>) and apply the layer only on the relevant part (<code>X[:L,:]</code>).  Shouldn’t it be quite a common task?</li>
<li>after I have done that and trained the model, how can I use the model to classify a single item, but get the hidden state (and classification after the <code>nn.Linear</code> layer) after every step?  In the end that would be my case - I will have the data coming in step by step, and classify it in real time.</li>
</ol>
<p>Thank you very much!</p>
LibTorch example link error

lrrm: <p>I am following this exmaple, <a href="https://pytorch.org/cppdocs/installing.html" rel="noopener nofollow ugc">Installing C++ Distributions of PyTorch — PyTorch master documentation</a></p>
<p>The LibTorch is 1.9.0, <a href="https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.9.0%2Bcpu.zip" rel="noopener nofollow ugc">https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.9.0%2Bcpu.zip</a></p>
<p>I got below error message after cmake … and cmake --build .</p>
<pre><code class="lang-auto">/usr/bin/ld: cannot find -lmkl_intel_ilp64
/usr/bin/ld: cannot find -lmkl_core
/usr/bin/ld: cannot find -lmkl_intel_thread
collect2: error: ld returned 1 exit status
gmake[2]: *** [example] Error 1
gmake[1]: *** [CMakeFiles/example.dir/all] Error 2
gmake: *** [all] Error 2
</code></pre>
<p>cmake output</p>
<pre><code class="lang-auto">$ cmake -DCMAKE_PREFIX_PATH=/usr/local/libtorch ..
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- MKL_ARCH: None, set to ` intel64` by default
-- MKL_ROOT /opt/intel/oneapi/mkl/2021.3.0
-- MKL_LINK: None, set to ` dynamic` by default
-- MKL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
-- MKL_THREADING: None, set to ` intel_thread` by default
-- MKL_MPI: None, set to ` intelmpi` by default
-- Found Torch: /usr/local/libtorch/lib/libtorch.so  
-- Configuring done
-- Generating done
-- Build files have been written to: /xxx/yyy/example/build
</code></pre>

zhxia: <p>Hello, I got the same question.Did you have fixed it?</p>
Dataloaders with variable batch size and accessing specfic features in a dataloader

Tehreem: <p>We are training a Physics informed deep learning network. We have a time series dataset and want to give data in batches, and our dataset consists of five excel sheets with different lengths. We want to use four excel sheets as training and a fifth excel sheet as a test dataset. We are doing this because these experiments are performed sequentially; we can not give them randomly to our network. I need help with these two questions.<br>
How can we make data loaders with variable batch sizes?<br>
How can we access specific columns and features in the data loader?</p>
<p>We want to do batch training, for that we need to to give our data in the form of batches, but we need to extract specific columns from dataloader as we are training physics informed neural network.</p>
<pre><code class="lang-auto">t_train = torch.tensor(train[:, :, 0])  
u_train = torch.tensor(train[:, :, 1:-1]) 
x_train = torch.tensor(train[:, :, 3])            
t_test = torch.tensor(test[:, :, 0])  
u_test = torch.tensor(test[:, :, 1:-1])  
x_test = torch.tensor(test[:, :, 3]) 
</code></pre>
<p>I appreciate any help you can provide. Thank you.</p>
I can't compile libtorch

rkudo: <p>I’m trying to do the a simple tutorial to display the tensor of libtorch. I can’t compile. Please help me.<br>
I get an error saying The CUDA compiler identification is unknown.<br>
I added the path to nvcc, but it didn’t work.</p>
<p>Errror</p>
<pre><code class="lang-auto">CMake Error at /home/rkudo/.pyenv-gpu/versions/miniconda3-4.7.12/envs/torch/share/cmake-3.22/Modules/CMakeTestCUDACompiler.cmake:56 (message):
  The CUDA compiler

    "/usr/local/cuda-11.6/bin/nvcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/rkudo/libtorch_test/build/CMakeFiles/CMakeTmp
    
    Run Build Command(s):/bin/gmake -f Makefile cmTC_e78a6/fast &amp;&amp; /bin/gmake  -f CMakeFiles/cmTC_e78a6.dir/build.make CMakeFiles/cmTC_e78a6.dir/build
    gmake[1]: Entering directory `/home/rkudo/libtorch_test/build/CMakeFiles/CMakeTmp'
    Building CUDA object CMakeFiles/cmTC_e78a6.dir/main.cu.o
    /usr/local/cuda-11.6/bin/nvcc      -c /home/rkudo/libtorch_test/build/CMakeFiles/CMakeTmp/main.cu -o CMakeFiles/cmTC_e78a6.dir/main.cu.o
    Linking CUDA executable cmTC_e78a6
    /home/rkudo/.pyenv-gpu/versions/miniconda3-4.7.12/envs/torch/bin/cmake -E cmake_link_script CMakeFiles/cmTC_e78a6.dir/link.txt --verbose=1
    "" CMakeFiles/cmTC_e78a6.dir/main.cu.o -o cmTC_e78a6 
    Error running link command: No such file or directory
    gmake[1]: *** [cmTC_e78a6] Error 2
    gmake[1]: Leaving directory `/home/rkudo/libtorch_test/build/CMakeFiles/CMakeTmp'
    gmake: *** [cmTC_e78a6/fast] Error 2
    
   
  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:7 (project)
</code></pre>
<p>CMakeLists.txt</p>
<pre><code class="lang-auto">set(CMAKE_CUDA_COMPILER /usr/local/cuda-11.6/bin/nvcc)
set(CMAKE_CUDA_COMPILER_VERSION "11.6")
cmake_minimum_required(VERSION 3.0 FATAL_ERROR)

# set(CMAKE_C_COMPILER "/opt/gcc/8.5.0/bin/gcc")
# set(CMAKE_CXX_COMPILER "/opt/gcc/8.5.0/bin/g++")
project(example-app LANGUAGES C CXX CUDA)

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)

# The following code block is suggested to be used on Windows.
# According to https://github.com/pytorch/pytorch/issues/25457,
# the DLLs need to be copied to avoid memory errors.
if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET example-app
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $&lt;TARGET_FILE_DIR:example-app&gt;)
endif (MSVC)

</code></pre>
Torchdata next(iter) datapipe?

Samuel_Lewis: <p>Hi,</p>
<p>I have been using the new torchdata datapipes recently. I only just noticed when I moved from a small subset of training data to training on the entire dataset each batch was taking a considerably longer time to load.</p>
<p>I did some testing and realised each time I call next(iter) on the datapipe it is iterating through the entire dataset before outputting one batch. This is extremely slow as it is streaming data and doing heavy operations on each item in the dataset.</p>
<p>I am therefor not actually sure on how next(iter) is working on iterdatapipes and would like some guidance on proper use.</p>
<p>Cheers</p>