akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
AttributeError when using Dataset and DataLoader

Francesco_Lee: <p>Hi, I am trying to load the audio files using DataSet and Dataloader, however it shows the following error, it seems to link the iter of the dataset. Could you help me solve it?</p>
<pre><code class="lang-auto">import torch
import numpy as np
import os
import matplotlib.pyplot as plt
import torchaudio
from torch.utils.data import Dataset, DataLoader

</code></pre>
<p>class MyData(Dataset):<br>
def <strong>init</strong>(self, root_dir, transform):<br>
self.root_dir = root_dir<br>
self.list_files = os.listdir(self.root_dir)<br>
self.transform = transform</p>
<pre><code> def __getitem__(self,idx):
      wave_file = self.list_files[idx]
      data_path = os.path.join(self.roo_dir,wave_file)
      waveform, samp_freq = torchaudio.load(data_path)
      return waveform, samp_freq

 def __len__(self):
     return len(self.list_files)
</code></pre>
<pre><code class="lang-auto">
if __name__ == "__main__":
   dataset = MyData(root_dir='E:\SSR\AudioAugmentation\AudioFile', transform = None)
   loader = DataLoader(dataset, batch_size=1)

   for waveform,f in loader:
       print(waveform.size())
       break
</code></pre>
<p>The error shows below:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "E:/SSR/AudioAugmentation/dataset.py", line 28, in &lt;module&gt;
    for waveform,f in loader:
  File "C:\Users\.conda\envs\lib\site-packages\torch\utils\data\dataloader.py", line 521, in __next__
    data = self._next_data()
  File "C:\Users\.conda\envs\lib\site-packages\torch\utils\data\dataloader.py", line 561, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "C:\Users\.conda\envs\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\.conda\envs\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "E:/SSR/AudioAugmentation/dataset.py", line 17, in __getitem__
    data_path = os.path.join(self.roo_dir,wave_file)
  File "C:\Users\.conda\envs\lib\site-packages\torch\utils\data\dataset.py", line 83, in __getattr__
    raise AttributeError
AttributeError
</code></pre>
<p>Thanks!!!</p>

ptrblck: <p>You have a typo here:</p>
<pre><code class="lang-python">data_path = os.path.join(self.roo_dir,wave_file)
</code></pre>
<p>and would need to use <code>self.root_dir</code> to avoid the <code>AttributeError</code>.</p>
The difference between windows and Linux

Hikary: <p>One project works normally on windows. I migrated the project from windows to Linux server, but the project gradient exploded on Linux. The data set has been checked and is available. What’s the problem?</p>

AlphaBetaGamma96: <p>Have you checked the versions of PyTorch you used? (Among other packages). Are you using the same environment? Same initial weights? Fixed the pseudorandom seeds between versions to rule out any randomness?</p>

Hikary: <p>Only the CUDA versions of the two are different.Linux is11.7，windows is 11.1</p>

AlphaBetaGamma96: <p>Can you share a minimal reproducible example for your error?</p>

Hikary: <p>This may be difficult, but now pycharm reports an error on the data set, which is not reflected in the windows Runtime: “the number of dimensions for input and target should be the same.” I am trying to solve this problem</p>

AlphaBetaGamma96: <p>Check the shape of your loss function inputs, and also share some code and the full error message (including the stacktrace). I can’t help if you can’t show what the problem is.</p>
How we know that our estimated function is differentiable?

Mahdi_Amrollahi: <p>In deep neural networks, sometimes we have a complicated estimated function at the end including many multiplication and applying activation functions. How do we know that is differentiable or not? We just add layers and changing activation functions without knowing it.</p>

ptrblck: <p>If you are unsure if a specific operation is differentiable you could check if it’s output has a valid <code>.grad_fn</code> attribute.</p>

Mahdi_Amrollahi: <p>Thanks, but I still have problem.<br>
Suppose that we have a derivative loss function by degree=3( loss= x^3), so here we cannot find the min of the function with gradient descend.<br>
To express more, let’s suppose we are tuning a deep nn model to find the better model. We try to add layers, changing hyperparams, changing activation function, adding or removing some neurons and … without caring about  what the final function we are supposed to achieve. We do not know how the shape of the loss function will be. Is that convex or concave or none of them? So, how we can understand that we can find the min of the loss function with gradient descend?<br>
I appreciate.</p>

smth: <p>you could apply the function on toy (or artificially constructed) datasets with known solutions – and test if descent methods bring you to these solutions.<br>
If the function + some effort in tuning hyperparameters can find the known solutions on these carefully constructed datasets, you might have a better understanding of your function’s curvature and it’s feasibility to apply gradient methods.</p>

Mahdi_Amrollahi: <p>Most of the time our loss function is nonconvex, so we may have local minimum. But this is not a problem when we are working on high dimensional space. Because any parameter can relieve us from the local minimum</p>
Pybind dataloader pickle

Lin1: <p>Hello everyone,<br>
Suggested by @ <a href="https://discuss.pytorch.org/u/rvarm1">rvarm1</a>, I change the topic category (distribute–&gt;dataloader) and re-post it here. Followings is the original problem:</p>
<p>A c++ preprocessor class is implemented with pybind11. The c++ class is imported in a custominzed dataset(torch.utils.data.Dataset) and one of its funtion is called by <em>getitem</em>. Once I tried to use DDP for multi-GPU training, the errror “<strong>cannot pickle preprocessor object</strong>” occurs. It works correctly, if i use one gpu without DDP(num_worker&gt;2, num_batch&gt;2).</p>
<p>It seems ddp/dataloader need to pickle everything contained by the dataset class for sharing the dataloader between different processes (<a href="https://stackoverflow.com/questions/69797634/how-to-pickle-an-object-which-is-written-in-c-through-pybind11" rel="noopener nofollow ugc">link</a>).</p>
<p>I also checked the official pybind11 doc about pickling support<br>
But there is no cues about how to pickle a general c++ classes with pointers (fill numpy arrays).</p>
<p>Am I using ddp wrongly? or is there any solution can solve it?</p>
<p>Thanks,<br>
Lin</p>
CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

ZHUANG_XIONG: <p>When I use F.conv2d function, The above problem occurred.</p>
<pre><code class="lang-auto">(pt1.9.0-cuda11.1) &gt; $ python -m torch.utils.collect_env                                                                                                                                                     [±dev_PHRDV2 ●●]
Collecting environment information...
PyTorch version: 1.9.0+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.9 (64-bit runtime)
Python platform: Linux-5.15.0-43-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 470.141.03
cuDNN version: Probably one of the following:
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn.so.8.2.1
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.2.1
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.2.1
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.2.1
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.2.1
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.2.1
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.2.1
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.2
[pip3] torch==1.9.0+cu111
[pip3] torchaudio==0.9.0
[pip3] torchfile==0.1.0
[pip3] torchvision==0.10.0+cu111
[conda] numpy                     1.21.2                   pypi_0    pypi
[conda] torch                     1.9.0+cu111              pypi_0    pypi
[conda] torchaudio                0.9.0                    pypi_0    pypi
[conda] torchfile                 0.1.0                    pypi_0    pypi
[conda] torchvision               0.10.0+cu111             pypi_0    pypi

</code></pre>

ptrblck: <p>Could you update to the latest stable release with the latest CUDA runtime and check if you are still seeing the issue, please?</p>

ZHUANG_XIONG: <p>Thanks, This method solves the problem.</p>
[Pytorch Android] How can I predict with two inputs?

SungmanHong: <blockquote>
<p>Tensor outputTensor_head = mModule.forward((IValue.from(inputTensor),IValue.from(inputTensor))).toTensor();</p>
</blockquote>
<p>I just made my pytorch android model, that takes two tensors.<br>
But In java, How can define two inputs?? (with array?)</p>
<blockquote>
<p>error: ‘)’ expected<br>
Tensor outputTensor_head = mModule.forward((IValue.from(inputTensor),IValue.from(inputTensor))).toTensor();</p>
</blockquote>
<p>Above error is came out.<br>
Help me…T.T</p>

ptrblck: <p>Is this a double post from <a href="https://discuss.pytorch.org/t/pytorch-android-two-input-is-possible/158992">here</a> with a new syntax error or what’s the difference between these posts?</p>

SungmanHong: <p>That was python code (converting pytorch model to pytorch android model) qustion.<br>
Now I just got some idea that was bug of pytorch mobile, the swap opearation or concat operation<br>
can not be able to made a pytorch mobile model.<br>
After adding some action(like CNN or ReLU) at python model’s forward, the conversion is passed.</p>
<p>But I still remain qustion, How Can I predict with converted model with two tensor is JAVA?<br>
The example code only contains one tensor… So I added this question.</p>
Pytorch QAT quantisation slows down the training of ViT significantly (reposting the question)

puranjay_mishra: <p>Hey, I apologize for reposting this question but I need the answer to this for a submission and it’s still unanswered <img src="https://discuss.pytorch.org/images/emoji/apple/frowning_face.png?v=12" title=":frowning_face:" class="emoji" alt=":frowning_face:" loading="lazy" width="20" height="20"><br>
I am using pytorch’s QAT to compress ViT model. The model does shrink to a significant extent however the training slows down. A single without the QAT takes 149.9 secs and when trained with the same hyper-parameters (QAT on) takes 273 secs.<br>
This shouldn’t happen as the only thing that changed was model becoming quantised. The memory shrinked 3.89 x which was expected. Can someone help me figure out why this is happening?</p>
<p>link to the original thread - <a href="https://discuss.pytorch.org/t/pytorch-qat-quantisation-slows-down-the-training-of-vit-significantly/157490" class="inline-onebox">Pytorch QAT quantisation slows down the training of ViT significantly</a></p>

ddang: <p>why not just ask in the original thread? seems there’s still an active discussion in there</p>

puranjay_mishra: <p>Hey. Yes, the thread is active now. It did go cold turkey for a bit and I’m in a rush to figure this out <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12" title=":sweat_smile:" class="emoji" alt=":sweat_smile:" loading="lazy" width="20" height="20"></p>
[Pytorch Android] Two input is possible?

SungmanHong: <p>Hi, I have two input tensor like below.</p>
<pre><code class="lang-auto">    input_blob_size = (1, 3, 1, 540, 960)
    print(input_blob_size)
    input_tensor = torch.randn(input_blob_size)
    input_tensor2 = torch.randn(input_blob_size)
    traced_model = torch.jit.trace(model, (input_tensor, input_tensor2), strict=False)
    optimized_traced__model = optimize_for_mobile(traced_model)
</code></pre>
<p>but I got this error…</p>
<blockquote>
<pre><code>optimized_traced__model = optimize_for_mobile(traced_model)
</code></pre>
<p>File “/home/hsm/anaconda3/envs/reproducibleresearch/lib/python3.6/site-packages/torch/utils/mobile_optimizer.py”, line 65, in optimize_for_mobile<br>
preserved_methods_str)<br>
RuntimeError: Method ‘forward’ is not defined.</p>
</blockquote>
<p>I want to use two inputs, rather than one input.<br>
Can I do it?</p>
<p>This is my model snipet.</p>
<pre><code class="lang-auto">class ResNet50_forward2(torch.nn.Module):
    def __init__(self):
        super(ResNet50_forward2, self).__init__()
        self.batch = nn.Sequential(*list(models.mobilenet_v3_small(pretrained=True).children())[0])

    def forward(self, pixel, pixel2):
        raw_read = torch.cat((pixel, pixel2), 0)
        print("DL1", raw_read.shape) #batch x chan x fr x w x h
        return raw_read
</code></pre>
DPP training incompatibility with checkpoint and detach

Pengzhangzhi: <p>I am using pytorch ddp to train my model. Turns out if I use ddp, then I can not use checkpoint or detach gradient. The incompatibility is a big problem, because these techniques are important for my use.</p>
<p>My model consists of two part roughly, a language model for generate representation, where weights are detached, another part of the model is trained with gradients.</p>
<p>the code of the language model:</p>
<p>if exists(config.msa_bert.msa_bert_config.model_weight) and not config.msa_bert.skip_load_msa_bert:<br>
self.bert_model = load_pretrain(self.bert_model, config.msa_bert.msa_bert_config.model_weight)</p>
<p>if config.msa_bert.msa_bert_config.freeze:<br>
print(’    frezze pretrained msa transformer’)<br>
for param in self.bert_model.parameters():<br>
param.detach_()<br>
self.bert_model.eval()<br>
Note in the other part of my model, there are recycles with detach.</p>
<p>for i in range(n_recycle):<br>
msa_fea, pair_fea = self.feat_extractor(msa_fea, pair_fea)<br>
msa_fea, pair_fea = msa_fea.detach_(), pair_fea.detach_()<br>
When using ddp, I have to turn on the find_unused_parameters=True , otherwise a error would be raised: RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one.<br>
Seems like if you have a model with detached params, you have to turn on this.<br>
Here comes the problem, if I keep find_unused_parameters=True and enable checkpoint, an error would be raised because a variable is marked twice.<br>
I conjecture that during forward, those detached parameters are marked as ready because of find_unused_parameters=True , and somehow they are marked ready again and causes this error.</p>
<p>I am wondering in what cases a param would be marked as ready again?<br>
And, what does it means for a param to be marked as ready? I think it is something to do with the autograd and the gradient compute map.</p>
<p>I accidentally find a solution that turn off the recycle ( i.e., turn off detach) and checkpoint while keep find_unused_parameters=True , the ddp training works.<br>
However, the problem is I can not turn off them as they are important for the efficiency. Without checkpoint, the gpu memory would explode.</p>

rvarm1: <p>Responded on the issue: <a href="https://github.com/pytorch/pytorch/issues/83074" class="inline-onebox" rel="noopener nofollow ugc">DPP training incompatibility with checkpoint and detach · Issue #83074 · pytorch/pytorch · GitHub</a></p>
Perform Transformation on Images based on Target

hasnainnaeem: <p>I intend to perform transformations on each image of the dataset. The transformations require the class label (I’m generating adversarial examples). For this, the target label must be passed to the adversarial algorithm for transformation. Is there any way to achieve this?</p>
<pre><code class="lang-auto">transform = transforms.Compose(
                    [
                        transforms.Lambda(adversarial_algorithm(target_label)),
                        transforms.ToTensor(),
                    ]
                )
</code></pre>
<p>Some people may recommend extending the dataset class and performing transformations there, but I want to achieve this using transformation.</p>
Loading googlenet model

rxvine: <p>I’m testing different models over a dataset but I have an issue with googlenet not loading. I have the latest version of pytorch and all other models work.<br>
I try to load googlenet using model = torchvision.models.googlenet(pretrained=True)<br>
But I get the following error: AttributeError: module ‘torchvision.models’ has no attribute ‘googlenet’.<br>
Could someone help me figure out what I’m doing wrong? Thanks</p>

ptrblck: <p><code>torchvision</code> might be too old. Could you try to update it or build from source? It’s quite simple and is described <a href="https://github.com/pytorch/vision/blob/master/README.rst#installation" rel="nofollow noopener">here</a>.</p>

Yong_Sun: <p><a href="https://stackoverflow.com/questions/55762706/how-to-load-pretrained-googlenet-model-in-pytorch" rel="noopener nofollow ugc">https://stackoverflow.com/questions/55762706/how-to-load-pretrained-googlenet-model-in-pytorch</a></p>
<pre><code class="lang-auto">import torchvision
google_net = torchvision.models.inception_v3(pretrained=True)</code></pre>
Mobile optimized model raised error "required rank 4 tensor to use channels_last format"

Mr-G: <p>I am trying to export a custom model (VGG + LSTM) from pytorch to an Android phone. I am using pytorch 1.9 both on the laptop and the android device.</p>
<p>The exported model is optimized with this:</p>
<pre><code class="lang-auto">traced_model = torch.jit.trace(rnn_model, example_inputs=x_dummy)
traced_model = optimize_for_mobile(traced_model)
</code></pre>
<p>But when I run model.forward(…) on the android device I get the following error:</p>
<pre><code class="lang-auto">Pending exception com.facebook.jni.CppException: required rank 4 tensor to use channels_last format
</code></pre>
<p>Everything works fine without optimization. Any idea?</p>

Linbin: <p>Have you tested the optimized model in Python? <a class="mention" href="/u/kimishpatel">@kimishpatel</a> any idea?</p>

kimishpatel: <p>HOw did you generate input? Can you provide repro steps? Also was the model saved for mobile or you just used torch.jit.save?</p>

balazsmorv: <p>Hi! I am having the same issue using a model on iOS. I found <a href="https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html" rel="noopener nofollow ugc">this page</a> explaining the difference between the memory formats, and indicating a conversion method:</p>
<p><strong>x = x.to(memory_format=torch.channels_last)</strong></p>
<p>However, applying this to the model output does not help, as I am getting the same error:</p>
<p><strong>required rank 4 tensor to use channels_last format</strong><br>
<strong>debug_handle:-1</strong></p>
<p>Anybody encountered the same issue? I have no idea what am I doing wrong.</p>

ptrblck: <p>As the error message states, a 4D tensor is required while it seems your <code>x</code> tensor has less or more dimensions.</p>

balazsmorv: <p>Thank you! I totally misunderstood the error message.</p>

SungmanHong: <p>Did you solve the error? It seems very common on the ImageNet based CNN models when converted to Pytorch Mobile…</p>
TracerWarning: Converting a tensor to a Python integer

SungmanHong: <p>I want to convert python model to pytorch lite, but I got above error.</p>
<blockquote>
<p>TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can’t record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!</p>
</blockquote>
<p>My code is simple,</p>
<pre><code class="lang-auto">class CNN_forward(torch.nn.Module):
    def __init__(self):
        super(CNN_forward, self).__init__()
        self.cnn= nn.Sequential(*list(models.mobilenet_v3_small(pretrained=True).children())[0])

    def forward(self, pixel):
        raw_data = pixel.transpose(1, 2)
        pixel= torch.squeeze(pixel, 0)
        preproc_data = self.batch(pixel)
        return preproc_data
</code></pre>
<p>I have to change axis of 1 and 2 because my java code generate tensor as<br>
[1 x RGB x Num frames x H x W] so it can not pass CNN<br>
so i changed to [1 x Num Frames x RGB x H x W]<br>
In my case Num Frames is only 1</p>
<p>But They say some warnning.<br>
Shold I change my python model?</p>
<p>Also, After run on mobile, I got this error, as same as below</p>
<h1><a name="mobile-optimized-model-raised-error-required-rank-4-tensor-to-use-channels_last-formathttpsdiscusspytorchorgtmobile-optimized-model-raised-error-required-rank-4-tensor-to-use-channels-last-format132385-1" class="anchor" href="#mobile-optimized-model-raised-error-required-rank-4-tensor-to-use-channels_last-formathttpsdiscusspytorchorgtmobile-optimized-model-raised-error-required-rank-4-tensor-to-use-channels-last-format132385-1"></a><a href="https://discuss.pytorch.org/t/mobile-optimized-model-raised-error-required-rank-4-tensor-to-use-channels-last-format/132385">Mobile optimized model raised error “required rank 4 tensor to use channels_last format”</a></h1>
<p>What is this error comes from?<br>
There is no batch size if run on mobile?<br>
OR when num frames is one, there dimemsion is deleted on mobile?</p>
<p>Thx</p>
PyTorch Profiler Per Layer

yzhao062: <p>Hi there,</p>
<p>Just wonder the current PyTorch Profiler (1.8.1+) supports the per layer profiling other than the lower-level operator profiling?</p>
<p>I check the documentation (<a href="https://pytorch.org/docs/stable/profiler.html" class="inline-onebox" rel="noopener nofollow ugc">torch.profiler — PyTorch 1.8.1 documentation</a>) and looks like in no place it mentions per-layer operations.</p>
<p>Thanks!</p>

Yao_Xue: <p>I have the same question, did you find any answer yet?</p>
DDP for Generative Models / Sample Generation

leonepavone: <p>Hello,</p>
<p>I am trying to train a new type of generative network called a <a href="https://arxiv.org/abs/2106.04399" rel="noopener nofollow ugc">Generative Flow Network</a>.</p>
<p>Often, the most expensive part of training these networks is generating samples from the networks. I looked into DDP, but what I’d really like is to parallelize the training loop or <code>train_step</code> in pytorch lightning.</p>
<p>For instance, if we were to use DDP on this Deep Energy Model <a href="https://pytorch-lightning.readthedocs.io/en/latest/notebooks/course_UvA-DL/07-deep-energy-based-generative-models.html" rel="noopener nofollow ugc">here</a>, would the sampling that happens in the <code>training_step</code> also be parallelized?</p>
<p>I also looked into building a <code>DataLoader</code> that depends on the model/network, but it appears as though each worker would copy the model at step 0 and never update it (since there is no model parameter consistency protocol for those copies).</p>
<p>What is the most effective way to parallelize sample generation (i.e. RL trajectories) in generative models?</p>
How to fix: "AttributeError: 'numpy.ndarray' object has no attribute 'cpu'"

danielmanu93: <p>Hi All I have data from a dataloader which I get a data and corresponding label from it. I am trying to normalize the labels within a certain range using <code>T.tonumpy_denormalize</code> , however I get this error <code>AttributeError: 'numpy.ndarray' object has no attribute 'cpu'</code>. I am not sure how to clear or override this error. Below is my sample code:</p>
<pre><code class="lang-auto">val = "./data.txt"
val_data = torch.load(val)
dataloader = torch.utils.data.DataLoader(val_data, batch_size=1, shuffle=True)

def evaluate(model):
      label_min = 1500
      label_max = 4500
      data, label = iter(dataloader).next()
      label_np = T.tonumpy_denormalize(label, label_min, label_max, exp=False)   # I get the error on this line.

# full error message
label_np = T.tonumpy_denormalize(label, label_min, label_max, exp=False)
  File "/home/pi/Desktop/exp/fcnvmb/transforms.py", line 97, in tonumpy_denormalize
    vid = minmax_denormalize(vid.cpu().numpy(), vmin, vmax, scale)
AttributeError: 'numpy.ndarray' object has no attribute 'cpu'
</code></pre>

ptrblck: <p>It seems your method expects a PyTorch tensor as it wants to call <code>.cpu()</code> on it while a numpy array is passed, which does not support the <code>.cpu()</code> operation so check and fix your input.</p>
Please help me understand this section of a paper

animesh22: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/a/6a583b6f05233e8ba5fbb0d7ddac93d9fac72832.png" data-download-href="https://discuss.pytorch.org/uploads/default/6a583b6f05233e8ba5fbb0d7ddac93d9fac72832" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/a/6a583b6f05233e8ba5fbb0d7ddac93d9fac72832_2_456x500.png" alt="image" data-base62-sha1="faLEcsXeDaaPki7Y6IFXvStSn0S" width="456" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/a/6a583b6f05233e8ba5fbb0d7ddac93d9fac72832_2_456x500.png, https://discuss.pytorch.org/uploads/default/original/3X/6/a/6a583b6f05233e8ba5fbb0d7ddac93d9fac72832.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/a/6a583b6f05233e8ba5fbb0d7ddac93d9fac72832.png 2x" data-dominant-color="ECEBEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">467×512 77.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Image dimension is h, w, 3<br>
Optical flow generates h, w, 2<br>
How have they been added in equation 1?</p>
Torchscript - saving and loading modules makes different outputs than scripting it directly

may1: <p>Hello,<br>
I thought scripting a module with same precision (fp32) would make the exact same output as the original eager module.<br>
It did when I converted directly but when I stored scripted modules using torch.jit.save() and then loaded again using torch.jit.load(), the output of loaded scripted modules became slightly different.</p>
<p>Does torch.jit.save() and load() change some operations?</p>

ptrblck: <p>Scripting a model tries to optimize the model e.g. by fusing different operations which would then create the expected small numerical errors caused by the limited floating point precision.</p>
Why my self defined layer does not enter backward?

Ziyu_Huang: <pre><code class="lang-auto">class layer(torch.autograd.Function):
    @staticmethod
    def forward(ctx, input,factors, bias=None):
        ctx.save_for_backward(input, factors, bias)
        print('fwd')
        output = .....
        return output

    @staticmethod
    def backward(ctx, grad_output):
        # input, weight, bias = ctx.input, ctx.weight, ctx.bias
        grad_input = grad_weight = grad_bias = None
        print('enter here!')  # &lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;-------------------This does not printed!!!!
        grads, dx = bwd(factors, grad_output, input)  
        if ctx.needs_input_grad[0]:
            grad_input = dx
        if ctx.needs_input_grad[1]:
            grad_weight = grads
        if bias is not None and ctx.needs_input_grad[2]:
            grad_bias = grad_output.sum(0)
        return grad_input, grad_weight, grad_bias
</code></pre>
<p>As shown above, the print does not work!!! But the forward’s print can work!!! Why???<br>
Thank you!!!</p>

ptrblck: <p>Your code works for me:</p>
<pre><code class="lang-python">class layer(torch.autograd.Function):
    @staticmethod
    def forward(ctx, input,factors, bias=None):
        ctx.save_for_backward(input, factors, bias)
        print('fwd')
        output = input.clone()
        return output

    @staticmethod
    def backward(ctx, grad_output):
        # input, weight, bias = ctx.input, ctx.weight, ctx.bias
        grad_input = grad_weight = grad_bias = None
        print('enter here!')  # &lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;-------------------This does not printed!!!!
        return grad_input, grad_weight, grad_bias
    
f = layer.apply

x = torch.randn(1, 1, requires_grad=True)
out = f(x, torch.randn(1, 1)) # prints: fwd
out.mean().backward() # prints: enter here!
</code></pre>
Performance issue with pytorch-v1.12.0

wuziyou199217: <p>Hello,</p>
<p>Recently I’ve tried to install Pytorch from source with CUDA11.7.0 and perform a Bert training.<br>
I found that with torch-v1.11.0, the performance could attain 450 examples/sec, which outspeeds the one trained with torch-v1.12.0, 150 examples/sec.</p>
<p>Below is installation command:<br>
<code>CFLAGS="-g0 -fno-gnu-unique" USE_CUPTI_SO=1 USE_KINETO=1 CMAKE_PREFIX_PATH="$(dirname $(which conda))/../" MAX_JOBS=80 USE_SYSTEM_NCCL=1 CUDA_HOME=/usr/local/cuda python setup.py install</code></p>
<p>Is there anyone who could help me solving this issue?</p>

ptrblck: <p>Which GPU are you using? If it’s from the Ampere family, you might want to re-enable TF32 via <code>torch.backends.cuda.matmul.allow_tf32 = True</code> or via <code>torch.set_float32_matmul_precision</code>.</p>

wuziyou199217: <p>Thanks a lot, problem solved.</p>
<p>But <code>torch.set_float32_matmul_precision</code> is not working.</p>

wuziyou199217: <p>BTW, Is there any reason that in the latest Pytorch version it disables TF32 by default?</p>

ptrblck: <p>Yes, you can read more about it <a href="https://dev-discuss.pytorch.org/t/pytorch-and-tensorfloat32/504">here</a>.</p>
Cuda 11.7 with pytorch command including "cuda toolkit 11.6"

Nicos: <p>Hello, I’ve installed pytorch with the following command</p>
<p>pip3 install torch torchvision torchaudio --extra-index-url <a href="https://download.pytorch.org/whl/cu116" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu116</a></p>
<p>However, I’ve installed (from the nvidia website) the nvida CUDA 11.7 toolkit.</p>
<p>I’ve run these commands:</p>
<p>print(torch.cuda.is_available())<br>
print(torch.cuda.device_count())<br>
print(torch.cuda.current_device())<br>
print(torch.cuda.device(0))<br>
print(torch.cuda.get_device_name(0))</p>
<p>and they show that my GPU was detected.</p>
<p>So, I assume there are no problems with the fact that my installed version of CUDA is 11.7, whereas I’ve used toolkit 11.6 in the pip command to install pytorch.<br>
Am I right?</p>
<p>Thanks in advance for the help.</p>

ptrblck: <aside class="quote no-group" data-username="Nicos" data-post="1" data-topic="158928">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/7feea3/48.png" class="avatar"> Nicos:</div>
<blockquote>
<p>However, I’ve installed (from the nvidia website) the nvida CUDA 11.7 toolkit.</p>
</blockquote>
</aside>
<p>Your local CUDA toolkit won’t be used unless you build PyTorch from source or custom CUDA extensions, as the pre-built PyTorch binaries ship with their own CUDA runtime (11.6 in your case).</p>
<p>Yes, there won’t be any issue as only your NVIDIA driver will be used if you just run PyTorch scripts. If you want to build custom extensions, I would recommend to install a matching CUDA toolkit.</p>