akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Use MaskedTensor with Embedding layer

lopez: <p>Hi all,</p>
<p>I want to train a binary classifier that takes as input sequences of variable length. The sequences are padded when loading the batches using <code>torch.nn.utils.rnn.pad_sequence(inputs)</code> inside of a user-defined function <code>_collate_fn_padd</code>  used with <code>DataPipes</code>. I can create the data loading module with MaskedTensor successfully. The problem comes when defining the architecture and trying to do a forward pass.</p>
<p>My architecture is a transformer model that starts with an embedding layer. The problem is I can’t find a way to use a MaskedTensor object in this setup. The output of</p>
<pre><code class="lang-auto">model_args = dict(
    ntoken=len(_MOL_WEIGHTS) + 1 - 8,
    d_model=512,
    nhead=8,
    nlayers=2,
    d_hid=128,
    dropout=0.2,
)
model = TransformerModule(**model_args)
summary(model, example_masked[0]["sequence"], batch_size=-1)
</code></pre>
<p>is</p>
<pre><code class="lang-auto">.../env/lib/python3.10/site-packages/torch/masked/maskedtensor/core.py:299: UserWarning: embedding is not implemented in __torch_dispatch__ for MaskedTensor.
If you would like this operator to be supported, please file an issue for a feature request at https://github.com/pytorch/maskedtensor/issues with a minimal reproducible code snippet.
In the case that the semantics for the operator are not trivial, it would be appreciated to also include a proposal for the semantics.
</code></pre>
<p>Is there something I can do or I have to simply forget about MaskedTensor and use regular masks with buffer?</p>
<p>Any help is greatly appreciated,</p>
<p>Cheers.</p>
Getting build errors and more on getting detail on build

kamathhrishi: <p>Hello, I get this error when building, how do I get more detailed info on why its failing? Also guesses as to why it maybe failing.</p>
<pre><code class="lang-auto">[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/qu8-vaddc/gen/qu8-vaddc-minmax-avx512skx-mul32-ld128-x32.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512skx-vpshufb-x64.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512skx-vpshufb-x128.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512skx-vpshufb-x192.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512skx-vpshufb-x256.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512vbmi-vpermx2b-x64.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512vbmi-vpermx2b-x128.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512vbmi-vpermx2b-x192.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/x8-lut/gen/x8-lut-avx512vbmi-vpermx2b-x256.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2-k-over-64.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2-k-over-2048.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-4.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-8.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-16.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-64.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-2048.c.o
[ 73%] Building C object confu-deps/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/vlog.c.o
[ 73%] Built target microkernels-all
make: *** [all] Error 2
</code></pre>

ptrblck: <p>The actual error might be reported a bit further up in the logs, so check for the error message and post it here.</p>
Adaptive Resonance Theory (ART)

gjshannon: <p>I need to develop an Adaptive Resonance Theory (ART)-based neural network for research.</p>
<p>Is there interest in adding ART-based neural networks to PyTorch?</p>
<p>This is a family of networks that implement the adaptive resonance theories of cognition (Carpenter and Grossberg).  They avoid the “catastrophic loss” issue that exist with other network architectures, i.e., they do not have to be completely re-trained from scratch if a new pattern is uncovered but can add this pattern on-the-fly.</p>
Loading data from a bucket hosted by netapp instead of aws?

stwerner97: <p>Hi folks! <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"></p>
<p>I want to train a model on several TBs of data. Since the dataset is stored in a non-public NetApp cloud bucket, that is, a provider other than AWS S3, I am having issues to use the <a href="https://pytorch.org/data/main/generated/torchdata.datapipes.iter.S3FileLoader.html" rel="noopener nofollow ugc">S3FileLoader</a> as well as other utilities of pytorch data.</p>
<p>How can I access the data and authenticate my requests?</p>
<p>Thanks for the help!</p>
Reset values less than threshold to 0 inplace

vadimkantorov: <p>Hi!</p>
<p>Is it possible in vanilla eager PyTorch (without Inductor) to express the following inplace without allocating the output of <code>x &lt; 0.5</code>?:</p>
<pre><code class="lang-python">x = torch.empty(1000).uniform_(-10, 10)
y = torch.where(x &lt; -1.5, 0, x, out = x)
</code></pre>
<p>torch.clamp, torch.hardswish, torch.hardtanh don’t cut it in an easy way</p>

JuanFMontesinos: <pre><code class="lang-auto">import torch

x = torch.empty(1000).uniform_(-10, 10)
print(x.data_ptr())  

x += 1.5
print(x.data_ptr())  # Should be the same if operation was in-place

x.clamp_(0)
print(x.data_ptr())  

x -= 1.5
print(x.data_ptr())  

x=x+1.5
print(x.data_ptr())
</code></pre>
<p>Would this work?</p>
<pre><code class="lang-auto">94261703466240
94261703466240
94261703466240
94261703466240
94261703474752
</code></pre>

vadimkantorov: <p>I don’t think this is correct computation. E.g. value <code>-5</code> will be flushed to <code>-1.5</code> and not to 0 after <code>x -= 1.5</code></p>

JuanFMontesinos: <p>Oh, you are right. But right now I don’t think it’s possible. It’s a piecewise function and even if you don’t store the states, any math op needs to internally allocate some memory.<br>
So you need a vectorized operator that really does if x&lt;-1.5</p>
<p>Maybe you can achieve it with <a href="https://pytorch.org/functorch/stable/generated/functorch.vmap.html" class="inline-onebox" rel="noopener nofollow ugc">functorch.vmap — functorch nightly documentation</a><br>
Otherwise you can write a quick cuda kernel.</p>

KFrank: <p>Hi Vadim!</p>
<aside class="quote no-group" data-username="vadimkantorov" data-post="1" data-topic="184520" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vadimkantorov/48/365_2.png" class="avatar"> vadimkantorov:</div>
<blockquote>
<pre data-code-wrap="python"><code class="lang-python">y = torch.where(x &lt; -1.5, 0, x, out = x)
</code></pre>
</blockquote>
</aside>
<p>Use:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.nn.functional.threshold (x, -1.5, 0.0, inplace = True)
</code></pre>
<p>Best.</p>
<p>K. Frank</p>
Encounter "aten::resize is not implemented" runtime error during backward of loss

wongkenchiang97: <p>I’m using C++ frontend API to implement the training loop.</p>
<ul>
<li>“pred” are the the predictions of a 2D Unet model from a batch of training data.</li>
<li>“target” are the label of the training data.</li>
<li>“loss_f” is the torch::nn::BCEWithLogitsLoss().</li>
<li>“aten::resize is not implemented” runtime error appear at line 134.</li>
</ul>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/c/ec88eb8c63acf9eec7d328d6f36e91094b09c141.png" data-download-href="https://discuss.pytorch.org/uploads/default/ec88eb8c63acf9eec7d328d6f36e91094b09c141" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec88eb8c63acf9eec7d328d6f36e91094b09c141_2_690x421.png" alt="image" data-base62-sha1="xKu22TnlBqDY8RyqyWLLzOKyCAx" width="690" height="421" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec88eb8c63acf9eec7d328d6f36e91094b09c141_2_690x421.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec88eb8c63acf9eec7d328d6f36e91094b09c141_2_1035x631.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec88eb8c63acf9eec7d328d6f36e91094b09c141_2_1380x842.png 2x" data-dominant-color="242625"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1418×867 122 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Any idea to solve this issue?</p>
Any tips for speeding up the source build process?

kjp4155: <p>Hi, I would like to make some modification to the PyTorch source code for my research.</p>
<p>The problem is, it takes 15~20min to build the whole source code on my 32-core machine. This is annoying when I make small changes in the C++ code and see the effect.</p>
<p>I wonder how the other developers deal with such long build time.<br>
Are there any tips to minimize the build process?</p>

ptrblck: <p>After a full rebuild <code>python setup.py develop</code> should only build all changed files and modules depending on it. <a href="https://github.com/pytorch/pytorch/blob/main/CONTRIBUTING.md#c-development-tips">This section</a> of the contributing guide has more tips such as using <code>ccache</code> etc.</p>
Does nn.DataParallel allow multiple forward passes?

Entropy: <p>I have a use case where I need to run forward passes on two different tensors before calling a backwards pass. I could potentially concatenate the two tensors and run a single forwards pass but that would lead to information from both tensors mixing in the batchnorm layers — I would ideally like to avoid this behavior.</p>
<p>Would nn.DataParallel function correctly if I call it twice in a row? I will admit that my knowledge of both DP and DDP is very limited at the moment.</p>

ptrblck: <p>Yes, I believe <code>nn.DataParallel</code> allows this, but note that this module in in maintenance and won’t be developed any further.<br>
If you are using the recommended <code>DDP</code> approach, you might want to use <a href="https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html#torch.nn.parallel.DistributedDataParallel.no_sync"><code>no_sync</code></a>.</p>

Entropy: <p>Thank you for answering my previous question!</p>
AssertionError: daemonic processes are not allowed to have children while doing parallel processing inside the dataloader's __getitem__ method

Sourabh_Patil: <p>Hi! I need to do some image processing operations for let’s say n number of times on one image that we get in our dataset through <strong>getitem</strong> method. So to make it fast rather than using a for loop I am doing multiprocessing after I read the image. But while doing so I get the error mentioned above. I am using multiple workers. So, when I use multiple workers, I get this error but when I use a single worker, I do not get that error. So, is there any possible way to use multiple workers as well as do multiprocessing to do image processing in parallel? Thanks in advance.</p>
How to correctly use the "pad" function in TorchScript?

aoussou: <p>The following code works fine in regular Pytorch</p>
<pre><code>def pad_audio(self,tensor,n_max=50000):
    
    diff_pad = n_max - len(tensor)    
    
    tensor = F.pad(tensor,(int(diff_pad/2),diff_pad - int(diff_pad/2)),'constant',0)
</code></pre>
<p>But when using JIT script, I get the following error:</p>
<pre><code class="lang-auto">
_pad(Tensor input, int[] pad, str mode="constant", float value=0.) -&gt; (Tensor):
Expected a value of type 'List[int]' for argument 'pad' but instead found type 'Tuple[int, Tensor]'.
:
        diff_pad = n_max - len(tensor)    
        
        tensor = F.pad(tensor,(int(diff_pad/2),diff_pad - int(diff_pad/2)),'constant',0)
                 ~~~~~ &lt;--- HERE
</code></pre>
<p>What is the correct way of using pad in TorchScript?</p>

ptrblck: <p>Make sure to use <code>List[int]</code> as described in the error message. Afterwards, you would run into another type mismatch, since the <code>value</code> is expected as a <code>float</code> and lastly you would have to annotate the inputs since <code>n_max</code> would be expected as a <code>Tensor</code> otherwise.<br>
This should work:</p>
<pre><code class="lang-python">@torch.jit.script
def pad_audio(tensor, n_max=50000):
    # type: (Tensor, int)  -&gt; Tensor
    diff_pad = int(n_max - len(tensor))
    tensor = F.pad(tensor, [int(diff_pad/2), diff_pad - int(diff_pad/2)], 'constant', 0.)
    return tensor
</code></pre>

lam_vu_Nguyen: <p>all parameters which are taken to F.pad() should have type integer, you can try this:</p>
<pre><code class="lang-auto">tensor = F.pad(tensor, [int(diff_pad/2), int(diff_pad) - int(diff_pad/2)], 'constant', 0.)
</code></pre>
Efficient slicing and stacking of tensors

jwb95: <p>Hello,<br>
I would like to write some type of 2d-convolution-layer with the possibility to store intermediate tensors.</p>
<p>Precisely, I would like to store slices of the input-tensor to the layer in a tensor with shape (batch, slices, ch_in, k, k) such that one could multiply it pointwise with a filters-tensor.</p>
<p>The snippet down below yields what I need but due to the forloops is intractable.<br>
How would it be possible to retrieve the described large tensor in an efficient way?</p>
<blockquote>
<p>ch_in = 3<br>
bs = 4<br>
k = 3<br>
h, w = 512, 512</p>
<p>x = torch.randn(bs, ch_in, h, w, requires_grad=True) # input tensor</p>
</blockquote>
<h6><a name="define-the-start-and-end-indices-of-each-3x3-slice-along-height-and-width-axis-1" class="anchor" href="#define-the-start-and-end-indices-of-each-3x3-slice-along-height-and-width-axis-1"></a>Define the start and end-indices of each 3x3-slice along height- and width-axis.</h6>
<blockquote>
<p>s_x = torch.arange(x.shape[2] - k + 1)<br>
e_x = s_x + k<br>
s_y = torch.arange(x.shape[3] - k + 1)<br>
e_y = s_y + k</p>
</blockquote>
<h6><a name="iterate-over-these-indices-and-append-each-slice-retrieved-this-way-to-an-empty-list-2" class="anchor" href="#iterate-over-these-indices-and-append-each-slice-retrieved-this-way-to-an-empty-list-2"></a>Iterate over these indices and append each slice retrieved this way to an empty list.</h6>
<blockquote>
<p>z = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
for s_x_, e_x_ in zip(s_x, e_x):<br>
for s_y_, e_y_ in zip(s_y, e_y):<br>
z.append(x[:, :, s_x_ : e_x_, s_y_ : e_y_])</p>
<p>z = torch.stack(z, dim=1)</p>
</blockquote>
<p>Any help would be appreciated! Thanks.</p>

ptrblck: <aside class="quote no-group" data-username="jwb95" data-post="1" data-topic="184491">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/e68b1a/48.png" class="avatar"> jwb95:</div>
<blockquote>
<p>The snippet down below yields what I need but due to the forloops is intractable.</p>
</blockquote>
</aside>
<p>That’s not the case as for loops don’t break the computation graph.<br>
For your use case you might want to consider using <code>unfold</code> to avoid the loops, but as already mentioned, the for loop should not break your code and the error might come from another place.</p>

jwb95: <p>Sorry, I did a wording mistake here.<br>
It’s not about breaking the computation graph but about the speed.<br>
Using loops over large tensors are not feasible for my case.</p>
<p>Anyways, unfold looks like what I need.</p>
<p>Thank you!</p>

ptrblck: <p>Ah, thanks for clarifying as I misunderstood the “intractable” wording.</p>
Fusion resnet3d

Maths_Electronics_Tu: <p>Advise please</p>
<p>I trained resnet50 (mmaction2) on rgb frames i got 90 val , optical flow 88 % , pretrained torchvision , when i fuse models with new head, tthe new model don’t improve well than each alone i got max 85%, i expected when concatenate features as space and temporal the model will be better</p>
<p>How. Can i improve it,</p>
Upsample module crashes on forward call

Andrej: <p>Hi,<br>
I was trying to use torch::nn::Upsample module in torchlib 1.12 (1.12.0+cu116). It crashes on the forward call.</p>
<p>The minimal reproducable example. I tested this on the cpu.</p>
<pre><code class="lang-auto">	torch::Tensor tensor = torch::zeros({ 1, 3, 512, 512 }); 
	std::vector&lt;double&gt; scl = {2.0};
	torch::nn::Upsample upsample_module(torch::nn::UpsampleOptions().scale_factor(scl).mode(torch::kNearest).align_corners(false));
	torch::Tensor output = upsample_module(tensor);
</code></pre>
<p>I am not sure if I am using it right way, however, the documentations says only</p>
<blockquote>
<p>Example:</p>
<pre><code class="lang-auto">Upsample model(UpsampleOptions().scale_factor({3}).mode(torch::kLinear).align_corners(false));
</code></pre>
</blockquote>
<p>Tested on:<br>
Windows 10<br>
torchlib 1.12<br>
Cuda 11.6<br>
RTX 3090</p>

Stijn_van_Esch: <p>I know it’s late, but perhaps useful for others as well.<br>
I implemented the upsample layer by:</p>
<pre><code class="lang-auto">torch::nn::Upsample(torch::nn::UpsampleOptions().scale_factor(std::vector&lt;double&gt;({2})).mode(torch::kBilinear))
</code></pre>
<p>Initially I had troubles with the input for .scale_factor(), which in the end needs to be a vector.<br>
Whereas in python it can just be a number like in the example you example provided.</p>
<p>Perhaps if you implement the upsample layer like the above code the issue will be fixed?</p>
Problems installing pytroch3D!

Devin_Bowler: <p>Okay so a few things, I am trying to work on this program which utilizes torch, cuda, and pytorch3d.</p>
<p>But no matter it seems what versions I download of Cuda toolkit and pytorch I can’t seem to install pytorch3d. Below I will show screenshots of current versions (CUDA 11.6, Python 3.10, Torch 1.13) of what I have running and the errors I am getting, but I am quite time sensitive to get this code running and I can’t seem to get the proper dependency running.</p>
<p>I have been using a few methods, trying to just run pip install pytorch or conda.</p>
<p>Also, I have tried cloning the repo and running the local install.</p>
<p>I am not familiar with conda environments or anaconda, not sure if that’s the approach but it’s just frustrating I can’t get this installed to continue. So any help is very much appreciated!!!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/9/b91f1f6bcc8ccf15c2457f2293cf584d8a753775.png" data-download-href="https://discuss.pytorch.org/uploads/default/b91f1f6bcc8ccf15c2457f2293cf584d8a753775" title="helpimg"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/9/b91f1f6bcc8ccf15c2457f2293cf584d8a753775_2_690x336.png" alt="helpimg" data-base62-sha1="qpF0KRWTeZ1OKu7BERbN2zsQu0Z" width="690" height="336" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/9/b91f1f6bcc8ccf15c2457f2293cf584d8a753775_2_690x336.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/9/b91f1f6bcc8ccf15c2457f2293cf584d8a753775_2_1035x504.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/9/b91f1f6bcc8ccf15c2457f2293cf584d8a753775_2_1380x672.png 2x" data-dominant-color="5A5C5E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">helpimg</span><span class="informations">1914×933 128 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>The <code>pip</code> and <code>conda</code> install commands fail, as binaries are not available for Windows as explained <a href="https://github.com/facebookresearch/pytorch3d/blob/main/INSTALL.md#installing-prebuilt-binaries-for-pytorch3d">here</a> in their <code>INSTALL.md</code>.<br>
The last error is raised as it seems <a href="https://github.com/facebookresearch/pytorch3d/blob/57f6e79280e78b6e8308f750e64d32984ddeaba4/pytorch3d/csrc/ball_query/ball_query.cu">this file</a> is missing in your seutp.</p>
Core bus dumped error when train wav2vec2 model

Yihong_Theis: <p>Hello, <a class="mention" href="/u/patrickvonplaten">@patrickvonplaten</a> <span class="mention">@valhalla</span><br>
I’m fine-tuning wav2vec model with <a href="https://huggingface.co/blog/fine-tune-xlsr-wav2vec2" rel="noopener nofollow ugc">Fine-Tune XLSR-Wav2Vec2 for low-resource ASR with <img src="https://discuss.pytorch.org/images/emoji/apple/hugs.png?v=12" title=":hugs:" class="emoji" alt=":hugs:" loading="lazy" width="20" height="20"> Transformers </a> by using Chinese common voice dataset at local machine with 4XA40 Nvidia. The memory is 46068MiB.<br>
I have some problems with training.<br>
I used the tutorial to train on Colab with another customer dataset is fine but not the Chinese common Voice. So I used it to train on a remote GPU (4 by A40) with docker setting.<br>
Every time when it reached to train the model. It always crashed by core bus dumped error. All of my GPU does not look like running at all. I don’t know why Colab can run but the local can’t run and I didn’t change any code. I tried my code on Juputer notebook and terminal. Both of them has core dumped error.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/9/398b4f20631cbf67ee12474ec7f8c1f0ad4fa5a9.png" data-download-href="https://discuss.pytorch.org/uploads/default/398b4f20631cbf67ee12474ec7f8c1f0ad4fa5a9" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/9/398b4f20631cbf67ee12474ec7f8c1f0ad4fa5a9.png" alt="image" data-base62-sha1="8d3GpUtNnqOAxVCartzRcSdRy9b" width="660" height="500" data-dominant-color="1A1A1A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">974×737 20.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/6/6666d39153e55138535a7ad175efda33198765cc.png" data-download-href="https://discuss.pytorch.org/uploads/default/6666d39153e55138535a7ad175efda33198765cc" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/6/6666d39153e55138535a7ad175efda33198765cc.png" alt="image" data-base62-sha1="eBT0ppe1scMGq9z18OwwhOtnZEg" width="690" height="177" data-dominant-color="18181A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1864×479 39.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Do I need add any code to let it access gpu or can the model run on multiple gpu?<br>
Thanks</p>
Trying to backward through the graph

mhr_azizi: <p>Hi,<br>
I tried to train an MLP model using the following code, but for the third data batch in the dataloader, I received this error msg:</p>
<p><strong>“Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad().”</strong></p>
<p>def train_nn(self, dataloader, epoch_number):<br>
model.train()<br>
for epoch in range(epoch_number):<br>
epoch_loss = 0<br>
for input, target in dataloader:<br>
Q_pred = model(input)<br>
loss = criterion(Q_pred, target)<br>
optimizer.zero_grad()<br>
loss.backward(retain_graph=True)<br>
optimizer.step()<br>
epoch_loss += loss<br>
mean_epoch_loss = epoch_loss / len(dataloader)<br>
print(f"[Epoch: {epoch+1:&gt;02}]  →  loss = {mean_epoch_loss:&gt;7.4f}")</p>
<p><strong>where the dataloader is defined in a separate function as follows:</strong></p>
<p>def create_dataloader(self):<br>
inputs, targets = self.create_dataset()<br>
n = targets.shape[0]<br>
inputs = torch.reshape(inputs, (n, state_size + input_number))<br>
targets = torch.reshape(targets, (n, 1))<br>
inputs = inputs.type(torch.float32)<br>
targets = targets.type(torch.float32)<br>
train_dataset = data.TensorDataset(inputs, targets)<br>
train_dataloader = data.DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True)<br>
self.dataloader = train_dataloader</p>
<p><strong>I also used “loss.backward(retain_graph=True)” instead, but it resulted in the following error msg:</strong></p>
<p>one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [128, 1]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).</p>
<p>can anyone help me how I can solve this issue.<br>
Thank you</p>

soulitzer: <p>You are probably trying to backprop through your data loader, make sure your tensors do not require grad when you manipulate them in your data loader.</p>
Execution Time Measurement in GPU

MayThet: <p>I would like to measure the execution time of my code using PyTorch on the GPU. I have tried using the following code, but I noticed that the execution time is similar to when I run the code on the CPU using <code>starttime = time.time()</code> . Could you please guide me if there are any other ways to measure the execution time accurately on the GPU? Thank you in advance.</p>
<blockquote>
<blockquote>
<p>start_event = torch.cuda.Event(enable_timing=True)<br>
end_event = torch.cuda.Event(enable_timing=True)</p>
</blockquote>
<pre><code>    start_event.record()
    imnoisy = imnoisy.to(device)
    nsigma = nsigma.to(device)
    outim = torch.clamp(
        imnoisy[:, :1, :, :] - model(imnoisy, nsigma).to(device), 0., 1.)
    end_event.record()
    torch.cuda.synchronize()
    execution_time = start_event.elapsed_time(end_event) / 1000  # Convert to seconds
    print(f"Execution time: {execution_time} seconds")
</code></pre>
</blockquote>