akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Validation part slow down as epoch progress

username: cbd
cooked: <p>As epoch progress validation part  slow down extremely. I run the code for 20 epoch. For validation i follow the usual method of <strong>“model.eval()”</strong> and <strong>" with torch.no_grad()"</strong>. I just calculating PSNR and SSIM in validation without calling backward().</p>
<p>After validation when i switch to <strong>“model.train()”</strong> part, execution speed is quite good and constant for all the epochs.</p>

username: ptrblck
cooked: <p>Are you using larger inputs during validation or have you changed anything else in the validation pass which would take more compute resources?</p>

username: cbd
cooked: <p>Switch to custom PSNR function solved the issue. Thanks.</p>
Error on install on Raspberry PI

username: ArshadIram
cooked: <p>I am following this  link to install pytorch on Raspberry PI. I am installing by wheel.</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://qengineering.eu/install-pytorch-on-raspberry-pi-4.html">
  <header class="source">
      <img src="https://qengineering.eu/install-pytorch-on-raspberry-pi-4.html/favicon.png?2019-3-17-1-637833980699058009" class="site-icon" width="" height="">

      <a href="https://qengineering.eu/install-pytorch-on-raspberry-pi-4.html" target="_blank" rel="noopener nofollow ugc">qengineering.eu</a>
  </header>

  <article class="onebox-body">
    <img src="https://qengineering.eu/favImage.png" class="thumbnail onebox-avatar" width="500" height="500">

<h3><a href="https://qengineering.eu/install-pytorch-on-raspberry-pi-4.html" target="_blank" rel="noopener nofollow ugc">Install PyTorch on Raspberry Pi 4 - Q-engineering</a></h3>

  <p>A thorough guide on how to install PyTorch on the Raspberry Pi 4</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<pre><code class="lang-auto">
Kindly, help me out in solving this issue.</code></pre>

username: tom
cooked: <p>I think just using pip install will work these days, see this tutorial: <a href="https://pytorch.org/tutorials/intermediate/realtime_rpi.html" class="inline-onebox" rel="noopener nofollow ugc">Real Time Inference on Raspberry Pi 4 (30 fps!) — PyTorch Tutorials 1.11.0+cu102 documentation</a></p>
<p>Best regards</p>
<p>Thomas</p>

username: ArshadIram
cooked: <p>I was getting the issue because my Pi os was 32 bit. This torch will work on OS bit-64.</p>
Mixing mixed-precision training and full precision inference in dataloader

username: mart_fire
cooked: <p>Hi, I have a problem.<br>
So far I was using mixed precision on GPU in my training loop with <code>amp.autocast</code> and everything went well.<br>
However I started using a Pytorch network in my dataloader (I use RetinaFace to extract landmarks and align my images). RetinaFace runs on full precision on CPU. But now I can’t use mixed-precision in my training loop anymore.<br>
I got the error <code>element 0 of tensors does not require grad and does not have a grad_fn</code> when I call <code>backward()</code> of my loss. But when I print the attribute <code>requires_grad</code> of my loss, it is set to True.<br>
I don’t get any error if I run my training loop without mixed precision.</p>

username: ptrblck
cooked: <p>Could you post a minimal, executable code snippet to reproduce the issue, please?</p>

username: mart_fire
cooked: <p>I tried to make an minimal example but I didn’t get the error.<br>
I resolved my error but I don’t know why.<br>
Instead of initializing the network RetinaFace (actually it is a class with contains the true neural network (a nn.Module) and loads it in its constructor ) in the constructor of my dataset, I set it to None and initialize it during the loading of the data (I check if it is equals to None). Because there are multiple workers it is initialized multiple times (I guess the network is erased multiple times because it’s an attribute of the Dataset) , from the printing it looks like it is initialized 2 times the number of workers (why 2 times ?).</p>
<p>So I don’t know if it was actually linked to mixed precision or due to different processing times.</p>