Is this required_grad unpropagation possible?

stepelu: <p>x.requires_grad<br>
| True<br>
x.is_leaf<br>
| True<br>
y = x * 2<br>
y.requires_grad<br>
| False<br>
x<br>
| tensor([[ 0.2089, -0.0671],<br>
|        [ 0.5331, -0.9865],<br>
|        [-0.4298,  0.7363],<br>
|        …,<br>
|        [ 0.2710, -0.1222],<br>
|        [-0.4925, -0.3631],<br>
|        [ 0.2648, -0.0363]], device=‘cuda:0’, requires_grad=True)</p>

ptrblck: <p>I don’t know if the posted code is what you are seeing right now or what you would like to achieve. In the latter case wrap <code>y = x * 2</code> into <code>with torch.no_grad()</code> and <code>y.requires_grad</code> will show <code>False</code>.</p>

stepelu: <p>Apologies, I made this post in a rush and it was unclear…<br>
This is the code I am seeing, and it is not wrapped in a torch.no_grad() context.<br>
Running the code itself works fine.<br>
In the actual use case I have:</p>
<ol>
<li>the usual training loop with loss.backward()</li>
<li>every x steps testing happens, some computations are carried out inside of a with torch.no_grad() scope</li>
<li>after this scope exits, the code above is called (not wrapped in anything) and I get the unexpected result</li>
</ol>
<p>I seem to have ruled out that 2. makes any difference, so I was wondering whether a prior backward pass (via tensor.backward() without any arguments, no retain_graph, …) could somehow result in the issue I am seeing.</p>
<p>And of course the code I posted is only “representative”, I guess I’ll have to reduce and see where exactly the issue pops up…</p>
<p>For reference, the real called code is a copy paste of:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/ngushchin/EntropicOTBenchmark/blob/main/notebooks/mixtures_benchmark_visualization_sb.ipynb">
  <header class="source">

      <a href="https://github.com/ngushchin/EntropicOTBenchmark/blob/main/notebooks/mixtures_benchmark_visualization_sb.ipynb" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/ngushchin/EntropicOTBenchmark/blob/main/notebooks/mixtures_benchmark_visualization_sb.ipynb" target="_blank" rel="noopener nofollow ugc">ngushchin/EntropicOTBenchmark/blob/main/notebooks/mixtures_benchmark_visualization_sb.ipynb</a></h4>


      <pre><code class="lang-ipynb">{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plot Benchmark processes from P to Q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "from torch.distributions.mixture_same_family import MixtureSameFamily\n",
    "from torch.distributions.multivariate_normal import MultivariateNormal\n",
    "from torch.distributions.categorical import Categorical\n",
</code></pre>



  This file has been truncated. <a href="https://github.com/ngushchin/EntropicOTBenchmark/blob/main/notebooks/mixtures_benchmark_visualization_sb.ipynb" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
up to cell [5], then calling get_drift(…) function results in log_p failng to have required_grad=True.</p>