Why multiprocess behavior differ between v1.12 and v1.9

samson-wang: <p>When I use torch==1.9.0, the following code runs fine.</p>
<pre><code class="lang-auto">import torch
from multiprocessing import Process
import multiprocessing

def run():
    print('in proc', torch.cuda.is_initialized())
    print('in proc', torch.cuda._is_in_bad_fork())
    torch.zeros((5,)).cuda()

def fk_run():
    print('in fork proc', torch.cuda.is_initialized())
    print('in fork proc', torch.cuda._is_in_bad_fork())
    torch.zeros((5,)).cuda()

if __name__ == "__main__":
    print('in main', torch.cuda.is_initialized())
    print('in main', torch.cuda._is_in_bad_fork())

    sp_ctx = multiprocessing.get_context('spawn')
    a = sp_ctx.Process(target=run)
    a.start()

    fk_ctx = multiprocessing.get_context('fork')
    b = fk_ctx.Process(target=fk_run)
    b.start()
    a.join()
    b.join()
</code></pre>
<p>However, when run with torch=1.12 get a RuntimeError</p>
<pre><code class="lang-auto">RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
</code></pre>
<p>I check the source code of 1.9 and 1.12 and find no difference.</p><aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/v1.12.0/torch/csrc/cuda/Module.cpp#L42">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/v1.12.0/torch/csrc/cuda/Module.cpp#L42" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/v1.12.0/torch/csrc/cuda/Module.cpp#L42" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/v1.12.0/torch/csrc/cuda/Module.cpp#L42</a></h4>



    <pre class="onebox"><code class="lang-cpp">
      <ol class="start lines" start="32" style="counter-reset: li-counter 31 ;">
          <li>#ifndef WIN32</li>
          <li>#include &lt;pthread.h&gt;</li>
          <li>#endif</li>
          <li>
          </li>
<li>using namespace torch;</li>
          <li>
          </li>
<li>static bool in_bad_fork = false;  // True for children forked after cuda init</li>
          <li>
          </li>
<li>#ifndef WIN32</li>
          <li>// Called in the forked child if cuda has already been initialized</li>
          <li class="selected">static void forked_child() {</li>
          <li>  in_bad_fork = true;</li>
          <li>  torch::utils::set_run_yet_variable_to_false();</li>
          <li>}</li>
          <li>#endif</li>
          <li>
          </li>
<li>// Should be called before the first cuda call.</li>
          <li>// Note: This is distinct from initExtension because a stub cuda implementation</li>
          <li>// has some working functions (e.g. device_count) but cannot fully initialize.</li>
          <li>static void poison_fork() {</li>
          <li>#ifndef WIN32</li>
      </ol>
    </code></pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/v1.9.0/torch/csrc/cuda/Module.cpp#L37">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/v1.9.0/torch/csrc/cuda/Module.cpp#L37" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/v1.9.0/torch/csrc/cuda/Module.cpp#L37" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/v1.9.0/torch/csrc/cuda/Module.cpp#L37</a></h4>



    <pre class="onebox"><code class="lang-cpp">
      <ol class="start lines" start="27" style="counter-reset: li-counter 26 ;">
          <li>#include &lt;pthread.h&gt;</li>
          <li>#endif</li>
          <li>
          </li>
<li>using namespace torch;</li>
          <li>
          </li>
<li>THCState *state = nullptr;</li>
          <li>static bool in_bad_fork = false;  // True for children forked after cuda init</li>
          <li>
          </li>
<li>#ifndef WIN32</li>
          <li>// Called in the forked child if cuda has already been initialized</li>
          <li class="selected">static void forked_child() {</li>
          <li>  in_bad_fork = true;</li>
          <li>  torch::utils::set_run_yet_variable_to_false();</li>
          <li>  state = nullptr;</li>
          <li>}</li>
          <li>#endif</li>
          <li>
          </li>
<li>// Should be called before the first cuda call.</li>
          <li>// Note: This is distinct from initExtension because a stub cuda implementation</li>
          <li>// has some working functions (e.g. device_count) but cannot fully initialize.</li>
          <li>static void poison_fork() {</li>
      </ol>
    </code></pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Besides, the demo code doesn’t init CUDA in the main process, why prompts <code> Cannot re-initialize CUDA in forked subprocess</code> ?</p>

ptrblck: <p>In PyTorch 2.0.0 I’m seeing:</p>
<pre><code class="lang-python">in main False
in main False
in fork proc False
in fork proc False
in proc False
in proc False
</code></pre>

samson-wang: <p>Ok. Still confused by the v1.12.0’s RuntimeError.</p>

ptrblck: <p>I don’t know what might have caused this issue in this older version, but are you still seeing the same issue in <code>2.0.0</code> or a recent nightly?</p>

samson-wang: <p>No，2.0.0 works fine.</p>

ptrblck: <p>OK, great! Thanks for confirming.</p>