Worker blocked in forward pass when parallelized

username: driou
cooked: <p>Hi everyone,</p>
<p>I’m facing a deadlock I can’t solve by myself.<br>
I’m trying to implement Reactor algorithm (<a href="https://openreview.net/forum?id=rkHVZWZAZ" rel="nofollow noopener">https://openreview.net/forum?id=rkHVZWZAZ</a>) to train an AI playing at French Tarot card game.<br>
When I tried to implement training worker parallelisation, things seemed to work well until I tried to increase the size of some hidden layer.<br>
Then worker seemed to block (like infinity loop) when passing through the first layer of the first forward pass (more precisely, the first matmul -&gt; checked with pudb debugger).</p>
<p>I tried a few things:</p>
<ul>
<li>When worker is called from the main process, everything is fine, whatever the layers size are</li>
<li>When worker exploration is performed through a separate thread (inside the secondary process, the main thread of that secondary process being the training), the exploration is ok, and the blocking occures into the training</li>
<li>At the contrary, if the exploration &amp; training are performed alternatively, the blocking occures into the first exploration step</li>
</ul>
<p>The multiprocess (and multithread) are performed with fork (linux environment), by subclassing multiprocess and multithread classes, and then by calling start method.<br>
Forward pass is only performed on local copy of the shared network (one copy for exploration, another for training).<br>
I verified copy process : local networks seem to match perfectly shared network.</p>
<p>I suspect some secondary process memory issue, but I have no other clue or direction to follow.<br>
The network dimension are pretty reasonable I think :</p>
<ul>
<li>one input linear layer 78 -&gt; n</li>
<li>two parallel recurrent layer (GRU) n,m -&gt; n (m = hidden state dimension)</li>
<li>three head with two linear layers each
<ul>
<li>actor layer m -&gt; n -&gt; 78</li>
<li>advantage layer  m -&gt; n -&gt; 78</li>
<li>value layer m -&gt; n -&gt; 1<br>
the degrees of freedom are n &amp; m, which where 80 initially. Blocking occures in input layer when n or m is greater than 110… (although m does not appear in input layer dimension …)<br>
training process is performed on 14 successive steps x batchs of 5 trajectories.</li>
</ul>
</li>
</ul>
<p>Does that kind of issue seems familiar to anyone ?</p>
<p>Thanks by advance</p>

username: mrshenli
cooked: <p>Hey <a class="mention" href="/u/driou">@driou</a></p>
<p>Are you using <code>torch.multiprocessing</code> only or does your code also used any feature from <code>torch.distributed</code> or <code>torch.nn.parallel</code>?</p>
<p>If it just <code>torch.multiprocessing</code>, it might relates to forking the processing. If the parent process used any CUDA-related feature, there will be a CUDA context on it which does not fork. Even if it is CPU only, the fork could have broken the OMP internal states see discussion here: <a href="https://github.com/pytorch/pytorch/issues/41197" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/41197</a></p>
<p>Can you check if using spawn from <code>torch.multiprocessing</code> solves the problem?</p>

username: driou
cooked: <p>Hey,</p>
<p>I’m indeed using torch.multiprocessing (and not torch.distributed nor torch.nn.parallel).<br>
I’m not using any CUDA-related feature : it’s CPU only (in fact I did not performed the full pytorch install with CUDA).<br>
I will try with spawn, but I choosed fork on purpose, because it was more practical with the ability to init process before starting it.</p>
<p>Before that, let’s have a look on that thread you send me: the symptom looks like very similar !</p>
<p>Thanks a lot for your help</p>