Subprocess terminated without any exception when passing the multidimensional tensor to the model

username: Fangyuan
cooked: <p>I set the model hidden layer a litter bigger(like 256), and use <code>load_state_dict</code> in the subprocess.<br>
When I give a multi-dimensional tensor to the model,  the process will be terminated without any exception.</p>
<p>I write a demo to reproduce the error.</p>
<pre><code class="lang-auto">import torch.multiprocessing as mp
import torch
import torch.nn as nn


class AC(nn.Module):
    def __init__(self, features_n, actions_n):
        super(AC, self).__init__()
        # if the hidden layer cells are lower, for example, 128, no error occurs.
        self.hidden_layer_cells = 256
        self.l1 = nn.Linear(features_n, self.hidden_layer_cells)
        self.l2 = nn.Linear(self.hidden_layer_cells, self.hidden_layer_cells)
        self.actor_linear = nn.Linear(self.hidden_layer_cells, actions_n)
        self.critic_linear = nn.Linear(self.hidden_layer_cells, 1)

    def forward(self, inputs):
        x = torch.tanh(self.l1(inputs))
        x = torch.tanh(self.l2(x))
        pi = self.actor_linear(x)
        q = self.critic_linear(x)
        return pi, q


class Worker2(mp.Process):
    def __init__(self) -&gt; None:
        super(Worker2, self).__init__()
        self.t = AC(10, 10)
        self.tt = AC(10, 10)
        # if I load state dict from exist model, it will be terminated when passing Multidimensional tensor
        self.tt.load_state_dict(self.t.state_dict())

    def run(self):
        while True:
            s = torch.ones(size=(1, 10))
            a = self.t(s)
            ss = torch.cat((s, s))
            # this line will terminate the process
            aa = self.t(ss)


w = Worker2()
w.start()
w.join()
</code></pre>

username: cbalioglu
cooked: <p>I am not able to reproduce your issue. Do you mind giving a bit more detail about your execution environment? Which version of PyTorch? Which operating system?</p>

username: Fangyuan
cooked: <p><a class="mention" href="/u/cbalioglu">@cbalioglu</a> I’m running it in a docker container.<br>
Operation system:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/0/8030e46bac324faa1cac5f2abffac6dbdfa9763d.png" data-download-href="https://discuss.pytorch.org/uploads/default/8030e46bac324faa1cac5f2abffac6dbdfa9763d" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/0/8030e46bac324faa1cac5f2abffac6dbdfa9763d.png" alt="image" data-base62-sha1="ii1RMm6FbwyxWk6Vq4w9f1Tto4Z" width="517" height="346" data-dominant-color="393834"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">860×576 38.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Pytorch version:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/a/6acb339cada2d0cd41e41bf6935b812e00b4f542.png" data-download-href="https://discuss.pytorch.org/uploads/default/6acb339cada2d0cd41e41bf6935b812e00b4f542" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/a/6acb339cada2d0cd41e41bf6935b812e00b4f542_2_517x136.png" alt="image" data-base62-sha1="feJY61fkT46wajVXsc51WSX5ixQ" width="517" height="136" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/a/6acb339cada2d0cd41e41bf6935b812e00b4f542_2_517x136.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/a/6acb339cada2d0cd41e41bf6935b812e00b4f542_2_775x204.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/a/6acb339cada2d0cd41e41bf6935b812e00b4f542_2_1034x272.png 2x" data-dominant-color="32312F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1682×444 32.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>When I run it on my MacBook, indeed, no problem occurs.</p>

username: cbalioglu
cooked: <p>I tested it on a Fedora machine and although it does not get terminated, I does get stuck on <code>aa = self.t(ss)</code> line.</p>
<p>The problem is that you are calling <code>self.tt.load_state_dict()</code> in the parent process and then use <code>self.tt</code> in the forked process. There are known issues in “implicitly” passing tensors via fork to child processes. If you move the logic of <code>__init__()</code> into <code>run()</code>, you will mitigate the issue.</p>
<p>Overall my advice would be to avoid forks in all circumstances. Make sure to spawn your child processes so that they have their own clean state. As you have already experienced forking can be very fragile and unpredictable (and historically it long precedes multi-threading and was never meant to be used with multithreaded processes).</p>