Questions about using shared models

fedetask: <p>I have some questions about using the <code>torch.multiprocessing</code> module. Let’s say I have a <code>torch.nn.Module</code> called <code>model</code> and I call <code>model.share_memory()</code> on it.</p>
<ol>
<li>What happens if two threads call the <code>forward()</code>, i.e. <code>model(input)</code> at the same time? Is it safe? Or should I use <code>Lock</code> mechanisms to be sure that <code>model</code> is not accessed at the same time by multiple threads?</li>
<li>Similarly, what happens if two or more threads have an <code>optimizer</code> working on <code>model.parameters()</code> and they call <code>optimizer.step()</code> at the same time?</li>
</ol>
<p>I ask these questions because I often see the <code>optimizer.step()</code> being called on shared models without lock mechanisms (i.e. in RL implementations of A3C or ACER) and I wonder if it is a safe thing to do.</p>

pritamdamania87: <aside class="quote no-group" data-username="fedetask" data-post="1" data-topic="104996">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fedetask/48/21636_2.png" class="avatar"> fedetask:</div>
<blockquote>
<ul>
<li>What happens if two threads call the <code>forward()</code>, i.e. <code>model(input)</code> at the same time? Is it safe? Or should I use <code>Lock</code> mechanisms to be sure that <code>model</code> is not accessed at the same time by multiple threads?</li>
</ul>
</blockquote>
</aside>
<p>This really depends on the implementation of your forward function. Typically a forward function doesn’t modify any state so it is safe to call forward from two different threads. However if your forward function is modifying some state for some reason, there might be a race here if you have two threads calling forward.</p>
<aside class="quote no-group" data-username="fedetask" data-post="1" data-topic="104996">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fedetask/48/21636_2.png" class="avatar"> fedetask:</div>
<blockquote>
<ol start="2">
<li>Similarly, what happens if two or more threads have an <code>optimizer</code> working on <code>model.parameters()</code> and they call <code>optimizer.step()</code> at the same time?</li>
</ol>
</blockquote>
</aside>
<p>In this case, the optimizers would step on each other and as you mention without any lock mechanisms there might be some inconsistency. However, many frameworks still do this without any lock mechanisms because they are leveraging <a href="https://arxiv.org/abs/1106.5730" rel="noopener nofollow ugc">HOGWILD!</a>, which is basically a paper that showed your training can converge even if you don’t have strict locking around your parameter updates as long as your parameter updates are sparse. You can refer to the paper for more details on why and how this works.</p>
<p>The PyTorch Hogwild example does something similar: <a href="https://pytorch.org/docs/stable/notes/multiprocessing.html#hogwild" class="inline-onebox" rel="noopener nofollow ugc">Multiprocessing best practices — PyTorch 2.1 documentation</a>.</p>