Copying weights from one net to another

username: rbrigden
cooked: <p>Is there a canonical method to copy weights from one network to another of identical structure?</p>

username: fmassa
cooked: <p>You can use <code>load_state_dict</code> and <code>state_dict</code> for that.</p>
<pre><code class="lang-auto">net1.load_state_dict(net2.state_dict())
</code></pre>
<p>You can also deep copy a model via <code>copy.deepcopy</code>.</p>

username: FuriouslyCurious
cooked: <p>How does deep copy / canonical copy differ from normal weights loading?</p>

username: fmassa
cooked: <p>it avoids you from having to do</p>
<pre><code class="lang-auto">model1 = Model()
model2 = Model()

model2.load_state_dict(model1.state_dict())
</code></pre>
<p>and instead you only do</p>
<pre><code class="lang-auto">model1 = Model()
model2 = copy.deepcopy(model1)
</code></pre>

username: shubhamjain0594
cooked: <p>As far as I have seen the code “load_state_dict copies only parameters and buffers”.</p>
<p>Does deepcopy also copies only _parameters and _buffers or the hooks as well?</p>

username: qq456cvb
cooked: <p>deep copy will recursively copy every member of an object, so it copies everything</p>

username: spnova12
cooked: <p>If using deepcopy in my experience, optimizer does not work …</p>

username: Navneet_M_Kumar
cooked: <p>I want to copy a part of the weight from one network to another.<br>
Using something like polyak averaging</p>
<p>Example:</p>
<p>weights_new = k*weights_old + (1-k)*weights_new</p>
<p>How can I do this?</p>

username: bhushans23
cooked: <p>Right.  How should we go about it then?<br>
deepcopying optimizer as well?</p>

username: D-X-Y
cooked: <p>Hi, have you found a effective way to do this thing?</p>

username: Navneet_M_Kumar
cooked: <p>t = polyak_constant<br>
target_dqn_model.conv1.weight.data = t*(dqn_model.conv1.weight.data) + <br>
(1-t)*(target_dqn_model.conv1.weight.data)</p>
<p>I am doing this for each layer. I believe there must be a better method but this works for now.</p>

username: Navneet_M_Kumar
cooked: <aside class="quote" data-post="1" data-topic="14199">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/navneet_m_kumar/48/744_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/copying-part-of-the-weights/14199">Copying part of the weights</a> <a class="badge-category__wrapper " href="/c/reinforcement-learning/6"><span data-category-id="6" data-drop-close="true" class="badge-category " title="A section to discuss RL implementations, research, problems"><span class="badge-category__name">reinforcement-learning</span></span></a>
  </div>
  <blockquote>
    I want to copy a part of the weight from one network to another. 
Using something like polyak averaging 
Example: 
weights_new = k*weights_old + (1-k)*weights_new 
This is required to implement DDPG. 
How can I do this?
  </blockquote>
</aside>


username: roee
cooked: <p>I think you need to reinitialize the optimizer using the new copied model and then you can copy the optimizer inner values from one to the other, a bit of a mess. I would probably stop at reinitializing the optimizer.</p>

username: bhushans23
cooked: <p>Yes. Here is way to do so - <a href="https://discuss.pytorch.org/t/does-deepcopying-optimizer-of-one-model-works-across-the-model-or-should-i-create-new-optimizer-every-time/14359/2?u=bhushans23">Does deepcopying optimizer of one model works across the model? or should I create new optimizer every time?</a></p>

username: roee
cooked: <p>Less messy than what I assumed, good to know! thx!</p>

username: Brando_Miranda
cooked: <p>does one know to to properly solve the polykia averaging issue mentioned? The solution mentioned doesn’t work I believe:</p>
<aside class="quote" data-post="1" data-topic="14199">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/navneet_m_kumar/48/744_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/copying-part-of-the-weights/14199">Copying part of the weights</a> <a class="badge-category__wrapper " href="/c/reinforcement-learning/6"><span data-category-id="6" data-drop-close="true" class="badge-category " title="A section to discuss RL implementations, research, problems"><span class="badge-category__name">reinforcement-learning</span></span></a>
  </div>
  <blockquote>
    I want to copy a part of the weight from one network to another. 
Using something like polyak averaging 
Example: 
weights_new = k*weights_old + (1-k)*weights_new 
This is required to implement DDPG. 
How can I do this?
  </blockquote>
</aside>


username: Navneet_M_Kumar
cooked: <pre><code class="lang-auto">def polyak_update(polyak_factor, target_network, network):
    for target_param, param in zip(target_network.parameters(), network.parameters()):
        target_param.data.copy_(polyak_factor*param.data + target_param.data*(1.0 - polyak_factor))
</code></pre>
<p>Source: <a href="https://github.com/navneet-nmk/pytorch-rl/blob/master/train_ddpg.py" rel="nofollow noopener">https://github.com/navneet-nmk/pytorch-rl/blob/master/train_ddpg.py</a></p>

username: Krishna_Garg
cooked: <p>load_state_dict worked for me.</p>
<p>copy.deepcopy failed for me with run time error “Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment”</p>