Get 1D tensor of all trainable parameters

lucac: <p>Hello!</p>
<p>In Torch, I could use the following command:</p>
<p>cnn_params, cnn_grad_params = cnn:getParameters()</p>
<p>to get a 1D tensor of all the trainable parameters of a given model (and corresponding gradients).<br>
Is it possible to do something like that in PyTorch so that cnn_params shares the same memory of the corresponding model? I should mention that I only care about the trainable parameters (i.e., weights and biases) and not their gradients.</p>
<p>Thanks a lot for any help!</p>

ptrblck: <p>You can get the parameters by calling <code>model.parameters()</code>. This will return a generator, so if you need a list just call <code>list(model.parameters())</code>.</p>

lucac: <p>Thanks for your answer ptrblck. My idea, if possible, would be to manipulate a 1D tensor with all the parameters so that I can avoid looping through the list and perform the same operation several times (e.g., replacing parameters above a certain value in each individual module) in order to speed up the algorithm.</p>
<p>Thanks again!</p>

justusschock: <pre><code class="lang-auto">torch.cat([param.view(-1) for param in model.parameters()])
</code></pre>
<p>Should work for you.</p>

lucac: <p>Thank you. I tried the solution that you suggested but it seems that this object does not share the same memory as the initial model.</p>
<p>Here is an example code:</p>
<pre><code class="lang-auto">net = nn.ModuleList()
net.append(nn.Conv2d(3, 8, 3))
net.append(nn.Conv2d(8, 8, 3))
net.append(nn.Conv2d(8, 1, 3)) 

unrolled = torch.cat([param.view(-1) for param in net.parameters()])
unrolled.fill_(3.14)
print(net[0].weight[0,0,0,0])
print(unrolled[0])
</code></pre>
<p>Outputs:<br>
tensor(0.0684, grad_fn=SelectBackward)<br>
tensor(3.1400, grad_fn=SelectBackward)</p>

lucac: <p>In case anyone else is interested, the closest solution I was able to find involves using<br>
<code>torch.nn.utils.parameters_to_vector()</code> to get the parameters vector and then calling <code>torch.nn.utils.vector_to_parameters()</code> when I’m done modifying it.</p>

ARGHYA_PAL: <p>May be this link will give a proper explanation: <a href="http://cat2.mit.edu/dh_upload/backup/transfer/source/pytorch/test/test_nn.py" rel="nofollow noopener">http://cat2.mit.edu/dh_upload/backup/transfer/source/pytorch/test/test_nn.py</a></p>

ARGHYA_PAL: <p>from torch.nn.utils import parameters_to_vector, vector_to_parameters</p>

ARGHYA_PAL: <p>Also, this link may be a help:<br>
</p><aside class="onebox githubpullrequest">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/pull/2795/files" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Pull Request">
    <svg width="60" height="60" class="github-icon" viewBox="0 0 12 16" aria-hidden="true"><path d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/pull/2795" target="_blank" rel="nofollow noopener">New features:  parameters to vector and vector to parameters</a>
    </h4>

    <div class="branches">
      <code>pytorch:master</code> ← <code>zuoxingdong:patch-5</code>
    </div>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2017-09-20" data-time="01:26:11" data-timezone="UTC">01:26AM - 20 Sep 17 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/zuoxingdong" target="_blank" rel="nofollow noopener">
          <img alt="zuoxingdong" src="https://avatars0.githubusercontent.com/u/18168681?v=4" class="onebox-avatar-inline" width="20" height="20">
          zuoxingdong
        </a>
      </div>

      <div class="lines" title="11 commits changed 3 files with 105 additions and 0 deletions">
        <a href="https://github.com/pytorch/pytorch/pull/2795/files" target="_blank" rel="nofollow noopener">
          <span class="added">+105</span>
          <span class="removed">-0</span>
        </a>
      </div>
    </div>

  </div>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>