Practical usage of nn.Variable() and nn.Parameter()

Federico_Ottomano: <p>I’m trying to figure out the difference and the <strong>practical</strong> usage one could make of <code>nn.Parameter()</code> and <code>nn.Variable()</code>.</p>
<p>For now, I’ve only got some experience in using <code>nn.Embedding()</code> which provides embeddings of specified dimension for labels/words in a dictionary.</p>
<p>But how <code>nn.Parameter()</code> and<code> nn.Variable()</code> can be used in <em>practice</em>? Could anyone provide me some use-case example to improve my understanding?</p>

ptrblck: <p><code>Variable</code>s are deprecated since PyTorch 0.4.0 (so for 4 years now <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> ).<br>
<code>nn.Parameter</code>s wrap tensors and are trainable. They are initialized in <code>nn.Module</code>s and trained afterwards.<br>
If you are writing a custom module, this would be an example how <code>nn.Parameter</code> is used:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.param = nn.Parameter(torch.randn(1, 1))
        
    def forward(self, x):
        x = x * self.param
        return x

model = MyModel()
print(dict(model.named_parameters()))
# {'param': Parameter containing:
# tensor([[0.6077]], requires_grad=True)}

out = model(torch.randn(1, 1))
loss = out.mean()
loss.backward()

print(model.param.grad)
# tensor([[-1.3033]])
</code></pre>

Federico_Ottomano: <p>many thanks!</p>
<p>So, if I find code where <code>Variable</code> is used, like this one I’m currently looking at <a href="https://locuslab.github.io/qpth/" rel="noopener nofollow ugc">locuslab/qpth: A fast and differentiable QP solver for PyTorch</a>, how should I consider/modify it?<br>
Have they been absorbed by <code>nn.Parameter()</code> ?</p>

KFrank: <p>Hi Frederico!</p>
<aside class="quote no-group" data-username="Federico_Ottomano" data-post="3" data-topic="148644" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/federico_ottomano/48/43102_2.png" class="avatar"> Federico_Ottomano:</div>
<blockquote>
<p>Have they been absorbed by <code>nn.Parameter()</code> ?</p>
</blockquote>
</aside>
<p>At the most basic technical level, <code>Variable</code> has been absorbed by<br>
<code>Tensor</code>.</p>
<p>In the old days (pre-deprecation), a <code>Variable</code> wrapped a <code>Tensor</code>,<br>
adding the structure (such as the <code>requires_grad</code> property) necessary<br>
for the <code>Tensor</code> to participate in autograd.  Now <code>Tensor</code> has that<br>
structure build in directly.  (The cost in overhead is negligible if you<br>
don’t use that structure.)</p>
<p>So you can use autograd to train a <code>Tensor</code> without further ado.</p>
<p><code>Parameter</code> wraps <code>Tensor</code> to help <code>Module</code>s and <code>Optimizer</code>s keep<br>
track of what <code>Tensor</code>s you want to train.  If you have a trainable<br>
<code>Tensor</code> in a <code>Module</code> you will typically want to wrap it in a <code>Parameter</code><br>
so that, for example, it will show up automatically when you call<br>
<code>my_module.parameters()</code>, a useful convenience.  (But, again, it<br>
doesn’t need to be a <code>Parameter</code> to work with autograd.)</p>
<p>Best.</p>
<p>K. Frank</p>