akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Optimizer parameters missing in Pytorch

username: han.liu
cooked: <p>I have 2 nets sharing one optimizer using different learning rate. Simple code shown as below:</p>
<pre><code class="lang-auto">optim = torch.optim.Adam([
    {'params': A.parameters(), 'lr': args.A},
    {'params': B.parameters(), 'lr': args.B}])
</code></pre>
<p>Is this right? I ask this because when I check parameters in optimizer (using code below), I found only 2 parameters.</p>
<pre><code class="lang-auto">for p in optim.param_groups:
    outputs = ''
    for k, v in p.items():
        if k is 'params':
            outputs += (k + ': ' + str(v[0].shape).ljust(30) + ' ')
        else:
            outputs += (k + ': ' + str(v).ljust(10) + ' ')
    print(outputs)
</code></pre>
<p>Only 2 parameters are printed:</p>
<pre><code class="lang-auto">params: torch.Size([16, 1, 80])        lr: 1e-05      betas: (0.9, 0.999) eps: 1e-08      weight_decay: 0          amsgrad: False
params: torch.Size([30, 10])           lr: 1e-05      betas: (0.9, 0.999) eps: 1e-08      weight_decay: 0          amsgrad: False
</code></pre>
<p>Actually, 2 nets have more than 100 parameters.  I thought all parameters will be printed. Why is this happening? Thank you!</p>

username: tom
cooked: <aside class="quote no-group" data-username="han.liu" data-post="1" data-topic="59352">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/han.liu/48/16578_2.png" class="avatar"> han.liu:</div>
<blockquote>
<p><code>outputs += (k + ': ' + str(v[0].shape).ljust(30) + ' ')</code></p>
</blockquote>
</aside>
<p>So at that point, <code>v</code> is the list of paramters in this group. You add the first parameter’s shape (<code>v[0].shape</code>). You would want to loop over ‘v’ to process all parameters.</p>
<p>Best regards</p>
<p>Thomas</p>