Manually manipulating model gradients and updating parameters

arya47: <p>Currently i have two instances of the same model. modelA and modelB. with different optimizers</p>
<p>i have a training loop where a forward pass happens on each model and gradients are accumulated. (No optim.step() yet).</p>
<p>Now before doing the backward pass i want to sum up the gradients of each model<br>
modelA.gradients = modelA.gradient + modelB.gradients<br>
modelB.gradients = modelA.gradients + modelB.gradients</p>
<p>optimA.step()<br>
optimB.step()</p>
<p>How can i do this in pytorch??</p>
<p>Thanks</p>
<p>More details</p>
<pre><code class="lang-auto">    # Zero grads
    optim1.zero_grad()
    optim2.zero_grad()

    # Model 1
    for batch_idx, (X, y_true) in enumerate(dl1):
        X = X.to(device)
        y_true = y_true.to(device)
        y_pred = model_p1(X)
        loss1 = loss_1(y_pred, y_true)
        loss1.backward()

   # Model 2
    for batch_idx, (X, y_true) in enumerate(dl2):
        X = X.to(device)
        y_true = y_true.to(device)
        y_pred = model_p2(X)
        loss2 = loss_2(y_pred, y_true)
        loss2.backward()

   # Combine (SUM gradients)
    for pA, pB in zip(model_p1.parameters(), model_p2.parameters()):
        sum_grads = pA.grad + pB.grad
        pA.grad = sum_grads
        pB.grad = sum_grads.clone()

   # Update model parameters using summed gradients
    optim1.step()
    optim2.step()
</code></pre>
<p>This is what i currently have but it dosen’t seem to be working</p>
<pre><code class="lang-auto">for pA, pB in zip(model_p1.parameters(), model_p2.parameters()):
        sum_grads = pA.grad + pB.grad
        pA.grad = sum_grads
        pB.grad = sum_grads.clone()
</code></pre>
<p>I want to update parameters based on summed gradients.</p>
<p>Thanks</p>

ptrblck: <aside class="quote no-group" data-username="arya47" data-post="1" data-topic="154568">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arya47/48/41192_2.png" class="avatar"> arya47:</div>
<blockquote>
<p>This is what i currently have but it dosen’t seem to be working</p>
</blockquote>
</aside>
<p>What’s the issue you are seeing using this approach?</p>

arya47: <p>The model isn’t training. My training loss stays constant</p>

ptrblck: <p>Could you check (some) parameter values before the <code>optimizer.step()</code> is called and compare them to the values afterwards? Do you see any changes or are they static?<br>
In the latter case, are you sure you’ve passed these parameters to the optimizer?</p>

InnovArul: <aside class="quote no-group" data-username="arya47" data-post="1" data-topic="154568">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arya47/48/41192_2.png" class="avatar"> arya47:</div>
<blockquote>
<p>Currently i have two instances of the same model. modelA and modelB. with different optimizers</p>
</blockquote>
</aside>
<p>Are the models initialized with the same weights?<br>
Also, what is the motivation to follow this approach?</p>

arya47: <p>Models are initiallised with random weights, the motivation behind this is i’m using this for a federated learning environment, in this case datasets for models are subsets of a bigger dataset.</p>

InnovArul: <p>I might be completely wrong, as I have not worked in this domain. But I think you should initialize the models with the same weights. When the models are initialized differently/randomly, the order of filters/weights don’t align between them.<br>
Then, When the gradients of non-aligned weights are added at the end, the resulting summed gradients might just be the corrupted version of original gradients/noise and may not provide any useful signal to the weight update step.</p>

arya47: <p>Nah i believe we should get similar convergence to optimum however i can see that parameter values are changing and will try and train for a lot longer</p>