Combining two models

TinfoilHat0: <p>Say I have 2 independently trained models (with identical architecture) with parameters <strong>params1</strong> and <strong>params2.</strong> I’d like to find out if there exists real values <strong>w1</strong> and <strong>w2</strong> s.t. the model with parameters <strong>(w1 x params1</strong> + <strong>w2 x params2) / 2</strong> performs well on some validation set.</p>
<p>To test this, I’ve written the following piece of code.</p>
<pre><code class="lang-auto">w = torch.ones(2, 1, requires_grad=True, device=args.device)
optimizer = optim.SGD([w], lr=1e-1, momentum=0.0)

for rnd in tqdm(range(1, args.epochs+1)):
    model.train()
    val_loss, val_acc = 0.0, 0.0 
    for _, (inputs, labels) in enumerate(val_loader):
        # pass inputs to device, clear gradients
        inputs, labels = inputs.to(device=args.device, non_blocking=True),\
                        labels.to(device=args.device, non_blocking=True)
        optimizer.zero_grad()
        
        # compute aggregated params
        aggr_params = (w[0]*model_1_params + w[1]*model_2_params) / 2
        # Load params to the model
        vector_to_parameters(aggr_params, model.parameters())
        # forward-backward pass 
        outputs = model(inputs)
        minibatch_loss = criterion(outputs, labels)
        minibatch_loss.backward()
        
        optimizer.step()
        
        # keep track of round loss/accuracy
        val_loss += minibatch_loss.item()*outputs.shape[0]
        _, pred_labels = torch.max(outputs, 1)
        val_acc += torch.sum(torch.eq(pred_labels.view(-1), labels)).item()
        
    # inference after epoch
    print(w)
    val_loss, val_acc = val_loss/len(val_dataset), val_acc/len(val_dataset)       
    print(f'| Valid Loss:{val_loss:.3f}|', end='--')
    print(f'| Valid Acc: {val_acc:.3f}|', end='\r')
</code></pre>
<p>However, this does not update weights <strong>w</strong>  at all.  I suspect the line</p>
<blockquote>
<p>vector_to_parameters(aggr_params, model.parameters())</p>
</blockquote>
<p>might be breaking autograd. However, I’m not sure.</p>

albanD: <p>Hi,</p>
<p>Can you share your <code>vector_to_parameters</code> function please?</p>
<p>What most likely happens is that you put that aggr_params back into an <code>nn.Parameter</code> which cannot have history and so it breaks the gradient graph.<br>
To fix this, the simplest solution is to delete the <code>nn.Parameter</code> (this is not a parameter anymore as you don’t want to learn it) and replace it with a regular Tensor containing the value you want.</p>

TinfoilHat0: <p>Hi,</p>
<p><code>vector_to_parameters</code> is not my own implementation. It is a PyTorch function.</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/convert_parameters.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/convert_parameters.py" target="_blank" rel="nofollow noopener">pytorch/pytorch/blob/master/torch/nn/utils/convert_parameters.py</a></h4>
<pre><code class="lang-py">import torch


def parameters_to_vector(parameters):
    r"""Convert parameters to one vector

    Arguments:
        parameters (Iterable[Tensor]): an iterator of Tensors that are the
            parameters of a model.

    Returns:
        The parameters represented by a single vector
    """
    # Flag for the device where the parameter is located
    param_device = None

    vec = []
    for param in parameters:
        # Ensure the parameters are located in the same device
        param_device = _check_param_device(param, param_device)
</code></pre>

  This file has been truncated. <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/convert_parameters.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


albanD: <p>Wow, I didn’t knew we had that. That is a very very very very dangerous function <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:">  We need to fix that.</p>
<p>The issue is slightly different here then: it uses <code>.data</code> (that should never be used!) and so break the computational graph. I’m afraid you won’t be able to use this function if you want gradient flowing back throw this update.</p>

TinfoilHat0: <p>I see, that’s ok I guess <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> In that case, how can I load the aggregated parameters to the model ?</p>

albanD: <p>It’s not going to be very clean. But the following should work:</p>
<pre><code class="lang-python">import torch
from torch import nn


model = nn.Sequential(nn.Linear(2, 2))
# Save the location of the parameters now
# Since we delete them later, we won't be able to call this function anymore
model_params = list(model.named_parameters())

agreg = torch.rand(6, requires_grad=True)

def get_last_module(model, indices):
    mod = model
    for idx in indices:
        mod = getattr(mod, idx)
    return mod

def replace_weights(agreg, model, model_params):
    pointer = 0
    for name, p in model_params:
        indices = name.split(".")
        mod = get_last_module(model, indices[:-1])
        p_name = indices[-1]
        if isinstance(p, nn.Parameter):
            # We can override Tensors just fine, only nn.Parameters have custom logic
            delattr(mod, p_name)

        num_param = p.numel()
        setattr(mod, p_name, agreg[pointer:pointer + num_param].view_as(p))
        pointer += num_param


print(model)

print(model[0].weight)
replace_weights(agreg, model, model_params)
print(model[0].weight)

agreg = agreg * 10
replace_weights(agreg, model, model_params)
print(model[0].weight)
</code></pre>

TinfoilHat0: <aside class="quote no-group" data-username="albanD" data-post="6" data-topic="77716">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>eplace_weights(agreg, model, model_params)</p>
</blockquote>
</aside>
<p>Perfect, works like a charm. Thank you so much for your time.</p>