To ignore during Fine-tuing of a Pruned module in torch.nn.utils.prune

Miura-code: <p><strong>What I want to do</strong><br>
I am currently trying to Fine-Tuning after pruning a deep learning model using convolutional layers.<br>
I want to ignore pruned modules when retraining.</p>
<p><strong>What I did</strong></p>
<ul>
<li>I pruned a nn.Conv2d module with amount=1.0 using nn.utils.prune.random_structured(). Then retrain the model.</li>
</ul>
<pre><code class="lang-auto">""" 各演算候補の定義 """
class SimpleModel(nn.Module):
  def __init__(self, C_in, C_out, kernel_size, stride, padding, affine=True):
    super(SimpleModel, self).__init__()
    self.process = nn.Sequential(
      nn.ReLU(inplace=False),
      nn.Conv2d(C_in, C_out, kernel_size, stride=stride, padding=padding, bias=False),
      nn.Conv2d(C_out, C_out, kernel_size, stride=stride, padding=padding, bias=False),
      nn.BatchNorm2d(C_out, affine=affine)
    )
    self.classifier = nn.Linear(5*32*32, 10)

  def forward(self, x):
    x = self.process(x)
    x = self.classifier(x.view(x.size(0),-1))
    return x
  
model = SimpleModel(3, 5, 3, 1, 1).cuda()
optimizer = torch.optim.SGD(
      model.parameters(),
      lr=0.1,
      momentum=0.9,
      weight_decay=3e-4)
criterion = nn.CrossEntropyLoss().cuda()

for i, (name, module) in enumerate(model.named_modules()):
  if name == 'process.1':
    prune.random_structured(module, name='weight', amount=1.0, dim=1)
    # prune.remove(module, name='weight')

train_data = dataset.CIFAR10(root='../data', train=True, download=True, transform=transforms.ToTensor(),)
train_queue = torch.utils.data.DataLoader(train_data, batch_size=1)

for step, (data, target) in enumerate(train_queue):
    for i, (name, module) in enumerate(model.named_modules()):
    data, target = next(iter(train_queue))
    data = Variable(data).cuda()
    target = Variable(target).cuda()
    optimizer.zero_grad()

    logits = model(data.cuda())
    loss = criterion(logits, target)
    loss.backward()
    optimizer.step()

    
</code></pre>
<ul>
<li>I Observed <code>weight</code> and <code>weight_orig</code> during experimental retraining.<br>
Then, when retraining, <code>weight_orig</code> was updated and <code>weight</code> remained zero.<br>
At that time the output of that module is completely zero</li>
<li>When <code>nn.utils.prune.remove()</code> was executed, <code>weight_orig</code> was removed and only <code>weight</code> remained.<br>
Retraining confirmed that <code>weight</code> was updated.</li>
</ul>
<p><strong>Here is a question.</strong></p>
<ul>
<li>
<p>If <code>remove()</code> is not done, is forward propagation computed using <code>weight_orig</code> or <code>weight</code>? And why is <code>weight_orig</code> updated?<br>
In fact, ideally, <code>weight_orig</code> should not be updated either, since we want BP to ignore the pruned module when retraining. (Because if <code>weight_orig</code> is being updated, it means that <code>weight_orig</code> is being used for gradient calculation, which is far from ideal. )</p>
</li>
<li>
<p>If remove() is executed, how do I stop updating the weights of the pruned module?</p>
</li>
<li>
<p>After pruning, how can I ignore the pruned module and retrain it?</p>
</li>
</ul>