ValueError: optimizer got an empty parameter list when using quantize_fx

Taghreed: <p>I want to do Federated learning Quantization using PyTorch. I followed some tutorial for centralized learning and tried to apply torch.quantization.quantize_fx to federated learning, but I got this error:<br>
ValueError: optimizer got an empty parameter list</p>
<p>Here is the code:<br>
import torch.quantization.quantize_fx as quantize_fx<br>
import copy</p>
<p>quantized_models_all=[ ]  # quantize all clients models<br>
for i in range(3):<br>
model= client_models[i]<br>
model.train()<br>
model_to_quantize = copy.deepcopy(model)<br>
qconfig_dict = {"": torch.quantization.get_default_qat_qconfig(‘qnnpack’)}<br>
model_prepared = quantize_fx.prepare_qat_fx(model_to_quantize, qconfig_dict)<br>
model_quantized = quantize_fx.convert_fx(model_prepared)<br>
quantized_models_all.append(model_quantized)</p>
<h1>
<a name="quantize-global-model-1" class="anchor" href="#quantize-global-model-1"></a>quantize global model</h1>
<p>model_fp = global_model</p>
<h1>
<a name="quantization-aware-training-for-static-quantization-2" class="anchor" href="#quantization-aware-training-for-static-quantization-2"></a>quantization aware training for static quantization</h1>
<p>model_to_quantize = copy.deepcopy(model_fp)<br>
qconfig_dict = {"": torch.quantization.get_default_qat_qconfig(‘qnnpack’)}<br>
model_to_quantize.train()</p>
<h1>
<a name="prepare-3" class="anchor" href="#prepare-3"></a>prepare</h1>
<p>model_prepared = quantize_fx.prepare_qat_fx(model_to_quantize, qconfig_dict)</p>
<h1>
<a name="quantize-4" class="anchor" href="#quantize-4"></a>quantize</h1>
<p>gmodel_quantized = quantize_fx.convert_fx(model_prepared)</p>
<p>def server_aggregate(gmodel_quantized, quantized_models_all):<br>
global_dict = gmodel_quantized.state_dict()</p>
<p>for k in global_dict.keys():<br>
global_dict[k] = torch.stack([quantized_models_all[i].state_dict()[k].float() for i in range(len(quantized_models_all))], 0).mean(0)</p>
<p>gmodel_quantized.load_state_dict(global_dict)<br>
for model in quantized_models_all:<br>
model.load_state_dict(gmodel_quantized.state_dict())</p>
<p>def test(gmodel_quantized, test_loader):<br>
model.eval()<br>
test_loss = 0<br>
correct = 0<br>
with torch.no_grad():<br>
for data, target in test_loader:<br>
data, target = data.cuda(), target.cuda()<br>
output = gmodel_quantized(data)<br>
test_loss += F.nll_loss(output, target, reduction=‘sum’).item()<br>
pred = output.argmax(dim=1, keepdim=True)  # get the index of the max log-probability<br>
correct += pred.eq(target.view_as(pred)).sum().item()<br>
test_loss /= len(test_loader.dataset)<br>
acc = correct / len(test_loader.dataset)<br>
return test_loss, acc</p>
<h1>
<a name="train-the-model-5" class="anchor" href="#train-the-model-5"></a>train the model</h1>
<h1>
<a name="global-model-6" class="anchor" href="#global-model-6"></a>global model</h1>
<p>gmodel_quantized =  gmodel_quantized.cuda()</p>
<h1>
<a name="client-models-7" class="anchor" href="#client-models-7"></a>client models</h1>
<p>for i in range(3):<br>
cmodel= quantized_models_all[i]<br>
cmodel.cuda()</p>
<p>quantized_models_all = [cmodel.cuda() for _ in range(num_selected)]<br>
for model in quantized_models_all:<br>
model.load_state_dict(gmodel_quantized.state_dict()) # initial synchronizing with global model</p>
<h2>
<a name="opt-optimsgdmodelparameters-lr01-for-model-in-quantized_models_all-8" class="anchor" href="#opt-optimsgdmodelparameters-lr01-for-model-in-quantized_models_all-8"></a>opt = [optim.SGD(model.parameters(), lr=0.1) for model in quantized_models_all]</h2>
<p>ValueError Traceback (most recent call last)</p>
<p><a href="https://ue1tumoqq97-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20220526-060044-RC00_451112829#" rel="noopener nofollow ugc"></a> in () 14 model.load_state_dict(gmodel_quantized.state_dict()) # initial synchronizing with global model 15 —&gt; 16 opt = [optim.SGD(model.parameters(), lr=0.1) for model in quantized_models_all]</p>
<p>2 frames</p>
<p><a href="https://ue1tumoqq97-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20220526-060044-RC00_451112829#" rel="noopener nofollow ugc">/usr/local/lib/python3.7/dist-packages/torch/optim/optimizer.py</a> in <strong>init</strong>(self, params, defaults) 47 param_groups = list(params) 48 if len(param_groups) == 0: —&gt; 49 raise ValueError(“optimizer got an empty parameter list”) 50 if not isinstance(param_groups[0], dict): 51 param_groups = [{‘params’: param_groups}]</p>
<p>ValueError: optimizer got an empty parameter list</p>
<p>I want to quantize models when clients send their model to the central server, and I want to print all loos and accuracy values for each rounds.</p>
<p>I’m using CIFAR100 dataset, and VGG19 model for this training.<br>
This the model architecture:<br>
cfg = {<br>
‘VGG11’: [64, ‘M’, 128, ‘M’, 256, 256, ‘M’, 512, 512, ‘M’, 512, 512, ‘M’],<br>
‘VGG13’: [64, 64, ‘M’, 128, 128, ‘M’, 256, 256, ‘M’, 512, 512, ‘M’, 512, 512, ‘M’],<br>
‘VGG16’: [64, 64, ‘M’, 128, 128, ‘M’, 256, 256, 256, ‘M’, 512, 512, 512, ‘M’, 512, 512, 512, ‘M’],<br>
‘VGG19’: [64, 64, ‘M’, 128, 128, ‘M’, 256, 256, 256, 256, ‘M’, 512, 512, 512, 512, ‘M’, 512, 512, 512, 512, ‘M’],<br>
}</p>
<p>class VGG(nn.Module):<br>
def <strong>init</strong>(self, vgg_name):<br>
super(VGG, self).<strong>init</strong>()<br>
self.features = self._make_layers(cfg[vgg_name])<br>
self.classifier = nn.Sequential(<br>
nn.Linear(512, 512),<br>
nn.ReLU(True),<br>
nn.Linear(512, 512),<br>
nn.ReLU(True),<br>
nn.Linear(512, 100)<br>
)</p>
<p>def forward(self, x):<br>
out = self.features(x)<br>
out = out.view(out.size(0), -1)<br>
out = self.classifier(out)<br>
output = F.log_softmax(out, dim=1)<br>
return output</p>
<p>def _make_layers(self, cfg):<br>
layers = []<br>
in_channels = 3<br>
for x in cfg:<br>
if x == ‘M’:<br>
layers += [nn.MaxPool2d(kernel_size=2, stride=2)]<br>
else:<br>
layers += [nn.Conv2d(in_channels, x, kernel_size=3, padding=1),<br>
nn.BatchNorm2d(x),<br>
nn.ReLU(inplace=True)]<br>
in_channels = x<br>
layers += [nn.AvgPool2d(kernel_size=1, stride=1)]<br>
return nn.Sequential(*layers)</p>
<p>Appreciate any help!</p>
<p>NOTE: there is no problem with the model when train it without quantization!</p>