AttributeError: 'Conv2d' object has no attribute '_modules'

sbht_sbht: <p>Trying to save entire model like in this docs <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html#save-load-entire-model" rel="noopener nofollow ugc">Save/Load Entire Model</a> and then load it, but got this error <code>AttributeError: 'Conv2d' object has no attribute '_modules'</code>.</p>
<p>I saved model like this (I post only part of model, becaus it’s to big):</p>
<pre><code class="lang-auto">torch.save(model_static_quantized, "full_model.pth")
</code></pre>
<p>And load model like this:</p>
<pre><code class="lang-auto">model_filename = "D:/3divi_cv/hw_9/model/full_model.pth"

checkpoint = torch.load(model_filename) #, map_location=torch.device('cpu')))
checkpoint.eval()
</code></pre>
<p>What is important is that I used quantization before saving the model and my <code>deeplabv3_resnet50</code> model starts look like this:</p>
<blockquote>
<p>QuantDeQuantNet(<br>
(quant): Quantize(scale=tensor([0.0079]), zero_point=tensor([0]), dtype=torch.quint8)<br>
(dequant): DeQuantize()<br>
(model): DeepLabV3(<br>
(backbone): IntermediateLayerGetter(<br>
(conv1): QuantizedConv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), scale=0.07399188727140427, zero_point=53, padding=(3, 3), bias=False)<br>
(bn1): QuantizedBatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)<br>
(layer1): Sequential(<br>
(0): Bottleneck(<br>
(conv1): QuantizedConv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.04221147298812866, zero_point=63, bias=False)<br>
(bn1): QuantizedBatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): QuantizedConv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.03571641072630882, zero_point=62, padding=(1, 1), bias=False)<br>
(bn2): QuantizedBatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): QuantizedConv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.007678185123950243, zero_point=68, bias=False)<br>
(bn3): QuantizedBatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)</p>
</blockquote>
<p>How can I fix given error?</p>

ptrblck: <p>Storing the model directly can easily break as you would need to restore the same data/file structure of your project and we thus recommend storing the <code>state_dict</code> instead and to recreate the model.<br>
Did you try to use this approach and would it work?</p>

sbht_sbht: <p>I tried to save state_dict, but I don’t understande, how can I load it as model with architecture. I guess that state_dict save only weights. I load the model in this order:</p>
<blockquote>
<ol>
<li>model = deeplabv3_resnet50()</li>
<li>model.load_state_dict(‘my_saved_model_dict’)</li>
</ol>
</blockquote>
<p>And get this error:</p>
<blockquote>
<p>1600             0, 'Missing key(s) in state_dict: {}. '.format(<br>
1601                 ‘, ‘.join(’“{}”’.format(k) for k in missing_keys)))<br>
1603 if len(error_msgs) &gt; 0:<br>
 → 1604     raise RuntimeError(‘Error(s) in loading state_dict for {}:\n\t{}’.format(<br>
1605                        self.<strong>class</strong>.<strong>name</strong>, “\n\t”.join(error_msgs)))<br>
1606 return _IncompatibleKeys(missing_keys, unexpected_keys)</p>
<p>RuntimeError: Error(s) in loading state_dict for DeepLabV3:<br>
Unexpected key(s) in state_dict: “backbone.conv1.bias”, “backbone.conv1.scale”, “backbone.conv1.zero_point”, “backbone.bn1.scale”, “backbone.bn1.zero_point”, “backbone.layer1.0.conv1.bias”, “backbone.layer1.0.conv1.scale”, “backbone.layer1.0.conv1.zero_point”, “backbone.layer1.0.bn1.scale”, “backbone.layer1.0.bn1.zero_point”, “backbone.layer1.0.conv2.bias”, “backbone.layer1.0.conv2.scale”, “backbone.layer1.0.conv2.zero_point”, “backbone.layer1.0.bn2.scale”, “backbone.layer1.0.bn2.zero_point”, “backbone.layer1.0.conv3.bias”, “backbone.layer1.0.conv3.scale”, “backbone.layer1.0.conv3.zero_point”, “backbone.layer1.0.bn3.scale”, “backbone.layer1.0.bn3.zero_point”</p>
</blockquote>
<p>After the model is quantized, its modules become different from the modules of the original model and it is not possible to load the weights</p>
<p>How can I save ALL model with architecture and weights, to be able to simply load it all and simply assign it to a <strong>model</strong> variable like this:</p>
<blockquote>
<p>model = torch.load(‘path_to_saved_model’)</p>
</blockquote>
<p>Also thank you for your help. I realy appreciate it</p>

ptrblck: <aside class="quote no-group" data-username="sbht_sbht" data-post="3" data-topic="178565">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sbht_sbht/48/46999_2.png" class="avatar"> sbht_sbht:</div>
<blockquote>
<p>After the model is quantized, its modules become different from the modules of the original model and it is not possible to load the weights</p>
</blockquote>
</aside>
<p>You should create the model in the same way, i.e. also with the quantization, before trying to load the <code>state_dict</code>.</p>
<aside class="quote no-group" data-username="sbht_sbht" data-post="3" data-topic="178565">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sbht_sbht/48/46999_2.png" class="avatar"> sbht_sbht:</div>
<blockquote>
<p>How can I save ALL model with architecture and weights, to be able to simply load it all and simply assign it to a <strong>model</strong> variable like this:</p>
</blockquote>
</aside>
<p>Again, I would not recommend it but using a plain <code>torch.save(model, PATH)</code> should work assuming you are restoring <em>all</em> files and definitions in the same way.</p>

sbht_sbht: <p>I’m trying load my model like you said, but this time I have an error when I’m trying to use this model to predict. I load model like in this code:</p>
<pre><code class="lang-auto">class QuantDeQuantNet(torch.nn.Module):
    def __init__(self, model):
        super(QuantDeQuantNet, self).__init__()
        self.quant = torch.quantization.QuantStub()
        self.dequant = torch.quantization.DeQuantStub()
        self.model = model

    def forward(self, x):
        x = self.quant(x)
        x = self.model(x)
        x = self.dequant(x)
        return x


class Solver:
    def __init__(self, weights_filename):
        self.model = self.init_model(weights_filename)
        self.device = 'cpu'


    def createDeepLabv3(self, outputchannels=2):
        model = models.segmentation.deeplabv3_resnet50(pretrained=True, progress=True, aux_loss=True)
        model.classifier = DeepLabHead(2048, outputchannels)

        model.aux_classifier[-1] = nn.Conv2d(256, 2, kernel_size=(1, 1), stride=(1, 1))
        model_fp32 = QuantDeQuantNet(model).to('cpu')

        backend = "fbgemm"
        model_fp32.qconfig = torch.quantization.get_default_qconfig(backend)
        torch.backends.quantized.engine = backend

        model_static_quantized = torch.quantization.prepare(model_fp32, inplace=False)
        model_static_quantized = torch.quantization.convert(model_static_quantized,
                                                            inplace=False)
        

        return model_static_quantized


    def init_model(self, weights_filename):
        model = self.createDeepLabv3()

        if torch.cuda.is_available():
            checkpoint = (torch.load(weights_filename))
        else:
            checkpoint = (torch.load(weights_filename, map_location=torch.device('cpu')))

        model.model.load_state_dict(checkpoint)
        print(model)
        return model
</code></pre>
<p>And get this huge error</p>
<blockquote>
<p>NotImplementedError: Could not run ‘aten::add.out’ with arguments from the ‘QuantizedCPU’ backend. This could be because the operator doesn’t exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit <a href="https://fburl.com/ptmfixes" class="inline-onebox" rel="noopener nofollow ugc">Internal Login</a> for possible resolutions. ‘aten::add.out’ is only available for these backends: [Dense, FPGA, Conjugate, VmapMode, FuncTorchGradWrapper, Functionalize, MPS, IPU, UNKNOWN_TENSOR_TYPE_ID, QuantizedXPU, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseCPU, SparseCUDA, SparseHIP, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseXPU, UNKNOWN_TENSOR_TYPE_ID, SparseVE, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, NestedTensorCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID].</p>
<p>CPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterCPU.cpp:37386 [kernel]<br>
CUDA: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterCUDA.cpp:51977 [kernel]<br>
Meta: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterMeta.cpp:31637 [kernel]<br>
MkldnnCPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterMkldnnCPU.cpp:690 [kernel]<br>
SparseCPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCPU.cpp:1858 [kernel]<br>
SparseCUDA: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCUDA.cpp:2018 [kernel]<br>
SparseCsrCPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCsrCPU.cpp:1507 [kernel]<br>
SparseCsrCUDA: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCsrCUDA.cpp:1657 [kernel]<br>
BackendSelect: fallthrough registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\core\BackendSelectFallbackKernel.cpp:3 [backend fallback]<br>
Python: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\core\PythonFallbackKernel.cpp:133 [backend fallback]<br>
Named: fallthrough registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\core\NamedRegistrations.cpp:11 [kernel]<br>
Conjugate: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\ConjugateFallback.cpp:18 [backend fallback]<br>
Negative: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\NegateFallback.cpp:18 [backend fallback]<br>
ZeroTensor: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\ZeroTensorFallback.cpp:86 [backend fallback]</p>
</blockquote>
<p>I tried to run the model using both cpu and cuda. Nothing helps.</p>
<p>I also noticed that when trying to quantize my other model, this two blocks</p>
<blockquote>
<p>(quant_m): Quantize(scale=tensor([0.0157]), zero_point=tensor([64]), dtype=torch.quint8)<br>
(dequant_m): DeQuantize()</p>
</blockquote>
<p>placed in the end of model:</p>
<blockquote>
<p>QuantizedSiameseNetwork(<br>
(backbone): Backbone(<br>
(conv1): QuantizedConv2d(3, 8, kernel_size=(3, 3), stride=(1, 1), scale=0.03301658481359482, zero_point=68, padding=(1, 1))<br>
(pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(conv2): QuantizedConv2d(8, 16, kernel_size=(3, 3), stride=(1, 1), scale=0.04873496666550636, zero_point=46, padding=(1, 1))<br>
(conv3): QuantizedConv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), scale=0.06440312415361404, zero_point=85, padding=(1, 1))<br>
(conv4): QuantizedConv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.04743369296193123, zero_point=93, padding=(1, 1))<br>
(fc1): QuantizedLinear(in_features=3136, out_features=1024, scale=0.015503729693591595, zero_point=72, qscheme=torch.per_channel_affine)<br>
(fc2): QuantizedLinear(in_features=1024, out_features=2, scale=0.024672511965036392, zero_point=80, qscheme=torch.per_channel_affine)<br>
)<br>
(quant_m): Quantize(scale=tensor([0.0157]), zero_point=tensor([64]), dtype=torch.quint8)<br>
(dequant_m): DeQuantize()<br>
)</p>
</blockquote>
<p>But in my current model that I’m trying quantize thit to blocks place at the beginning of model:</p>
<blockquote>
<p>QuantDeQuantNet(<br>
(quant): Quantize(scale=tensor([0.0079]), zero_point=tensor([0]), dtype=torch.quint8)<br>
(dequant): DeQuantize()<br>
(model): DeepLabV3(<br>
(backbone): IntermediateLayerGetter(</p>
</blockquote>

Devv4m: <p>Hi, sbht_sbht, have you solved this problem? I also encounter these bugs when I load the quantizationed model and try to evaluate it.</p>

Kristine_Xu: <p>I also got this problem when saved and load the whole quantized model.<br>
AttributeError: ‘ConvReLU1d’ object has no attribute ‘_modules’</p>