Issue with DeepLabv3 inbuilt model and custom backbones

justin-larking-pk: <p>When creating a deeplabv3 model using torchvision inbuilt model i’ve noticed it appears to not work the same when instantiating the model with a prebuilt setup like deeplabv3_mobilenet_v3_large().</p>
<p>Specifically when building a deeplabv3 model like so:</p>
<pre><code class="lang-auto">def createdeeplabv3_mobilenet(outputchannels=1):
    """DeepLabv3 class with custom head
    Args:
        outputchannels (int, optional): The number of output channels
        in your dataset masks. Defaults to 1.
    Returns:
        model: Returns the DeepLabv3 model with the MobileNetV3 backbone.
    """
    model = torchvision.models.segmentation.deeplabv3_mobilenet_v3_large(pretrained=True, progress=True)
    #model = torchvision.models.segmentation.deeplabv3_resnet50(pretrained=True, progress=True)
    model.classifier = torchvision.models.segmentation.deeplabv3.DeepLabHead(960, outputchannels)
    # Set the model in training mode
    model.train()
    return model
</code></pre>
<p>It works perfectly fine in training and converges.</p>
<p>However if I try to put a different backbone on the network (even even the exact same one), like so:</p>
<pre><code class="lang-auto">def createdeeplabv3_effnetv2(outputchannels=1):
    """DeepLabv3 class with custom head
    Args:
        outputchannels (int, optional): The number of output channels
        in your dataset masks. Defaults to 1.
    Returns:
        model: Returns the DeepLabv3 model with the Efficienetv2s backbone.
    """
    #backbone = torchvision.models.efficientnet_v2_s(weights="DEFAULT").features
    #backbone.out_channels = 1280
    #classifier = torchvision.models.segmentation.deeplabv3.DeepLabHead(1280, outputchannels)
    #model = torchvision.models.segmentation.DeepLabV3(backbone=backbone, classifier=classifier)
    model = torchvision.models.segmentation.deeplabv3_mobilenet_v3_large(pretrained=True, progress=True)
    model.backbone = torchvision.models.mobilenet_v3_large(weights="DEFAULT").features
    model.classifier = torchvision.models.segmentation.deeplabv3.DeepLabHead(1280, outputchannels)
    # Set the model in training mode
    print(model)
    model.train()
    return model
</code></pre>
<p>This error is returned when I try to train:</p>
<blockquote>
<p>File “/home/justin/Documents/Code/computer-vision-utils/deeplearning/DeepLabV3Plus/main.py”, line 90, in main<br>
model_trained, history = trainer.train_segmentation(model, dataloaders, criterion, optimizer, DEVICE, args.Checkpoint, scheduler=scheduler, num_epochs=EPOCHS, project=project)<br>
File “/home/justin/Documents/Code/computer-vision-utils/deeplearning/DeepLabV3Plus/trainer.py”, line 70, in train_segmentation<br>
preds = model(inputs.to(device))<br>
File “/home/justin/miniconda3/envs/pytorch_env/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1130, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/home/justin/miniconda3/envs/pytorch_env/lib/python3.9/site-packages/torchvision/models/segmentation/_utils.py”, line 26, in forward<br>
x = features[“out”]<br>
TypeError: new(): invalid data type ‘str’</p>
</blockquote>
<p>This error is not returned with the base deeplabv3 models provided by trochvision (resnet or mobilenet)</p>
<p>When printing these models (the base deeplbav3_mobilenetv3_large and the custom one) with print(model), as far as I can tell they look exactly the same. Yet one works fine and the other doesn’t.</p>
<p>I have tried multiple other ways of instantiating a custom backbone i.e.:</p>
<pre><code class="lang-auto">backbone = torchvision.models.efficientnet_v2_s(weights="DEFAULT").features
classifier = torchvision.models.segmentation.deeplabv3.DeepLabHead(1280, outputchannels)
model = torchvision.models.segmentation.DeepLabV3(backbone, classifier, None)
</code></pre>
<p>It gives the same error.</p>
<p>Where might I be going wrong here?</p>