I'm having a problem training a face detection model and would like to ask for help

jinghui: <p>Here is the part of my model code where RuntimeError occurs: Given groups=16, weight of size [16, 1, 3, 3], expected input[32, 3, 256, 256] to have 16 channels, but got 3 channels instead, what should I do.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F


class hswish(nn.Module):
    def forward(self, x):
        out = x * F.relu6(x + 3, inplace=True) / 6
        return out


class hsigmoid(nn.Module):
    def forward(self, x):
        out = F.relu6(x + 3, inplace=True) / 6
        return out


def conv_bn(inp, oup, kernel, stride, padding=1):
    return nn.Sequential(
        nn.Conv2d(inp, oup, kernel, stride, padding, bias=False),
        nn.BatchNorm2d(oup),
        nn.ReLU(inplace=True))


def conv_1x1_bn(inp, oup):
    return nn.Sequential(
        nn.Conv2d(inp, oup, 1, 1, 0, bias=False),
        nn.BatchNorm2d(oup),
        nn.ReLU(inplace=True))


class SE(nn.Module):
    def __init__(self, in_channels, se_channels):
        super(SE, self).__init__()
        self.global_pool = nn.AdaptiveAvgPool2d(1)
        self.conv1 = nn.Conv2d(in_channels, se_channels, kernel_size=1, bias=False)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = nn.Conv2d(se_channels, in_channels, kernel_size=1, bias=False)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        out = self.global_pool(x)
        out = self.conv1(out)
        out = self.relu(out)
        out = self.conv2(out)
        out = self.sigmoid(out)
        return x * out


class MobileBottleneck(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, stride, expand_ratio, se=False, use_hswish=False):
        super(MobileBottleneck, self).__init__()
        self.stride = stride
        self.use_res_connect = self.stride == 1 and in_channels == out_channels
        self.expand_ratio = expand_ratio

        hidden_dim = int(round(in_channels * expand_ratio))
        self.use_se = se
        self.se = SE(hidden_dim, hidden_dim // 4) if se else None
        self.conv1 = conv_1x1_bn(in_channels, hidden_dim) if expand_ratio != 1 else None
        self.conv2 = nn.Conv2d(hidden_dim, hidden_dim, kernel_size, stride, (kernel_size - 1) // 2, groups=hidden_dim,
                               bias=False)
        self.bn2 = nn.BatchNorm2d(hidden_dim)
        self.conv3 = conv_1x1_bn(hidden_dim, out_channels) if expand_ratio != 1 else None
        self.hs = hswish() if use_hswish else nn.ReLU(inplace=True)

    def forward(self, x):
        out = x
        if self.expand_ratio != 1:
            out = self.conv1(out)
        out = self.hs(self.bn2(self.conv2(out)))
        if self.use_se:
            out = self.se(out)
        if self.expand_ratio != 1:
            out = self.conv3(out)
        if self.use_res_connect:
            return x + out
        else:
            return out


class MobileNetV3(nn.Module):
    def __init__(self, mode="small", width_mult=1.0, num_classes=1000):
        super(MobileNetV3, self).__init__()
        self.mode = mode
        self.width_mult = width_mult
        self.in_channels = 16  # Modify this for input channel compatibility
        self.num_classes = num_classes

        if mode == "large":
            self.cfg = [
                [3, 1, 16, False, False, 1],
                [3, 4, 24, False, False, 2],
                [3, 3, 24, False, False, 1],
                [5, 3, 40, True, False, 2],
                [5, 3, 40, True, False, 1],
                [5, 3, 40, True, False, 1],
                [3, 6, 80, False, True, 2],
                [3, 6, 80, False, True, 1],
                [3, 6, 80, False, True, 1],
                [3, 3, 112, True, True, 1],
                [3, 3, 112, True, True, 1],
                [5, 6, 160, True, True, 2],
                [5, 6, 160, True, True, 1],
                [5, 6, 160, True, True, 1],
                [3, 6, 320, False, True, 1]
            ]
        elif mode == "small":
            self.cfg = [
                [3, 1, 16, True, False, 2],
                [3, 4, 24, False, False, 2],
                [3, 3, 24, False, False, 1],
                [5, 3, 40, True, False, 2],
                [5, 3, 40, True, False, 1],
                [5, 3, 40, True, False, 1],
                [3, 6, 80, False, True, 2],
                [3, 3, 80, False, True, 1],
                [3, 6, 80, False, True, 1],
                [3, 3, 112, True, True, 1],
                [3, 3, 112, True, True, 1]
            ]
        else:
            raise NotImplementedError

        self.features = nn.Sequential(*[MobileBottleneck(self.in_channels, int(c * self.width_mult), k, s, t, se, hs)
                                        for k, t, c, se, hs, s in self.cfg])

        self.conv_head = conv_1x1_bn(int(320 * self.width_mult), 1280) if mode == "large" else conv_1x1_bn(
            int(112 * self.width_mult), 576)
        self.hs2 = hswish()
        self.avgpool = nn.AdaptiveAvgPool2d(1)
        self.classifier = nn.Sequential(
            nn.Linear(1280, num_classes) if mode == "large" else nn.Linear(576, num_classes)
        )
        self.initialize_weights()

    def forward(self, x):
        out = self.features(x)
        out = self.conv_head(out)
        out = self.hs2(out)
        out = self.avgpool(out)
        out = out.view(out.size(0), -1)
        out = self.classifier(out)
        return out

    def initialize_weights(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight, mode='fan_out')
                if m.bias is not None:
                    nn.init.zeros_(m.bias)
            elif isinstance(m, nn.BatchNorm2d):
                nn.init.ones_(m.weight)
                nn.init.zeros_(m.bias)
            elif isinstance(m, nn.Linear):
                nn.init.normal_(m.weight, 0, 0.01)
                if m.bias is not None:
                    nn.init.zeros_(m.bias)


class PNet_MobileNetV3(nn.Module):
    def __init__(self, mode="small", width_mult=1.0):
        super(PNet_MobileNetV3, self).__init__()
        self.conv_input = nn.Conv2d(3, 3, kernel_size=1, stride=1)
        self.backbone = MobileNetV3(mode=mode, width_mult=width_mult, num_classes=2)
        self.conv4_3 = nn.Conv2d(3, 2, kernel_size=1, stride=1, padding=0, bias=False)
        self.conv4_4 = nn.Conv2d(3, 4, kernel_size=1, stride=1, padding=0, bias=False)

    def forward(self, x):
        x = self.conv_input(x)
        x = self.backbone.features(x)
        cls = self.conv4_3(x)
        bbox = self.conv4_4(x)
        return cls, bbox

class RNet_MobileNetV3(nn.Module):
    def __init__(self, mode="small", width_mult=1.0):
        super(RNet_MobileNetV3, self).__init__()
        self.conv_input = nn.Conv2d(3, 16, kernel_size=1, stride=1) 
        self.backbone = MobileNetV3(mode=mode, width_mult=width_mult, num_classes=2)
        self.conv3_1 = nn.Conv2d(16, 2, kernel_size=1, stride=1, padding=0, bias=False)
        self.conv3_2 = nn.Conv2d(16, 4, kernel_size=1, stride=1, padding=0, bias=False)

    def forward(self, x):
        x = self.conv_input(x)  
        x = self.backbone.features(x)
        cls = self.conv3_1(x)
        bbox = self.conv3_2(x)
        return cls, bbox

class ONet_MobileNetV3(nn.Module):
    def __init__(self, mode="small", width_mult=1.0):
        super(ONet_MobileNetV3, self).__init__()
        self.conv_input = nn.Conv2d(3, 16, kernel_size=1, stride=1)
        self.backbone = MobileNetV3(mode=mode, width_mult=width_mult, num_classes=2)
        self.conv3_1 = nn.Conv2d(16, 2, kernel_size=1, stride=1, padding=0, bias=False)
        self.conv3_2 = nn.Conv2d(16, 4, kernel_size=1, stride=1, padding=0, bias=False)

    def forward(self, x):
        x = self.conv_input(x)  
        x = self.backbone.features(x)
        cls = self.conv3_1(x)
        bbox = self.conv3_2(x)
        return cls, bbox

</code></pre>

J_Johnson: <p>Your model is for black and white(1 channel) but your input is color(3 channels).</p>
<p>You could change this line:</p>
<aside class="quote no-group" data-username="jinghui" data-post="1" data-topic="211087">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/f05b48/48.png" class="avatar"> jinghui:</div>
<blockquote>
<p><code>[3, 1, 16, False, False, 1],</code></p>
</blockquote>
</aside>
<p>to:</p>
<pre><code class="lang-auto">[3, 3, 16, False, False, 1]
</code></pre>
<p>Or convert the input images to black and white.</p>

jinghui: <p>Thank you for your reply. I modified the code, but I still get a RuntimeError: “Given groups=16, weight of size [16, 1, 3, 3], expected input[32, 3, 256, 256] to have 16 channels, but got 3 channels instead.” Could you please tell me what the problem is?</p>

J_Johnson: <p>I see. You have <code>self.in_channels = 16</code>. Change that to 3. You can revert back the previous change in my last message.</p>

jinghui: <p>Thank you for your follow-up reply. I modified the code, but I’m still getting a RuntimeError: “Given groups=1, expected weight to be at least 1 at dimension 0, but got weight of size [0, 3, 1, 1] instead.” Could you please explain what’s causing this error?</p>

J_Johnson: <p>Please show how you’re instantiating the model.</p>

jinghui: <p>Sorry, I don’t quite understand what you mean. I have three parts of code: defining data set, defining algorithm and training data set. What do I need to provide?</p>

J_Johnson: <p>Should be something like:</p>
<pre><code class="lang-auto">model = MobileNetV3( ... )

</code></pre>
<p>Need to see what you’re putting in for the model arguments.</p>

jinghui: <pre><code class="lang-auto">model = MobileNetV3(mode='small', width_mult=1.0, num_classes=2)
</code></pre>
<p>The three parameters passed in here are:</p>
<p>mode=‘small’: The model type, where ‘small’ MobileNetV3 is selected.<br>
width_mult=1.0: The model width multiplier, where 1.0 is set, indicating the use of the default model width.<br>
num_classes=2: The number of output classes for the classification layer, where 2 is set, representing positive and negative sample classification in the face detection task.</p>
<p>The training algorithm code contains:</p>
<pre><code class="lang-auto">model = PNet_MobileNetV3(mode='small', width_mult=1.0) 
model = RNet_MobileNetV3(mode='small', width_mult=1.0)
model = ONet_MobileNetV3(mode='small', width_mult=1.0)
</code></pre>
<p>Here is the complete code that defines the dataset:</p>
<pre><code class="lang-auto">import os
import cv2
import numpy as np
import torch
from torch.utils.data import Dataset

class WiderFaceDataset(Dataset):
    def __init__(self, data_root, split='train', transform=None, label_file=None, target_size=(256, 256)):
        self.data_root = data_root
        self.split = split
        self.transform = transform
        self.target_size = target_size  
        self.image_paths = []
        self.labels = []
        self.bboxes = []

        if label_file is None:
            if split == 'train':
                label_file = os.path.join(data_root, 'wider_face_split', 'wider_face_train_bbx_gt.txt')
            elif split == 'val':
                label_file = os.path.join(data_root, 'wider_face_split', 'wider_face_val_bbx_gt.txt')
            else:
                raise ValueError(f'Unknown split: {split}')

        MAX_LENGTH = 256

        with open(label_file, 'r') as f:
            lines = f.readlines()
            i = 0
            while i &lt; len(lines):
                image_path = lines[i].strip()
                i += 1


                num_faces_str = lines[i].strip()
                if num_faces_str.isdigit():
                    num_faces = int(num_faces_str)
                else:
                    i += 1
                    continue

                i += 1  # 跳过空行
                if num_faces == 0:
                    i += num_faces 
                    continue

                bboxes = []
                labels = []
                for _ in range(num_faces):
                    bbox = [int(x) for x in lines[i].strip().split()]
                    if len(bbox) &gt;= 4:
                        x1, y1, w, h = bbox[0], bbox[1], bbox[2], bbox[3]
                        x2 = x1 + w
                        y2 = y1 + h
                        bboxes.append((x1, y1, x2, y2))
                        labels.append(1)  
                    i += 1

                if len(labels) &lt;= MAX_LENGTH and len(bboxes) &lt;= MAX_LENGTH:
                    self.image_paths.append(os.path.join(data_root, 'WIDER_' + split, 'images', image_path))
                    self.labels.append(np.array(labels, dtype=np.float32))
                    self.bboxes.append(np.array(bboxes, dtype=np.float32))

    def __len__(self):
        return len(self.image_paths)

    def __getitem__(self, idx):
        image_path = self.image_paths[idx]
        label = self.labels[idx]
        bbox = self.bboxes[idx]


        image = cv2.imread(image_path)
        if image is None:
            raise FileNotFoundError(f"Image not found: {image_path}")
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)


        original_height, original_width = image.shape[:2]


        image = cv2.resize(image, self.target_size, interpolation=cv2.INTER_LINEAR)
        print(f"Image shape: {image.shape}") 
        image = image.astype(np.float32) / 255.0  
        print(f"Image shape after resizing: {image.shape}")  
        image = image.transpose((2, 0, 1))  
        print(f"Image shape after transformation: {image.shape}")  


        assert image.shape[0] == 3, f"Expected 3 channels, but got {image.shape[0]}"


        height, width = image.shape[1:3]
        if len(bbox) &gt; 0:
            scale_x = width / original_width
            scale_y = height / original_height
            bbox[:, 0] *= scale_x  # x_min
            bbox[:, 1] *= scale_y  # y_min
            bbox[:, 2] *= scale_x  # x_max
            bbox[:, 3] *= scale_y  # y_max


        if len(label) == 0:
            labels = torch.zeros((1, 4), dtype=torch.float32) 
        else:
            labels = torch.tensor(label, dtype=torch.float32)
        return torch.tensor(image, dtype=torch.float32), labels, torch.tensor(bbox, dtype=torch.float32)
</code></pre>
<p>I used the Wider face data set that is publicly available on the Web.</p>

J_Johnson: <p>Can you copy the full error message including which line of code is problematic?</p>

jinghui: <p>This is the complete error message for running the code:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "C:\Users\Administrator\Desktop\facelandmarks-main\facelandmarks-main\train_mobilenetv3_mtcnn_end.py", line 155, in &lt;module&gt;
    train_pnet(train_dataset, val_dataset)
  File "C:\Users\Administrator\Desktop\facelandmarks-main\facelandmarks-main\train_mobilenetv3_mtcnn_end.py", line 113, in train_pnet
    train(train_dataset, val_dataset, model, optimizer, criterion, epochs=epochs, batch_size=batch_size,
  File "C:\Users\Administrator\Desktop\facelandmarks-main\facelandmarks-main\train_mobilenetv3_mtcnn_end.py", line 98, in train
    train_loss = train_epoch(model, train_loader, optimizer, criterion, device)
  File "C:\Users\Administrator\Desktop\facelandmarks-main\facelandmarks-main\train_mobilenetv3_mtcnn_end.py", line 57, in train_epoch
    cls_out, bbox_out = model(images)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Administrator\Desktop\facelandmarks-main\facelandmarks-main\mobilenetv3_mtcnn_end.py", line 171, in forward
    x = self.backbone.features(x)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\container.py", line 217, in forward
    input = module(input)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Administrator\Desktop\facelandmarks-main\facelandmarks-main\mobilenetv3_mtcnn_end.py", line 71, in forward
    out = self.hs(self.bn2(self.conv2(out)))
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\conv.py", line 460, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\conv.py", line 456, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=16, weight of size [16, 1, 3, 3], expected input[32, 3, 256, 256] to have 16 channels, but got 3 channels instead
</code></pre>

J_Johnson: <aside class="quote no-group" data-username="jinghui" data-post="11" data-topic="211087">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/f05b48/48.png" class="avatar"> jinghui:</div>
<blockquote>
<p><code>cls_out, bbox_out = model(images)</code></p>
</blockquote>
</aside>
<p>The above is in your error code, but it’s unclear which <code>model</code> is in fact being called since your previous code has all three instantiated under the same name:</p>
<pre><code class="lang-auto">model = PNet_MobileNetV3(mode='small', width_mult=1.0) 
model = RNet_MobileNetV3(mode='small', width_mult=1.0)
model = ONet_MobileNetV3(mode='small', width_mult=1.0)
</code></pre>
<p>If defined exactly as above, only the final <code>model</code> definition will be used further in latter code.</p>
<p>That aside, this is a good opportunity for you to learn how to debug and check dims in a convolution network.</p>
<p>What you want to do is to insert print statements throughout. First, try to understand what size of data you’re putting in. How many channels does it have? For example:</p>
<pre><code class="lang-auto">...
print(images.size()) #new line
cls_out, bbox_out = model(images)
...
</code></pre>
<p>That will give you something like <code>(32, 3, 256, 256)</code> which is describing batch size, channels, X, and Y dimensions. If the channels of images going into the model are 3, then you need to adjust the <code>self.in_channels</code> argument accordingly, as mentioned earlier. If that is then giving you later errors, like you mentioned, you’ll need to debug to find out why you’re passing zero channels into a Conv2d argument.</p>

jinghui: <p>Thank you very much for your response. This issue has consumed a great deal of my time and energy, and I feel that I may not be ready to learn about complex convolutional neural networks at this stage. I have decided to pursue a different direction. Once again, I appreciate your willingness to reply to my many messages.</p>