RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x92160 and 9216x120)

nassimus: <p>Having this model :</p>
<pre><code class="lang-auto">cnn = tv.models.efficientnet_b3(weights=weights)
for param in cnn.features.parameters():
    param.requires_grad = False    
cnn.classifier[1] = nn.Flatten(0)
torch.manual_seed(42)
torch.cuda.manual_seed(42)
class MultiInputs(nn.Module):
  def __init__(self):
    super(MultiInputs, self).__init__()
    self.cnn = cnn
    self.sec = torch.nn.Sequential()
    self.sec.append(torch.nn.Linear(9216,120))
    self.sec.append(torch.nn.ReLU())
    self.sec.append(torch.nn.Dropout(p=0.1))
    self.sec.append(torch.nn.Linear(120, 24))
    self.sec.append(torch.nn.Sigmoid())
    self.sec.append(torch.nn.Dropout(p=0.1))
    self.sec.append(torch.nn.Linear(24, 2, bias=True))
  def forward(self, input1, input2, input3, input4, input5, input6):
    c1 = self.cnn(input1)
    c2 = self.cnn(input2)
    c3 = self.cnn(input3)
    c4 = self.cnn(input4)
    c5 = self.cnn(input5)
    c6 = self.cnn(input6)
    all = torch.cat((c1, c2, c3, c4, c5, c6), 0)
    return self.sec(all)
model = MultiInputs().to(device)
</code></pre>
<p>I am getting this runtime error :</p>
<pre><code class="lang-auto">&lt;ipython-input-195-1b012765bfdd&gt; in forward(self, input1, input2, input3, input4, input5, input6)
     30     c6 = self.cnn(input6)
     31     all = torch.cat((c1, c2, c3, c4, c5, c6), 0)
---&gt; 32     return self.sec(all)
     33 model = MultiInputs().to(device)
     34 

/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in _wrapped_call_impl(self, *args, **kwargs)
   1734             return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1735         else:
-&gt; 1736             return self._call_impl(*args, **kwargs)
   1737 
   1738     # torchrec tests the code consistency with the following code

/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in _call_impl(self, *args, **kwargs)
   1745                 or _global_backward_pre_hooks or _global_backward_hooks
   1746                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1747             return forward_call(*args, **kwargs)
   1748 
   1749         result = None

/usr/local/lib/python3.10/dist-packages/torch/nn/modules/container.py in forward(self, input)
    248     def forward(self, input):
    249         for module in self:
--&gt; 250             input = module(input)
    251         return input
    252 

/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in _wrapped_call_impl(self, *args, **kwargs)
   1734             return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1735         else:
-&gt; 1736             return self._call_impl(*args, **kwargs)
   1737 
   1738     # torchrec tests the code consistency with the following code

/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in _call_impl(self, *args, **kwargs)
   1745                 or _global_backward_pre_hooks or _global_backward_hooks
   1746                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1747             return forward_call(*args, **kwargs)
   1748 
   1749         result = None

/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py in forward(self, input)
    123 
    124     def forward(self, input: Tensor) -&gt; Tensor:
--&gt; 125         return F.linear(input, self.weight, self.bias)
    126 
    127     def extra_repr(self) -&gt; str:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x92160 and 9216x120)
</code></pre>
<p>Here’s all the layers</p>
<pre><code class="lang-auto">============================================================================================================================================
Layer (type:depth-idx)                                            Input Shape               Output Shape              Param #
============================================================================================================================================
MultiInputs                                                       [1, 3, 320, 320]          [2]                       --
├─EfficientNet: 1-1                                               [1, 3, 320, 320]          [1536]                    --
│    └─Sequential: 2-1                                            [1, 3, 320, 320]          [1, 1536, 10, 10]         --
│    │    └─Conv2dNormActivation: 3-1                             [1, 3, 320, 320]          [1, 40, 160, 160]         (1,160)
│    │    └─Sequential: 3-2                                       [1, 40, 160, 160]         [1, 24, 160, 160]         (3,504)
│    │    └─Sequential: 3-3                                       [1, 24, 160, 160]         [1, 32, 80, 80]           (48,118)
│    │    └─Sequential: 3-4                                       [1, 32, 80, 80]           [1, 48, 40, 40]           (110,912)
│    │    └─Sequential: 3-5                                       [1, 48, 40, 40]           [1, 96, 20, 20]           (638,700)
│    │    └─Sequential: 3-6                                       [1, 96, 20, 20]           [1, 136, 20, 20]          (1,387,760)
│    │    └─Sequential: 3-7                                       [1, 136, 20, 20]          [1, 232, 10, 10]          (4,628,964)
│    │    └─Sequential: 3-8                                       [1, 232, 10, 10]          [1, 384, 10, 10]          (3,284,218)
│    │    └─Conv2dNormActivation: 3-9                             [1, 384, 10, 10]          [1, 1536, 10, 10]         (592,896)
│    └─AdaptiveAvgPool2d: 2-2                                     [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-3                                            [1, 1536]                 [1536]                    --
│    │    └─Dropout: 3-10                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-11                                         [1, 1536]                 [1536]                    --
├─EfficientNet: 1-2                                               [1, 3, 320, 320]          [1536]                    (recursive)
│    └─Sequential: 2-4                                            [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-12                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-13                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-14                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-15                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-16                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-17                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-18                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-19                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-20                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-5                                     [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-6                                            [1, 1536]                 [1536]                    --
│    │    └─Dropout: 3-21                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-22                                         [1, 1536]                 [1536]                    --
├─EfficientNet: 1-3                                               [1, 3, 320, 320]          [1536]                    (recursive)
│    └─Sequential: 2-7                                            [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-23                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-24                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-25                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-26                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-27                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-28                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-29                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-30                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-31                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-8                                     [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-9                                            [1, 1536]                 [1536]                    --
│    │    └─Dropout: 3-32                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-33                                         [1, 1536]                 [1536]                    --
├─EfficientNet: 1-4                                               [1, 3, 320, 320]          [1536]                    (recursive)
│    └─Sequential: 2-10                                           [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-34                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-35                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-36                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-37                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-38                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-39                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-40                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-41                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-42                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-11                                    [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-12                                           [1, 1536]                 [1536]                    --
│    │    └─Dropout: 3-43                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-44                                         [1, 1536]                 [1536]                    --
├─EfficientNet: 1-5                                               [1, 3, 320, 320]          [1536]                    (recursive)
│    └─Sequential: 2-13                                           [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-45                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-46                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-47                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-48                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-49                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-50                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-51                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-52                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-53                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-14                                    [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-15                                           [1, 1536]                 [1536]                    --
│    │    └─Dropout: 3-54                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-55                                         [1, 1536]                 [1536]                    --
├─EfficientNet: 1-6                                               [1, 3, 320, 320]          [1536]                    (recursive)
│    └─Sequential: 2-16                                           [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-56                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-57                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-58                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-59                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-60                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-61                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-62                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-63                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-64                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-17                                    [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-18                                           [1, 1536]                 [1536]                    --
│    │    └─Dropout: 3-65                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-66                                         [1, 1536]                 [1536]                    --
├─Sequential: 1-7                                                 [9216]                    [2]                       --
│    └─Linear: 2-19                                               [9216]                    [120]                     1,106,040
│    └─ReLU: 2-20                                                 [120]                     [120]                     --
│    └─Dropout: 2-21                                              [120]                     [120]                     --
│    └─Linear: 2-22                                               [120]                     [24]                      2,904
│    └─Sigmoid: 2-23                                              [24]                      [24]                      --
│    └─Dropout: 2-24                                              [24]                      [24]                      --
│    └─Linear: 2-25                                               [24]                      [2]                       50
============================================================================================================================================</code></pre>

Tony-Y: <aside class="quote no-group" data-username="nassimus" data-post="1" data-topic="212186">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nassimus/48/73605_2.png" class="avatar"> nassimus:</div>
<blockquote>
<p><code>all = torch.cat((c1, c2, c3, c4, c5, c6), 0)</code></p>
</blockquote>
</aside>
<p>This shape is 1x92160. So, it is not acceptable as input fed into <code>torch.nn.Linear(9216,120)</code>.</p>

nassimus: <p>Hi, i don’t think so, the ‘all’  has a shape of  9216 = 6 * 1536 (the output of the Flatten → cnn )</p>
<p>the 92160 seems equals to 9216 * batch_size, when i put 1 in the batch_size the error disapear</p>

Tony-Y: <p>Could you check the shape of <code>all</code>?</p>
<pre><code class="lang-auto">def forward(...):
  ...
  all = torch.cat((c1, c2, c3, c4, c5, c6), 0)
  print('Shape of all:', all.size())
  return self.sec(all)
</code></pre>

Tony-Y: <p>I think c1, c2, etc. should be (batch_size, 1536) in shape. So, the concatenation dimension should be 1.</p>
<pre><code class="lang-auto">def forward(self, input1, input2, input3, input4, input5, input6):
    c1 = self.cnn(input1).view(-1, 1536)
    c2 = self.cnn(input2).view(-1, 1536)
    c3 = self.cnn(input3).view(-1, 1536)
    c4 = self.cnn(input4).view(-1, 1536)
    c5 = self.cnn(input5).view(-1, 1536)
    c6 = self.cnn(input6).view(-1, 1536)
    all = torch.cat((c1, c2, c3, c4, c5, c6), 1)
    return self.sec(all)
</code></pre>

nassimus: <p>I changed to</p>
<pre><code class="lang-auto">all = torch.cat((c1, c2, c3, c4, c5, c6), 1)
    print('1_ all.shape ')
    print(all.shape)
    all = all.view(1, 1, 9216)
    print('2_ all.shape ')
    print(all.shape)
    
    return self.sec(all)
</code></pre>
<p>here’s the output, but it doesn’t fix the batch_size problem :</p>
<pre><code class="lang-auto">1_ all.shape 
torch.Size([1, 9216])
2_ all.shape 
torch.Size([1, 1, 9216])
</code></pre>

nassimus: <p>Here’s the new model, but still the sane batch size problem</p>
<pre><code class="lang-auto">

============================================================================================================================================
MultiInputs                                                       [1, 3, 320, 320]          [1, 2]                    --
├─EfficientNet: 1-1                                               [1, 3, 320, 320]          [1, 1536]                 --
│    └─Sequential: 2-1                                            [1, 3, 320, 320]          [1, 1536, 10, 10]         --
│    │    └─Conv2dNormActivation: 3-1                             [1, 3, 320, 320]          [1, 40, 160, 160]         (1,160)
│    │    └─Sequential: 3-2                                       [1, 40, 160, 160]         [1, 24, 160, 160]         (3,504)
│    │    └─Sequential: 3-3                                       [1, 24, 160, 160]         [1, 32, 80, 80]           (48,118)
│    │    └─Sequential: 3-4                                       [1, 32, 80, 80]           [1, 48, 40, 40]           (110,912)
│    │    └─Sequential: 3-5                                       [1, 48, 40, 40]           [1, 96, 20, 20]           (638,700)
│    │    └─Sequential: 3-6                                       [1, 96, 20, 20]           [1, 136, 20, 20]          (1,387,760)
│    │    └─Sequential: 3-7                                       [1, 136, 20, 20]          [1, 232, 10, 10]          (4,628,964)
│    │    └─Sequential: 3-8                                       [1, 232, 10, 10]          [1, 384, 10, 10]          (3,284,218)
│    │    └─Conv2dNormActivation: 3-9                             [1, 384, 10, 10]          [1, 1536, 10, 10]         (592,896)
│    └─AdaptiveAvgPool2d: 2-2                                     [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-3                                            [1, 1536]                 [1, 1536]                 --
│    │    └─Dropout: 3-10                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-11                                         [1, 1536]                 [1, 1536]                 --
├─EfficientNet: 1-2                                               [1, 3, 320, 320]          [1, 1536]                 (recursive)
│    └─Sequential: 2-4                                            [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-12                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-13                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-14                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-15                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-16                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-17                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-18                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-19                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-20                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-5                                     [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-6                                            [1, 1536]                 [1, 1536]                 --
│    │    └─Dropout: 3-21                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-22                                         [1, 1536]                 [1, 1536]                 --
├─EfficientNet: 1-3                                               [1, 3, 320, 320]          [1, 1536]                 (recursive)
│    └─Sequential: 2-7                                            [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-23                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-24                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-25                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-26                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-27                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-28                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-29                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-30                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-31                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-8                                     [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-9                                            [1, 1536]                 [1, 1536]                 --
│    │    └─Dropout: 3-32                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-33                                         [1, 1536]                 [1, 1536]                 --
├─EfficientNet: 1-4                                               [1, 3, 320, 320]          [1, 1536]                 (recursive)
│    └─Sequential: 2-10                                           [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-34                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-35                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-36                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-37                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-38                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-39                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-40                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-41                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-42                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-11                                    [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-12                                           [1, 1536]                 [1, 1536]                 --
│    │    └─Dropout: 3-43                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-44                                         [1, 1536]                 [1, 1536]                 --
├─EfficientNet: 1-5                                               [1, 3, 320, 320]          [1, 1536]                 (recursive)
│    └─Sequential: 2-13                                           [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-45                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-46                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-47                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-48                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-49                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-50                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-51                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-52                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-53                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-14                                    [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-15                                           [1, 1536]                 [1, 1536]                 --
│    │    └─Dropout: 3-54                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-55                                         [1, 1536]                 [1, 1536]                 --
├─EfficientNet: 1-6                                               [1, 3, 320, 320]          [1, 1536]                 (recursive)
│    └─Sequential: 2-16                                           [1, 3, 320, 320]          [1, 1536, 10, 10]         (recursive)
│    │    └─Conv2dNormActivation: 3-56                            [1, 3, 320, 320]          [1, 40, 160, 160]         (recursive)
│    │    └─Sequential: 3-57                                      [1, 40, 160, 160]         [1, 24, 160, 160]         (recursive)
│    │    └─Sequential: 3-58                                      [1, 24, 160, 160]         [1, 32, 80, 80]           (recursive)
│    │    └─Sequential: 3-59                                      [1, 32, 80, 80]           [1, 48, 40, 40]           (recursive)
│    │    └─Sequential: 3-60                                      [1, 48, 40, 40]           [1, 96, 20, 20]           (recursive)
│    │    └─Sequential: 3-61                                      [1, 96, 20, 20]           [1, 136, 20, 20]          (recursive)
│    │    └─Sequential: 3-62                                      [1, 136, 20, 20]          [1, 232, 10, 10]          (recursive)
│    │    └─Sequential: 3-63                                      [1, 232, 10, 10]          [1, 384, 10, 10]          (recursive)
│    │    └─Conv2dNormActivation: 3-64                            [1, 384, 10, 10]          [1, 1536, 10, 10]         (recursive)
│    └─AdaptiveAvgPool2d: 2-17                                    [1, 1536, 10, 10]         [1, 1536, 1, 1]           --
│    └─Sequential: 2-18                                           [1, 1536]                 [1, 1536]                 --
│    │    └─Dropout: 3-65                                         [1, 1536]                 [1, 1536]                 --
│    │    └─Flatten: 3-66                                         [1, 1536]                 [1, 1536]                 --
├─Sequential: 1-7                                                 [1, 1, 9216]              [1, 2]                    --
│    └─Flatten: 2-19                                              [1, 1, 9216]              [1, 9216]                 --
│    └─Linear: 2-20                                               [1, 9216]                 [1, 120]                  1,106,040
│    └─ReLU: 2-21                                                 [1, 120]                  [1, 120]                  --
│    └─Dropout: 2-22                                              [1, 120]                  [1, 120]                  --
│    └─Linear: 2-23                                               [1, 120]                  [1, 24]                   2,904
│    └─Sigmoid: 2-24                                              [1, 24]                   [1, 24]                   --
│    └─Dropout: 2-25                                              [1, 24]                   [1, 24]                   --
│    └─Linear: 2-26                                               [1, 24]                   [1, 2]                    50
</code></pre>

Tony-Y: <p><code>all</code> should be (batch_size, 9216) in shape. Did you try my code in the previous reply?</p>

nassimus: <p>Thank you tony, now I did, and it solve the batch issue, Many thanks</p>