NotImplementedError: Module [ModuleList] is missing the required "forward" function

Askerag: <p>Im trying to build network which extract features by backbone network (ViT) and classify by linear layer.<br>
So i get error when trying to predict/forward-pass model:<br>
raise NotImplementedError(f"Module [{type(self).<strong>name</strong>}] is missing the required "forward" function")<br>
NotImplementedError: Module [ModuleList] is missing the required “forward” function</p>
<pre><code class="lang-auto">class FTNet(nn.Module):
    def __init__(self):
        super(FTNet, self).__init__()
        self.features = torch.nn.Sequential(*list(model_bb.children())[:-2])
        self.linear = torch.nn.Linear(320, 10)
        
    def forward(self, x):
        x = self.features(x)
        x = self.linear(x)
        return x
</code></pre>

ptrblck: <p>It seems you are using an <code>nn.ModuleList</code> in your model and are trying to call it directly which won’t work as it’s acting as a <code>list</code> but properly registers trainable parameters:</p>
<pre><code class="lang-python">modules = nn.ModuleList([
    nn.Linear(10, 10),
    nn.ReLU(),
    nn.Linear(10, 10),
])
x = torch.randn(1, 10)
out = modules(x)
# NotImplementedError: Module [ModuleList] is missing the required "forward" function
</code></pre>
<p>You should iterate the modules instead:</p>
<pre><code class="lang-python">out = x
for module in modules:
    out = module(out)
</code></pre>
<p>or use <code>nn.Sequential</code>:</p>
<pre><code class="lang-python">model = nn.Sequential(
    nn.Linear(10, 10),
    nn.ReLU(),
    nn.Linear(10, 10),
)
x = torch.randn(1, 10)
out = model(x)
</code></pre>
<p>or via:</p>
<pre><code class="lang-python">model = nn.Sequential(*modules)
out = model(x)
</code></pre>

Askerag: <p>Hi. I tried, and it didnt work.</p>
<pre><code class="lang-auto">import torch
from src.models.backbone.load_models import init_zoo_model
model_bb = init_zoo_model(name_model='tiny_vit_5m_224', pretrained=True, device=torch.device('cpu'))
mod = list(model_bb.children())[:-2]
mod.append(torch.nn.Linear(320, 10))
model = torch.nn.Sequential(*mod)
x = torch.randn(1, 3, 224, 224)
res = model(x)
</code></pre>

Askerag: <p>The main goal to remove 2 last layers from pretrained network:</p>
<ol>
<li>use it for finetune, so I can just replace last linear layer</li>
<li>use nn as feature extractor as part of another neural architecture.</li>
</ol>
<p>I can use this case:</p>
<pre><code class="lang-auto">model_bb = init_zoo_model(name_model='tiny_vit_5m_224', pretrained=True, device=torch.device('cpu'))
model_bb.norm_head = torch.nn.Identity()
model_bb.head = torch.nn.Linear(320, 10)
x = torch.randn(1, 3, 224, 224)
res = model_bb(x)
</code></pre>
<p>But i have no idea how to use it with proper nn class creation with forward call .</p>

Askerag: <p>This solution is working</p>
<pre><code class="lang-auto">model_bb.norm_head = torch.nn.Identity()
model_bb.head = torch.nn.Identity()

# PyTorch models inherit from torch.nn.Module
class FTNet(torch.nn.Module):
    def __init__(self):
        super(FTNet, self).__init__()
        self.features = model_bb
        self.linear = torch.nn.Linear(320, 10)

    def forward(self, x):
        x = self.features(x)
        x = self.linear(x)
        return x

model = FTNet()
x = torch.randn(1, 3, 224, 224)
res = model(x)
</code></pre>