Using pretrained 3D ResNet 50 without classifier layer

slim_dog: <p>Hello,</p>
<p>I am using 3D ResNet 50 in PyTorch for the first time and have loaded the model pretrained on Kinetics using:</p>
<p>model = torch.hub.load(‘facebookresearch/pytorchvideo’, ‘slow_r50’, pretrained=True)</p>
<p>I want to be able to extract the “video features” calculated by the model for a given input (i.e. the outputs of the last layer before the linear and pool layers that calculate the Kinetics classification). However, when I do the following, I get an output size that is the same as my input size instead of a feature vector:</p>
<p>model = torch.hub.load(‘facebookresearch/pytorchvideo’, ‘slow_r50’, pretrained=True)<br>
modules=list(model.children())[:-2]<br>
model = nn.Sequential(*modules)<br>
input = torch.ones(1, 3, 8, 256, 256) <span class="hashtag">#dummy</span> input<br>
print(model(input).shape)</p>
<p>What is the correct code for creating a new 3D ResNet 50 model with the pretrained weights that doesn’t include the last two layers? Since this model seems to have many embedded ModuleLists, I think I’m not being able to index into the right layers.</p>
<p>Thanks!</p>

ptrblck: <p>Trying to recreate a model by wrapping its internal modules into an <code>nn.Sequential</code> container assumes that <code>model.children()</code> returns modules in the exact same order they were used in the forward pass and that the actual model uses a strict sequential execution of these modules without e.g. conditions, multiple paths, concatenations etc. Both assumptions could of course easily break if you are not careful and check the implementation of the model.<br>
Instead of trying to recreate the model you could try to replace the unwanted layers with <code>nn.Identity</code> modules, which would keep the original <code>forward</code> pass, but would just return the activations for these layers.</p>

slim_dog: <p>Thanks so much! My bad for asking a potentially trivial next question, but what would be the right way to replace the last two layers in the model? The layers I want to replace are in this construction:</p>
<p>(5): ResNetBasicHead(<br>
(pool): AvgPool3d(kernel_size=(8, 7, 7), stride=(1, 1, 1), padding=(0, 0, 0))<br>
(dropout): Dropout(p=0.5, inplace=False)<br>
(proj): Linear(in_features=2048, out_features=400, bias=True)<br>
(output_pool): AdaptiveAvgPool3d(output_size=1)<br>
)</p>
<p>I would just want to change the proj and output_pool layers to identity layers.</p>

ptrblck: <p>Access the desired attributes of the model and assign new <code>nn.Identity</code> layers to these.<br>
Something like this should work:</p>
<pre><code class="lang-python">model[5].proj = nn.Identity()
model[5[.output_pool = nn.Identity()
</code></pre>
<p>assuming <code>(5): ResNetBasicHead</code> can be accessed via <code>model[5]</code>.</p>

slim_dog: <p>Hmm… the strange thing is that the model isn’t subscriptable:</p>
<blockquote>
<blockquote>
<blockquote>
<p>model[5].proj = torch.nn.Identity()</p>
</blockquote>
</blockquote>
</blockquote>
<p>Traceback (most recent call last):<br>
File “”, line 1, in <br>
TypeError: ‘Net’ object is not subscriptable</p>

ptrblck: <p>You should check the model architecture to see how modules are registered.<br>
If I print the model I see:</p>
<pre><code class="lang-python">Net(
  (blocks): ModuleList(
    (0): ResNetBasicStem(
      (conv): Conv3d(3, 64, kernel_size=(1, 7, 7), stride=(1, 2, 2), padding=(0, 3, 3), bias=False)
      (norm): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (activation): ReLU()
      (pool): MaxPool3d(kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=[0, 1, 1], dilation=1, ceil_mode=False)
    )
    (1): ResStage(
      (res_blocks): ModuleList(
        (0): ResBlock(
          (branch1_conv): Conv3d(64, 256, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)
          (branch1_norm): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (branch2): BottleneckBlock(
...
</code></pre>
<p>so, this would work:</p>
<pre><code class="lang-python">model.blocks[5].proj = nn.Identity()
model.blocks[5].output_pool = nn.Identity()
</code></pre>

slim_dog: <p>Ah I see what you’re saying! That ended up working - thanks so much for your help!</p>

Ayanokoji: <p><a class="mention" href="/u/slim_dog">@slim_dog</a>, with<br>
<code>input = torch.ones(1, 3, 8, 256, 256) #dummy input</code><br>
and</p>
<pre><code class="lang-auto">model.blocks[5].proj = nn.Identity()
model.blocks[5].output_pool = nn.Identity()
</code></pre>
<p>What is the  feature vector dimension you got?<br>
I expected dimension 1, 2048, 8; but I got 1, 2048, 1, 1, 1.</p>
<p>Can you share you code for finetuning that model?</p>