Pretrained Resnet3d giving reversed model

username: Dhruv_Jain
cooked: <p>Hi,<br>
I am trying to use a pretrained model of resnet3d on my gpu. But somehow it is getting reversed. I am not sure how this is possible. I am using</p>
<p><code>self.resnet3d = torch.hub.load('facebookresearch/pytorchvideo', 'slow_r50', pretrained=True)</code></p>
<p>And I am getting,</p>
<blockquote>
<p>Net(<br>
(blocks): ModuleList(<br>
(0): ResNetBasicStem(<br>
(pool): MaxPool3d(kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=[0, 1, 1], dilation=1, ceil_mode=False)<br>
(activation): ReLU()<br>
(norm): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv): Conv3d(3, 64, kernel_size=(1, 7, 7), stride=(1, 2, 2), padding=[0, 3, 3], bias=False)<br>
)<br>
(1): ResStage(<br>
(res_blocks): ModuleList(<br>
(0): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(64, 256, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(64, 64, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(64, 64, kernel_size=(1, 1, 1), stride=(1, 1, 1), padding=[0, 0, 0], bias=False)<br>
)<br>
(branch1_norm): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(branch1_conv): Conv3d(64, 256, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
)<br>
(1): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(64, 256, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(64, 64, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(256, 64, kernel_size=(1, 1, 1), stride=(1, 1, 1), padding=[0, 0, 0], bias=False)<br>
)<br>
)<br>
(2): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(64, 256, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(64, 64, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(256, 64, kernel_size=(1, 1, 1), stride=(1, 1, 1), padding=[0, 0, 0], bias=False)<br>
)<br>
)<br>
)<br>
)<br>
(2): ResStage(<br>
(res_blocks): ModuleList(<br>
(0): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(128, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(128, 128, kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(256, 128, kernel_size=(1, 1, 1), stride=(1, 1, 1), padding=[0, 0, 0], bias=False)<br>
)<br>
(branch1_norm): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(branch1_conv): Conv3d(256, 512, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False)<br>
)<br>
(1): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(128, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(128, 128, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(512, 128, kernel_size=(1, 1, 1), stride=(1, 1, 1), padding=[0, 0, 0], bias=False)<br>
)<br>
)<br>
(2): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(128, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(128, 128, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(512, 128, kernel_size=(1, 1, 1), stride=(1, 1, 1), padding=[0, 0, 0], bias=False)<br>
)<br>
)<br>
(3): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(128, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(128, 128, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(512, 128, kernel_size=(1, 1, 1), stride=(1, 1, 1), padding=[0, 0, 0], bias=False)<br>
)<br>
)<br>
)<br>
)<br>
(3): ResStage(<br>
(res_blocks): ModuleList(<br>
(0): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(256, 1024, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(256, 256, kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(512, 256, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
(branch1_norm): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(branch1_conv): Conv3d(512, 1024, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False)<br>
)<br>
(1): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(256, 1024, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(256, 256, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(1024, 256, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
)<br>
(2): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(256, 1024, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(256, 256, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(1024, 256, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
)<br>
(3): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(256, 1024, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(256, 256, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(1024, 256, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
)<br>
(4): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(256, 1024, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(256, 256, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(1024, 256, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
)<br>
(5): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(256, 1024, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(256, 256, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(1024, 256, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
)<br>
)<br>
)<br>
(4): ResStage(<br>
(res_blocks): ModuleList(<br>
(0): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(512, 2048, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(512, 512, kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(1024, 512, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
(branch1_norm): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(branch1_conv): Conv3d(1024, 2048, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False)<br>
)<br>
(1): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(512, 2048, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(512, 512, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(2048, 512, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
)<br>
(2): ResBlock(<br>
(activation): ReLU()<br>
(branch2): BottleneckBlock(<br>
(norm_c): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_c): Conv3d(512, 2048, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)<br>
(act_b): ReLU()<br>
(norm_b): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_b): Conv3d(512, 512, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), bias=False)<br>
(act_a): ReLU()<br>
(norm_a): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv_a): Conv3d(2048, 512, kernel_size=(3, 1, 1), stride=(1, 1, 1), padding=[1, 0, 0], bias=False)<br>
)<br>
)<br>
)<br>
)<br>
(5): ResNetBasicHead(<br>
(output_pool): AdaptiveAvgPool3d(output_size=1)<br>
(proj): Linear(in_features=2048, out_features=400, bias=True)<br>
(dropout): Dropout(p=0.5, inplace=False)<br>
(pool): AvgPool3d(kernel_size=(8, 7, 7), stride=(1, 1, 1), padding=(0, 0, 0))<br>
)<br>
)<br>
)</p>
</blockquote>
<p>As you can see the Convolutions have been reversed ie(conv_c, conv_b, conv_a) along with the batchnorm and while in google collab it is coming straight forward.<br>
Can someone please help me out.<br>
Thanks</p>

username: ptrblck
cooked: <p>The order of execution is defined in the <code>forward</code> method of each module while printing the model should print the modules in the order of their initialization (at least that’s what I’ve seen so far).<br>
Are you seeing the <code>forward</code> pass “reversed” or just the printing of the submodules?</p>

username: Dhruv_Jain
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> for your reply. Yes the forward is working correctly. But I don’t understand that in the google collab it is getting initialised the right way while in my gpu it is getting reversed. How can that be possible as I am using importing the model the same way.</p>

username: ptrblck
cooked: <p>I don’t know as I’m not familiar enough with the order of initialization of Python objects. I would assume they are created in the same order as used in the <code>__init__</code> method and that’s what I’ve seen in all my setups so far. However, based on your description it seems as if that’s not the case for your setups so check the Python version etc. to narrow down if some internal init orders were changed.</p>