Visualizing 3D filters from r3d_18

username: Subhankar_Ghosh
cooked: <p>So I have trained an r3d_18 model on a video dataset. I want to visualize the filters of the first convolution layer.<br>
I tried this:<br>
<code>model.state_dict().keys()</code><br>
This gives:</p>
<blockquote>
<p>odict_keys([‘stem.0.weight’, ‘stem.1.weight’, ‘stem.1.bias’, ‘stem.1.running_mean’, ‘stem.1.running_var’, ‘stem.1.num_batches_tracked’, ‘layer1.0.conv1.0.weight’, ‘layer1.0.conv1.1.weight’, ‘layer1.0.conv1.1.bias’, ‘layer1.0.conv1.1.running_mean’, ‘layer1.0.conv1.1.running_var’, ‘layer1.0.conv1.1.num_batches_tracked’, ‘layer1.0.conv2.0.weight’, ‘layer1.0.conv2.1.weight’, ‘layer1.0.conv2.1.bias’, ‘layer1.0.conv2.1.running_mean’, ‘layer1.0.conv2.1.running_var’, ‘layer1.0.conv2.1.num_batches_tracked’, ‘layer1.1.conv1.0.weight’, ‘layer1.1.conv1.1.weight’, ‘layer1.1.conv1.1.bias’, ‘layer1.1.conv1.1.running_mean’, ‘layer1.1.conv1.1.running_var’, ‘layer1.1.conv1.1.num_batches_tracked’, ‘layer1.1.conv2.0.weight’, ‘layer1.1.conv2.1.weight’, ‘layer1.1.conv2.1.bias’, ‘layer1.1.conv2.1.running_mean’, ‘layer1.1.conv2.1.running_var’, ‘layer1.1.conv2.1.num_batches_tracked’, ‘layer2.0.conv1.0.weight’, ‘layer2.0.conv1.1.weight’, ‘layer2.0.conv1.1.bias’, ‘layer2.0.conv1.1.running_mean’, ‘layer2.0.conv1.1.running_var’, ‘layer2.0.conv1.1.num_batches_tracked’, ‘layer2.0.conv2.0.weight’, ‘layer2.0.conv2.1.weight’, ‘layer2.0.conv2.1.bias’, ‘layer2.0.conv2.1.running_mean’, ‘layer2.0.conv2.1.running_var’, ‘layer2.0.conv2.1.num_batches_tracked’, ‘layer2.0.downsample.0.weight’, ‘layer2.0.downsample.1.weight’, ‘layer2.0.downsample.1.bias’, ‘layer2.0.downsample.1.running_mean’, ‘layer2.0.downsample.1.running_var’, ‘layer2.0.downsample.1.num_batches_tracked’, ‘layer2.1.conv1.0.weight’, ‘layer2.1.conv1.1.weight’, ‘layer2.1.conv1.1.bias’, ‘layer2.1.conv1.1.running_mean’, ‘layer2.1.conv1.1.running_var’, ‘layer2.1.conv1.1.num_batches_tracked’, ‘layer2.1.conv2.0.weight’, ‘layer2.1.conv2.1.weight’, ‘layer2.1.conv2.1.bias’, ‘layer2.1.conv2.1.running_mean’, ‘layer2.1.conv2.1.running_var’, ‘layer2.1.conv2.1.num_batches_tracked’, ‘layer3.0.conv1.0.weight’, ‘layer3.0.conv1.1.weight’, ‘layer3.0.conv1.1.bias’, ‘layer3.0.conv1.1.running_mean’, ‘layer3.0.conv1.1.running_var’, ‘layer3.0.conv1.1.num_batches_tracked’, ‘layer3.0.conv2.0.weight’, ‘layer3.0.conv2.1.weight’, ‘layer3.0.conv2.1.bias’, ‘layer3.0.conv2.1.running_mean’, ‘layer3.0.conv2.1.running_var’, ‘layer3.0.conv2.1.num_batches_tracked’, ‘layer3.0.downsample.0.weight’, ‘layer3.0.downsample.1.weight’, ‘layer3.0.downsample.1.bias’, ‘layer3.0.downsample.1.running_mean’, ‘layer3.0.downsample.1.running_var’, ‘layer3.0.downsample.1.num_batches_tracked’, ‘layer3.1.conv1.0.weight’, ‘layer3.1.conv1.1.weight’, ‘layer3.1.conv1.1.bias’, ‘layer3.1.conv1.1.running_mean’, ‘layer3.1.conv1.1.running_var’, ‘layer3.1.conv1.1.num_batches_tracked’, ‘layer3.1.conv2.0.weight’, ‘layer3.1.conv2.1.weight’, ‘layer3.1.conv2.1.bias’, ‘layer3.1.conv2.1.running_mean’, ‘layer3.1.conv2.1.running_var’, ‘layer3.1.conv2.1.num_batches_tracked’, ‘layer4.0.conv1.0.weight’, ‘layer4.0.conv1.1.weight’, ‘layer4.0.conv1.1.bias’, ‘layer4.0.conv1.1.running_mean’, ‘layer4.0.conv1.1.running_var’, ‘layer4.0.conv1.1.num_batches_tracked’, ‘layer4.0.conv2.0.weight’, ‘layer4.0.conv2.1.weight’, ‘layer4.0.conv2.1.bias’, ‘layer4.0.conv2.1.running_mean’, ‘layer4.0.conv2.1.running_var’, ‘layer4.0.conv2.1.num_batches_tracked’, ‘layer4.0.downsample.0.weight’, ‘layer4.0.downsample.1.weight’, ‘layer4.0.downsample.1.bias’, ‘layer4.0.downsample.1.running_mean’, ‘layer4.0.downsample.1.running_var’, ‘layer4.0.downsample.1.num_batches_tracked’, ‘layer4.1.conv1.0.weight’, ‘layer4.1.conv1.1.weight’, ‘layer4.1.conv1.1.bias’, ‘layer4.1.conv1.1.running_mean’, ‘layer4.1.conv1.1.running_var’, ‘layer4.1.conv1.1.num_batches_tracked’, ‘layer4.1.conv2.0.weight’, ‘layer4.1.conv2.1.weight’, ‘layer4.1.conv2.1.bias’, ‘layer4.1.conv2.1.running_mean’, ‘layer4.1.conv2.1.running_var’, ‘layer4.1.conv2.1.num_batches_tracked’, ‘fc.weight’, ‘fc.bias’])</p>
</blockquote>
<p>I think the first layers weights would be ‘layer1.0.conv1.0.weight’.</p>
<p>I did that using:<br>
<code>cnn_weights = model.state_dict()['layer1.0.conv1.0.weight'].cpu()</code></p>
<p>The shape:<br>
<code>cnn_weights.shape</code></p>
<p>Result:</p>
<blockquote>
<p>torch.Size([64, 64, 3, 3, 3])</p>
</blockquote>
<p>Now how do I plot this? Both XY and XT.</p>

username: ptrblck
cooked: <p>You could iterate all input channels, filters, and the depth of each filter to visualize the <code>3x3</code> kernel e.g. via <code>matplotlib.pyplot.imshow</code>.<br>
I’m not aware of any good techniques to visualize 4D kernels.</p>