Implement a 3D custom model

username: robsver
cooked: <p>Hi everyone, I would like to implement a 3D, single channel, PyTorch model having two 3D convolutions, followed by two linear layers, independently from the batch size and shape of the input volume.</p>
<p>Any suggestions?</p>
<p>Thanks</p>

username: J_Johnson
cooked: <p>Something like this is similar to what you’re asking:</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/xmuyzz/3D-CNN-PyTorch/blob/master/models/C3DNet.py">
  <header class="source">

      <a href="https://github.com/xmuyzz/3D-CNN-PyTorch/blob/master/models/C3DNet.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/xmuyzz/3D-CNN-PyTorch/blob/master/models/C3DNet.py" target="_blank" rel="noopener nofollow ugc">xmuyzz/3D-CNN-PyTorch/blob/master/models/C3DNet.py</a></h4>


      <pre><code class="lang-py">import math
import torch
import torch.nn as nn
import torch.nn.init as init
import torch.nn.functional as F
from torch.autograd import Variable
from functools import partial


class C3D(nn.Module):

    """
    This is the c3d implementation with batch norm.

    [1] Tran, Du, et al. "Learning spatiotemporal features with 3d convolutional networks."
    Proceedings of the IEEE international conference on computer vision. 2015.
    """

    def __init__(self, sample_size, sample_duration, num_classes=600, in_channels=1):

</code></pre>



  This file has been truncated. <a href="https://github.com/xmuyzz/3D-CNN-PyTorch/blob/master/models/C3DNet.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Except that has more than 2 channels and some added pooling/dropout/batchnorm.</p>
<p>Should help you on the right track.</p>

username: robsver
cooked: <p>Thank you. Only one question what are <code>sample_duration</code> and <code>sample_size</code> ?</p>

username: J_Johnson
cooked: <p>Not my code, but that is used for calculating the transition between the final conv layer and the first Linear layer. However, you can just use <code>nn.LazyLinear</code>:<br>
<a href="https://pytorch.org/docs/stable/generated/torch.nn.LazyLinear.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.LazyLinear.html</a></p>
<p>Just specify the <code>out_features</code> you want and it will figure out the input size during the first run.</p>

username: robsver
cooked: <p>Thanks a lot <a class="mention" href="/u/j_johnson">@J_Johnson</a> !<br>
The LazyLinear was really useful.</p>
<p>I wrote this snippet of code (batch = 2, 1 channel, 130x130x130 image) and I expected to get a tensor having shape (2,3), but I got a (2,64,64,64,3).</p>
<p>What am I missing?<br>
Thanks a lot.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F

class CustomNet(nn.Module):
    def __init__(self):
        super(CustomNet, self).__init__()
        self.seq1 = nn.Sequential(
            nn.Conv3d(in_channels=1, out_channels=64, kernel_size=3, stride=1),
            nn.BatchNorm3d(64),
            nn.ReLU(),
            nn.MaxPool3d(kernel_size=(2,2,2))
        )
        self.l1 = nn.LazyLinear(out_features=64, bias=False)
        self.l2 = nn.Linear(64, 3, bias=False)

    def forward(self, x):
        x1 = self.seq1(x)
        x2 = self.l1(x1)
        x3 = self.l2(x2)

        return x3

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
fake_input = torch.randn((2,1,130,130,130)).to(device) # Batch = 2, 1 channel, 130x130x130 image

model = CustomNet().to(device)

outputs = model(fake_input)

print(outputs.size())
</code></pre>

username: J_Johnson
cooked: <p>This <a href="https://discuss.pytorch.org/t/size-mismatch-error-in-cnn-fc-layer/174553/3">post</a> has the output size calculation from the docs written as a Python function. You can enter in the arguments on one dimension for your Conv3d and MaxPool3d as well as input size on one dimension to determine what the output size is you should expect from that layer, given a certain input size.</p>

username: robsver
cooked: <p>Thanks <a class="mention" href="/u/j_johnson">@J_Johnson</a>, I really appreciate your reply. I solved by adding a Flatten before the LazyLinear layer to my custom model.</p>
<p>Have a nice day.</p>