Conv1d or Conv2d for 1d vectors with batch size?

username: Peter_Phan
cooked: <p>I know this is not new, however, after reading many explanations I am still really confused about the parameters which are required for Conv1D. Specifically, I have a dataset which contains 154k rows, and each rows is a 1D array of 289 floats. Now I want to train my model using batches with batch size = 50 (this is dynamic). In my model, there are some other type of layers such as Linears. The source code like this:</p>
<pre><code>    self.linear1 = nn.Linear(noise_dim, sequenceLength)
    self.linear2 = nn.Linear(sequenceLength, 2*sequenceLength)
    self.linear3 = nn.Linear(2*sequenceLength, 2*sequenceLength)
    self.linear4 = nn.Conv1d(2*sequenceLength, sequenceLength,kernel_size=sequenceLength+1, stride=1, padding=0)
</code></pre>
<p>I expect to use Conv1D to reduce the size of input from 2*SequenceLength to a SequenceLength. However, it display the error of (289 is my sequenceLength in execution)</p>
<pre><code>    Given groups=1, weight of size [289, 578, 290], expected input[1, 50, 578] to have 578 channels, but got 50 channels instead
</code></pre>
<p>It took my batch size into account, if so, in the training section, if I need to change the batch size, then we need to upgrade our models?<br>
Moreover, if taking the batch size together with the input, I think it should be Conv2d, am I right?</p>
<p>Thank you</p>

username: ptrblck
cooked: <p><code>nn.Conv1d</code> expects a batched 3-dimensional input in the shape <code>[batch_size, in_channels, seq_length]</code> or an unbatched 2-dimensional input in the shape <code>[in_channels, seq_length]</code>. Based on the error message you are using the second approach and the layer unsqueezes the batch dimension for you. Note that conv layers use a kernel and apply a convolution (or rather cross-correlation) on the input. To simply reduce the dimension of the input you could still apply a linear layer. In case your activation has a temporal dimension and a kernel should be applied over it you might want to <code>unsqueeze</code> the channel dimension manually.</p>

username: ptrblck
cooked: <p>The <code>in_channels</code> argument should be a single <code>int</code> value not a <code>list</code>.</p>

username: Peter_Phan
cooked: <p>I think I got your idea, thank you very much</p>

username: Peter_Phan
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
This is my new update</p>
<pre><code>def __init__(self, noise_dim, sequenceLength):
    super(Generator,self).__init__()
    
    self.linear1 = nn.Linear(noise_dim, sequenceLength)
    self.linear2 = nn.Linear(sequenceLength, 2*sequenceLength)
    self.linear3 = nn.Linear(2*sequenceLength, 2*sequenceLength)
    self.conv4 = nn.Conv1d(2*sequenceLength, sequenceLength,kernel_size=3, padding=0)
    self.lrelu = nn.LeakyReLU(0.01)
    self.tanh = nn.Tanh()
    self.sigmoid = nn.Sigmoid()

def forward(self, x):
    out = self.linear1(x)
    out = self.lrelu(out)
    out = self.linear2(out)
    out = self.tanh(out)
    out = self.linear3(out)
    out = self.lrelu(out)
    out = out.unsqueeze(2)
    out = self.conv4(out)
    out = out.squeeze()
    out = self.sigmoid(out)
    return out
</code></pre>
<p>I add unsqueeze(2) in the forward functions and it can work fine except for the kernel size.<br>
If I increase the kernel size to any number greater than 1, I will receive the error:</p>
<pre><code class="lang-auto">Calculated padded input size per channel: (1). Kernel size: (3). Kernel size can't be greater than actual input size
</code></pre>
<p>I wonder my sequence length is 289 (in execution), with 1 dimension, why the kernel cannot be some thing like 3 or 4,… so on.</p>

username: ptrblck
cooked: <p>Because you are unsqueezing the temporal dimension and use the sequence length as the channels. Check the expected input shape described in my previous post.</p>

username: Peter_Phan
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , I change the padding to 1, and it work (with kernel = 3).<br>
But according to your answer, if the sequence turn into the channels, I wonder the if the kernel keep working correctly or not? Or it only return without stride?<br>
Thank you</p>

username: ptrblck
cooked: <p>Using a padded input would avoid running into functionality issues, but note that only the “center” pixel of the conv kernel will be used as seen here:</p>
<pre><code class="lang-python">conv = nn.Conv1d(10, 1, kernel_size=3, padding=1, bias=False)
x = torch.ones(1, 10, 1)

out = conv(x)
print(out)
# tensor([[[0.7653]]], grad_fn=&lt;ConvolutionBackward0&gt;)
print(conv.weight[:, :, 1].sum())
# tensor(0.7653, grad_fn=&lt;SumBackward0&gt;)
</code></pre>