Understanding Convolution 1D output and Input

Rajesh_Karmakar: <p>Hi,</p>
<p>I have input of dimension 32 x 100 x 1 where 32 is the batch size.</p>
<p>I wanted to convolved over 100 x 1 array in the input for each of the 32 such arrays i.e. a single data point in the batch has an array like that.</p>
<p>I hoped that conv1d(100, 100, 1) layer will work.</p>
<p>How does this convolves over the array ? How many filters are created? Does this convolve over 100 x 1 dimensional array? or is a filter created for each of the 100 dimensions?</p>
<p>There are so many terminologies I found searching for this function Filter, Length, Channel_in Channel out.</p>
<p>Any help in explaining what the above layer does to the above output will be of great help.</p>
<p>Cheers!</p>

krishnavishalv: <p>In your example of conv1d(100, 100, 1).</p>
<p><code>in_channels = 100</code><br>
<code>out_channels = 100</code><br>
<code>kernel_size = 1</code><br>
By default <code>stride = 1</code>.</p>
<p>100 filters are created and it does convolve over a 100x1 dimensional array. A filter is created for each of the 100 dimensions. Number of filters = Number of <code>out_channels</code>.</p>

Rajesh_Karmakar: <p>if each of the 100 filters convolves over 100x1 dimensional array, then there should be 100x 100 outputs right? what will be the output size? and what is the filter size?</p>

krishnavishalv: <p>execute the following code.</p>
<pre><code class="lang-python">import torch
from torch import nn

a = torch.randn(32, 100, 1)  
m = nn.Conv1d(100, 100, 1) 
out = m(a)
print(out.size())
print(m)
</code></pre>
<p>See the link below for more clear explanation.</p>
<p><a href="https://pytorch.org/docs/stable/nn.html#torch.nn.Conv1d" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.html#torch.nn.Conv1d</a></p>

Rajesh_Karmakar: <p>thanks, i am not getting hwo the convolutions are done: if each of the 100 filters convolves over 100x1 dimensional array, then there should be 100x 100 outputs right?</p>

ptrblck: <p>Well, not really. Currently you are using a signal of shape <code>[32, 100, 1]</code>, which corresponds to <code>[batch_size, in_channels, len]</code>.<br>
Each kernel in your conv layer creates an output channel, as <a class="mention" href="/u/krishnavishalv">@krishnavishalv</a> explained, and convolves the “temporal dimension”, i.e. the <code>len</code> dimension.<br>
Since <code>len</code> is in your case set to 1, there won’t be much to convolve, as you basically passed a single time stamp with 100 channels.<br>
Try to think about your signal as a sound source. In a simple use case you would have 2 channels (left and right) and a certain length, e.g. 1000 time stamps. Your input would thus have the shape <code>[batch_size, 2, 1000]</code>.<br>
Now if you setup a conv layer, you would have to use <code>in_channels=2</code> and an arbitrary number of <code>out_channels</code>. Remember, the <code>out_channels</code> just define the number of kernels. Each kernel is applied separately on the input.<br>
The kernel size defines, how much of the temporal dimension is used in a sliding window fashion.<br>
E.g. if you set <code>kernel_size=5</code>, 5 time stamps will be used for the convolution for each position.</p>
<p>In your use case, however, we only have one single time stamp,  so that you could easily use a linear layer instead.</p>
<p><a href="http://cs231n.github.io/convolutional-networks/">CS231n</a> explains this concept really well.</p>

smu226: <aside class="quote no-group" data-username="krishnavishalv" data-post="4" data-topic="30764">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/krishnavishalv/48/1756_2.png" class="avatar"> krishnavishalv:</div>
<blockquote>
<p>a = torch.randn(32, 100, 1) m = nn.Conv1d(100, 100, 1) out = m(a) print(out.size()) print(m)</p>
</blockquote>
</aside>
<p>I tried this, but I am getting this error <code>TypeError: argument 0 is not a Variable</code> Do you know what can be the cause? Thank you!</p>

InnovArul: <p>I guess you are using pytorch &lt;= 0.4.</p>
<p>Try to wrap the tensors with autograd.Variable.</p>
<pre><code class="lang-auto">from torch.autograd import Variable
input = Variable(input)
</code></pre>

shlomiamitai: <p>following their documantation, the arithmetic is accumulating the convolution results of all channels, which is weird…</p>

caglar_demir: <p>The following chunk of code might complement the answer of <a class="mention" href="/u/ptrblck">@ptrblck</a> .</p>
<pre><code class="lang-auto">import torch
from torch import nn
import numpy as np
np.random.seed(1)

def convolve_slice(X,W,b,stride=2):
    xH,xW=X.shape
    wH,wW=W.shape
    vH,vW= (xH-wH)//stride +1,(xW-wW)//stride +1    
    assert vH==vW
    assert wH==wW
    k=wH
    v=np.zeros((vH,vW)) # output shape
    for i in range(vH):
        for j in range(vH):
            v[i,j]=np.sum(X[stride*i:(stride*i)+k,stride*j:(stride*j)+k] *W) +b    
    return v


xH,xW=7,7
stride=2
x=np.random.randn(xH,xW)

# show it if you will.
#plt.imshow(x, interpolation='none')
#plt.show()


k = 3 # kernel/filter shape.
conv = nn.Conv1d(in_channels=1,
                 kernel_size=(k, k),
                 out_channels=1, stride=stride)
# from numpy to torch
x_torch = torch.from_numpy(x)
x_torch = x_torch.float()
x_torch = x_torch.view(1, 1, xH, xW)
print(conv(x_torch))

w0 = conv.weight.data.numpy()
b0 = conv.bias.data.numpy()
w0 = w0.reshape(k, k)

m=convolve_slice(x,w0,b0,2)
print(m)
</code></pre>

Alice_NL: <p>Dear Piotr,</p>
<p>I read your detailed reply for this example but I am still “lost in dimentions”. I would like to try my luck and ask you to help me out.</p>
<p>My data is about patients, it has a shape of 239 (number of patients), 49 (rows per each patient that is time stamps), 5 (features).  The output is a binary label - one per patient, thus the shape is 239.</p>
<p>I assume for my case I should use Conv1d. I am then lost with what is what… as per your reply I start thinking that number of in-channels is the number of features in my case and the length is 49 (rows). If that is right does it mean I have to somehow transpose the tensor so that my shape is 239,5,49? I am getting lost here.</p>
<p>Thank you in advance for any comments…</p>
<p>Best regards,<br>
Alice</p>

Nikronic: <p>Hi,</p>
<p>You are right. Convolution operation works on spatial/temporal data (in our examples) and you can think of your data in this way, that you have 5 features for each time stamp, not 5 time staps for each feature.</p>
<p>A better way is to assume that your input data is the output of another operation, in this case, 5 features corresponds to 5 different kernels that recorded features for all timestamps. When you are defining conv layer, you need to specify output channel which eactly could have the same definition as 5 features.</p>
<p>I do not know you are familiar with image or not, but for a 2D image (1D can be true to but does not make much sense), then shape of input would be <code>[number of images, channel, height, width]</code> which for 1 RGB image would be <code>[1, 3, 10, 10]</code>. 3 corresponds to Red, Green and Blue, same as your 5 features. Then 10x10 is the spatial domain, which in your case, it is a 1D temporal domain. So, it could be like <code>[1, 3, 10]</code>.</p>
<p>You can transpose the dimensions using <code>tensor.permute()</code>.</p>
<p>Bests</p>

Alice_NL: <p>Dear Doosti and all,</p>
<p>First of all, thank you for your attention to my question and your time to answer me.<br>
I will now describe my understanding, please correct me if I am still not getting it:</p>
<p>I need to first transpose my current tensor (239 groups each containing 49 rows/time stamps having 5 columns/features)</p>
<p>into a tensor of</p>
<p>5 rows (features) having 49 * 239 = 11 711 columns/time stamps (that represent 239 groups each having 49 columns)?</p>
<p>Then, as I wish to use 1 patient per batch (239 patients), for the batching I should slice the tensor 238 times, meaning that each batch dimension will be 5 (rows) x 49 (columns) and I will have 239 of these batches.</p>
<p>In this case my understanding is that the Conv1d would be:</p>
<p>torch.nn. Conv1d( in_channels: <strong>5</strong> (features/rows), out_channels: <strong>5</strong>, kernel_size: I <strong>can try different numbers here, the kernel will be sliding over 49 time stamps in each out of 239 batches</strong>, stride: <strong>I can change, it is a kernel “step”</strong>, padding: <strong>depending on the kernel size might use padding</strong>, dilation: <strong>another hyperparameter I might tune</strong>, groups: <strong>not applicable for me</strong>, bias: bool = True, padding_mode: str = ‘zeros’* )</p>
<p>Thank you in advance for any comments and your reply, if any.</p>
<p>Sincerely,<br>
Alice</p>

Nikronic: <p>Yes, you are absolutely correct.</p>
<p>Here is a snippet for your case:</p>
<pre><code class="lang-auto">x = torch.ones((239, 49, 5))
x = x.permute((0, 2, 1))
print(x.shape)
model = nn.Conv1d(5, 5, 3)
output = model(x)
print(output.shape)
</code></pre>
<p>If you want to know a little more what happens in each channel, you can play with above example like this:</p>
<pre><code class="lang-auto">x = torch.ones((239, 49, 5))
x = x.permute((0, 2, 1))  # permuate feature and temporal channels
print(x.shape)
model = nn.Conv1d(5, 5, 49, groups=5, bias=False)  # remove bias and set window same as whole sequence of temporal data
nn.init.constant_(model.weight, 1.)  # set kernel to one (instead of random)
output = model(x)
print(output.shape)
</code></pre>