How to apply 1d convolution on data having batch size 1?

username: Noorulain_Islam
cooked: <p>hi, I want to apply 1d convolution on this input data of (128, 64). But, I am finding an error, indeed I provided the input of shape (1, 128, 64) to CNN model. Can you help me to fix?</p>
<p>import torch<br>
import torch.nn as nn</p>
<h1><a name="define-our-input-data-128-samples-each-with-a-sequence-of-64-features-1" class="anchor" href="#define-our-input-data-128-samples-each-with-a-sequence-of-64-features-1"></a>Define our input data (128 samples, each with a sequence of 64 features)</h1>
<p>input_data = torch.randn(128, 64)<br>
input_data1 = input_data.unsqueeze(0)<br>
print(input_data.shape)</p>
<h1><a name="define-the-1d-cnn-model-2" class="anchor" href="#define-the-1d-cnn-model-2"></a>Define the 1D CNN model</h1>
<p>conv1d = nn.Conv1d(in_channels=64, out_channels=32, kernel_size=3)</p>
<p>output = conv1d(input_data1)<br>
print(output.shape)<br>
print(output.unsqueeze(0).shape)</p>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in &lt;cell line: 13&gt;()<br>
11<br>
12<br>
—&gt; 13 output = conv1d(input_data)<br>
14 # Apply the convolution to the input data<br>
15 <span class="hashtag-raw">#output</span> = conv1d(input_data.unsqueeze(0))  # Add a batch dimension with .unsqueeze(0)</p>
<p>3 frames<br>
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)<br>
304                             weight, bias, self.stride,<br>
305                             _single(0), self.dilation, self.groups)<br>
 → 306         return F.conv1d(input, weight, bias, self.stride,<br>
307                         self.padding, self.dilation, self.groups)<br>
308</p>
<p>RuntimeError: Given groups=1, weight of size [32, 64, 3], expected input[1, 128, 64] to have 64 channels, but got 128 channels instead</p>

username: ptrblck
cooked: <p><code>nn.Conv1d</code> expects either a batched input in the shape <code>[batch_size, channels, seq_len]</code> or an unbatched input in the shape <code>[channels, seq_len]</code>.<br>
In your example you are using the first approach by explicitly unsqueezing the batch dimension and the <code>128</code> samples will be interpreted as the channel dimension.<br>
I’m unsure if you want to treat the input as a single sample containing <code>128</code> time steps, each with a channel dimension of <code>64</code>, but if so this should work:</p>
<pre><code class="lang-python">input1 = input1.permute(0, 2, 1).contiguous()
# input1 should now have the shape [1, 64, 128]
</code></pre>

username: Noorulain_Islam
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , I have unbatched input. In my case I have total 128 samples and every single sample has a dimension of 64. By providing the unbatched input, I have got the output of Con1D of shape (32, 126) .</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Noorulain_Islam" data-post="3" data-topic="190653">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/noorulain_islam/48/56831_2.png" class="avatar"> Noorulain_Islam:</div>
<blockquote>
<p>I have unbatched input. In my case I have total 128 samples</p>
</blockquote>
</aside>
<p>Commonly you would refer to the number of elements in the batch dimension as samples, so I assume your “samples” refer to a temporal dimension instead.</p>
<aside class="quote no-group" data-username="Noorulain_Islam" data-post="3" data-topic="190653">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/noorulain_islam/48/56831_2.png" class="avatar"> Noorulain_Islam:</div>
<blockquote>
<p>I have got the output of Con1D of shape (32, 126)</p>
</blockquote>
</aside>
<p>I assume you got this shape using my approach and this shape matches your expectation?</p>

username: Noorulain_Islam
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , yes this shape matches my expectation.</p>

username: Noorulain_Islam
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , if we have an input sample of shape (128, 64), for every single sample I want to predict its class. Do you think after applying the convolution 1d layer pooling layer is helpful? Pooling layer reduce the sample size, I want to predict exact same 128 classes.</p>

username: ptrblck
cooked: <p>After permuting the input as seen in my previous post the shape would be <code>[batch_size=1, channels=64, seq_len=128]</code>, so the batch contains a single sample with 64 channels and 128 time steps.<br>
The shape you’ve posted is again wrong as you are treating the 128 time steps as “samples”, while you’ve previously defined a batch size of 1.<br>
If you keep the shape as shown in my post as <code>[1, 64, 128]</code> you will be able to classify each time step with a target in the shape <code>[1, 128]</code> containing values in the range <code>[0, 64]</code> (assuming <code>64</code> represents the class dimension).</p>

username: Noorulain_Islam
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , I am treating it as an unbatched element. My data is not in batched. The time when I post the question, I was thinking CNN doesn’t accept unbatched data.</p>

username: ptrblck
cooked: <p>It doesn’t matter since the model will <code>unsqueeze</code> <code>dim0</code> internally if you pass an unbatched input.<br>
The shapes are still as explained before.</p>

username: J_Johnson
cooked: <aside class="quote no-group" data-username="Noorulain_Islam" data-post="6" data-topic="190653">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/noorulain_islam/48/56831_2.png" class="avatar"> Noorulain_Islam:</div>
<blockquote>
<p>Do you think after applying the convolution 1d layer pooling layer is helpful? Pooling layer reduce the sample size, I want to predict exact same 128 classes.</p>
</blockquote>
</aside>
<p>In Conv1d, a pooling layer will reduce the size of dim = -1. This typically has the effect of filtering out important vs. unimportant data from the sequences.</p>
<p><a class="mention" href="/u/noorulain_islam">@Noorulain_Islam</a> reading through the thread, it seems there may be a difference in the terms you are using and what are typically used, and I want to make sure I’m on the same page as you. Can you provide the physical description of what 128 and 64 represent in your input data size?</p>

username: Noorulain_Islam
cooked: <p><a class="mention" href="/u/j_johnson">@J_Johnson</a> , in my case, 128 are the total number of training examples (total training instances), 64 is the feature dimension for every single training example. When I pass this data to con1d I reshape it with shape (64, 128) and then pass this unbatched data to Conv1D layer to extract features. Conv1D provide me output of shape (32, 126). However, if I use padding same then then I get output (32, 128).</p>

username: J_Johnson
cooked: <p>Noted that the 0 dim represents 128 samples and the 1 dim represents 64 features.</p>
<p>My next question regards the features dim only.</p>
<p>Are these</p>
<ol>
<li>Temporally/spatially related to one another, i.e. sequential; OR</li>
<li>Are they not sequentially related to one another?</li>
</ol>
<p>Examples of sequentially related data would be stock close price over time, notes in a song, glucose levels over time, word tokens in a sentence, etc. In other words, the order is as important as the values.</p>
<p>Non-sequentially related data would be like (height, weight, hair color) of a person, (temperature, radius, luminosity, mass) of a star, etc. In other words, order of the values is not important.</p>

username: Noorulain_Islam
cooked: <p><a class="mention" href="/u/j_johnson">@J_Johnson</a> , my features are not sequentially related to one another.</p>

username: J_Johnson
cooked: <p>Based on them not being sequentially related, I would suggest you use <a href="https://pytorch.org/docs/stable/generated/torch.nn.Linear.html" rel="noopener nofollow ugc">nn.Linear</a>. The Linear layer is best suited for features that do not contain a spatial or temporal relation to one another, i.e. are non-sequential.</p>

username: Noorulain_Islam
cooked: <p>Thankyou for your suggestion.</p>