Correct way of using Conv1d on time series data

trebor_latam: <p>Hi everyone, i am pretty new in the Pytorch world, and in 1D convolution. I am working with some time series data, and i am trying to make a convolutive neural network that predicts the next value, given a window size of for example 10. So i want my model to train so that  given 10 time steps in input, it predicts the next value at time step t+1. To begin i started with a simple toy dataset, generating 400 datapoints of a sine curve and windowing the data so that the format should look like this.</p>
<p>input window:</p>
<p>tensor([[-0.4532],<br>
[ 0.9988],<br>
[ 2.3085],<br>
[ 3.2892],<br>
[ 3.8014],<br>
[ 3.7718],<br>
[ 3.2049],<br>
[ 2.1813],<br>
[ 0.8469],<br>
[-0.6081]])</p>
<p>output:<br>
tensor([-1.9765])</p>
<p>Considering that this is an univariate time series, window lenght of 10 and 390 (400-10) data to train with, in order to use the convolution in the appropriate way, what should i put in the parameters in the Conv1D function? The model i created gives really good results, but my issue is that i think that maybe it is just because the dataset is very simple and he is not actually using the convolution but just learning from each point individually.</p>
<p>My model is this:</p>
<pre><code>class CNN1d(nn.Module):
    def __init__(self, input_size, output_size):
        super(CNN1d, self).__init__()

        self.conv1 = nn.Conv1d(input_size, 64, kernel_size=3, stride=1, padding=2)
        self.apply(init_params)  #function to initiate model parameters
        self.pool1 = nn.MaxPool1d(2)
        self.conv2 = nn.Conv1d(64,64,kernel_size=3, stride = 1, padding=1)

        self.fc1 = nn.Linear(64, output_size)
        self.fc2 = nn.Linear(1, 1)

    def forward(self, x):
        in_size1 = x.size(0)  # one batch
        x = F.relu(self.conv1(x))
        x = self.pool1(x)
        x = F.relu(self.conv2(x))
        x = x.view(in_size1, -1)  # flatten the tensor
        x = self.fc1(x)
        output = self.fc2(x)

        return output
</code></pre>
<p>And this is how i train it:</p>
<p>loss_fn = nn.MSELoss()</p>
<p>EPOCH = 300<br>
train_losses = []  # Collect training losses</p>
<pre><code>for epoch in range(EPOCH):

    train_pred = cnn1d(X)
    train_loss = loss_fn(train_pred, y)
    train_losses.append(train_loss)

    optimizer.zero_grad()
    train_loss.backward()
    optimizer.step()
</code></pre>
<p>The shape of the X i am passing is:<br>
torch.Size([390, 10, 1])<br>
where 390 is the dimension of the training set, 10 is the time steps, which i interpreted as the window size and 1 is the number of features, as i am using a univariate time series.</p>
<p>The output_size in the end is 1 as i am trying to predict a single value, but what should the input size be? should it be 1 as there is only one single variable or 10 which is the size of the window on which i am training? And how is the convoution moving across my data?<br>
For conext, my model does very well in predicting the values in the time steps of the test sets, which are the continuation of the training ones, and i use as input_size the size of the window, but is this correct?<br>
I am sorry if my questions might seem trivial, but i am having some difficulties understanding why and how this works</p>

googlebot: <p>It is incorrect to do unrolling manually, kernel_size is like window size.</p>
<p>Conv1d(1,ch,kernel_size=10) on tensor [batch=1, channels=1, time=400]</p>
<p>does the same as</p>
<p>nn.Linear(10,ch) on an unrolled tensor [time=390,window=10]</p>
<p>You can sometimes interpret such inputs as [batch=390, channels=1, time=10], but it is not the usual case, esp. with overlapping.</p>

trebor_latam: <p>Thank you for the reply,<br>
Unfortunately I did not understand clearly what you said. So as i was afraid i am not convolving in the right way over the dataset right? But in time series forecasting, i saw in every tutorial that the data should be split in windows so that you use the first n samples to predict the sample at time n+1, is the data format wrong too? Sorry for bothering, but could you tell me how should i organize my data and set the convolution parameters in order to perform the convolution correctly?</p>

googlebot: <p>You’d create data windows for sampling purposes, but Conv &amp; RNN inputs have no “window” dimension - it is just “time”.</p>
<p>With your toy problem, different batching/sampling schemes would work, but general approach would be:</p>
<ol>
<li>sample batch_size sequences of length t</li>
<li>use input shape [batch, channels_in=1, t]</li>
<li>in general, you can predict multiple values in parallel (one step into the future), and this is the natural approach with convolutions. but it is possible to use many-to-one mode too (keywords: dilated causal convolutions).</li>
</ol>

trebor_latam: <p>Thank you, i think i understood.</p>