Convert data to serve as CNN input

Pablo_Garcia_Sant: <p>The problem I am facing is as follows:</p>
<p>I have a dataset with time series of the following type:</p>
<pre><code>datetime;Accelerometer1RMS;Accelerometer2RMS;Current;Pressure;Temperature;Thermocouple;Voltage;Volume Flow RateRMS
2020-03-09 10:14:33;0.026587799999999998;0.0401113;1.3302;0.054711;79.3366;26.0199;233.062;32.0
2020-03-09 10:14:34;0.0261697;0.040452499999999995;1.35399;0.38263800000000003;79.5158;26.0258;236.04;32.0
</code></pre>
<p>For reasons of the project I am working on, I need this data to serve as input for a convolutional network, which is:</p>
<pre><code>class ConvAutoencoder(nn.Module):
    def __init__(self):
        super(ConvAutoencoder, self).__init__()

        # Encoder
        self.conv1 = nn.Conv2d(3, 16, 3, padding=1) 
        self.conv2 = nn.Conv2d(16, 4, 3, padding=1)
        self.pool = nn.MaxPool2d(2, 2)
        # Decoder
        self.t_conv1 = nn.ConvTranspose2d(4, 16, 2, stride=2)
        self.t_conv2 = nn.ConvTranspose2d(16, 3, 2, stride=2)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = self.pool(x)
        x = F.relu(self.conv2(x))
        x = self.pool(x)
        x = F.relu(self.t_conv1(x))
        x = F.sigmoid(self.t_conv2(x))

        return x
</code></pre>
<p>I guess I have to make matrices of size nx8 to feed into the neural network. But I have doubts on how to convert the data to make this possible as I am somewhat new to pytorch.</p>
<p>Thanks for your attention!</p>

ptrblck: <p>You could try to use <code>pandas</code> to read the data and create the corresponding tensor.<br>
Here is a small example using your time series data:</p>
<pre><code class="lang-python">import pandas as pd

data = '''datetime;Accelerometer1RMS;Accelerometer2RMS;Current;Pressure;Temperature;Thermocouple;Voltage;Volume Flow RateRMS
2020-03-09 10:14:33;0.026587799999999998;0.0401113;1.3302;0.054711;79.3366;26.0199;233.062;32.0
2020-03-09 10:14:34;0.0261697;0.040452499999999995;1.35399;0.38263800000000003;79.5158;26.0258;236.04;32.0'''

with open("tmp.csv", "w") as f:
    f.write(data)
    
dataframe = pd.read_csv("tmp.csv", delimiter=";")
print(dataframe["Accelerometer1RMS"])
</code></pre>

Pablo_Garcia_Sant: <p>Hi! Thanks for answering, so kind. My finally approach was the following: I divide my df in subarrays of a size, for example 32. Then I initialize a tensor of size (32,8), (8 is the number of features) and I assign the values:</p>
<pre><code class="lang-auto">zeros = torch.zeros(32,8)
for i in range (0,31):
    for j in range (0,8):
        zeros[i][j] = arr_train[i][j]
</code></pre>
<p>(I assigned the values this way, because in my little search I didn’t find a way to convert lists/arrays of different sizes to tensor)</p>
<p>Then I resize the tensor, expanding, just so that it can be processed by the conv2d layers:</p>
<p><code>zeros = zeros.expand(1,32,8) #32x8 dimensions to 1x32x8</code></p>
<p>I am not sure if this is a good solution. Maybe using conv1d would make more sense in this case, but for restrictions of the project I am working on, I need to use conv2d.</p>
<p>I also change the input of the model to:</p>
<pre><code class="lang-auto">        self.conv1 = nn.Conv2d(1, 16, 3, padding=1) #batch_size, channels, height, width
</code></pre>
<p>Because there is only 1 channel</p>