Convolutional 1D NN with different sequence lengths

Ditlev_Jorgensen: <p>Hi.</p>
<p>I have a hard time understanding the size of tensors and how they change shape thorughout my network.</p>
<p>My data consist of 1000s of .csv files which each have 1000s of data samples. Some have few samples like 10 some have 10.000 samples. Due to memory a single track(sequence/.csv file) must be loaded at a time. I would prefer to have the network train on a single timestep on a single file. But atm i will settle for just getting a basic working network up and run with my data. I have 3 folders, 1 folder with .csv files from each class. My end goal is to produce a classifier which can classify ship types from movement patterns. First running the through a CNN to extract features and then a RNN to produce the predicted classes. However i start out very simply with just 2 conv-layers.</p>
<p>My custom data.Dataset class is the following (shown here is from Cargo ships):</p>
<pre><code class="lang-auto">class cargoShipDataset(data.Dataset):
    def __init__(self, root_dir, transform=None):
        self.transform = transform
        self.list_of_data_files = glob.glob(root_dir) #Load list of all filenames in directory
        self.len = len(self.list_of_data_files)
        
    def __getitem__(self, index):
        single_track = torch.from_numpy(genfromtxt(self.list_of_data_files[index], delimiter=',')[1:,:]) #a single track is loaded.
        target = torch.ones(single_track.size(0)) # Class Cargo = 1 ? Class Passenger = 2 ?
        
        #print(single_track)
        
        return (single_track, target)

    def __len__(self):
        return self.len
</code></pre>
<p>My network (stolen from <a href="https://github.com/MorvanZhou/PyTorch-Tutorial/blob/master/tutorial-contents/401_CNN.py" rel="nofollow noopener">https://github.com/MorvanZhou/PyTorch-Tutorial/blob/master/tutorial-contents/401_CNN.py</a>):<br>
It inputs a timeseries with three channels (latitude, longitude and timestamp).</p>
<pre><code class="lang-auto">class CNN(nn.Module):
    def __init__(self):
        super(CNN,self).__init__()
        self.conv1 = nn.Sequential(         
            nn.Conv1d(
                in_channels=3,              #Inputs time series, with Latitude, Longitude and Timestamp
                out_channels=16,            #Some random number
                kernel_size=5,              
                stride=1,                   
                padding=2,                  
            ),                              
            nn.ReLU(),                      
            nn.MaxPool1d(kernel_size=2),   #size after pooling?
        )
        self.conv2 = nn.Sequential(         
            nn.Conv1d(
                in_channels=16,             
                out_channels=8,             
                kernel_size=5,              
                stride=1,                   
                padding=2,                  
            ),                              
            nn.ReLU(),                      
            nn.MaxPool1d(kernel_size=2),    
            
        )
        self.out = nn.Linear(8, 3)  #output 3 classes: Cargo, Passenger or Fishing
        
    def forward(self,x):
        #print(x.size())
        x = self.conv1(x)
        #print(x.size())
        x = self.conv2(x)
        #print(x.size())
        output = self.out(x)
        return output
</code></pre>
<p>My train loop:</p>
<pre><code class="lang-auto">cnn = CNN()
#print(cnn)
optimizer = torch.optim.Adam(cnn.parameters(), lr=0.0001)   # optimize all cnn parameters
loss_func = nn.CrossEntropyLoss()                       # the target label is not one-hotted

for epoch in range(1):
    for step, data in enumerate(train_set_cargo):   # gives batch data, normalize x when iterate train_loader
        #print(data)
        b_x = data[0] #here b_x is a tensor with size [sequence length, 3] Where the values are: lat, lon, time.
        #print(b_x)
        b_y = data[1] #b_y is a tensor with size [sequence length, 1] of simply ones.
        #print(b_y)
        output = cnn(b_x)               # cnn output
        loss = loss_func(output, b_y)   # cross entropy loss
        optimizer.zero_grad()           # clear gradients for this training step
        loss.backward()                 # backpropagation, compute gradients
        optimizer.step()                # apply gradients
</code></pre>
<p>The problem is that b_x is of size [sequence length , 3] when printing it pre output = cnn(b_x). However, when printing it inside the Network code, pre x = self.conv1(x), the size is suddenly [1, sequence length, 3].</p>
<p>Since my network is set to convolve 1D over 3 channels is returns error cause it gets “sequence length” channels. See error below.</p>
<p>My traceback error snippet:</p>
<pre><code class="lang-auto">Traceback (most recent call last):

  File "&lt;ipython-input-14-af3912e69f8b&gt;", line 1, in &lt;module&gt;
    runfile('C:/Users/AWESOME-O 4000/Desktop/Speciale/Data/NN.py', wdir='C:/Users/AWESOME-O 4000/Desktop/Speciale/Data')

  File "C:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 678, in runfile
    execfile(filename, namespace)

  File "C:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 106, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/AWESOME-O 4000/Desktop/Speciale/Data/NN.py", line 107, in &lt;module&gt;
    output = cnn(b_x)               # cnn output

  File "C:\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)

  File "C:/Users/AWESOME-O 4000/Desktop/Speciale/Data/NN.py", line 87, in forward
    x = self.conv1(x)

  File "C:\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)

  File "C:\Anaconda3\lib\site-packages\torch\nn\modules\container.py", line 91, in forward
    input = module(input)

  File "C:\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)

  File "C:\Anaconda3\lib\site-packages\torch\nn\modules\conv.py", line 176, in forward
    self.padding, self.dilation, self.groups)

RuntimeError: Given groups=1, weight of size [16, 3, 5], expected input[1, 1429, 3] to have 3 channels, but got 1429 channels instead
</code></pre>
<p>I am very new to pytorch (still havn’t done a single network yet). And have a hard time understanding when tensors change size. Once this is fixed my next problem is that my sequences have different length, but this i will fix later <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=6" title=":smiley:" class="emoji" alt=":smiley:"></p>
<p>Any help/insight is greatly appreciated. I feel like its most likely an error stemming from my custom data.Dataset or the network/training loop. Which is why i have that code pasted in.<br>
My plan is to do 3 data.Dataset classes, one for each class. And then concanate them into one data set.</p>
<p>If any information is missing let me know and i will provide as best as i can.</p>
<p>regards</p>

ptrblck: <p><code>nn.Conv1d</code> expects your input to be <code>[batch_size, channels, length]</code>, see the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.Conv1d">docs</a>. Could you try to swap the axes using:</p>
<pre><code class="lang-auto">b_x = b_x.permute(0, 2, 1)
</code></pre>
<p>Alternatively you could reshape it in your <code>Dataset</code>'s <code>__getitem__</code> so that your training loop stays a bit cleaner.</p>

Ditlev_Jorgensen: <p>Thank you so much for your reply.</p>
<p>I fixed the error by transposing the numpy array in the Dataset’s <strong>getitem</strong> as you requested <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=6" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>The line is now:</p>
<pre><code class="lang-auto">single_track = torch.from_numpy(np.transpose(genfromtxt(self.list_of_data_files[index], delimiter=','))[:,1:])
</code></pre>
<p>Unfortunately a new error popped up now calling an error in the type of tensor. I use FloatTensor but it expects a DoubleTensor.</p>
<pre><code class="lang-auto">RuntimeError: Expected object of type torch.DoubleTensor but found type torch.FloatTensor for argument #2 'weight'
</code></pre>
<p>For what i understand its an error that occurs when it uses both cpu and gpu? I don’t know how to set which processor to use so will have to look into that <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=6" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

InnovArul: <aside class="quote no-group" data-username="Ditlev_Jorgensen" data-post="3" data-topic="23816">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ditlev_jorgensen/48/12622_2.png" class="avatar"> Ditlev_Jorgensen:</div>
<blockquote>
<pre><code class="lang-auto">RuntimeError: Expected object of type torch.DoubleTensor but found type torch.FloatTensor for argument #2 'weight'
</code></pre>
</blockquote>
</aside>
<p>Probably,  you may need to call <code>.float()</code> on your data and models to solve this.</p>

Ditlev_Jorgensen: <p>Thank you! Now it works (except for my different sequence lengths, but i will look into padding now <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=6" title=":smile:" class="emoji" alt=":smile:">  )</p>

QuirkyDataScientist1: <p>Hello I am new to pytorch as well - where would I insert the .float()?</p>

QuirkyDataScientist1: <p>So what does the final code that works look like?</p>