Difficulty loading data/running model on custom dataset derrived from DNA sequence data - TypeError: conv1d() received an invalid combination of arguments

Moss-And-Bone: <p>Hello,</p>
<p>I am a student who has some limited experience with keras, and for a new project recently decided to learn how to use pytorch to implement my models. I’m a beginner with both, so apologies in advance for my inexperience, I am doing my best to follow tutorials, but my limited experience combined with most examples being in different uses has resulted in slower comprehension.</p>
<p>I have built a custom dataset class for my data, the X data is genetic sequence 256 bases long (i.e. “AGCTGGAGCT…”), so the resulting array after one-hotting to four channels for each of the four bases looks like<code> [[[1,0,0,0],[0,1,0,0]...], [0,0,1,0], ...]]</code> and has shape 48,976, 256, 4. I read that Conv1d looks for channels first, so I permuted the channels in the dataset’s tensor to read in that way, resulting in<code> torch.Size([48976, 4, 256])</code>. The Y data is 2 values for a given sequence of X, ESC and TSC, each numeric values derived from other source data. The dataset code is as follows:</p>
<pre><code class="lang-auto">device = "cuda" if torch.cuda.is_available() else "cpu"

def onehotseq(dataset, input_shape):
    
    onehot = np.zeros(input_shape)
    
    for i in range(0, dataset.shape[0]):
        seq = dataset.iloc[i,1]
    
        for c in range(0,len(seq)):
            if (seq[c] == "A"):
                onehot[i,c,:] = [1,0,0,0]
            elif (seq[c] == "C"):
                onehot[i,c,:] = [0,1,0,0]
            elif (seq[c] == "G"):
                onehot[i,c,:] = [0,0,1,0]
            elif (seq[c] == "T"):
                onehot[i,c,:] = [0,0,0,1]
    
    return onehot

            
            
class EpiDataset(torch.utils.data.Dataset):
    
    def __init__(self, Seq_filepath="path_to_sequence_data", Y_data_filepath="path_to_output_data"):
        
        self.seq_data = pd.read_csv(Seq_filepath, sep="\t", header=None)
        
        self.seq_data.rename(columns={0:"id", 1:"seq"}, inplace=True)
        
        self.y_data = pd.read_csv(Y_data_filepath, sep="\t", header = 0)
        
        self.y_data["ESC"] = np.log2((self.y_data["ESC.H3K27ac"].values+1)/(self.y_data["ESC.input"].values+1))
        self.y_data["TSC"] = np.log2((self.y_data["TSC.H3K27ac"].values+1)/(self.y_data["TSC.input"].values+1))
        
        self.dataset = self.seq_data.merge(self.y_data, on="id")
        
        self.list_IDs = self.dataset["id"]
        
        self.seq = self.dataset["seq"]
        
        self.esc = self.dataset["ESC"]
        
        self.tsc = self.dataset["TSC"]
        
        self.input_shape = (self.dataset.shape[0], 256, 4)
        
        self.onehotseq = onehotseq(self.dataset, self.input_shape)
        
        self.tensorX = torch.from_numpy(self.onehotseq)
        
        self.tensorX = self.tensorX.permute(0, 2, 1)
        
        self.labels = self.dataset[["ESC","TSC"]].to_numpy()
        
        self.tensorY = torch.from_numpy(self.labels)
        
        
    
        
    def __len__(self):
                return len(self.list_IDs)
                
    def __getitem__(self, index):
                
                ID = self.list_IDs[index]
                
                seq = self.seq[index]
                
                esc = self.esc[index]
                
                tsc = self.tsc[index]
                
                return {
                    
                    "ID: ": ID,
                    "sequence: ": seq,
                    "ESC: ": esc,
                    "TSC: ": tsc
                }               

</code></pre>
<p>This all seems to work as intended, and I was able to design a Module class, which also seems to be functionally correct, but I get a type error whenever I try to use the model. The code and error are:</p>
<pre><code class="lang-auto">def nin_block(out_channels, kernel_size, padding="same"):
    
    return nn.Sequential(
        
        nn.LazyConv1d(out_channels, kernel_size, padding),
        
        nn.ReLU(),
        
        nn.LazyConv1d(out_channels, kernel_size=1), nn.ReLU(),
        
        nn.LazyConv1d(out_channels, kernel_size=1), nn.ReLU()
    
    
    )

class NeuralNetwork(nn.Module):
    
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        
        self.flatten = nn.Flatten()
        
        self.NiN = nn.Sequential(
                nin_block(32, kernel_size=11,padding="same"),
        
                nn.MaxPool1d(3, stride=2),
        
                nin_block(64, kernel_size=4, padding="same"),
        
                nn.MaxPool1d(3, stride=2),
        
                nin_block(128, kernel_size=4, padding="same"),
        
                nn.MaxPool1d(3, stride=2),
        
                nin_block(256, kernel_size=3, padding="same"),
        
                nn.MaxPool1d(3, stride=2),
        
                nn.Dropout(0.4),
        
                nin_block(4, kernel_size=3, padding="same"),
        
                nn.AdaptiveAvgPool1d(2),
        
                nn.Flatten(),
        )
        
    def forward(self, x):
            
        x = self.flatten(x)
            
        logits = self.NiN(x)
            
        return logits

</code></pre>
<p>Error message, resulting from running <code>model = NeuralNetwork().to(device)</code> and then</p>
<p><code>logit = model(x.tensorX)</code></p>
<pre><code class="lang-auto">TypeError: conv1d() received an invalid combination of arguments - got (Tensor, Parameter, Parameter, tuple, tuple, tuple, int), but expected one of:
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (Tensor, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (Tensor, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)
</code></pre>
<p>My question is, what am I doing wrong either in building my module or dataset, or am I missing a step? The data loaded in is the prepared data for initial exploration/training of different model architectures.</p>

ptrblck: <p>You are passing the <code>padding="same"</code> argument to the <code>stride</code> of the conv layer:</p>
<pre><code class="lang-python">(0): Conv1d(32, 32, kernel_size=(11,), stride=('s', 'a', 'm', 'e'))
</code></pre>
<p>Use <code>nn.LazyConv1d(out_channels, kernel_size, padding=padding)</code> and it should work.</p>

Moss-And-Bone: <p>Great, this seems to have solved this error! The model can be applied to the x tensor, but unfortunately I am still getting issues when trying to train the model. I can move this to a new thread if necessary!</p>
<p>As the goal is to get a prediction for both ESC and TSC, should I have my forward return two values? Currently, I have modified the <strong>getitem</strong> as follows:</p>
<pre><code class="lang-auto"> def __getitem__(self, index):
                
                
                seq = self.tensorX[index]
                
                val1 = self.escTensor[index]
                
                val2 = self.tscTensor[index]
                
                label = [val1, val2]
                
                return seq, label

</code></pre>
<p>I tried the dataloader iteration example to compare labels etc for the batch:</p>
<pre><code class="lang-auto">print(f"Feature batch shape: {train_x.size()}")
print(f"Labels 1 batch shape: {train_y1[0].size()}")
print(f"Labels 2 batch shape: {train_y1[1].size()}")
&gt;&gt;&gt;&gt;&gt; OUTPUT BELOW THIS LINE &gt;&gt;&gt;&gt;&gt;

Feature batch shape: torch.Size([64, 4, 256])
Labels 1 batch shape: torch.Size([64])
Labels 2 batch shape: torch.Size([64])


</code></pre>
<p>Which seems to make sense: 64 sequences, 4 channels, 256 bases/characters/etc long, and two output labels, 64 of each for the batch.</p>
<p>But when I try and pass this to a training loop, I get a warning on the training, as well as a new error when just calling model(x.tensorX).</p>
<p>This is the loss and training code I am using:</p>
<pre><code class="lang-auto">import torch.optim as optim
import time

criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)


num_epochs = 100

train_start = program_starts = time.time()

for epoch in range(num_epochs):
    
    running_loss = 0.0
    
    start = time.process_time()
    
    now = time.time()
    
    for i, data in enumerate(x):
        
        
        inputs, labels1 = data
        
        output1 = model(inputs)
        
        #print(inputs.shape)
        
        #print(labels1)
        
        #print(output1)
        
        #print(output1[0].shape)
        
        
        loss1 = criterion(output1[0], labels1[0])
        
        loss2 = criterion(output1[1], labels1[1])
        
        
        loss = loss1 + loss2
        
        loss.backward()
        
        optimizer.step()
        
        running_loss += loss.item()
        
        if i % 2000 == 1999:
            print(f'[{epoch + 1}, {i+1:5d}] loss: {running_loss / 2000:.3f}')
            running_loss = 0.0
            
    print(f'Epoch {epoch+1} ran in {time.process_time() - start}')
    print("It has been {0} seconds since training began".format(now-train_start))

end_time = time.time()
print("Finished training in {0} seconds".format(end_time-train_start))
</code></pre>
<p>The warning when starting training is:</p>
<pre><code class="lang-auto">/home/~username~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/loss.py:530: UserWarning: Using a target size (torch.Size([])) that is different to the input size (torch.Size([2])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

</code></pre>
<p>And the model error is (from <code>model(x.tensorX)</code>):</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [251], in &lt;cell line: 1&gt;()
----&gt; 1 model(x.tensorX)

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/module.py:1130, in Module._call_impl(self, *input, **kwargs)
   1126 # If we don't have any hooks, we want to skip the rest of the logic in
   1127 # this function, and just call forward.
   1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129         or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1130     return forward_call(*input, **kwargs)
   1131 # Do not call functions when jit is used
   1132 full_backward_hooks, non_full_backward_hooks = [], []

Input In [246], in NeuralNetwork.forward(self, x)
     34 def forward(self, x):
     36     x = self.flatten(x)
---&gt; 38     logits = self.NiN(x)
     40     #x2 = self.flatten(x)
     41     
     42     #logits2 = self.NiN(x)
     44     return logits

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/module.py:1130, in Module._call_impl(self, *input, **kwargs)
   1126 # If we don't have any hooks, we want to skip the rest of the logic in
   1127 # this function, and just call forward.
   1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129         or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1130     return forward_call(*input, **kwargs)
   1131 # Do not call functions when jit is used
   1132 full_backward_hooks, non_full_backward_hooks = [], []

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/container.py:139, in Sequential.forward(self, input)
    137 def forward(self, input):
    138     for module in self:
--&gt; 139         input = module(input)
    140     return input

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/module.py:1130, in Module._call_impl(self, *input, **kwargs)
   1126 # If we don't have any hooks, we want to skip the rest of the logic in
   1127 # this function, and just call forward.
   1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129         or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1130     return forward_call(*input, **kwargs)
   1131 # Do not call functions when jit is used
   1132 full_backward_hooks, non_full_backward_hooks = [], []

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/container.py:139, in Sequential.forward(self, input)
    137 def forward(self, input):
    138     for module in self:
--&gt; 139         input = module(input)
    140     return input

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/module.py:1130, in Module._call_impl(self, *input, **kwargs)
   1126 # If we don't have any hooks, we want to skip the rest of the logic in
   1127 # this function, and just call forward.
   1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129         or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1130     return forward_call(*input, **kwargs)
   1131 # Do not call functions when jit is used
   1132 full_backward_hooks, non_full_backward_hooks = [], []

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/conv.py:307, in Conv1d.forward(self, input)
    306 def forward(self, input: Tensor) -&gt; Tensor:
--&gt; 307     return self._conv_forward(input, self.weight, self.bias)

File ~/anaconda3/envs/d2l/lib/python3.8/site-packages/torch/nn/modules/conv.py:303, in Conv1d._conv_forward(self, input, weight, bias)
    299 if self.padding_mode != 'zeros':
    300     return F.conv1d(F.pad(input, self._reversed_padding_repeated_twice, mode=self.padding_mode),
    301                     weight, bias, self.stride,
    302                     _single(0), self.dilation, self.groups)
--&gt; 303 return F.conv1d(input, weight, bias, self.stride,
    304                 self.padding, self.dilation, self.groups)

RuntimeError: Given groups=1, weight of size [32, 4, 11], expected input[1, 48976, 1024] to have 4 channels, but got 48976 channels instead

</code></pre>
<p>Although the model now trains, the warning along with that error when simply applying the model to the X tensor is concerning. I tried having forward return an X1 and X2 previously, as well as <strong>getitem</strong> returning three values instead of x, [y1,y2], but had other issues before I was able to even get the potential start with training to work.</p>
<p>An interesting update, I just ran the training step again, but it did not give the warning. Normally it is the warning followed by the loss outputs.</p>
<p>A follow up to this, I am now getting loss of NaN at each step, which was not previously occuring, whether the warning occurs or not! I am attempting troubleshooting these issues on the forums, but have not had luck so far. The loss returns NaN only, sometimes numbers and NaN, and only actual numbers on other runs (manual interrupts between since I’m unsure the model can actually be trained in its current state). I’ve only followed it through one epoch at present for troubleshooting but will leave it longer now. Is it possibly due to issues with batch sizes with &gt;64 samples?</p>