RuntimeError: Given groups=1, weight of size [1, 3, 1], expected input[1, 1, 3] to have 3 channels, but got 1 channels instead

AP_M: <p>For Conv1d, using time series data set, I am trying to implement CNN and getting the below error:</p>
<pre><code class="lang-auto">"RuntimeError: Given groups=1, weight of size [1, 3, 1], expected input[1, 1, 3] to have 3 channels, but got 1 channels instead".
</code></pre>
<p>I tried to make a lot of modifications in the layers but still the error is there</p>
<p>Code:</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv1d(3, 1, kernel_size=1)
        self.dropout = nn.Dropout()
        self.fc1 = nn.Linear(1, 3)
        self.fc2 = nn.Linear(3, 1)
        

    def forward(self, x):
        print(x)
        x = F.relu(self.conv1(x))
        x = F.max_pool1d(x, 1)
        x = self.dropout(x)
        x = x.view(1, -1)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
       return (x)

model.eval()
    with torch.no_grad():
        for x1,x2 in train_ldr:
            print(x1, x2)
            x1 = x1.unsqueeze(0)
            print(x1.shape)
            out = model(x1)
            print(out.shape)

And the shapes are:
tensor([[ 1.0302, -0.7415, -1.0918]]) tensor([1.])
torch.Size([1, 1, 3])</code></pre>

Matias_Vasquez: <p>The expected input for <code>conv1d</code> is either <code>(N, C_in, L_in)</code> or <code>(C_in, L_in)</code> as shown in the <a href="https://pytorch.org/docs/stable/generated/torch.nn.Conv1d.html" rel="noopener nofollow ugc">documentation</a>.</p>
<p>In your case, <code>x1</code> has the shape <code>torch.Size([1, 1, 3])</code> when you feed it to your model. This means that <code>C_in=1</code>.</p>
<p>However, when you defined <code>self.conv1</code>, the first parameter, which corresponds to <code>C_in</code> is 3. This is why you are getting the error.</p>
<p>If you want to leave your model like that, then you can simply permute the dimensions.</p>
<pre><code class="lang-python">x1 = x1.permute(0, 2, 1)
model(x1)
</code></pre>
<p>However, I do not know your exact application, so you might want to change the model instead.</p>
<p>Hope this helps <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=12" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>

AP_M: <p>Thank you for the solution. But if the error is:</p>
<pre><code class="lang-auto">RuntimeError: Given groups=1, weight of size [1, 3, 1], expected input[1, 1, 1] to have 3 channels, but got 1 channels instead
</code></pre>
<p>how do we solve the error keeping the  model same. How can we change the channels in a tensor ?</p>

Matias_Vasquez: <p>Since you are leaving the model as it is, the expected <code>C_in</code> for <code>self.conv1</code> is <code>3</code>.</p>
<p>In this new error that you get, you can see the shape of the <code>input</code> you are giving to the model, which is <code>torch.Size([1, 1, 1])</code>. This means that <code>N=1</code>, <code>C_in=1</code> and <code>L_in=1</code>.</p>
<p>Again, since I do not know your specific application, a possible solution would be to copy the tensor in the dimension needed.</p>
<pre><code class="lang-python"># Random generated input
x1 = torch.rand(1, 1, 1)

# This is what you can do
x1 = x1.repeat(1, 3, 1)

# This should now work
model(x1)
</code></pre>
<p>If instead you want to modify the model, then you need to specify the correct number of channels <code>C_in</code> for the <code>conv1</code>.</p>
<pre><code class="lang-python"># C_in = 1 instead of 3
self.conv1 = nn.Conv1d(1, 1, kernel_size=1)
</code></pre>
<p>But it all depends on how your actual data looks like and what you want to do.</p>

AP_M: <p>Thank you for the help. I am trying to use simCLR with my model but getting a lot of dimension errors. How should I change my model to correct the dimensions. The error is in the loss function :</p>
<pre><code class="lang-auto">RuntimeError: shape '[8, 1]' is invalid for input of size -4
</code></pre>
<pre><code class="lang-auto">for epoch in range(10):
        
    print(f"Epoch [{epoch}/{epochs}]\t")
    stime = time.time()

    model.train()
    tr_loss_epoch = 0
    
    for step, (x_i, x_j) in enumerate(train_ldr):
        optimizer.zero_grad()
        x_i = x_i.unsqueeze(0)
        x_j = x_j.unsqueeze(0)
        x_j = x_j.unsqueeze(1) 
        print(x_i.shape)
        print(x_j.shape)
        # positive pair, with encoding
        x_i = x_i.permute(0, 2, 1)
        x_j = x_j.permute(0, 2, 1)
        print('Permute',x_i.shape)
        print('Permute',x_j.shape)
        z_i = model(x_i)
        x_j = x_j.repeat(1, 3, 1)
        z_j = model(x_j)
        print('z_i shape', z_i.shape)
        print('z_j shape', z_j.shape)

        loss = criterion(z_i, z_j)
        loss.backward()

        optimizer.step()
        
        if nr == 0 and step % 50 == 0:
            print(f"Step [{step}/{len(X_train)}]\t Loss: {round(loss.item(), 5)}")

        tr_loss_epoch += loss.item()

criterion = SimCLR_Loss(batch_size = 4, temperature = 0.5)
</code></pre>
<pre><code class="lang-auto">class SimCLR_Loss(nn.Module):
    def __init__(self, batch_size, temperature):
        super(SimCLR_Loss,self).__init__()
        self.batch_size = batch_size
        self.temperature = temperature

        self.mask = self.mask_correlated_samples(batch_size)
        self.criterion = nn.CrossEntropyLoss(reduction="sum")
        self.similarity_f = nn.CosineSimilarity(dim=2)

    def mask_correlated_samples(self, batch_size):
        N = 2 * batch_size
        mask = torch.ones((N, N), dtype=bool)
        mask = mask.fill_diagonal_(0)
        
        for i in range(batch_size):
            mask[i, batch_size + i] = 0
            mask[batch_size + i, i] = 0
        return mask

    def forward(self, z_i, z_j):

        N = 2 * self.batch_size

        z = torch.cat((z_i, z_j), dim=0)

        sim = self.similarity_f(z.unsqueeze(1), z.unsqueeze(0)) / self.temperature

        sim_i_j = torch.diag(sim, self.batch_size)
        sim_j_i = torch.diag(sim, -self.batch_size)
              
        positive_samples = torch.cat((sim_i_j, sim_j_i), dim=0).reshape(N, 1)
        negative_samples = sim[self.mask].reshape(N, -1)
        
        #SIMCLR
        labels = torch.from_numpy(np.array([0]*N)).reshape(-1).to(positive_samples.device).long() #.float()
        
        logits = torch.cat((positive_samples, negative_samples), dim=1)
        loss = self.criterion(logits, labels)
        loss /= N
        
        return loss
</code></pre>
<p>The shapes of z_i and z_j are:<br>
z_i shape torch.Size([1, 1])<br>
z_j shape torch.Size([1, 1])</p>

Matias_Vasquez: <p>I have not looked at it in that much detail but I have some questions.</p>
<p>When you are getting your data from your dataloader, what is <code>x_i</code> and <code>x_j</code>?</p>
<p>I assume <code>x_i</code> is supposed to be your data and <code>x_j</code> the label for this data.</p>
<aside class="quote no-group" data-username="AP_M" data-post="5" data-topic="150214">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ap_m/48/32203_2.png" class="avatar"> AP_M:</div>
<blockquote>
<p><code>for step, (x_i, x_j) in enumerate(train_ldr):</code></p>
</blockquote>
</aside>
<p>If that is the case, then I do not quite understand why you also pass the label through the model?</p>
<aside class="quote no-group quote-modified" data-username="AP_M" data-post="5" data-topic="150214">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ap_m/48/32203_2.png" class="avatar"> AP_M:</div>
<blockquote>
<pre><code class="lang-auto">z_i = model(x_i)            # ← This would be the prediction of your data 
x_j = x_j.repeat(1, 3, 1)
z_j = model(x_j)            # ← But here you also pass the label through the model
</code></pre>
</blockquote>
</aside>
<p>The label is your ground truth for your loss. You might need to change its dimensions or something, but passing it through the model will give you something other than the actual label.</p>
<aside class="quote no-group" data-username="AP_M" data-post="5" data-topic="150214">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ap_m/48/32203_2.png" class="avatar"> AP_M:</div>
<blockquote>
<p><code>loss = criterion(z_i, z_j)</code></p>
</blockquote>
</aside>

AP_M: <p>Sorry for the late reply.</p>
<p>Since I am using SimCLR model, x_i and x_j are the two data augmented representations of a sample x on which data transformations have been applied. We have to check the similarity between them and calculate the loss between them. But in my program as you said, it is considering x_j as the label x_i as the sample and thus giving different dimensions.</p>

Matias_Vasquez: <p>Oh yeah, sorry I am not that familiar with SimCLR.</p>
<p>However, by skimming over the <a href="https://arxiv.org/pdf/2002.05709.pdf" rel="noopener nofollow ugc">paper</a> I see that they resize the augmented images to the original image size.</p>
<p>So in this case x_i and x_j should have the same size <strong>before</strong> entering your model.</p>