RuntimeError: Given groups=1, weight of size [6, 1, 5, 5], expected input[128, 3, 218, 178] to have 1 channels, but got 3 channels instead

Mashood: <p>Yes, I have looked and tried solution from other related questions. Let me explain.</p>
<p><strong>Description:</strong> So, I am doing adversarial training and I’ve used code from <a href="https://github.com/ttchengab/FGSMAttack" rel="noopener nofollow ugc">this Github</a>  repo. It uses Pytorch to train model on MNIST dataset.<br>
What want is that instead of MNIST I want to use CelebeA dataset. And when I run this exact code on that it is giving me the above error.<br>
<strong>What I’ve tried:</strong> I googled the error and got 2,3 related issues in which the issue got resolved when people changed their input-size in first conv layer. When I changed <code>self.conv1 = torch.nn.Conv2d(1, 6, 5, padding=2)</code>  to <code>self.conv1 = torch.nn.Conv2d(3, 6, 5, padding=2)</code> it results into another error on line <code>(x=x.view(-1, 16*5*5))</code> saying: “<code>RuntimeError: shape '[-1, 400]' is invalid for input of size 4472832</code>” .<br>
I even tried changing this -1 value to 3, but its resulted into another different error and to be honest I don’t know what I’m doing so I stopped messing around.<br>
Part of the code that I think is related to this error. You can see the whole code <a href="https://github.com/ttchengab/FGSMAttack/blob/master/FGSMAttack.ipynb" rel="noopener nofollow ugc">here</a>:</p>
<pre><code class="lang-auto"># Creating a simple network
class LeNet5(torch.nn.Module):          
     
    def __init__(self):     
        super(LeNet5, self).__init__()
        self.conv1 = torch.nn.Conv2d(1, 6, 5, padding=2)
        self.conv2 = torch.nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16*5*5, 120)   
        self.fc2 = nn.Linear(120, 84)       
        self.fc3 = nn.Linear(84, 10)    
        
    def forward(self, x):
        x = F.relu(self.conv1(x))  
        x = F.max_pool2d(x, 2) 
        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 2)
        x = x.view(-1, 16*5*5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        
        return F.log_softmax(x,dim=-1)

FLAGS = flags.FLAGS
NB_EPOCHS = 2
BATCH_SIZE = 128
LEARNING_RATE = .001

#Training the Network
def trainTorch(torch_model, train_loader, test_loader,
        nb_epochs=NB_EPOCHS, batch_size=BATCH_SIZE, train_end=-1, test_end=-1, learning_rate=LEARNING_RATE, optimizer=None):

    train_loss = []
    total = 0
    correct = 0
    step = 0
    for _epoch in range(nb_epochs):
      for xs, ys in train_loader:
        xs, ys = Variable(xs), Variable(ys)
        if torch.cuda.is_available():
          xs, ys = xs.cuda(), ys.cuda()
          print('Cuda is available')
        optimizer.zero_grad()
        preds = torch_model(xs)
        # print("HI")
        loss = F.nll_loss(preds, ys)
        # print("HADSFSDF")
        loss.backward()  # calc gradients
        train_loss.append(loss.data.item())
        optimizer.step()  # update gradients

        preds_np = preds.cpu().detach().numpy()
        correct += (np.argmax(preds_np, axis=1) == ys.cpu().detach().numpy()).sum()
        total += train_loader.batch_size
        step += 1
        if total % 1000 == 0:
          acc = float(correct) / total
          print('[%s] Training accuracy: %.2f%%' % (step, acc * 100))
          total = 0
          correct = 0



model1 = LeNet5()
if torch.cuda.is_available():
  model1 = model1.cuda()
nb_epochs = 4
batch_size = 128
learning_rate = 0.001
train_end = -1
test_end = -1
report = AccuracyReport()
train_loader = torch.utils.data.DataLoader(
    datasets.CelebA('data', split='train', transform=transforms.ToTensor(), download="True"),
    batch_size=batch_size, shuffle=True)

test_loader = torch.utils.data.DataLoader(
    datasets.CelebA('data', split='test', transform=transforms.ToTensor()),
    batch_size=batch_size)

# In[ ]:


#Training the model
print("Training Model")
optimizer = optim.Adam(model1.parameters(), lr=learning_rate)
trainTorch(model1, train_loader, test_loader, nb_epochs, batch_size, train_end, test_end, learning_rate, optimizer = optimizer)

</code></pre>
<p><strong>What I want to achieve:</strong><br>
1- To understand this and resolve this error and get my CelebA dataset working on this code.<br>
2- Would I require any other change (for adversarial training on CelebA instead of MNIST) even if this is resolved?</p>
<p><strong>3- My ultimate goal is to do adversarial training on any facial recognition dataset (ideally with pytorch and FGSM attack). If my current approach is not right, can you guys suggest me any other sources where it is done or can provide information on how to do this?</strong><br>
Thanks!</p>

Mashood: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , need your feedback please.</p>

ptrblck: <p>Replace <code>self.conv1</code> with a new <code>nn.Conv2d</code> layer accepting 3 input channels, then replace:</p>
<pre><code class="lang-python">x = x.view(-1, 16*5*5)
</code></pre>
<p>with</p>
<pre><code class="lang-python">x = x.view(x.size(0), -1)
</code></pre>
<p>and fix the <code>in_features</code> of <code>self.fc1</code> in case you are running into a shape mismatch afterwards.</p>

Mashood: <aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>x = x.view(x.size(0), -1)</code></p>
</blockquote>
</aside>
<p>First of all thanks for the prompt reply, man.<br>
I applied the change but now it throws the following error at line: <code>preds = torch_model(xs)</code></p>
<blockquote>
<p>File ~/anaconda3/envs/pytorch/lib/python3.9/site-packages/torch/nn/functional.py:1690 in linear<br>
ret = torch.addmm(bias, input, weight.t())</p>
<p>RuntimeError: mat1 and mat2 shapes cannot be multiplied (128x34944 and 400x120)</p>
</blockquote>

ptrblck: <p>Yes, that was expected and matches my second point:</p>
<aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>and fix the <code>in_features</code> of <code>self.fc1</code> in case you are running into a shape mismatch afterwards.</p>
</blockquote>
</aside>
<p>Change <code>self.fc1</code> to:</p>
<pre data-code-wrap="python"><code class="lang-python">self.fc1 = nn.Linear(34944, 120)
</code></pre>

Mashood: <p>I also want to know that what I am trying to do is right or not?<br>
Like should the code that’s been running on MNIST initially run on CelebA dataset with minor changes ?</p>

ptrblck: <p>No, it should not run directly as MNIST uses grayscale images (this the <code>in_channels</code> is set to <code>1</code> in the first conv layer) while CelebA uses RGB images which is why you need to replace the first conv layer with a new one accepting 3 input channels.<br>
Also, if the spatial size changes (MNIST uses <code>28x28</code>) you need to adapt the <code>in_features</code> of the first linear layer to account for this increase in features.</p>

Mashood: <aside class="quote no-group" data-username="ptrblck" data-post="7" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>No, it should not run directly as MNIST uses grayscale images (this the <code>in_channels</code> is set to <code>1</code> in the first conv layer) while CelebA uses RGB images which is why you need to replace the first conv layer with a new one accepting 3 input channels.</p>
</blockquote>
</aside>
<p>Forgive my inexperience, but how would I “adapt the in_features of the first linear layer to account for this increase in features” ?<br>
Also, what would be the class size in case of CelebA dataset? <code>self.fc3 = nn.Linear(84, 10) </code>, here they have added 10(I think its because of MNIST), but what would it be in my case?<br>
I really appreciate your guidance. Thanks!</p>

ptrblck: <aside class="quote no-group" data-username="Mashood" data-post="8" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9f8e36/48.png" class="avatar"> Mashood:</div>
<blockquote>
<p>Forgive my inexperience, but how would I “adapt the in_features of the first linear layer to account for this increase in features” ?</p>
</blockquote>
</aside>
<p>Check my <a href="https://discuss.pytorch.org/t/runtimeerror-given-groups-1-weight-of-size-6-1-5-5-expected-input-128-3-218-178-to-have-1-channels-but-got-3-channels-instead/158741/5">previous post</a> where I explain how to change the <code>in_features</code> based on the reported activation shape in the error message.</p>
<aside class="quote no-group" data-username="Mashood" data-post="8" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9f8e36/48.png" class="avatar"> Mashood:</div>
<blockquote>
<p>Also, what would be the class size in case of CelebA dataset? <code>self.fc3 = nn.Linear(84, 10) </code>, here they have added 10(I think its because of MNIST), but what would it be in my case?</p>
</blockquote>
</aside>
<p>It depends on your use case and <a href="https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html">CelebA</a> provides:</p>
<blockquote>
<ul>
<li>10,177 number of identities,</li>
<li>202,599 number of face images, and</li>
<li>5 landmark locations, 40 binary attributes annotations per image.</li>
</ul>
</blockquote>

Mashood: <aside class="quote no-group" data-username="ptrblck" data-post="9" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Check my <a href="https://discuss.pytorch.org/t/runtimeerror-given-groups-1-weight-of-size-6-1-5-5-expected-input-128-3-218-178-to-have-1-channels-but-got-3-channels-instead/158741/5">previous post</a> where I explain how to change the <code>in_features</code> based on the reported activation shape in the error message.</p>
</blockquote>
</aside>
<p>Yeah sorry about that. I think I missed that message somehow. I have updated that. But still there are errors…<br>
Now, I’m seeing error : <code>RuntimeError: 1D target tensor expected, multi-target not supported</code> on line:<br>
<code>loss = F.nll_loss(preds, ys)</code><br>
I googled the error and every solution says that the label tensor that should be 1-D instead of multiple dimension tensor. In my case I’ve printed the <code>ys</code> and it is a multi-dimension. So, to fox that I’ve tried <code>ys = ys.squeeze_()</code> but its still the same.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="9" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>It depends on your use case</p>
</blockquote>
</aside>
<p>My use case is that first I want to run simple facial recognition on the dataset and see the results. Then, the code attacks the model with some adversaries(which is created on same dataset) and then train the model using the original data mixed with the adversarial images. So, basically it is a Multi-class Classification problem.<br>
Also, for that reason I have set the size of my class to 10177 like:<br>
<code>self.fc3 = nn.Linear(84, 10177)</code></p>

ptrblck: <p>For a multi-class classification use case with <code>nn.NLLLoss</code> the model outputs should have the shape <code>[batch_size, nb_classes]</code> and the targets should have the shape <code>[batch_size]</code> containing class indices in the range <code>[0, nb_classes-1]</code>.</p>

Mashood: <p>Hey man, I understand to some extent what you are saying, but I don’t know how and what exact changes are needed… Like what exactly should I do here <code>loss = F.nll_loss(preds, ys)</code> now?</p>

ptrblck: <p>Here is a small but complete code snippet using a single layer as a model showing the expected shapes:</p>
<pre><code class="lang-python">batch_size = 2
nb_classes = 10
in_features = 20

model = nn.Linear(in_features, nb_classes)
x = torch.randn(batch_size, in_features)
target = torch.randint(0, nb_classes, (batch_size,))

criterion = nn.NLLLoss()

output = model(x)
output = F.log_softmax(output, dim=1)
loss = criterion(output, target)
loss.backward()
</code></pre>

Mashood: <p>Sorry man. I really appreciate your help. But, I’m still clueless. I don’t know how to relate this to my code…</p>
<aside class="quote no-group" data-username="Mashood" data-post="1" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9f8e36/48.png" class="avatar"> Mashood:</div>
<blockquote>
<pre><code class="lang-auto">FLAGS = flags.FLAGS
NB_EPOCHS = 2
BATCH_SIZE = 128
LEARNING_RATE = .001

#Training the Network
def trainTorch(torch_model, train_loader, test_loader,
        nb_epochs=NB_EPOCHS, batch_size=BATCH_SIZE, train_end=-1, test_end=-1, learning_rate=LEARNING_RATE, optimizer=None):

    train_loss = []
    total = 0
    correct = 0
    step = 0
    for _epoch in range(nb_epochs):
      for xs, ys in train_loader:
        xs, ys = Variable(xs), Variable(ys)
        if torch.cuda.is_available():
          xs, ys = xs.cuda(), ys.cuda()
          print('Cuda is available')
        optimizer.zero_grad()
        preds = torch_model(xs)
        # print("HI")
        loss = F.nll_loss(preds, ys)
        # print("HADSFSDF")
        loss.backward()  # calc gradients
        train_loss.append(loss.data.item())
        optimizer.step()  # update gradients

        preds_np = preds.cpu().detach().numpy()
        correct += (np.argmax(preds_np, axis=1) == ys.cpu().detach().numpy()).sum()
        total += train_loader.batch_size
        step += 1
        if total % 1000 == 0:
          acc = float(correct) / total
          print('[%s] Training accuracy: %.2f%%' % (step, acc * 100))
          total = 0
          correct = 0
</code></pre>
</blockquote>
</aside>

ptrblck: <p>In your code you are using the default <code>target_type</code> of the <code>CelebA</code> dataset, which is <code>attr</code>, i.e. a multi-label classification as described in the <a href="https://pytorch.org/vision/main/generated/torchvision.datasets.CelebA.html">docs</a>.<br>
Based on your description you want to use the <code>identity</code> target, so this code should work:</p>
<pre><code class="lang-python">dataset = datasets.CelebA('./data', split='train', transform=transforms.ToTensor(), download="True", target_type='identity')
batch_size = 8
train_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=True)

model = models.resnet18()
model.fc = nn.Linear(512, 10177)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

for xs, ys in train_loader:
    optimizer.zero_grad()
    output = model(xs)
    output = F.log_softmax(output, dim=1)
    loss = F.nll_loss(output, ys)
    loss.backward()
    optimizer.step()
    print(loss.item())
</code></pre>

Mashood: <aside class="quote no-group" data-username="ptrblck" data-post="15" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>model = models.resnet18()</code></p>
</blockquote>
</aside>
<p>I’m trying this snippet separate from my existing messed up code. One thing though, I’m getting the following error on the above line: <code>Undefined name 'models'</code></p>

ptrblck: <p>You are most likely missing:</p>
<pre><code class="lang-python">import torchvision.models as models
</code></pre>

Mashood: <p>Hey ptr. I tried your code and there are 2 things that seem wrong. First the training accuracy is 0% for all of the iterations and it says: <code>IndexError: Target 10177 is out of bounds.</code> Same error in case of 10176 as well.<br>
My new code snippet:</p>
<pre><code class="lang-auto">def trainTorch(torch_model, train_loader, test_loader,
        nb_epochs=NB_EPOCHS, batch_size=BATCH_SIZE, train_end=-1, test_end=-1, learning_rate=LEARNING_RATE, optimizer=None):

    train_loss = []
    total = 0
    correct = 0
    step = 0
    for _epoch in range(nb_epochs):
      for xs, ys in train_loader:        
        optimizer.zero_grad()
        preds = torch_model(xs)
        output = F.log_softmax(preds, dim=1)
        loss = F.nll_loss(output, ys)
        loss.backward()
       
        train_loss.append(loss.data.item())
        optimizer.step()  # update gradients

        preds_np = preds.cpu().detach().numpy()
        correct += (np.argmax(preds_np, axis=1) == ys.cpu().detach().numpy()).sum()
        total += train_loader.batch_size
        step += 1
        if total % 1000 == 0:
          acc = float(correct) / total
          print('[%s] Training accuracy: %.2f%%' % (step, acc * 100))
          total = 0
          correct = 0
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">Training Model
[125] Training accuracy: 0.00%
[250] Training accuracy: 0.00%
[375] Training accuracy: 0.00%
[500] Training accuracy: 0.00%
[625] Training accuracy: 0.00%
[750] Training accuracy: 0.00%
[875] Training accuracy: 0.10%
[1000] Training accuracy: 0.00%
[1125] Training accuracy: 0.00%
[1250] Training accuracy: 0.00%
[1375] Training accuracy: 0.00%
</code></pre>
<p>Please, need your suggestions. Thanks.</p>

ptrblck: <aside class="quote no-group" data-username="Mashood" data-post="18" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9f8e36/48.png" class="avatar"> Mashood:</div>
<blockquote>
<p><code>IndexError: Target 10177 is out of bounds.</code></p>
</blockquote>
</aside>
<p>That’s true, as the <code>out_features</code> have to be <code>10178</code> for <code>10177</code> classes.</p>
<aside class="quote no-group" data-username="Mashood" data-post="18" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9f8e36/48.png" class="avatar"> Mashood:</div>
<blockquote>
<p>Same error in case of 10176 as well.</p>
</blockquote>
</aside>
<p>That’s not the case and my code works up to class indices of <code>10176</code>.</p>
<aside class="quote no-group" data-username="Mashood" data-post="18" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9f8e36/48.png" class="avatar"> Mashood:</div>
<blockquote>
<p>First the training accuracy is 0% for all of the iterations</p>
</blockquote>
</aside>
<p>That might be the case as I didn’t provide a code snippet containing a model with already tuned hyperparameters for this use case, but provided a code snippet which shows how to fix your previous errors as you were stuck even after explaining what is causing the issues.<br>
You can thus the the class index issue by using <code>nn.Linear(512, 10178)</code> as the final classifier in your model and could then reuse parts of my code to fix your multi-target errors in your model and training code.</p>

Mashood: <aside class="quote no-group" data-username="Mashood" data-post="18" data-topic="158741">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9f8e36/48.png" class="avatar"> Mashood:</div>
<blockquote>
<p><code>F.log_softmax(preds, dim=1)</code></p>
</blockquote>
</aside>
<p>Hey man. I think we are really close here. Sorry to ask you again and again. I have reuse your parts with my code and now the training accuracy is not 0 but its still low as 2% or 5%. I did changed the <code>batch_size</code> to 128 instead of 8 which increased the accuracy and speed. But, its still not right. I have googled how to adjust these hyperparameters but did not find relative help. Can you please advise if I’ve missed something here in <code>self.conv</code> or <code>self.fc</code> layers.<br>
*note: I changed the second param of <code>self.fc2</code> to 512 because when I used your code for <code>self.fc3</code> it gave “mat1 and mat2 missmatch error”.<br>
Thank you for all the help!<br>
This is what the code looks like now:</p>
<pre><code class="lang-auto">class LeNet5(torch.nn.Module):          
     
    def __init__(self):     
        super(LeNet5, self).__init__()
        self.conv1 = torch.nn.Conv2d(3, 6, 5, padding=2)
        self.conv2 = torch.nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(34944, 120)
        self.fc2 = nn.Linear(120, 512)       
        self.fc3 = nn.Linear(512, 10178) # change class size according to celebA dataset 
        
    def forward(self, x):
        x = F.relu(self.conv1(x))  
        x = F.max_pool2d(x, 2) 
        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 2)
        #x = x.view(-1, 16*5*5)
        x = x.view(x.size(0), -1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        
        return F.log_softmax(x,dim=-1)


model1 = LeNet5()

nb_epochs = 4
batch_size = 128
learning_rate = 0.001
train_end = -1
test_end = -1
report = AccuracyReport()
train_loader = torch.utils.data.DataLoader(
    datasets.CelebA('data', split='train', target_type='identity', transform=transforms.ToTensor(), download="True"),
    batch_size=batch_size, shuffle=True)
</code></pre>