ValueError: Expected input batch_size (512) to match target batch_size (6815744)

Henryaa: <p>Please help me fix this issue. Thanks in advance.</p>
<p>My code:<br>
import torch<br>
import torch.nn as nn<br>
import torch.optim as optim</p>
<p>class MLP(nn.Module):<br>
def <strong>init</strong>(self, input_dim, hidden_dim, output_dim):<br>
super(MLP, self).<strong>init</strong>()<br>
self.input_dim = input_dim<br>
self.hidden_dim = hidden_dim<br>
self.output_dim = output_dim</p>
<pre><code>    self.fc1 = nn.Linear(input_dim, hidden_dim)
    self.fc2 = nn.Linear(hidden_dim, output_dim)
    
def forward(self, x):
    x = x.view(-1, self.input_dim)
    x = nn.functional.relu(self.fc1(x))
    x = self.fc2(x)
    print(x.shape)
    return x
</code></pre>
<h1>
<a name="load-features-and-labels-1" class="anchor" href="#load-features-and-labels-1"></a>Load features and labels</h1>
<p>features_list = []<br>
for i in range(13):<br>
xx = “{:02d}”.format(i)<br>
feature_file = f’C:/Users/prave/AppData/Local/Programs/Python/Python39/Scripts/assigment_data/assigment/feats/feats_{xx}.pt’<br>
feature_tensor = torch.load(feature_file)<br>
features_list.append(feature_tensor)</p>
<p>labels_list = []<br>
for i in range(13):<br>
xx = “{:02d}”.format(i)<br>
label_file = f’C:/Users/prave/AppData/Local/Programs/Python/Python39/Scripts/assigment_data/assigment/labels/y_{xx}.pt’<br>
label_tensor = torch.load(label_file)<br>
labels_list.append(label_tensor)</p>
<p>print(labels_list[0].shape)</p>
<h1>
<a name="concatenate-the-labels-into-a-single-tensor-2" class="anchor" href="#concatenate-the-labels-into-a-single-tensor-2"></a>Concatenate the labels into a single tensor</h1>
<p>all_labels = torch.cat(labels_list, dim=0)<br>
print(all_labels.shape)</p>
<h1>
<a name="find-the-number-of-unique-labels-the-number-of-classes-3" class="anchor" href="#find-the-number-of-unique-labels-the-number-of-classes-3"></a>Find the number of unique labels (the number of classes)</h1>
<p>num_classes = len(torch.unique(all_labels))<br>
print(f"Number of classes: {num_classes}")</p>
<h1>
<a name="calculate-input-dimension-4" class="anchor" href="#calculate-input-dimension-4"></a>Calculate input dimension</h1>
<p>input_dim = 524288<br>
print(“Input dimension:”, input_dim)</p>
<h1>
<a name="define-the-model-and-optimizer-5" class="anchor" href="#define-the-model-and-optimizer-5"></a>Define the model and optimizer</h1>
<p>model = MLP(input_dim=input_dim, hidden_dim=128, output_dim=18)<br>
optimizer = optim.Adam(model.parameters(), lr=0.001)</p>
<h1>
<a name="train-the-model-6" class="anchor" href="#train-the-model-6"></a>Train the model</h1>
<p>criterion = nn.CrossEntropyLoss()</p>
<p>for epoch in range(5):<br>
running_loss = 0.0<br>
total_pixels = 0<br>
correct_pixels = 0</p>
<pre><code>for i in range(len(features_list)):
    features = features_list[i]
    labels = labels_list[i]

    # Upscale the features to match the size of the labels
    features = features.unsqueeze(0)[:64]  # Add a batch dimension
    features = nn.functional.interpolate(features, scale_factor=4, mode='bilinear', align_corners=False)
    features = features.squeeze(0)  # Remove the batch dimension
    features = features[:labels.size(0)]  # Match the batch size of the labels
    
    # Reshape the features tensor to (batch_size, number_channels*height*width)
    features = features.view(features.size(0), -1)
    assert features.size(0) == labels.size(0)

    print(f"Shape of the feature tensor before pass: {features.shape}")
    print(f"Shape of the label tensor before pass: {labels.shape}")
    
    batch_size = features.size(0)
    labels = labels.view(batch_size, -1)
    
    # Flatten the labels tensor
    labels = torch.cat(labels_list, dim=0).view(-1)

    # Print the shape of the flattened labels tensor
    print(f"Shape of the label tensor after flattening: {labels.shape}")

    # Forward pass
    outputs = model(features)
    
    # Compute loss
    loss = criterion(outputs, labels)

    # Backward and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    # Compute accuracy
    predicted = torch.max(outputs.data, 1)[1]
    total_pixels += labels.numel()
    correct_pixels += (predicted == labels).sum().item()

    running_loss += loss.item()

epoch_loss = running_loss / len(features_list)
epoch_acc = correct_pixels / total_pixels

print('Epoch [{}/{}], Loss: {:.4f}, Pixel Accuracy: {:.4f}'.format(epoch+1, 10, epoch_loss, epoch_acc))
</code></pre>
<h2>
<a name="error-7" class="anchor" href="#error-7"></a>Error:</h2>
<p>ValueError                                Traceback (most recent call last)<br>
Cell In[2], line 94<br>
91 outputs = model(features)<br>
93 # Compute loss<br>
—&gt; 94 loss = criterion(outputs, labels)<br>
96 # Backward and optimize<br>
97 optimizer.zero_grad()</p>
<p>File c:\users\prave\appdata\local\programs\python\python39\lib\site-packages\torch\nn\modules\module.py:1194, in Module._call_impl(self, *input, **kwargs)<br>
1190 # If we don’t have any hooks, we want to skip the rest of the logic in<br>
1191 # this function, and just call forward.<br>
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks<br>
1193         or _global_forward_hooks or _global_forward_pre_hooks):<br>
 → 1194     return forward_call(*input, **kwargs)<br>
1195 # Do not call functions when jit is used<br>
1196 full_backward_hooks, non_full_backward_hooks = [], []</p>
<p>File c:\users\prave\appdata\local\programs\python\python39\lib\site-packages\torch\nn\modules\loss.py:1174, in CrossEntropyLoss.forward(self, input, target)<br>
1173 def forward(self, input: Tensor, target: Tensor) → Tensor:<br>
 → 1174     return F.cross_entropy(input, target, weight=self.weight,<br>
1175                            ignore_index=self.ignore_index, reduction=self.reduction,<br>
1176                            label_smoothing=self.label_smoothing)</p>
<p>File c:\users\prave\appdata\local\programs\python\python39\lib\site-packages\torch\nn\functional.py:3026, in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing)<br>
3024 if size_average is not None or reduce is not None:<br>
3025     reduction = _Reduction.legacy_get_string(size_average, reduce)<br>
 → 3026 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)</p>
<p>ValueError: Expected input batch_size (512) to match target batch_size (6815744).</p>
<p>Some of the print statement output:<br>
torch.Size([512, 1024])<br>
Number of classes: 18<br>
Input dimension: 524288<br>
Shape of the feature tensor before pass: torch.Size([512, 524288])<br>
Shape of the label tensor before pass: torch.Size([512, 1024])<br>
Shape of the label tensor after flattening: torch.Size([6815744])<br>
torch.Size([512, 18]) → x</p>