SMAI Assignment 4: MLP on CIFAR-10 (PyTorch, true labels)
MLP trained in Task 1.4.2 with p = 0 (no label noise) and compared with the IOC-NN in Task 1.5.
Architecture
3072 -> 512 -> 256 -> 10, ReLU in the two hidden layers, no activation on the output (10 logits, apply softmax yourself). 1,707,274 parameters.
File mlp_p0_torch.npz: arrays W0, b0, W1, b1, W2, b2; W{i} has shape (in, out) (the transpose of the PyTorch weight), so logits = relu(relu(x @ W0 + b0) @ W1 + b1) @ W2 + b2.
Preprocessing
- Flatten each 32 x 32 x 3 image (
np.array(img).reshape(3072)). - Convert to float64 and divide by 255.
- For each of the 3072 columns subtract the mean and divide by the standard deviation of the 50,000 CIFAR-10 training images (divisor 1 if the standard deviation is 0).
Training
CIFAR-10 train part (50,000 images), cross-entropy loss, plain SGD, learning rate 0.05, batch size 64, 100 epochs, no early stopping. Weights ~ Normal(0, 2/n_in) in the hidden layers and Normal(0, 1/n_in) in the output layer, biases 0. Seed 13011 (torch.manual_seed(seed + 1), DataLoader generator seed + 2).
Results (CIFAR-10 test part, 10,000 images)
Train accuracy 1.0000, test accuracy 0.5490, test loss 4.9792, mean F1 0.5496.
WandB project: https://forge.coreweave.com/wandb/sushanthmantrala-iiith/SMAI-A4-MLP (run mlp-p0).