Aakarsh14/smai-a4-mlp

MLP on CIFAR-10 (SMAI Assignment 4, Q1)

The MLP trained on the true labels (p = 0) in Task 1.4.2 and compared in Section 1.5 of my SMAI Assignment 4 (mlp.ipynb).

Architecture

nn.Sequential(nn.Linear(3072, 256), nn.ReLU(), nn.Linear(256, 10)) in float64, 789,258 parameters. The output is the ten logits; the loss is softmax cross-entropy.

• Constraint: none; all weights are free.
• Initialisation: weights N(0, 2/3072) in layer 1 and N(0, 1/256) in layer 2, zero biases.

Preprocessing

1. Load uoft-cs/cifar10 and take the img column as uint8 arrays of shape (n, 32, 32, 3).
2. Flatten each image to 3072 values in (row, column, channel) order: img.reshape(-1, 3072).
3. Divide by 255 (in float64).
4. Standardise every column with the mean and standard deviation of the 50,000 images of the train split (preprocessing.npz: mean, std, 3072 values each; a zero std would be set to 1, but none occurs).

Training

Train split (50,000 images) with the true labels, 100 epochs of torch.optim.SGD (lr 0.01, no momentum), batch size 64, the batch order from torch.Generator().manual_seed(11009), and torch seed 11008 before building the networks.

Results (after epoch 100)

	train	test (10,000 images)
accuracy	0.9938	0.5083
mean CCE loss	0.0565	2.8182

Best test accuracy over the 100 epochs: 0.5257 at epoch 16.

Files

• model.pt: the state dict (0.weight, 0.bias, 2.weight, 2.bias).
• model.npz: the same weights as W0 (3072 x 256), b0, W1 (256 x 10), b1, with activations = ['relu', 'identity'] and loss = cce, the format of Model.save_to in the notebook.
• preprocessing.npz: mean and std of step 4.

Loading

import numpy as np
import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download

repo = "Aakarsh14/smai-a4-mlp"
net = nn.Sequential(nn.Linear(3072, 256), nn.ReLU(), nn.Linear(256, 10)).double()
net.load_state_dict(torch.load(hf_hub_download(repo, "model.pt")))
pre = np.load(hf_hub_download(repo, "preprocessing.npz"))
x = (images.reshape(-1, 3072) / 255.0 - pre["mean"]) / pre["std"]    # images: uint8 (n, 32, 32, 3)
predicted_class = net(torch.from_numpy(x)).argmax(dim=1)