sanyamseac/smai-a4-mlp

CIFAR-10 MLP (SMAI Assignment 4, Q1)

This is the MLP from Section 1.4.2, trained on the true labels (p = 0). It is one of the two models compared in Section 1.5.

Architecture

• 3072-64-10, ReLU hidden layer, no activation on the output layer. The 10 outputs are logits, and softmax turns them into class probabilities.
• 197,322 parameters.
• Initialisation: weights from N(0, 2/n_in) for the ReLU layer and N(0, 1/n_in) for the output layer, biases 0.

Preprocessing

1. Flatten the 32×32×3 uint8 image, in (height, width, channel) order, to 3072 values.
2. Divide by 255.
3. Subtract feature_mean and divide by feature_std. Both are stored in model.npz and were computed on the 50,000 images of the CIFAR-10 train part.

Class order: 0 airplane, 1 automobile, 2 bird, 3 cat, 4 deer, 5 dog, 6 frog, 7 horse, 8 ship, 9 truck.

Training

• Data: the 50,000 images of the CIFAR-10 train part (uoft-cs/cifar10).
• Plain SGD with learning rate 0.02, batch size 64 and no gradient accumulation.
• Categorical cross-entropy loss.
• 100 epochs without early stopping, in float64, with seed 13006.

Results after epoch 100

	accuracy	CCE loss
train (50,000 images)	0.7697	0.6530
test (10,000 images)	0.4613	2.2429

File

model.npz holds:

• W0, b0, W1, b1. Every W is stored as (inputs, outputs), so a layer computes h @ W + b.
• activations (one per layer) and loss.
• widths, feature_mean, feature_std and preprocessing.

Loading with NumPy

import numpy as np

d = np.load("model.npz")
h = (pixels.reshape(-1, 3072) / 255 - d["feature_mean"]) / d["feature_std"]  # pixels: uint8, shape (n, 32, 32, 3)
h = np.maximum(h @ d["W0"] + d["b0"], 0)  # ReLU
logits = h @ d["W1"] + d["b1"]
predicted_class = logits.argmax(axis=1)