vikeshkansal/smai-a4-mlp

MLP on CIFAR-10 (SMAI Assignment 4, Section 1.5)

A multi-layer perceptron trained in PyTorch with the architecture selected in Task 1.2.5.

Architecture

• Layers: 3072 → 128 → 10, 394,634 parameters.
• Hidden activation: ReLU. The output layer is linear and gives 10 logits; softmax gives the class probabilities.

Training

• Data: the train part of uoft-cs/cifar10 (50,000 images), true labels (noise level p = 0).
• Plain SGD, learning rate 0.01, batch size 64, 1 gradient accumulation step, cross-entropy loss, 100 epochs, no early stopping, float64.
• Seeds: torch.manual_seed(1127) before building the model, torch.Generator().manual_seed(1128) for the batch order.
• After the last epoch, on the test part (10,000 images): accuracy 0.4770, loss 2.6550, mean F1 0.4772. Training accuracy 0.8840.

Preprocessing

1. Flatten each 32 x 32 x 3 uint8 image to 3072 numbers (img.reshape(3072), height, width, channel order).
2. Convert to float64 and divide by 255.
3. Standardise every column: (x - mean) / std with mean and std from preprocessing.npz, computed on the 50,000 training images (population std; a zero std is replaced by 1).

Files

• model.npz: arrays W0, b0, W1, b1, ... in layer order. W has shape (n_in, n_out), so a layer computes x @ W + b.
• preprocessing.npz: arrays mean and std, each of length 3072.

Usage

import numpy as np

w, pre = np.load("model.npz"), np.load("preprocessing.npz")
x = (img.reshape(-1, 3072).astype(np.float64) / 255 - pre["mean"]) / pre["std"]
for i in range(2):
    x = x @ w[f"W{i}"] + w[f"b{i}"]
    if i < 2 - 1:
        x = np.maximum(0, x)  # ReLU
logits = x