yennik16/2026-24679-switch-vs-switch2-classifier

Switch OLED vs Switch 2 image classifier

Binary image classifier over photographs of two Nintendo handhelds: 0 = Switch OLED (TOTK edition), 1 = Switch 2. A frozen ImageNet backbone with a small trained head, selected by a budgeted random search. Built for Homework 2 (AutoML for neural networks) of Designing with AI by YOUR NAME.

Purpose

Given a 224 × 224 RGB photograph prepared as described below, predict which of the two consoles it shows. Intended use is coursework and demonstration of transfer learning on a small self-collected image set. It is not a general Switch-model identifier — see Known failure modes, which is the section that matters most here.

Data origin and splits

• Dataset: pakiino/2026-24679-switch-image-hw1, CC-BY-4.0. Photographed by a classmate; not my own dataset and not my Project 1 partner's.
• Original photographs were split with class stratification before any variant was created. Validation and test contain unaugmented parents only, and a source never crosses partitions. The notebook asserts all three.
• Splits used exactly as shipped: train 374 rows (22 original photographs + 352 offline variants), validation 5 photographs, test 5 photographs.

Input resolution and expected preprocessing

Input resolution: 224 × 224 RGB.

An input must reproduce this pipeline exactly, in order:

1. Apply EXIF orientation; convert to RGB. (done by the dataset for its own rows)
2. Resize preserving aspect ratio; pad to 224 × 224 with (128, 128, 128). (dataset)
3. Scale pixels to [0, 1]. (this model)
4. Normalize with ImageNet statistics — mean [0.485, 0.456, 0.406], std [0.229, 0.224, 0.225]. (this model)

No resize or center crop is applied at step 3–4: images arrive already prepared, and a center crop would discard the padding the dataset author added on purpose. Feeding an unpadded or differently padded image is out of distribution.

Augmentation pipeline

Augmentation is offline and ships with the dataset; this model applies no augmentation at inference and none of its own at training time. Each original training photograph contributes 4 independent variants per method, and transforms are never stacked:

method	parameter range
brightness	factor 0.4–0.7 or 1.4–2.0, chosen with equal probability
contrast	factor 0.35–0.65 or 1.5–2.2, chosen with equal probability
rotation	15–30° either direction on a fixed canvas; exposed corners filled from the parent, edges may clip
blur	Gaussian radius 2.0–4.5 px

Which methods enter training was itself a search axis (aug_set ∈ none / photometric / all). The selected configuration trains on: brightness, contrast.

Architecture

• Backbone: efficientnet_b0, ImageNet-pretrained, frozen (4,007,548 parameters, no gradients). The classifier is replaced with identity so the pooled embedding is the output.
• Head: linear(256) → ReLU → dropout → linear(2), preceded by feature standardization whose mean and standard deviation are stored as module buffers.
• Trainable parameters: 328,450 — this is the "compact network" the assignment asks for; the backbone contributes representation, not capacity.

AutoML search

Random search, 24 trials, sampled with seed 24679:

axis	values
backbone	mobilenet_v3_small, resnet18, efficientnet_b0
hidden	0, 64, 128, 256
dropout	0.0 – 0.6
optimizer	adam, adamw, sgd (momentum 0.9)
learning_rate	1e-4 – 1e-1, log scale
weight_decay	1e-6 – 1e-2, log scale
aug_set	none, photometric, all

Validation scheme — three splits, three jobs. Selection used 4-fold cross-validation over the 22 original training photographs, stratified by label; each fold trained on the other originals plus their variants, with every descendant of a held-out photograph removed, and scored on the held-out originals. Early stopping used the shipped 5-photograph validation split and nothing else, so the stopping signal never entered a fold's score. The test split was untouched until final evaluation.

Early stopping. Each fit ran to at most 60 epochs and stopped after 10 epochs without validation-loss improvement, restoring the best weights. The final refit ran 13 epochs and restored epoch 3; the median best epoch across CV folds was 3, which is the cross-check that the stopping point was not an artifact of one noisy run.

Selected hyperparameters

• backbone: efficientnet_b0
• hidden: 256
• dropout: 0.148
• optimizer: adam
• learning_rate: 0.01237
• weight_decay: 0.000233
• aug_set: photometric

Leaderboard (top 10 of 24 trials, by CV balanced accuracy)

backbone	hidden	dropout	optimizer	learning_rate	weight_decay	aug_set	cv_balanced_accuracy	cv_std
efficientnet_b0	256	0.148	adam	0.0124	0.0002	photometric	1	0
efficientnet_b0	256	0.113	adamw	0.0396	0.0002	none	0.9583	0.0833
efficientnet_b0	256	0.436	sgd	0.0024	0.0001	all	0.9583	0.0833
efficientnet_b0	256	0.053	adam	0.0002	0.002	none	0.9583	0.0833
efficientnet_b0	256	0.367	adamw	0.0005	0.0044	all	0.9583	0.0833
efficientnet_b0	64	0.523	adamw	0.0003	0.0007	all	0.9583	0.0833
efficientnet_b0	64	0.04	adamw	0.0018	0.0001	all	0.9583	0.0833
efficientnet_b0	128	0.579	adamw	0.0009	0.0073	none	0.9583	0.0833
efficientnet_b0	256	0.168	sgd	0.093	0	all	0.9583	0.0833
efficientnet_b0	64	0.517	adam	0.0059	0.0002	photometric	0.9167	0.1667

Metrics

All metrics are unitless fractions in [0, 1].

Selection metric — 4-fold CV over 22 original training photographs: balanced accuracy 1.000 ± 0.000 (mean ± s.d. across folds).

Held-out test set (5 photographs), used once:

	Accuracy	Balanced accuracy	Macro F1
Selected model	0.4	0.5	0.286
Training-majority baseline	0.6	0.5	0.375

Per class:

	precision	recall	f1-score	support
Switch OLED	0	0	0	3
Switch 2	0.4	1	0.571	2

Confusion matrix:

	pred Switch OLED	pred Switch 2
true Switch OLED	0	3
true Switch 2	0	2

Test accuracy 40.0% (2/5), 95% Clopper–Pearson interval 5.3% – 85.3%; balanced accuracy 0.500, macro F1 0.286. With 5 photographs one error moves accuracy by 20 points, so the interval, not the point estimate, is the honest summary — and the interval still understates the true uncertainty, for the reason in the next section.

Known failure modes

1. One physical unit per class — the dominant limitation. The dataset contains exactly one Switch OLED and one Switch 2. Every split holds photographs of those same two objects. A high test score means the model can re-recognize these two consoles, not that it learned what separates the models. It may be keying on stickers, wear, dock, background or room lighting. Expect it to fail on a different physical unit.
2. Views are not grouped. The dataset card notes front and back photographs of a unit were split by label rather than grouped, so a view seen in training can reappear in evaluation.
3. Effective training size is 22 photographs, not 374 rows. The offline variants are echoes; a large train/validation gap is expected and uninformative.
4. Augmentation can destroy evidence. Strong brightness and contrast erase surface detail, rotation can crop the distinguishing edge out of frame, and blur hides small features such as the Joy-Con rail and logo. Inputs resembling those extremes are unreliable.
5. Out-of-distribution inputs are unhandled. The model has two classes and always picks one. A Switch Lite, an original Switch, a different colorway, a boxed or docked console, a screenshot, or any other object will still receive a confident Switch-OLED-or-Switch-2 prediction. There is no abstain option and the softmax probability should not be read as a confidence calibrated on anything.
6. Backgrounds and lighting are a narrow sample. A few rooms, one phone camera, one photographer.
7. Frozen backbone. ImageNet features were never adapted to this domain; fine-tuning was deliberately out of scope at 22 training photographs, where it would overfit rather than help.

Ethical notes

The dataset contains no people, faces, or personally identifying information, and camera metadata was stripped by its author. The realistic harm is misplaced confidence — a model that looks near-perfect on five photographs of two consoles could be presented as a working product identifier and be wrong on essentially any other unit. Any downstream use should treat the output as a hint a human checks.

License

CC-BY-4.0, matching the source dataset. Attribution to the dataset author for the photographs and to YOUR NAME for the model.

Hardware and compute budget

• Hardware: Tesla T4 (device type cuda), Google Colab.
• Feature extraction, once per backbone: mobilenet_v3_small 13 s, resnet18 2 s, efficientnet_b0 15 s.
• Search: 24 trials × 4 folds, ≤ 60 epochs each, patience 10 — 25 s wall-clock.
• Total ≈ 56 s (0.9 minutes) of compute, excluding installs and dataset download.
• Versions: torch 2.11.0+cu128, torchvision 0.26.0+cu128, seed 24679.

AI usage disclosure

Generative AI (Claude, Anthropic) was used as a coding assistant to structure this notebook — the integrity checks, the cached-feature search loop, the grouped cross-validation, and the prose of this card. Every cell was executed and checked by me, and the interpretation of the results, including the one-unit-per-class limitation, is my own. No images were generated by AI; the dataset is a classmate's own photography, used unchanged.

How to use

import json, torch, numpy, huggingface_hub
from PIL import Image

repo = "yennik16/2026-24679-switch-vs-switch2-classifier"
config = json.load(open(huggingface_hub.hf_hub_download(repo, "config.json")))
state = torch.load(huggingface_hub.hf_hub_download(repo, "switch_classifier.pt"), map_location="cpu")

# Rebuild SwitchClassifier exactly as defined in the notebook, then:
model.load_state_dict(state)
model.eval()

# Preprocess: EXIF -> RGB -> aspect-preserving resize -> pad to 224 with (128,128,128)
# -> scale to [0,1] -> normalize with ImageNet mean/std.

The SwitchClassifier class definition lives in the training notebook; the weights alone are not a standalone package. Reload with the same torch and torchvision versions listed above.