PyTorch FoodVision & VisionTransformer Repo

This repo serves to be a hub for projects relating to the Zero to Mastery PyTorch FoodVision Projects and Learnigns.

Project Technical Overview

A deeper technical look at the three Milestone Projects from this repo, in the order they were built. Each section covers the architecture, training setup, and results at a glance — click through to the project's own README.md for the full write-up, code-level details, and file-by-file breakdown.

---

1. FoodVision Mini — Experiment Tracking

code/FoodVision_Mini_Experiment_Tracking/ · full README

A systematic experiment-tracking workflow to find the best training configuration for a pizza/steak/sushi image classifier, rather than relying on a single one-off run.

Setup

• Task: 3-class image classification (pizza, steak, sushi).
• Models: two pretrained torchvision backbones used as frozen feature extractors, each with a new Dropout + Linear classifier head trained on top — EfficientNet_B0 (dropout 0.2, 1280 in-features) and EfficientNet_B2 (dropout 0.3, 1408 in-features).
• Data: two dataset sizes (10% and 20% subsets of pizza_steak_sushi) sharing the same held-out test set, so results are directly comparable.
• Grid search: every combination of {10%, 20% data} × {5, 10 epochs} × {EffNetB0, EffNetB2} — 8 experiments total — each trained with Adam (lr=0.01) and CrossEntropyLoss, with per-epoch train/test loss and accuracy logged to TensorBoard via a SummaryWriter.

Results

• More data and more epochs generally improved test accuracy, at the cost of training time.
• EfficientNet_B2 on the 20% split trained for 10 epochs won out as the best of the 8 configurations, and that checkpoint correctly classified both held-out test images and an out-of-dataset custom photo — confirming it generalizes.
• This winning model/data combination is the one carried forward into both later projects.

---

2. Vision Transformer — Paper Replication

code/Vision_Transformer/ · full README

A from-scratch implementation of the Vision Transformer (ViT) architecture directly from the equations in An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, benchmarked against a pretrained ViT fine-tuned on the same classification task.

Setup

• Patch + position embedding (Eq. 1): input images split into 16×16 patches via a Conv2d acting as the patcher, flattened into a sequence, prepended with a learnable class token, and summed with learnable position embeddings — following ViT-Base hyperparameters (patch size 16, hidden size 768).
• Multihead self-attention (Eq. 2): LayerNorm → nn.MultiheadAttention (12 heads, batch_first=True) with a residual connection.
• MLP block (Eq. 3): LayerNorm → Linear → GELU → Dropout → Linear → GELU → Dropout, also residual.
• Full ViT model (Eq. 4): 12 stacked TransformerEncoderBlocks (MSA + MLP) with embedding layers and a LayerNorm + Linear head — ViT-Base defaults throughout (12 layers, 12 heads, embedding dim 768, MLP size 3072).
• From-scratch training: the custom ViT trained for 10 epochs on pizza_steak_sushi, using the paper's Section 4.1 optimizer settings (Adam, lr=3e-3, betas=(0.9, 0.999), weight_decay=0.3).
• Pretrained comparison: torchvision's ImageNet-pretrained vit_b_16, base frozen, with a fresh 3-class Linear head fine-tuned for 10 epochs (Adam, lr=1e-3).

Results

• The from-scratch ViT is architecturally correct per the paper, but — as the paper itself predicts — ViT lacks a CNN's convolutional inductive bias and needs large-scale pretraining data to generalize well; trained on only ~225 images it underperforms a comparably-trained CNN, as expected.
• The pretrained vit_b_16, already carrying ImageNet-learned visual features, converges quickly and reaches usable accuracy by fine-tuning only its classifier head — the practical "pretrain then fine-tune" justification that motivates transfer learning throughout this repo.

---

3. FoodVision Deployed — Model Deployment

code/FoodVision_Deployed/ · full README

Takes the best model candidates from the two prior projects and ships one as a live, interactive web demo.

Setup

• Candidate comparison: EfficientNetB2 (from project 1) and vit_b_16 (from project 2) were each retrained for 10 epochs on the winning 20%-data split, with test loss/accuracy, parameter count, and saved model file size reported for each — framing the choice as an accuracy-vs-size tradeoff rather than accuracy alone.
• Model selection: EfficientNetB2 was deployed — it reaches comparable accuracy to vit_b_16 at a fraction of the parameter count and on-disk size, which matters for a responsive, easily-hosted demo.
• Deployment package (demos/foodvision_mini/): a self-contained Gradio app —
  • model.py rebuilds the EfficientNetB2 architecture so the saved .pth weights can be reloaded independent of the training pipeline.
  • app.py loads those weights, defines a predict() function (image transform → inference → softmax class probabilities + timing), and wires it into a gr.Interface with example images.
  • Pinned requirements.txt so the environment reproduces exactly on a hosting platform.

Results

• The packaged demo runs standalone from just app.py, model.py, the saved weights, and requirements.txt, and is structured to deploy directly to a Hugging Face Gradio SDK Space for public access — turning the earlier two projects' best model into something a non-technical user can try in a browser.

---

For the full list of tutorial progress and a higher-level summary table, see the main README.md.