shalev396/foodvision-big

FoodVision Big

An image classifier that names the dish in a food photo out of the 101 categories of Food-101 (apple pie to waffles). torchvision's EfficientNet-B2, pretrained on ImageNet, gets a new 101-class head and is then fine-tuned end to end on all 75,750 Food-101 training photos.

Model

Architecture	torchvision efficientnet_b2 (initialised from EfficientNet_B2_Weights.IMAGENET1K_V1), classifier replaced by Dropout(0.3) -> Linear(1408, 101), every layer fine-tuned
Parameters	7,843,303
Input	an RGB image of any size
Preprocessing	model.get_transform(): resize to 288 (bicubic), center-crop 288x288, ImageNet mean/std
Output	softmax probabilities for all 101 classes ({"apple_pie": p, ..., "waffles": p}), or the top k with predict(image, top_k=5)
Files	model.safetensors (weights), config.json (class_names, dropout, pretrained), model.py (architecture + load() + Predictor), handler.py + requirements.txt (Inference Endpoint), metrics.json, assets/ (graphs)

Class names are stored in config.json in logit order (alphabetical, the order of torchvision's Food101). config.json records pretrained: true because the backbone started from ImageNet weights; model.load() always rebuilds the network with pretrained=False and then loads model.safetensors, so inference never downloads the torchvision weights.

Usage

Python

from huggingface_hub import hf_hub_download, snapshot_download
import sys
path = snapshot_download("shalev396/foodvision-big")
sys.path.insert(0, path)
import model
predictor = model.load(path, device="cpu")   # or "cuda"
image = hf_hub_download("shalev396/foodvision-big", "examples/2582289.jpg", repo_type="space")
print(predictor.predict(image, top_k=3))     # a PIL image works too; omit top_k for all 101 classes
# {'pizza': 0.9284, 'bruschetta': 0.006, 'lasagna': 0.0046}

Requirements (requirements.txt): torch, torchvision, pillow, huggingface_hub, safetensors.

Space API (free): the Space exposes /predict (image in, [top-5 label, seconds, device] out). curl and @gradio/client examples are in its README.

Inference Endpoint: deploy this repo from its page (Deploy -> Inference Endpoints, CPU or GPU). handler.py loads the model once (on cuda when the endpoint has a GPU) and accepts an image sent as raw bytes (Content-Type: image/jpeg) or as JSON {"inputs": "<base64 image>", "parameters": {"top_k": 5}}. It returns {dish: probability} for all 101 dishes, or the top_k most likely ones.

curl $ENDPOINT_URL -H "Authorization: Bearer $HF_TOKEN" -H "Content-Type: image/jpeg" --data-binary @dish.jpg

Training

• Data: Food-101, 101 dishes with 750 training and 250 test photos each: 75,750 train / 25,250 test (the Hub copy calls the test split validation). The training images carry some label noise on purpose (from the dataset authors); the test images were cleaned by hand.
• Recipe: full fine-tune of every layer. Adam(lr=1e-4) with a constant learning rate, cross-entropy with label smoothing 0.1, batch 32, 5 epochs, seed 42. Training images get TrivialAugmentWide before the standard preprocessing; test images only the preprocessing. Mixed precision (fp16 AMP) and torch.compile on the GPU. The epoch with the lowest test loss is kept (it was the last one).
• This checkpoint: converted from the original training run (PyTorch Deep Learning bootcamp, foodvision_big/train.py, trained on 2026-05-09 on one NVIDIA RTX 2080 Ti, about 22 minutes from the start of logging to the last epoch). The original .pth state_dict was mapped 1:1 into FoodVisionNet and saved as safetensors; on the 3 example images the converted model's probabilities match the original's exactly (max |diff| 0.0).

Full code: training/ · Colab. The notebook retrains the recipe (a GPU job), evaluates the new run next to this checkpoint on the full test split, and exports whichever is better (the deployed one wins ties).

Experiments

variant	accuracy
original training run (deployed)	0.8746

One variant has been trained so far: the original 5-epoch full fine-tune. Its test accuracy per epoch was 80.6% -> 84.9% -> 86.1% -> 86.5% -> 87.5%; test loss fell every epoch, so the last epoch is the one kept. Train accuracy stays below test accuracy because training images are augmented and the training loss includes label smoothing and dropout. Both curves were still rising at epoch 5.

Local re-check of the converted weights. To make sure the conversion kept the model intact, the safetensors checkpoint was re-evaluated per image on CPU on a slice of the test split: 1,010 images (101 parquet row groups of ethz/food101 spread evenly over the split, 10 random images from each). All 101 classes are present, but not equally: the Hub files are grouped by class and a row group can straddle two classes, so most classes have 10 images and a few have between 2 and 19.

evaluation	images	top-1 accuracy	top-5 accuracy	macro F1
original run, final epoch (TensorBoard log)	25,250 (all)	0.8746	not logged	not logged
converted weights, local re-check	1,010 (slice)	0.8792	0.9752	0.8688

The slice result (888 of 1,010 correct) is within its sampling error (about +/-1 percentage point) of the full-test number: the converted weights behave like the original run reported. (The conversion itself is checked exactly on the example images, see Training.) The slice is a sanity check, not a replacement for the full-test figure.

Evaluation

metric (test)	value
accuracy	0.8746

Top-1 accuracy of the deployed weights on all 25,250 test images, from the original run's log. The logged value is a mean of per-batch accuracies (batch 32; the last batch holds 2 images), which can differ from the per-image accuracy by at most about 0.1 percentage points. Top-5 accuracy and macro F1 were not logged in that run; on the 1,010-image slice they are 0.975 and 0.869.

Mistakes on the slice mostly happen between similar-looking dishes. The most frequent ones (true -> predicted) were filet mignon -> steak (3 images), then pork chop -> steak, steak -> pork chop, prime rib -> steak, ice cream -> chocolate mousse, lobster bisque -> clam chowder, fish and chips -> french fries and bruschetta -> caprese salad (2 each).

Limitations

• 101 dishes only. Every photo gets one of the 101 Food-101 labels, even a photo of a dog. There is no "other" class or rejection threshold.
• Look-alike dishes. Dishes that look alike (steak / filet mignon / prime rib / pork chop, lobster bisque / clam chowder) are confused most often; the top-5 list is more useful than the top-1 label for those.
• No validation split. Food-101 has only train and test. The recipe keeps the epoch with the lowest test loss, so the reported test number is slightly optimistic (it picked the last of 5 epochs, which had the lowest loss anyway).
• Per-class numbers are noisy. The per-class chart and confusion matrix come from the 1,010-image slice (about 10 images per class; the chart only shows classes with at least 5), so one image moves a class by about 10 points.
• Domain. Food-101 photos come from foodspotting.com (user-uploaded restaurant and home photos). Other styles (packaged food, drawings, cuisines outside the 101 classes) are untested.