XiElonMAsk/cropguard-models

CropGuard - Crop Disease Classifier (38 classes)

ResNet50 fine-tuned on PlantVillage, exported to ONNX and dynamically quantised to INT8 for CPU-only serving. Part of CropGuard, an end-to-end MLOps pipeline.

Results (held-out test set, n=8,125)

Model	Accuracy	Macro-F1	Note
fp32	0.9911	0.9865	reference
INT8 (dynamic)	not evaluated	not evaluated	not served - see below

Macro-F1 is the metric to read here, not accuracy: the dataset is imbalanced ~36x, so accuracy is dominated by the largest classes.

Only cropguard.onnx (fp32) is published. Dynamic INT8 quantization was tried and is not shipped: quantize_dynamic rewrites every Conv into ConvInteger, which ONNX Runtime's CPU backend has no optimized kernel for. Measured on an Intel Alder Lake CPU it ran at 1567 ms/image against 19 ms/image for fp32 - a 75x regression in exchange for 4x less disk. Dynamic quantization suits MatMul-dominated models (Transformers, RNNs), not CNNs; the correct approach for a Conv-heavy network is static quantization with a calibration set, which emits the optimized QLinearConv. That is not built yet.

The split is grouped by leaf, and that matters

PlantVillage contains 54,305 images of only ~7,600 distinct physical leaves - roughly 7 photographs of each. A standard per-image stratified split scatters those near-duplicates across train and test, so a model can score well by memorising leaf identity rather than learning disease morphology. Measured on this dataset, a naive split leaves 74.2% of test images sharing a leaf with training.

This model was trained on a leaf-grouped split instead:

	Naive stratified	Grouped (used here)
test images sharing a leaf with train	74.2%	0.0%
train / val / test	-	38,008 / 8,172 / 8,125

So the accuracy above is measured on a holdout with no leaf overlap. Note that it is not much lower than typically published PlantVillage figures - the honest reading is that this dataset is genuinely easy, not that leakage was inflating everything.

Intended use

Identifying disease on single leaves photographed against a plain background, matching the PlantVillage capture protocol.

Limitations - read before deploying this

• Lab images, not field images. Every training image is a detached leaf on a uniform background under controlled lighting. Real photographs from a farm - variable lighting, occlusion, multiple leaves, soil backgrounds - are a different distribution, and published work on this dataset reports large drops there. This model has not been evaluated on field photographs.
• Per-class metrics for rare classes are noisy. Eight classes have fewer than 100 test images; the smallest (Potato___healthy) has 24. A recall of 0.833 there is 4 mistakes, and its confidence interval spans roughly +/-15 points. Do not read those per-class numbers as precise.
• Confidence is not calibrated. Training used label smoothing (0.1), which deliberately caps confidence, so predicted probabilities are expected to understate. No temperature scaling has been fitted.
• uncertainty is predictive entropy, not epistemic uncertainty. It cannot distinguish an ambiguous input from one far outside the training distribution - an out-of-distribution image can produce confidently wrong output with low entropy.
• 38 classes across 14 crops only. Anything outside that set is silently forced into one of them.

Training

ResNet50 (timm, ImageNet-pretrained), 224x224, batch 64, AdamW (lr 3e-4, weight decay 1e-4), cosine schedule, label smoothing 0.1, medium augmentation, 12 epochs, mixed precision. Checkpoint selected on val_f1_macro.

Usage

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download

path = hf_hub_download("XiElonMAsk/cropguard-models", "cropguard.onnx")
session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])

# Preprocessing must match training: resize short side to 256, centre-crop 224,
# scale to [0,1], normalise with ImageNet mean/std, NCHW.
logits = session.run(["logits"], {"input": batch})[0]

cropguard.serving.model_loader in the repo implements exactly that preprocessing.

Citation

Dataset: Mohanty, Hughes & Salathe (2016), Using deep learning for image-based plant disease detection, Frontiers in Plant Science.