SambaGuard AI — Fall Armyworm Detection Model v2
Overview
SambaGuard AI is a YOLOv8-based object detection model for the early detection of Fall Armyworm (FAW) infestations in smallholder maize fields. The model was developed as part of a research project at Dedan Kimathi University of Technology, Kenya, targeting edge deployment on resource-constrained devices such as the Raspberry Pi 4.
The system is designed to address a documented gap in FAW management: smallholder farmers cannot scout frequently enough to catch FAW in its narrow early intervention window, and existing tools do not provide severity-graded, localized guidance. SambaGuard AI aims to close this gap through automated, camera-based detection at the field level.
This repository contains the trained model weights, evaluation metrics, training visualizations, and full experiment outputs for version 2.
Detected Classes
The model detects four Fall Armyworm-related classes:
| Class ID | Class Name |
|---|---|
| 0 | Fall Armyworm Egg |
| 1 | Fall Armyworm Frass |
| 2 | Fall Armyworm Larva |
| 3 | Fall Armyworm Larval Damage |
Model Architecture
- Base architecture: YOLOv8s (Ultralytics)
- Framework: PyTorch
- Input resolution: 640 x 640 pixels
- Parameters: 11.1M
- GFLOPs: 28.4
Dataset
The model was trained on a cleaned and validated subset of the KaraAgro AI Maize dataset, accessed via Dataset Ninja (https://datasetninja.com/kara-agro-ai-maize) in Supervisely format. The original dataset was published by KaraAgro AI and is available on Harvard Dataverse (DOI: 10.7910/DVN/CXUMDS, License: CC0 1.0).
Dataset preparation steps for v2:
- Removal of non-FAW classes (healthy maize, maize streak disease) with 10% retained as background negative samples
- Detection and removal of duplicate file extensions
- Repair of invalid YOLO annotations with out-of-bounds bounding box coordinates
- Per-image bounding box coordinate clipping to the valid range of 0.0 to 1.0
- Image-label pair verification across all splits
- Stratified train/val/test split to preserve class proportions
- Oversampling of the egg class in the training split only to address class imbalance
Final dataset verification confirmed no missing images, no missing labels, no invalid annotations, and no unmatched image-label pairs across all splits.
| Split | Images | Labels |
|---|---|---|
| Train | 5,709 | 5,709 |
| Val | 1,320 | 1,320 |
| Test | 664 | 664 |
Training Configuration
| Parameter | Value |
|---|---|
| Base model | yolov8s.pt (COCO pretrained) |
| Epochs | 100 |
| Early stopping patience | 20 |
| Batch size | 16 |
| Image size | 640 |
| Optimizer | Auto (AdamW) |
| Learning rate schedule | Cosine annealing |
| Mixed precision (AMP) | Enabled |
| Mosaic augmentation | Enabled |
| HSV augmentation | Enabled |
| Horizontal flip | Enabled |
| Hardware | NVIDIA Tesla T4 GPU |
Validation Results
Best checkpoint obtained at epoch 55.
Overall performance:
| Metric | Value |
|---|---|
| Precision | 0.479 |
| Recall | 0.376 |
| mAP50 | 0.347 |
| mAP50-95 | 0.137 |
Per-class performance:
| Class | Precision | Recall | mAP50 | mAP50-95 |
|---|---|---|---|---|
| Fall Armyworm Egg | 0.336 | 0.250 | 0.198 | 0.085 |
| Fall Armyworm Frass | 0.374 | 0.233 | 0.196 | 0.066 |
| Fall Armyworm Larva | 0.782 | 0.716 | 0.726 | 0.303 |
| Fall Armyworm Larval Damage | 0.425 | 0.304 | 0.267 | 0.093 |
The larva class achieves the strongest detection performance (mAP50 = 0.726), which is consistent with its larger visual signature and stronger representation in the training data. Egg and frass detection remain areas for improvement in subsequent versions.
Comparison with v1 Baseline
| Metric | v1 Baseline | v2 Clean Dataset | Change |
|---|---|---|---|
| Precision | 0.486 | 0.479 | -0.007 |
| Recall | 0.401 | 0.376 | -0.025 |
| mAP50 | 0.368 | 0.347 | -0.021 |
| mAP50-95 | 0.144 | 0.137 | -0.007 |
| Larva mAP50 | 0.768 | 0.726 | -0.042 |
| Epochs trained | 50 | 100 (best at 55) | — |
Overall metrics show a slight decrease from v1. This is being investigated and attributed to differences in the effective training set composition after dataset cleaning — specifically the removal of out-of-bounds annotations that were previously counted as valid training signal. Further analysis is ongoing. See experiment notes below.
Repository Structure
SambaGuard-v2/
├── weights/
│ ├── best.pt
│ └── last.pt
├── metrics/
│ └── results.csv
├── plots/
│ ├── results.png
│ ├── confusion_matrix.png
│ ├── confusion_matrix_normalized.png
│ ├── PR_curve.png
│ ├── P_curve.png
│ ├── R_curve.png
│ ├── F1_curve.png
│ ├── labels.jpg
│ └── labels_correlogram.jpg
├── training_samples/
├── validation_samples/
└── README.md
Usage
Install the required library:
pip install ultralytics
Run inference on an image:
from ultralytics import YOLO
model = YOLO("weights/best.pt")
results = model.predict(
source="image.jpg",
imgsz=640,
conf=0.25
)
results[0].show()
Experiment Notes
This version introduced substantial dataset cleaning improvements over v1, including annotation repair, duplicate removal, bounding box validation, and egg class oversampling. Despite these improvements, overall mAP50 is marginally lower than v1 (0.347 vs 0.368).
This result is consistent with the hypothesis that dataset cleaning removed a number of noisy annotations that, while technically invalid, previously provided approximate training signal. The v2 dataset is cleaner and more reliable, but smaller in effective annotated objects. Subsequent experiments (v3, v4) will isolate the effect of augmentation and image resolution changes to determine the best path forward.
Planned Experiments
| Version | Key Change | Status |
|---|---|---|
| v1 | Baseline — 50 epochs, uncleaned dataset | Complete |
| v2 | Clean dataset — 100 epochs | Complete (this model) |
| v3 | Stronger augmentation for egg and frass classes | Planned |
| v4 | Image size increased to 960 | Planned |
Future Work
- Improving egg and frass detection through targeted augmentation
- Edge export to TFLite and ONNX for Raspberry Pi 4 deployment
- Real-time inference benchmarking on edge hardware
- Integration with a Swahili-language LLM advisory layer for farmer-facing SMS guidance
- Field validation with smallholder farmers in Kenya
Related Resources
- GitHub Repository: https://github.com/aneneahs-kanaks/SambaGuard
- Dataset Ninja: https://datasetninja.com/kara-agro-ai-maize
- Original Dataset (Harvard Dataverse): https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/CXUMDS
Author
Annastacia Ndunge Electrical and Electronics Engineering Dedan Kimathi University of Technology, Kenya
GitHub: https://github.com/aneneahs-kanaks Hugging Face: https://huggingface.co/ndunge23 LinkedIn: https://www.linkedin.com/in/annastacia-ndunge-809a21361--- license: apache-2.0
- Downloads last month
- 16