AI-MED-AGH/Recruitment-Task-3-Model

DeepWeeds weak recruitment baseline

This repository contains the intentionally weak model supplied for the optional AI-MED AGH DeepWeeds recruitment task. It is meant for reproducible audit and improvement work, not deployment. The checkpoint is the frozen flawed arm of a controlled comparison; the reference checkpoint is not included.

Model and fixed training

SmallDeepWeedsCNN is a 23,881-parameter CNN trained from scratch. It has three convolution blocks (1→16→32→64 channels) with ReLU activations, two 2×2 max-pooling layers, global adaptive average pooling, and a linear 64→9 classifier. It has no pretrained weights, augmentation, Batch Normalization, Dropout, class weights, class-aware sampling, focal loss, or scheduler.

Both comparison arms used the unchanged published DeepWeeds fold 0, seed 2026, 128×128 inputs, batch size 64, 12 epochs, Adam at 1e-3, ordinary unweighted cross-entropy, and checkpoint selection by minimum validation loss. This checkpoint was selected at epoch 9 (validation loss 1.254291292089491) and trained in 159.56 seconds with zero data-loader workers after a Windows worker-process failure.

Deliberately flawed preprocessing

The published checkpoint expects one-channel grayscale input. The pipeline resizes to 128×128, uses PILToTensor, casts raw uint8 values to float32 without dividing by 255, then applies mean 0.5 and standard deviation 0.5 normalization as though the input were already in [0,1]. It therefore maps the original [0,255] range to [-1,509]. This grayscale plus unscaled raw-float normalization is intentionally flawed.

Frozen fold-0 test results

The 3,507-image test split is imbalanced: class 8 has 1,822 examples (51.95%). A class-8-only predictor has accuracy 0.5195323638437411 and macro-F1 0.07597839911594838. This model predicts class 8 for 2,961 of 3,507 images and never predicts class 1.

Metric	Value
Accuracy	0.5420587396635301
Balanced accuracy	0.2073904927878915
Macro-F1	0.20664976855491451
Parameters	23,881
Recorded test inference time	2.753719700005604 s

The recorded runtime used CPython 3.12.14 on Windows 11.

Class	Support	Precision	Recall	F1
0	226	0.39361702127659576	0.16371681415929204	0.23125
1	213	0.0	0.0	0.0
2	207	0.45588235294117646	0.1497584541062802	0.22545454545454546
3	205	0.22448979591836735	0.05365853658536585	0.08661417322834646
4	213	0.75	0.09859154929577464	0.17427385892116182
5	202	0.29310344827586204	0.4207920792079208	0.34552845528455284
6	215	0.4	0.018604651162790697	0.035555555555555556
7	204	0.7142857142857143	0.024509803921568627	0.04739336492890995
8	1822	0.5764944275582573	0.9368825466520307	0.7137779636211583

The RGB reference arm scored macro-F1 0.21534935157900917; the flawed arm scored 0.20664976855491451. Their directional gap was only 0.00869958302409466, so the declared 0.10 publication gate failed. This small comparison does not establish meaningful measured degradation caused by the preprocessing flaw. Publication was subsequently authorized for recruitment use with these failures disclosed.

Load and run inference

Install the listed dependencies, then run from this repository directory:

python -m pip install -r requirements.txt

from pathlib import Path

import torch
from PIL import Image
from safetensors.torch import load_file

from data import build_comparison_transforms
from model import SmallDeepWeedsCNN

root = Path(".")
model = SmallDeepWeedsCNN(num_classes=9, in_channels=1)
model.load_state_dict(load_file(str(root / "model.safetensors"), device="cpu"), strict=True)
model.eval()

with Image.open("example.jpg") as image:
    batch = build_comparison_transforms()["bad_test"](image).unsqueeze(0)
with torch.inference_mode():
    logits = model(batch)  # shape: (1, 9)
    probabilities = torch.softmax(logits, dim=1)

Reproduce test evaluation

Download AI-MED-AGH/Recruitment-Task-3, extract original/images.zip, and keep the downloaded labels/ directory unchanged. From this model repository run:

python evaluate.py evaluate --run-kind bad --labels-dir /path/to/Recruitment-Task-3/labels --images-dir /path/to/extracted/images --checkpoint model.safetensors --output-dir evaluation --device cpu --num-workers 0

The command writes evaluation/metrics.json and evaluation/predictions.csv. The frozen outputs from the original evaluation are included here. SHA256SUMS covers every public file except the manifest itself.

Intended use and limitations

Use this model only as a weak, auditable starting point for the optional recruitment exercise. It is not suitable for agricultural decisions, safety-critical use, or claims about performance outside this one published fold. Limitations include the deliberately incorrect preprocessing, loss of color information, severe class imbalance and majority-class behavior, no prediction for class 1 on the frozen test split, low macro-F1 and balanced accuracy, a single seed/fold, no augmentation or imbalance handling, and evaluation on the source dataset only. The comparison includes one reference and one flawed run and does not isolate or quantify general causal effects beyond those runs.

Data, source, and license

The AI-MED AGH dataset mirror preserves 17,509 DeepWeeds images and the authors' published folds. Original source: AlexOlsen/DeepWeeds. Paper: Olsen et al., DeepWeeds. The images and annotations are CC BY 4.0 and are not redistributed here. The code and model repository contents are provided under Apache License 2.0; see LICENSE.