NexusDwin/sailswarm-yolo-thermal

SailSwarm thermal (LWIR) object detector, first round: a negative result with a signal

YOLOv8n detectors for the FLIR Lepton 3.0 frame (160 x 120, inferno palette inverted to intensity) of the SailSwarm obstacle-detection box, trained 2026-09-12/13 after the first night lake trials. Published so that the next round (a night session with targets at 3 to 8 m and about 500 pure-thermal audits) starts from these weights and numbers rather than from scratch. Not used in the ICRA 2027 submission and not deployed.

Weights

file	trained on	role
weights/yolov8n_llvip_massmind_lowres_best.pt	LLVIP (15,488 night RGB/LWIR pairs, person boxes) + MassMIND (2,916 harbour LWIR frames, boxes from the instance masks), downscaled to 160 px so that target scale matches the Lepton; 30 epochs at 640	pretrain (val mAP50 0.747 on the low-res external data)
weights/yolov8n_konstanz_thermal_ft2_best.pt	fine-tune of the above on 773 Konstanz thermal-space frames (2026-08-26 + 2026-09-08 audits warped into the Lepton frame, plus 100 boxes drawn directly on the thermal image), 150 epochs at 640	the model to start from
weights/yolov8n_konstanz_thermal_ft2_cocoinit_best.pt	same fine-tune from the COCO checkpoint	control

Classes as in NexusDwin/sailswarm-yolo-audited (boat, buoy, duck, other, person, structure).

Evaluation on the held-out night window (20:41 to 21:00 on 2026-09-08, 74 frames, 54 boxes, RGB black)

init	conf	recall (person / boat)	precision	FP per frame
LLVIP+MassMIND low-res pretrain	0.25	0.11 (3/39, 3/15)	1.00	0.00
LLVIP+MassMIND low-res pretrain	0.10	0.35 (13/39, 6/15)	0.84	0.11
COCO	0.10	0.15 (6/39, 2/15)	0.88	0.01
classical hot-blob detector (reference, 101 night frames)		0.77	0.17	3.3

runs/thermal_det_eval_*.json hold the per-class counts; runs/*/results.csv the training curves (the ultralytics val mAP of the fine-tunes is near zero because the in-training val split is the tiny thermal-space set; the table above is the audited night test).

What this says

• The external pretrain is worth about +0.20 recall over COCO at the same precision.
• The 100 thermal-space boxes lifted night recall from 0.10 (fisheye-warped labels only) to 0.35 at 0.84 precision: a high-precision, low-recall operating point, the opposite of the hot-blob detector. Neither passes the recall gate for a thermal voter.
• Cause: 3 to 6 px targets in an AGC-normalised 160 x 120 frame and a few hundred labelled frames. What changes it is closer targets and pure-thermal annotation, not more of the same labels.

Provenance / license

Ultralytics YOLOv8 (AGPL-3.0). External data: LLVIP (Jia et al., 2021) and MassMIND (Nirgudkar et al., 2023; CC BY-NC-SA, derived weights inherit its terms). Konstanz thermal frames and audits from the SailSwarm project (University of Konstanz, Cluster of Excellence in Collective Behaviour). Recipe and evaluation: scripts/gpu_finetune/train_yolo.py, scripts/eval/thermal_gt_eval.py and the node scripts archived with the 2026-09-12 session in the SailSwarm-ObstacleDetection repository (docs/history/2026-09-12_night_retrain_aime.md, sections 9 and 10).