Thermostatic/community-forensics-frontier-detector-2026-08

Community Forensics frontier detector (August 2026)

This is an independently fine-tuned whole-image AI-generated-image detector based on the MIT-licensed OwensLab Community Forensics 384 model. It is not an official OwensLab release.

The release is intentionally accompanied by complete positive and negative validation results. It performs strongly on ordinary, web-laundered, and moderately degraded images, but it is not robust to every declared attack. In particular, very small synthetic composites and very-low-resolution inputs remain serious weaknesses.

Files

• model.safetensors: canonical EMA inference weights.
• community_forensics_frontier_fp16.onnx: recommended compact deployment artifact.
• config.json: architecture, preprocessing, labels, and decision threshold.
• calibrator.json: frozen calibration parameters.
• reports/final_report.json and .md: complete final evaluation.
• reports/redteam_report.json and .md: complete 33-condition robustness evaluation.
• metadata/training.json and metadata/data_audit.json: training and dataset-audit records.
• metadata/preparation.json: construction record for the combined manifest.
• metadata/dataset_composition.json: row counts by role, label, source, license, repository, and revision.
• metadata/canary_summary.json: equal-exposure architecture comparison used before the final run.
• metadata/ranking.json: frozen candidate selection and calibrator.
• metadata/training_recipe.json: machine-readable optimizer, augmentation, sampling, and validation recipe.
• configs/redteam.yaml: exact 33-condition robustness registry.

Intended use

Binary whole-image classification:

• 0: real
• 1: AI-generated

This model is a screening signal, not proof of provenance. Do not use it as the sole basis for accusations, moderation sanctions, employment decisions, or legal conclusions. It was not trained as a localization model and should not be expected to reliably identify an image containing only a tiny synthetic patch.

Preprocessing and decision rule

• RGB input
• resize the short edge to 440 pixels
• center-crop to 384 × 384
• normalize using the values in config.json
• output: one raw logit
• frozen PyTorch-reference decision rule: raw_logit >= 1.359375 means AI-generated
• equivalent calibrated probability threshold: 0.65

The ONNX model uses FP16 weights with FP32 input and output. The PyTorch calibration ledger was produced with CUDA bfloat16 autocast. This numerical-runtime difference is reported below rather than hidden.

Dataset construction

The final manifest combined an older 73,371-row forensic corpus with 40,101 newly selected frontier-generator images. The public frontier collection is available at Thermostatic/frontier-synthetic-images-2026; the public dataset is the new frontier component, not a redistribution of every image in the combined training manifest.

Split	Real	AI-generated	Total	Groups
Train	41,313	68,247	109,560	1,676
Calibration	1,983	1,929	3,912	61
Competition test	0	0	0	0

The five real-image training sources were:

Source	Rows
COCO train2017	14,998
OpenFake real / LAION	9,287
OpenFake real / Pexels	8,764
ImageNet	6,000
WikiArt	2,264

The 68,247 synthetic training rows came from 131 source buckets. They include older generators and forensic benchmarks—such as SD 1.5, GLIDE, Wukong, Midjourney, SynthBuster, and OpenFake—plus 40,101 selected frontier images from 42 named generator buckets. Those frontier buckets include GPT Image generations, DALL·E 3, FLUX 1/2 variants, Imagen 3/4, Seedream 3/4/5, Qwen Image variants, Hunyuan Image, Midjourney 5.2, Recraft, Ideogram, Janus, Sana, Z-Image, and others. Exact per-source counts and source revisions are in metadata/dataset_composition.json and metadata/preparation.json.

The calibration split contains 1,983 LAION/Pexels real images and 1,929 synthetic images across 59 OpenFake generator sources. It was kept group-disjoint from training. A further 189 recent Hugging Face images were held out for positive-only diagnostics, and the 90-image OpenRouter cohort was protected from training. In total, 4,802 images from protected manifests were excluded from frontier-data selection.

The launch audit reported zero issues. It checked the finalized manifest with an exact-banded perceptual-hash search at Hamming distance 4, alongside the stored content hashes and group assignments. The combined manifest SHA-256 is af629187fca31ab2f4f7f4c81ea3a761c13ea7cb8d4ee8a9bb9e7bd8e1eb6f36.

Dataset licenses and terms remain source-specific. The full mixed corpus is not claimed to be reproducible from the public frontier dataset alone; the release publishes composition and provenance metadata so this limitation is explicit.

Architecture selection

Four candidates received equal canary exposure: 600 steps, batch size 48, and 28,800 presented samples each. Selection used the calibration role only; the competition test remained unopened.

Candidate	Status	Initial macro BA	Final/best macro BA	Initial → final median loss
Community Forensics ViT-S/16 384	Eligible	0.9214	0.9283	0.1458 → 0.0670
Pretrained ConvNeXt-Small	Needs organizer approval	0.5977	0.6888	0.6654 → 0.3203
Scratch ConvNeXt-Small	Eligible	0.5807	0.6185	0.7024 → 0.6693
Custom forensic ConvNeXt	Eligible	0.5716	0.6131	0.6828 → 0.6501

Community Forensics was both the strongest candidate and the healthiest eligible loss curve. A large-teacher/distillation experiment was considered but cancelled before any teacher outputs were used. This release contains no distillation. No bounty-participant detector weights, pseudo-labels, or outputs were used for training, selection, or calibration.

For clarity, the 0.9214 initial value in the Community Forensics canary row is the score of an earlier legacy-data fine-tune used to initialize that canary. It is not the untouched upstream OwensLab score. A direct comparison against the original upstream weights follows below.

Direct comparison with the original upstream model

The untouched OwensLab/commfor-model-384 revision and this release were scored with the same loader, bfloat16 inference path, images, deterministic views, and calibration procedure. Each model received its own bias-only calibrator fitted under the same protocol. The original raw-logit boundary was -9.4375; the fine-tuned boundary is 1.359375.

Calibration evaluation	Original	Fine-tuned	Absolute change
OOF macro balanced accuracy	0.7200	0.9328	+0.2128
Raw ROC AUC	0.8241	0.9790	+0.1549
Clean balanced accuracy	0.7905	0.9568	+0.1663
Web balanced accuracy	0.7922	0.9385	+0.1464
Hard balanced accuracy	0.6447	0.9047	+0.2600
Per-image worst of clean/web/hard	0.4606	0.8829	+0.4223

The protected positive-only recent-generator cohorts show the same pattern:

Diagnostic recall	Original	Fine-tuned	Absolute change
OpenRouter-90 clean	0.8556	0.8778	+0.0222
OpenRouter-90 web	0.4111	0.6556	+0.2444
OpenRouter-90 hard	0.0444	0.4778	+0.4333
OpenRouter-90 correct under all views	0.0444	0.4667	+0.4222
Recent-HF-189 clean	0.8836	0.9788	+0.0952
Recent-HF-189 web	0.5556	0.9312	+0.3757
Recent-HF-189 hard	0.1693	0.7407	+0.5714
Recent-HF-189 correct under all views	0.1693	0.7407	+0.5714

This is strong evidence that fine-tuning improved the original model, especially under laundering. It is not an unbiased competition-test estimate: the 3,912-image calibration set was used for development and checkpoint selection. The two recent-generator cohorts were protected from training, but they are positive-only and relatively small. The exact comparison is published in metadata/original_comparison.json.

Final training recipe

The final run initialized from the winning Community Forensics canary checkpoint, which itself initialized from OwensLab/commfor-model-384@6076002bf0d9dd37537f965ee2f06f826c333b61.

Parameter	Value
Hardware	1 × NVIDIA L40S
Input	384 × 384 RGB
Canary	600 steps × batch 48 = 28,800 presentations
Final run	6,000 steps × batch 96 = 576,000 presentations
Total fine-tuning presentations	604,800
Optimizer	fused AdamW
Backbone learning rate	1e-5
Classifier-head multiplier	1.0
Weight decay	0.05
Schedule	300-step linear warmup, then cosine decay to zero
EMA	0.999; EMA weights used for validation and release
Precision	CUDA bfloat16 autocast; TF32 enabled
Gradient clipping	global norm 1.0
Seed	323
Compilation	PyTorch reduce-overhead mode
Sampling	exact class-balanced batches; source-temperature exponent 0.35
Validation	all 3,912 calibration images under clean/web/hard every 750 steps
Checkpointing	every 500 steps

Augmentation was class-symmetric—real and synthetic images received the same transform distribution. It used:

• up to 8% crop on each edge with probability 0.18;
• scale 0.40–1.30 with probability 0.72, using nearest/bilinear/bicubic/Lanczos;
• Gaussian blur radius 0.15–0.75 with probability 0.08;
• a first JPEG/WebP round trip with probability 0.82 and an independent second round trip with probability 0.30;
• codec quality sampled uniformly from 30–95, with WebP selected 22% of the time;
• random 384 crop after resizing the short edge to 440;
• random horizontal flip with probability 0.5;
• ImageNet normalization: mean [0.485, 0.456, 0.406], standard deviation [0.229, 0.224, 0.225].

The best full-run checkpoint occurred at step 4,500. The run completed all 6,000 steps so the later validation trajectory could be checked rather than stopping immediately at the first peak.

Training and data audit summary

Item	Result
Combined manifest rows	113,472
New frontier training rows	40,101
Final-run optimization steps	6,000
Final-run samples presented	576,000
Selected checkpoint	step 4,500
Decode failures	0
Dataset audit issues	0
Locked competition test opened	No

The final run took 14,187.6 seconds (3 h 56 min) and reported zero decode failures.

Evaluation and calibration protocol

Candidate selection, checkpoint selection, and calibration used development/calibration data only. There was no competition-test manifest in the combined data, and the locked test was never opened.

The three recurring validation views were deterministic per sample:

• clean: decode, RGB conversion, resize/crop, and normalization only;
• web: limit the long edge to 768 pixels using bilinear resampling, then JPEG quality 60 with 4:2:0 subsampling;
• hard: limit the long edge to 512 pixels using bilinear resampling, then JPEG quality 40 with 4:2:0 subsampling.

At each 750-step checkpoint, the model scored 3,912 images × 3 views = 11,736 predictions. Selection used macro balanced accuracy across views, not clean accuracy alone. Step 4,500 was selected at 0.9328 macro balanced accuracy; step 6,000 finished lower at 0.9298, so the earlier checkpoint was retained.

Calibration compared supported calibrator families out-of-fold and selected a bias-only transform. The frozen parameters are slope 1.0, intercept -0.7403357915937764, target probability threshold 0.65, and equivalent raw-logit boundary 1.359375. The candidate-ranking ledger and its SHA-256 are published in metadata/ranking.json.

The additional evaluation layers were:

1. recent-generator positive-only diagnostics on OpenRouter-90 and recent-HF-189;
2. a 33-condition red-team registry over all 3,912 calibration images (129,096 predictions);
3. 2,000-replicate hierarchical bootstrap intervals over per-image worst variants;
4. FP16 and FP32 ONNX export smoke tests;
5. full 11,736-prediction PyTorch/ONNX ledger comparison.

The red-team suite is development evidence, not an independent test. Its exact operations and severities are published in configs/redteam.yaml.

Calibration results

All values below are development/calibration results. The locked competition test was never opened.

Evaluation	Balanced accuracy	Fake recall	Real specificity	ROC AUC
OOF macro selection metric	0.9328	—	—	0.9790 raw
Clean	0.9568	0.9585	0.9551	0.9890
Web	0.9385	0.9114	0.9657	0.9813
Hard	0.9047	0.8538	0.9556	0.9650
Per-image worst of clean/web/hard	0.8829	0.8455	0.9203	0.9431

Recent-generator diagnostics

These are small positive-only diagnostic cohorts, so recall is reported rather than balanced accuracy.

Cohort	Images	Clean recall	Web recall	Hard recall	Correct under all three views
OpenRouter recent generators	90	0.8778	0.6556	0.4778	0.4667
Recent Hugging Face holdout	189	0.9788	0.9312	0.7407	0.7407

The OpenRouter result is a material limitation: detection degrades substantially after the hard laundering pipeline.

Robustness results — failed gates

The full development red-team evaluation covered 3,912 images, 33 conditions, and 129,096 predictions with zero inference failures. The report is nevertheless marked valid: false because three declared gates failed.

Gate	Result
Attack protocol match	Pass: 0 mismatches
Worst declared attack BA	Fail: 0.5100 on composite_05
Minimum critical fake recall	Fail: 0.0492
External cohort coverage	Fail: generative laundering, physical recapture, and platform laundering were absent

composite_05 places only 5% synthetic foreground into an otherwise real image. It exposes the mismatch between whole-image classification and localized synthetic-content detection; it is not omitted or reclassified here. The per-image worst-of-all-attacks balanced accuracy was 0.2766 (95% hierarchical-bootstrap interval 0.2523–0.3121). Very-low-resolution cases at 96–128 pixels were also weak. See reports/redteam_report.md for the full case table.

ONNX parity

Recommended artifact: community_forensics_frontier_fp16.onnx

Item	Result
Size	43,778,110 bytes
SHA-256	d75791ba2fa59146025d342cfaafa9ddeab24af117642a94f752ee4c1619375d
Smoke-test max absolute logit difference	0.01693
Full-ledger mean absolute logit difference	0.10500
Full-ledger max absolute logit difference	2.86719
Threshold disagreements	34 / 11,736
Decision agreement	99.7103%

An FP32 ONNX export was also evaluated. It was twice as large and produced 35 threshold disagreements, so it is not published as the recommended artifact. ONNX balanced accuracy at the frozen boundary was 0.9571 clean, 0.9373 web, and 0.9047 hard—within 0.0013 of the PyTorch-reference values.

Limitations

• Weak on severe low-resolution laundering.
• Weak on small or diluted synthetic regions because this is a whole-image classifier.
• OpenRouter hard-view recall was only 0.4778 on a 90-image cohort.
• Three important external robustness families were not covered.
• Validation results are development evidence, not an untouched competition-test estimate.
• Detectors can age quickly as image generators and laundering pipelines change.

Lineage and license

Base model revision: OwensLab/commfor-model-384@6076002bf0d9dd37537f965ee2f06f826c333b61.

• Upstream code: JeongsooP/Community-Forensics
• Upstream paper: Community Forensics
• Base model license: MIT

This fine-tuned release is published under the MIT license. Dataset licenses and terms remain those of their respective sources.