EvidenceGate-VLM

Evidence-constrained multimodal reasoning for reliable visual inspection.

This repository contains an honest, executable vertical slice for researching whether independently verified visual evidence and abstention can improve multimodal inspection decisions. It is not certified safety equipment and it does not claim benchmark improvements.

Current implementation status

• FastAPI API with SQLite persistence and local artifact storage.
• Real image-byte validation and decoding.
• A deterministic, documented visual anomaly baseline that produces image scores, heatmaps, connected-component regions, and boxes from the submitted image. It is not presented as PatchCore.
• Optional local Qwen3-VL adapter using the current Transformers multimodal interface. If Transformers, weights, or compatible hardware are unavailable, the API records MODEL_UNAVAILABLE; it never fabricates a VLM answer.
• Independent region/claim grounding checks and a decision policy with ACCEPT, REJECT, REVIEW, and ABSTAIN outcomes.
• Inspection records, raw outputs, provenance, and audit events in SQLite.
• Next.js inspection workspace connected to the API, with truthful readiness, error, and empty states.
• VisA source registry and a safe, provenance-preserving ingestion/indexing CLI. Dataset files are not bundled or redistributed.
• Research protocol, related work, hypotheses, and novelty matrix in research/.

The default local environment is intentionally usable without model weights: CV inspection runs, while VLM-dependent decisions remain visibly abstained. This keeps unavailable capability distinguishable from a passing model test.

Quick start

API

py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e .
python -m uvicorn evidencegate_api.main:app --app-dir apps/api --reload

The API is available at http://127.0.0.1:8000, with OpenAPI at /docs.

Web app

cd apps/web
npm install
npm run dev

Open http://localhost:3000. Set NEXT_PUBLIC_API_URL if the API is not on the default URL.

Optional Qwen3-VL inference

Install the optional model dependencies and configure the model explicitly:

python -m pip install -e ".[vlm]"
$env:EVIDENCEGATE_VLM_ENABLED = "true"
$env:EVIDENCEGATE_VLM_MODEL_ID = "Qwen/Qwen3-VL-4B-Instruct"

The model is large and the repository does not download weights automatically. GPU, precision, revision, and generation settings are reported in the run record. Read the original Qwen3-VL model card and its license before use.

VisA ingestion

The source registry records the official AWS Open Data source. Downloading is explicit and opt-in:

python scripts/ingest_visa.py --output-dir data/visa --accept-license

The command records the source URL, download hash, license acknowledgement, image metadata, safe paths, and an index. It does not commit image files. See docs/datasets/visa.md.

Tests and status vocabulary

python -m pytest
python -m ruff check .
cd apps/web; npm run typecheck; npm run build

UNIT_TEST_PASSED, INTEGRATION_TEST_PASSED, REAL_MODEL_INFERENCE_VERIFIED, and FULL_BENCHMARK_COMPLETED are separate statuses. This empty repository has no benchmark result until a real dataset and compatible model execution are performed.

Safety and limitations

This is a research system, not a certified industrial safety device. The deterministic visual baseline is not a trained anomaly detector and has not been validated as a production inspection model. The VLM adapter is optional and model availability is hardware- and dependency-dependent. No accuracy, calibration, latency, or superiority claim is made without a recorded experiment.

See SECURITY.md, research/experimental_protocol.md, and CHANGELOG.md.