AGENTS.md

AGENTS.md — NeuroBridge Enterprise Pipeline

Read this file at the start of every session. It is the contract every agent (human or LLM) operates under in this repository.

1. Project Vision

NeuroBridge Enterprise is a B2B SaaS platform that solves three structural problems in real-world clinical/biomedical ML pipelines:

1. Data Drift between hospitals and acquisition sites (multi-center MRI).
2. Missing Modalities (a patient may have MRI but no EEG, or vice versa).
3. Artifacts in raw biosignals (eye blinks, line noise, motion in EEG).

The platform exposes three production pipelines behind a single FastAPI surface:

Modality	Pipeline	Core Technique
Image (MRI / fMRI)	src/pipelines/mri_pipeline.py	ComBat Harmonization for site-level domain shift
Signal (EEG)	src/pipelines/eeg_pipeline.py	MNE-Python + ICA for artifact removal
Tabular (BBB / molecules)	src/pipelines/bbb_pipeline.py	RDKit Morgan fingerprints from SMILES

All experiment runs are tracked in MLflow. All services ship as Docker images.

2. Directory Layout (load-bearing — do not violate)

.
├── AGENTS.md                 # This file
├── README.md
├── requirements.txt
├── pytest.ini
├── conftest.py               # Repo-wide pytest fixtures (autouse: pins MLFLOW_TRACKING_URI to tmp dir for test isolation)
├── Dockerfile                # Production image (FastAPI + pipelines)
├── docker-compose.yml        # api + mlflow services for local stack
├── .dockerignore
├── .streamlit/
│   └── config.toml           # Streamlit theme tokens
├── data/
│   ├── raw/                  # Untouched source data. NEVER train on this directly.
│   └── processed/            # Pipeline output as Parquet (preserves dtypes; overwritten each run; see §4).
├── src/
│   ├── api/                  # FastAPI surface
│   │   ├── main.py           # App factory + /health
│   │   ├── routes.py         # POST /pipeline/{bbb,eeg,mri} dispatch
│   │   └── schemas.py        # Shared Pydantic request/response models
│   ├── core/                 # Cross-cutting utilities
│   │   ├── logger.py         # Structured logger (mandatory in every pipeline)
│   │   ├── determinism.py    # Thread-pin env vars (OMP/OPENBLAS/MKL/pyarrow)
│   │   ├── storage.py        # Parquet read/write helpers (snappy, single-threaded, deterministic)
│   │   └── tracking.py       # MLflow `track_pipeline_run` context manager (see §7)
│   ├── pipelines/            # One file per modality. Pure functions + a `run_pipeline()` entry.
│   ├── models/               # Downstream decision-layer models
│   │   ├── bbb_model.py      # BBB-permeability classifier + SHAP explainer + trainer CLI
│   │   └── mri_model.py      # Volumetric MRI ONNX inference surface (external training)
│   ├── llm/                  # Natural-language explainers (template + OpenRouter fallback)
│   ├── rag/                  # Fastembed + FAISS retrieval layer
│   ├── agents/               # Tool registry + guarded OpenRouter orchestrator
│   └── frontend/
│       └── app.py            # Streamlit dashboard
└── tests/
    ├── core/
    ├── api/
    ├── frontend/
    ├── pipelines/            # incl. test_cross_pipeline_smoke.py for integration coverage
    └── fixtures/             # Tiny synthetic data files used by tests (NOT a Python package — no __init__.py)

Rules:

• New modality → new file under src/pipelines/. No mixing modalities in one file.
• Anything imported by 2+ pipelines → src/core/.
• Pipeline code (src/pipelines/, src/core/) must not read from or write to any path outside data/. Test code may read tests/fixtures/. The data/ boundary is the storage contract for production data.
• tests/fixtures/ holds CSV / numpy / DICOM blobs — do not add an __init__.py there.

3. Coding Standards

• Python 3.10–3.12 (the pinned native-extension dependencies do not yet ship cp313+ wheels). Use from __future__ import annotations when needed for forward refs.
• Type hints are mandatory on every public function/method (parameters and return).
• Modular structure. One responsibility per function. If a function exceeds ~40 lines or 3 levels of nesting, split it.
• TDD is the default workflow. Write the failing test first, watch it fail, then implement. Tests live in tests/ mirroring src/.
• Logging is mandatory for every pipeline. Use src.core.logger.get_logger(__name__). No print() in src/.
• Docstrings on every public function — one-line summary + Args/Returns when non-trivial.
• No hard-coded paths in business logic. Pass paths as arguments to run_pipeline(input_path, output_path).
• Format & lint: keep imports sorted; prefer pathlib.Path over os.path.
• Commits are small and frequent. Each green test → commit.

4. Data Readiness Principles

The Golden Rule: never train a model directly on raw data. Raw data must always pass through a pipeline first.

Every modality pipeline MUST guarantee, before writing to data/processed/:

1. Schema validity — required columns present, expected dtypes.
2. Domain validity — invalid records (e.g. unparseable SMILES, NaN-only EEG epochs, corrupted DICOMs) are logged with their identifier and dropped, never silently coerced.
3. Determinism — given the same data/raw/ input, the pipeline produces byte-identical data/processed/ output. No wall-clock, no random seeds without explicit seeding.
4. Traceability — log row count in, row count out, and percentage dropped at INFO level.
5. Idempotence — re-running the pipeline overwrites data/processed/ cleanly; no append, no partial writes.

Determinism environment: byte-identical output requires deterministic floating-point reductions. Each pipeline module sets OMP_NUM_THREADS=1, OPENBLAS_NUM_THREADS=1, MKL_NUM_THREADS=1, and pins pyarrow to single-threaded mode at import time. CI runners and developer machines do not need to set these manually — the pipeline modules handle it — but overriding them in the environment will break Determinism rule 3.

ComBat determinism boundary: the MRI pipeline's harmonize_combat wraps neuroHarmonize.harmonizationLearn and applies np.round(14) to its output. This is a defensive measure: with the thread-pinning above, harmonization is already bit-identical, but the rounding guarantees byte-identity even when the env-pin discipline is bypassed (e.g. a sub-process that re-exports a thread count). It discards ~5 trailing-mantissa bits of float64 — well below ComBat's biological effect-size precision floor.

A model training script is allowed to import from data/processed/ only. If a training script references data/raw/ directly, that is a bug and must be refactored into a pipeline.

5. How to Add a New Pipeline (checklist)

1. Add tests/pipelines/test_<name>_pipeline.py with the failing tests first.
2. Create src/pipelines/<name>_pipeline.py exposing run_pipeline(input_path: Path, output_path: Path) -> None.
3. Use get_logger(__name__) for all status output (per §3).
4. Apply the §4 Data Readiness contract: validate + drop invalid records with a logged WARNING (identifier + count), log row count in/out/dropped at INFO, write deterministically, and overwrite (do not append) on re-run.
5. Write deterministic output to output_path.
6. Document any new dependency in requirements.txt (pinned).
7. Add a one-line entry to this file's pipeline table.

6. Storage Format Convention

All data/processed/ outputs MUST be Parquet (pyarrow engine, compression="snappy"):

• Preserves dtypes (uint8 fingerprints stay uint8; float64 EEG features stay float64) — CSV silently widens numeric columns and is unsuitable for the high-dimensional float arrays produced by the EEG and MRI pipelines.
• Byte-deterministic with fixed compression and single-threaded writes (satisfies §4 Determinism).
• Read with pd.read_parquet(path); no dtype hints required.

The raw data/raw/ inputs may be in any vendor-supplied format (CSV for BBBP, EDF/FIF for EEG, NIfTI for MRI).

7. Experiment Tracking

Every run_pipeline() invocation logs to MLflow via src.core.tracking.track_pipeline_run:

• Experiment names match the pipeline module: bbb_pipeline, eeg_pipeline, mri_pipeline.
• Params: input/output paths and pipeline hyperparameters (e.g. BBB n_bits / radius, EEG epoch_duration_s / random_state, MRI intensity_threshold / n_roi_axes).
• Metrics: row counts (rows_in, rows_out, rows_dropped — or modality equivalent like subjects_in/out/dropped) and duration_sec.
• Artifact: the produced Parquet at data/processed/<modality>_features.parquet.

The tracking URI is read from MLFLOW_TRACKING_URI (defaults to ./mlruns/ when unset).

Live-demo lifeline: set NEUROBRIDGE_DISABLE_MLFLOW=1 to skip tracking entirely — the helper yields None and emits no MLflow calls. Use this when the tracking server is unreachable (offline demo, network outage, or CI without an MLflow service). Pipelines complete normally; only the run metadata is lost.

The repo-wide conftest.py autouse fixture pins MLFLOW_TRACKING_URI to a tmp directory for the test session, so the production mlruns/ directory is never written by the test suite. Tests that interact with MLflow (in tests/core/test_tracking.py and the per-pipeline Test<Modality>PipelineMLflow classes) all share this isolated store.

8. Decision Layer (Downstream Models)

Pipelines produce features (data/processed/<modality>_features.parquet). Downstream models live in src/models/ and consume processed features or a deterministic model-local preprocessing contract:

Model	File	Output	Endpoint
BBB permeability	src/models/bbb_model.py	data/processed/bbb_model.joblib	POST /predict/bbb
MRI image classifier	src/models/mri_model.py	data/processed/mri_model.onnx	POST /predict/mri

In-repo trainable downstream model modules expose a uniform surface:

• train(df, label_col, ...) → fitted classifier
• save(model, path) / load(path) → joblib artifact I/O
• predict_with_proba(model, smiles) → {label, confidence} (confidence is the max-class probability)
• explain_prediction(model, smiles, top_k) → SHAP top-k attributions sorted by |shap_value| descending

MRI DL exception: training happens outside this repo and exports ONNX, so it does not expose train() or SHAP. Runtime loads the ONNX artifact with mri_model.load(), preprocesses one NIfTI via the same deterministic resize + z-score contract used during training (preprocess_nifti()), then returns class probabilities via predict_nifti().

The API loads model artifacts at request time. If an artifact is missing, the endpoint returns HTTP 503 with a remediation hint instead of failing process startup. BBB points at the trainer CLI (python -m src.models.bbb_model); MRI points at the external ONNX export path.

Determinism: all in-repo classifiers are seeded (random_state=42 default), n_jobs=1 (no tree-parallelism races). Re-running the BBB trainer on the same Parquet produces identical predictions. MRI ONNX determinism is bounded by the exported model plus the fixed runtime preprocessing contract.

Override BBB_MODEL_PATH env var to point the API at a non-default artifact location (used by tests for tmp_path isolation).

Override MRI_MODEL_PATH env var to point the API at a non-default ONNX artifact location. If the ONNX artifact is missing, POST /predict/mri returns HTTP 503 with a remediation hint.

Calibration metadata (Day 6): train() does an 80/20 stratified split, computes precision-at-confidence-threshold bins on the held-out test set, and stashes them on model._neurobridge_calibration: list[dict] (sorted ascending by threshold). The API includes the bin matching each prediction's confidence in BBBPredictResponse.calibration. UI uses this to render an honest trust caption ("≥75% confident → 92% precision, n=18"). For tiny test fixtures where stratified split fails, calibration falls back to zero-support bins so the API contract is always populated.

9. Demo Features (Day 6)

The frontend includes three jury-day demo amplifiers that don't change the core contract:

• Edge-case dropdown (BBB tab): a curated catalog of 5 robustness probes — invalid SMILES, empty input, OOD macrocycle (cyclosporine-like), heavy halogenated aromatic. Each has a stated expectation; the UI visualizes graceful failure (HTTP 400 → recoverable warning, never a crash).
• Calibration trust caption (BBB decision card): renders the precision-at-confidence-threshold from BBBPredictResponse.calibration. Demonstrates that the system knows what it doesn't know.
• MRI ComBat diagnostics (MRI tab): POST /pipeline/mri/diagnostics runs the pipeline twice (pre + post ComBat) and returns long-format data + site-gap KPIs (Pre, Post, Reduction factor). The UI renders a faceted altair density plot — visual proof that ComBat removes site-driven domain shift.

10. Drift Surface (Day 7)

Each predict route maintains a per-worker rolling window of recent prediction confidences (collections.deque(maxlen=100)). Train-time median + std are stashed on model._neurobridge_train_stats (joblib roundtrip-safe). The drift z-score is (rolling_median − train_median) / max(train_std, 1e-9), computed only when the buffer holds ≥10 samples AND the model has the train-stats attribute. The /predict/bbb response carries drift_z: float | None and rolling_n: int. The UI renders a one-line caption with a magnitude tag (in-band, mild, significant). Worker restart clears the deque; this is acceptable for demo and removes the audit-trail concern.

11. LLM Explainer Surface (Day 7 + 9)

src/llm/explainer.py is the single entry point for natural-language rationales. explain(payload) always returns {rationale, source, model}. The deterministic template path is the source of truth for tests; the LLM path is OpenRouter via the openai==1.51.0 SDK and walks a smartest → smallest free-tier fallback chain (_DEFAULT_FREE_MODEL_CHAIN, 10 ids — head: inclusionai/ling-2.6-1t:free). The chain is overridable at runtime via OPENROUTER_FREE_MODELS (comma-separated). Status-code classification:

• 401 → key is bad → bail to template + actionable WARNING (rotate at https://openrouter.ai/keys, enable free-model data-sharing at https://openrouter.ai/settings/privacy).
• 400 → prompt-shape mismatch on this model → advance to next.
• 402 / 403 / 404 / 429 / 5xx → advance to next.
• Network/timeout → bail to template (switching models won't help).

Two env knobs control the gate:

• OPENROUTER_API_KEY — when absent, fallback to template.
• NEUROBRIDGE_DISABLE_LLM=1 — hard kill-switch; force template even if a key is set. Use this for demo days when you want fully deterministic, reproducible rationales.

Prompt design (_build_llm_prompt): two intent modes. When the caller supplies user_question, the model is instructed to language-match (Turkish question → Turkish answer), answer the question directly (not a canned paper-style summary), and respond conversationally to off-topic / greeting questions. When no user_question is supplied, falls back to the original 2-4 sentence paper-style rationale.

The POST /explain/bbb endpoint mirrors this contract. Pydantic enforces a non-empty top_features list (422 on empty); every other failure mode degrades to template + WARNING log + source="template".

Diagnostics: GET /diag/openrouter (src/api/main.py) returns key-presence (length + 12-char prefix only), kill-switch state, chain length, first model id, and the result of an 8-token probe call against that model. Surfaced in Streamlit as the sidebar "🔧 Diagnose LLM" button. Use it when the deployed Space shows source="template" unexpectedly — the most common causes are a missing/misnamed OPENROUTER_API_KEY Space secret or a revoked key.

12. Multi-Modal Explainer (Day 8)

src/llm/explainer.py exposes explain(payload, modality) where modality ∈ {"bbb", "eeg", "mri"}. Each modality has its own deterministic template (_template_explain_bbb / _eeg / _mri) and its own LLM prompt header. Unknown modality strings degrade to the BBB template with a warning log; the function never raises. The hybrid OpenRouter fallback contract from §11 applies uniformly.

The API exposes three matching endpoints — POST /explain/{bbb,eeg,mri} — each on the explain_router (/explain prefix). Streamlit surfaces the BBB version in the AI Assistant tab and the EEG/MRI versions as inline expanders inside their respective pipeline tabs.

13. Experiments Surface (Day 8)

GET /experiments/runs returns up to 50 most recent MLflow runs across the bbb/eeg/mri experiments, flattened into a list of MLflowRunSummary (run_id, experiment_name, start_time, status, metrics, params). POST /experiments/diff {run_id_a, run_id_b} returns a side-by-side metric+param diff (RunDiffRow).

When NEUROBRIDGE_DISABLE_MLFLOW=1, both endpoints return empty responses without raising — useful for deployments where there is no writable mlruns/ tree or the tracking server is unavailable. Unknown run ids → 404.

The Streamlit "Experiments" tab is the user-facing surface. Cached in session state with an explicit Refresh button.

14. Deploy Surface (Day 8)

Dockerfile.hf is the Hugging Face Spaces image. Single container, two processes (FastAPI :8000 + Streamlit :7860) launched via supervisord.conf. Build-time RUN python -m src.models.bbb_model bakes the BBB model artifact into the image so the first /predict/bbb call is instant on cold start. Build-time RAG ingest creates data/processed/faiss_index/.

docker-entrypoint.sh is the runtime guard for local Docker/Compose demos: when a mounted ./data volume hides image-built artifacts, it seeds fixture raw data, rebuilds missing BBB features/model artifacts, and rebuilds the FAISS index before starting supervisord. It does not bake NEUROBRIDGE_DISABLE_MLFLOW=1 into the image; operators may set that env at runtime if their tracking service is unavailable.

Default environment: DEPLOY_ENV=hf_spaces. The LLM kill-switch is not set — deployed Spaces use the real OpenRouter free-tier chain (§11) when OPENROUTER_API_KEY is configured in the Space's Secrets panel. Set NEUROBRIDGE_DISABLE_LLM=1 only when you want to force the deterministic template path for a fully-reproducible demo.

The README's YAML front-matter declares the Space metadata (SDK=docker, port=7860, app_file=src/frontend/app.py).

15. Orchestrator Agent Surface

src/agents/orchestrator.py exposes a single-agent function-calling loop over the openai SDK (no LangChain / framework dep). The API enables the guarded workflow mode: if the LLM skips or mis-shapes a required tool call, deterministic routing in src/agents/routing.py falls back to exactly one pipeline tool, then exactly one retrieval tool, then final synthesis. The agent holds 4 tools, defined in src/agents/tools.py:

• run_bbb_pipeline(smiles, top_k) — wraps POST /predict/bbb
• run_eeg_pipeline(input_path) — wraps POST /pipeline/eeg
• run_mri_pipeline(input_dir, sites_csv=None) — wraps POST /pipeline/mri and defaults sites_csv to <input_dir>/sites.csv
• retrieve_context(query, k) — wraps src/rag/retrieve.py

The system prompt (src/agents/prompts.py:ORCHESTRATOR_SYSTEM_PROMPT) describes the workflow: pick exactly one pipeline → run it → formulate a focused retrieval query → call retrieve_context → synthesize a 3-5 sentence response that cites at least one chunk. The API-side workflow guard enforces that order in code; the prompt is guidance, not the only control plane. Language of the final response is mirrored from the user's question.

POST /agent/run is the public surface. It accepts user_input, optional user_question, and optional MRI sites_csv. Default model is google/gemini-2.0-flash-exp:free on OpenRouter (function-calling support verified). Override via NEUROBRIDGE_AGENT_MODEL env var. Returns 503 when OPENROUTER_API_KEY is unset.

Diagnostics: GET /diag/agent returns key presence, configured model, RAG index status (chunk count), and the registered tool names.

16. RAG Surface

src/rag/ is the retrieval layer. Stack: fastembed (BAAI/bge-small-en-v1.5, 384-dim, ONNX, no torch dep) for embeddings + faiss-cpu (IndexFlatIP after L2-norm = cosine) for vector search.

Knowledge base lives at data/knowledge_base/ (gitignored; user-supplied .md / .txt / .pdf). Build the FAISS index with:

python -m src.rag.ingest [<input_dir> [<output_dir>]]

Defaults: input=data/knowledge_base/, output=data/processed/faiss_index/. The Dockerfile runs this at build time so deployed Spaces start with a populated index. docker-entrypoint.sh also rebuilds the index at startup when a mounted data/ volume hides the image-built artifacts. Empty KB → empty index → retrieve_context returns 0 chunks; the agent surfaces this and answers from the pipeline result alone.

tests/fixtures/kb_sample/ ships 3 seed markdown files (Lipinski, ComBat, MNE+ICA) — these double as test fixtures and as the demo seed if no user-supplied PDFs are added.