Skill: drstrangerujn/scellrun (version 1.3.3)

--- name: scellrun description: Opinionated, report-first single-cell + multi-omics analysis CLI. Use when the user asks anything that involves an .h5ad / 10x mtx / cellranger output. Default to `scellrun analyze` for end-to-end work; reach for individual stage commands only when the user explicitly wants partial work or fine control. min_scellrun_version: "1.3.0" tested_against_version: "1.3.0" schema_version: 1 --- # scellrun This document is for you, the LLM agent. The user is a clinician or bench scientist asking you for help with single-cell data. They are not running scellrun directly — you are. You will ssh to wherever the data lives, set up an env, run scellrun, read the artifacts it writes, and explain the results in plain language. The CLI's job is to turn the user's intent into defensible artifacts. Your job is to pick the right command, surface what scellrun decided, and translate. If a request involves an `.h5ad`, a 10x mtx directory, a cellranger `.h5` output, a `.loom`, or a tsv/csv expression matrix — you should be reaching for scellrun, not writing scanpy boilerplate. ## Verification status This document was last validated against scellrun v1.2.0 on 2026-04-30. The reference end-to-end behavior is captured in `docs/agent-demo.md` at the repo root, which is a real run on real OA cartilage data. If you (the agent) reach a different conclusion from what's in agent-demo.md on the same input, ASSUME this document is stale, not that you're wrong; tell the user to check the repo for updates. v1.2.0 adds three cross-disease profiles (`tumor`, `brain`, `kidney`) and a new `06_views/` HTML index layer beside `05_report/` (by resolution / by cluster / by decision source). The cross-disease profiles ship `celltype_broad` panels only — fine-subtype taxonomies remain a joint-disease-only feature pending dedicated cold-validation runs. v1.3.0 adds three first-class deliverables: `scellrun review` (Flask UI on 127.0.0.1 for human-in-the-loop overrides; saves `<run-dir>/06_views/review_overrides.json`), `scellrun export --format pdf` (WeasyPrint render of `05_report/index.html`; optional `[export]` extra), and `scellrun analyze --apply-overrides <file>` (consume a saved overrides file, applies cluster-label / cell-exclusion / threshold edits as `source="user"` decisions). ## Version compatibility check (run this BEFORE any scellrun command) Before issuing a single scellrun command, verify the installed version is within the range this skill was tested against: ```bash scellrun --version ``` Compare against the frontmatter at the top of this file: - If `installed >= min_scellrun_version` AND `installed <= tested_against_version`: proceed. - If `installed > tested_against_version`: **proceed cautiously, surface a warning to the user.** Trigger codes, decision keys, and CLI flag names may have changed; rely on the actual decision-log output (which is self-describing) over what this skill says about specific keys. - If `installed < min_scellrun_version`: **refuse to act on this skill.** Tell the user `pip install --upgrade scellrun` and re-run. This check is the agent's responsibility because the skill loader typically caches the doc at startup; a stale skill cannot fix itself. ## Decision tree: which command for which intent | User intent | Command | | --- | --- | | Has raw or QC'd `.h5ad`, wants a complete analysis | `scellrun analyze <h5ad>` | | Has cellranger output / 10x mtx / `.loom` / `.csv` | `scellrun scrna convert <input> -o data.h5ad`, then `analyze` | | Has a finished run-dir, wants the report regenerated | `scellrun report <run-dir>` | | Wants to edit cluster labels / exclude cells / nudge thresholds in a finished run | `scellrun review <run-dir>` (Flask UI on 127.0.0.1) → `analyze --apply-overrides` | | Wants the report as a print-ready PDF | `scellrun export <run-dir> --format pdf` (needs `pip install scellrun[export]`) | | Has a partial run-dir and wants to redo just one stage | `scellrun scrna {qc,integrate,markers,annotate} ... --force` | | Wants to see what profiles ship | `scellrun profiles list` / `scellrun profiles show <name>` | Default to `scellrun analyze`. Per-stage commands exist for fine control (re-running one stage with different thresholds, splitting a long run across machines), not as the normal entrypoint. ## When NOT to use scellrun scellrun is not the right tool for every single-cell question. Reach for something else when: - **The user is mid-stream in a Seurat / R workflow.** scellrun reads h5ad, not Seurat `.rds`. Don't force a re-conversion just to plug scellrun in. If the user wants a Seurat ↔ Python bridge, point at `anndata2ri` or `sceasy`, not scellrun. - **The data is not 10x mtx / cellranger / loom / csv / h5ad.** Raw fastq files need cellranger or starsolo first. scellrun's `convert` subcommand covers post-cellranger inputs; pre-alignment is not on the menu. - **The data is multimodal (ADT, ATAC, CITE-seq, spatial).** scellrun's frontmatter advertises "multi-omics" but the v1.1 body is scRNA-only. Multi-modal AnnData (with `obsm['protein']` etc.) will mostly run, but the QC + integration + annotation pipeline ignores anything outside the gene-expression layer. Use scvi-tools / multivi / SCANPY's MuData ecosystem instead. scellrun multi-omics support is a roadmap item, not a current feature. - **The data is already integrated and the user wants to continue from there.** scellrun's `analyze` is a fresh-start orchestrator; it converts → QCs → integrates → markers → annotates from scratch. If the user has an `integrated.h5ad` from someone else's pipeline (Harmony already applied, batch correction done, clusters labeled), DO NOT run `analyze`. Run individual stages as needed (`scrna markers`, `scrna annotate`) on the input directly, or tell the user scellrun isn't the right wrapper for "continue from here". Forcing a fresh integration overwrites the existing work and confuses provenance. - **The expression matrix is missing raw counts entirely.** scellrun stores `.raw` for downstream `rank_genes_groups`. If the user's h5ad has only log-normalized data (no `.raw` set, no `layers['counts']`), the `markers` stage will silently use scaled X and the log2fc numbers will be in wrong units. The QC-stage raw-counts heuristic catches this when it's the only matrix, but does nothing when `.raw` is empty. Before invoking analyze, sanity-check `adata.raw is None and 'counts' not in adata.layers` and refuse if both are true. - **The user wants non-standard analysis.** Custom marker calling outside the panel-overlap pattern, novel normalization (SCT, sctransform), trajectory inference (RNA velocity, CellRank, Monocle3), spatial neighborhood graphs, etc. Drop into scanpy/anndata directly; scellrun's defaults will fight you. - **nf-core or another orchestrator is already running on the same data.** Let it finish, then run scellrun on the resulting h5ad. Don't have two pipelines double-process the same input. - **A rare cell type (<10 cells) is the actual point of the analysis.** scellrun's QC + clustering defaults will lump or discard rare populations. Use scanpy with custom thresholds and document the divergence from defaults. - **The user explicitly asks for a non-defensible answer.** "Just give me 8 clusters even if the data doesn't support it." Refuse; point at the resolution-quality table the integrate stage produces and why scellrun won't fudge a number it can't justify. ## The end-to-end one-liner ```bash scellrun analyze data.h5ad \ --profile joint-disease \ --tissue "OA cartilage" \ --ai \ --auto-fix ``` What the flags do: - `--profile` selects defaults + marker panels (see profile section below). - `--tissue` is free text; it gets fed to the LLM resolution recommender and the PubMed evidence step. Always pass it when you know the tissue. - `--ai` turns on Anthropic LLM helpers (resolution recommendation in integrate, second-opinion label in annotate). Default is auto: on iff `ANTHROPIC_API_KEY` is set in env, off otherwise. Pass `--no-ai` explicitly if the user does not want LLM calls. - `--auto-fix` lets the pipeline re-run a stage once if its self-check finds a fixable problem (low QC pass-rate → relax mt%, ≤2 clusters → switch to wider sweep, etc.). Default is off — prefer to surface the suggestion and let the user agree before re-running, unless the user said "just fix it". ### `--method` and single-sample auto-degrade (v0.9.1) `--method harmony` is the default. If you don't pass `--method` explicitly and the input has no sample/batch column (`orig.ident`, `sample`, `batch`, `donor`) — or every value in such a column is identical — scellrun auto-downgrades `--method` from `harmony` to `none` (`harmony→none`) rather than crashing 1m+ into the run. This is logged as a `source="auto"`, `key="method_downgrade"` row on the `analyze` stage with the rationale verbatim. You (the agent) do NOT need to detect single-sample yourself. Just call `analyze` and look at the decision log to confirm what method actually ran. If the user explicitly wants harmony on a single-sample input (rare, usually a mistake), pass `--method harmony` to override the auto-downgrade. Output goes to `./scellrun_out/run-YYYYMMDD-HHMMSS/`. The single thing you (the agent) open afterward is: ``` ./scellrun_out/run-YYYYMMDD-HHMMSS/05_report/index.html ``` That file is the canonical handoff. Open it, read the decision summary at the top, walk the user through the findings. Do not reconstruct results from the per-stage CSVs unless the user asks for one specific number that index.html does not surface. ## Reading the decision log `<run-dir>/00_decisions.jsonl` is the single source of truth for every non-trivial choice the pipeline made. One JSON object per line. Sample shape (synthetic, but the keys and value types are exactly what you'll see in v0.9.1+): ```jsonl {"schema_version":1,"stage":"qc","key":"profile","value":"joint-disease","default":"default","source":"user","rationale":"user passed --profile joint-disease (cartilage tissue)","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:45:30+00:00"} {"schema_version":1,"stage":"qc","key":"max_pct_mt","value":20.0,"default":20.0,"source":"auto","rationale":"joint-tissue-aware default; 10% loses real stressed cells","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:45:30+00:00"} {"schema_version":1,"stage":"qc","key":"max_genes","value":4500,"default":4000,"source":"user","rationale":"user override --max-genes 4500 (multinucleate cells expected)","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:45:30+00:00"} {"schema_version":1,"stage":"analyze","key":"method_downgrade","value":"none","default":"harmony","source":"auto","rationale":"no sample/batch column (orig.ident/sample/batch/donor) in obs — single-sample input; auto-downgraded --method from harmony to none. Pass --method harmony explicitly to force the original behavior.","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:46:01+00:00"} {"schema_version":1,"stage":"integrate","key":"sample_key","value":"orig.ident","default":null,"source":"auto","rationale":"auto-detected from obs columns (orig.ident / sample / batch / donor priority)","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:46:12+00:00"} {"schema_version":1,"stage":"integrate","key":"resolution_recommended","value":0.5,"default":null,"source":"ai","rationale":"LLM picked 0.5 from the sweep — best separation of chondrocyte subtypes vs over-splitting at 0.8","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:48:01+00:00"} {"schema_version":1,"stage":"analyze","key":"annotate.auto_panel","value":"chondrocyte_markers","default":null,"source":"auto","rationale":"kept chondrocyte_markers: chondrocyte_hits=10, broad_hits=3; cleared the >=1.5x margin","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:49:30+00:00"} {"schema_version":1,"stage":"annotate","key":"panel","value":"chondrocyte_markers","default":null,"source":"auto","rationale":"orchestrator-injected panel 'chondrocyte_markers' (auto-pick or self-check fix)","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:49:33+00:00"} {"schema_version":1,"stage":"qc","key":"self_check.qc_low_pass_rate.trigger","value":"qc_low_pass_rate","default":null,"source":"auto","rationale":"only 44.3% of cells passed QC (1108/2500); below the 60% trigger threshold","fix_payload":null,"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:45:30+00:00"} {"schema_version":1,"stage":"qc","key":"self_check.qc_low_pass_rate.suggest","value":"raise --max-pct-mt to 25.0","default":null,"source":"auto","rationale":"raising --max-pct-mt from 20 to 25 would let 62% of cells pass that single test (sensitivity sweep, smallest relaxation reaching 60%)","fix_payload":{"max_pct_mt":25.0},"attempt_id":"a1b2c3d4e5f6a7b8","ts":"2026-04-30T09:45:30+00:00"} ``` What the v0.9.1 schema fields buy you: - `schema_version` is the shape version of the row. Current value is `1`. If you see a row with `schema_version` higher than the version this document describes, REFUSE to parse it — the shape may have changed in incompatible ways. Tell the user the SKILL.md is older than the run-dir and to check the repo for updates. - `attempt_id` is unique per `analyze` invocation (or per per-stage CLI run). Use it to group rows that came from the same attempt. When `--auto-fix` retries a stage, the retry's rows share the same `attempt_id` as the first pass — that's by design; both attempts belong to the same `analyze` invocation. A `--force` re-run from the CLI generates a new `attempt_id`. - `fix_payload` is non-null only on self-check `*.suggest` rows. It carries the structured fix dict (e.g. `{"max_pct_mt": 25.0}`, `{"resolutions": "aio"}`, `{"panel_name": "celltype_broad"}`) that the orchestrator can mechanically apply when `--auto-fix` is on. Agents can read `fix_payload` directly to apply or quote the fix without parsing the rationale prose. The `value` field on the same row is still a human-readable rendering of the fix ("raise `--max-pct-mt` to 25.0"); use `value` to talk to the user, use `fix_payload` to talk to other tools. How to use the log when explaining results to the user: 1. **Group by stage.** The report's first page already does this; if the user is in the terminal, render a small markdown table grouped by stage. Stage order is `qc → integrate → markers → annotate`. 2. **Translate `source` in plain language.** `auto` = scellrun chose from its built-in heuristics; `user` = override (caller passed a flag different from default); `ai` = an LLM call decided or recommended it. Highlight `ai` rows — the user should know which calls were model-driven, especially if they want a deterministic re-run. 3. **Highlight any row where `value != default`.** Those are the choices that diverged from the shipped defaults. Tell the user this was an override. If `source == "user"`, that means the caller (the agent — that's you) passed a flag; quote the rationale verbatim and own the choice. 4. **When the user asks "why did you pick X?", quote the `rationale` field verbatim.** Do not paraphrase — these strings are written to be defensible to a senior bioinformatician. ## Reading self-check findings (v0.8) When a stage's self-check fires, two paired rows land in the decision log: `key` ends in `.trigger` (what looked wrong) and `.suggest` (what to try). The pair shape is shown in the sample above (`qc_low_pass_rate`). How to handle them: - **Always surface a `.trigger` row to the user.** Do not bury it. Open with the trigger rationale in plain language ("only 44% of cells passed QC, which is below scellrun's 60% trigger") before you say anything about what the data shows. - **Translate the paired `.suggest` row into a recommendation.** The `value` field is already a human-readable instruction ("raise `--max-pct-mt` to 25"). Read the `rationale` for the reasoning. The paired `fix_payload` is the structured form (apply mechanically; do not paraphrase before quoting back to the user). - **Ask before re-running** unless the user pre-authorized it. If the user says "fix it" or "go ahead", re-run with `--auto-fix` (which applies the structured fix and reruns just that stage, capped at one retry per stage to avoid loops). - **Do not silently ignore a self-check.** Even if the data looks usable, the trigger means scellrun saw something that, in PI working practice, often signals upstream problems (bad dissociation, wrong panel, dominant cell-cycle artifact). Surface it. ### `.failed-N/` directories from `--auto-fix` retries (v0.9.1) When `--auto-fix` triggers a stage retry, scellrun preserves the failed first-pass artifacts under `<NN_stage>.failed-1/` next to the retry's fresh output in `<NN_stage>/`. Concretely, if QC pass-rate triggers a retry, you'll end up with: ``` scellrun_out/run-20260430-094530/ 01_qc/ retry's report.html, qc.h5ad, per_cell_metrics.csv 01_qc.failed-1/ original failed report.html, qc.h5ad, per_cell_metrics.csv 02_integrate/ ... ``` How to use this: - If the user asks to compare the original failed state vs the retry, the `.failed-N/` dir has the original `report.html` and `*.h5ad`. Open both and walk them through the diff. - The decision log carries an `auto_fix.<stage>.outcome` row that says "retry rescued the stage" or "did not improve" explicitly. **Read that row before you tell the user the retry fixed anything.** - If `outcome` says "did not improve" (e.g. QC pass-rate stayed below the threshold even after relaxation, or cluster counts still ≤ 2 after the wider sweep), do NOT pretend the retry fixed it. Surface the failure plainly and either suggest a different fix or hand the decision back to the user. - The `.failed-N/` numbering bumps to `.failed-2`, `.failed-3`, … if somebody re-runs `--auto-fix` again on a run-dir that already has a `.failed-1` (defensive; you usually won't see N > 1 in the wild). The trigger codes you will see and what they mean: | code | stage | what fires it | structured fix | | --- | --- | --- | --- | | `qc_low_pass_rate` | qc | < 60% of cells passed QC, and a single threshold relaxation could push past 60% | `{max_pct_mt: ...}` or `{max_genes: ...}` | | `qc_low_pass_rate_no_easy_fix` | qc | < 60% passed and no single relaxation gets to 60% | none — human review needed | | `integrate_too_few_clusters` | integrate | every requested resolution yielded ≤ 2 clusters (skipped on n_cells < 500) | `{resolutions: "aio"}` | | `integrate_dominant_cluster` | integrate | the largest cluster > 50% at every resolution and cell-cycle regression is off | `{regress_cell_cycle: true}` | | `annotate_ambiguous_panel` | annotate | every cluster's panel margin < 0.05; an alternate panel is available on the profile | `{panel_name: ...}` | | `annotate_ambiguous_no_alt_panel` | annotate | margins < 0.05 and no alternate panel | none — switch profiles | | `annotate_panel_tissue_mismatch` | annotate | chondrocyte panel chosen but most clusters have immune-marker top genes | `{panel_name: "celltype_broad"}` | The QC trigger ceiling was raised from 30% (v0.8) to 60% (v0.9.1) so the degraded-prep band fires too. Real OA samples routinely sit at 40-55% pass on default thresholds because joint tissue is stress-prone; that's prime suggestion territory. ## Profile selection guidance ```bash scellrun profiles list scellrun profiles show joint-disease ``` Available profiles ship with the package; pick by tissue, not by guess: - `default` — fresh-tissue 10x v3, human, mt% ≤ 20, hb% ≤ 5. Use for PBMC or any tissue without a dedicated profile. (Pre-1.2.0 this was the catch-all for tumor / brain / kidney too; with 1.2.0's profiles shipped, prefer the tissue-specific one.) - `joint-disease` — same plus hb% ≤ 2 (cartilage is avascular) and the Fan 2024 11-subtype chondrocyte panel + a 15-group broad cell-type panel. Use when the user mentions cartilage, synovium, subchondral bone, OA, RA, or "joint" in any form. - `tumor` (v1.2.0) — pan-cancer / TME panel (TAMs, CAFs, T/B/NK/plasma, endothelial, neutrophils, plus a proliferation-marker proxy for malignant cells). Default thresholds. Use for any tumor scRNA without a more specific cancer-type profile. No fine-subtype panel — malignant cell substructure is disease-specific (run inferCNV or a dedicated panel when needed). - `brain` (v1.2.0) — cortical / hippocampal panel from Tasic 2018 / Hodge 2019 (excitatory + GABAergic neurons with PV/SST/VIP/LAMP5 interneurons broken out, oligo lineage, astrocytes, microglia, vasculature). Tightens mt% to 10. Use for fresh brain scRNA / snRNA. - `kidney` (v1.2.0) — KPMP / Stewart 2019 nephron + vascular + immune panel (PT, DT, TAL, CD principal/intercalated, podocytes, parietal, endothelial, mesangial, broad immune). Tightens mt% to 15. Use for kidney biopsy scRNA / snRNA. If the user mentions a tissue keyword, pick the matching profile. Do not default to `default` and let the broad panel run when a tissue- specific profile exists. If the user's tissue has no matching profile, use `default` and tell them — adding a profile is a one-PR contribution. ### Panel auto-selection inside `joint-disease` (v1.1.0+) The `joint-disease` profile ships two annotation panels: the Fan 2024 `chondrocyte_markers` (11 chondrocyte subtypes) and a broad `celltype_broad` panel (15-group: chondrocytes, fibroblasts, endothelial, pericytes, macrophages, T/B/NK/plasma cells, etc.). Old behavior was to always prefer `chondrocyte_markers` on this profile, which mis-labels subchondral-bone or synovium datasets where most clusters are immune. v1.1.0 hardens the auto-pick step (`_autopick_panel_for_data`): at the chosen resolution, scellrun counts the number of clusters where the chondrocyte panel scores any hit and the number where the broad panel scores any hit. The chondrocyte panel is kept ONLY when chondrocyte hits >= 1.5x broad hits. Tie or smaller margin swaps to `celltype_broad`. (The pre-1.1.0 rule fired only when >50% of clusters hit broad-WITHOUT-chondrocyte; on BML_1 every immune cluster picked up at least one weak chondrocyte hit, so the swap-trigger was 0/13 and the chondrocyte panel won the tie. The 1.5x margin closes that loophole.) What this means for you (the agent): - `--profile joint-disease` is safe to use even when the dataset turns out to be immune-rich (subchondral bone, synovium, fluid). scellrun will swap panels rather than blindly run the chondrocyte panel. - Two related decision-log rows record this. Both are real, both matter, do not conflate them: - `stage="analyze"`, `key="annotate.auto_panel"` — the orchestrator's auto-pick step, with the cluster-level hit counts in the rationale. For a swap: `value=celltype_broad`, `rationale="swapped to celltype_broad: chondrocyte_hits=2, broad_hits=9; required >=1.5x margin to keep chondrocyte panel."` For the chondrocyte default: `value=chondrocyte_markers`, `rationale="kept chondrocyte_markers: chondrocyte_hits=10, broad_hits=3; cleared the >=1.5x margin..."`. - `stage="annotate"`, `key="panel"` — the per-stage record of which panel `run_annotate` actually matched against. `source="auto"` when the orchestrator (or a self-check fix) injected the name, `source="user"` when the per-stage CLI got `--panel` on argv. See the next subsection for the source-attribution rules. - If you (the agent) want to override the auto-pick, pass `--panel chondrocyte_markers` or `--panel celltype_broad` to the per-stage `scrna annotate` command. This shows up as a `source="user"` row on `annotate.panel`. - (`panel_name` is NOT a decision-log key. It appears only inside self-check `fix_payload` dicts as the structured-fix shape, e.g. `{"panel_name": "celltype_broad"}`. The decision-log row that the fix lands in is still `annotate.panel`.) ### `source="user"` vs `source="auto"` on `annotate.panel` (v1.1.0+) Pre-1.1.0 had a footgun: when `analyze` orchestrated the pipeline and the auto-pick (or a self-check fix) selected a panel name, the value was passed through to `run_annotate` as a string, and the recorded decision row was tagged `source="user"`. That broke hard rule 5 ("when value differs from default, tell the user this was an override") — the user did not in fact override anything, the orchestrator did. v1.1.0 separates intent from value: `run_annotate` now takes a `panel_name_user_supplied` flag distinct from `panel_name`. The orchestrator (`scellrun analyze`) always passes `panel_name_user_supplied=False` for its auto-picked or self-check fixed-up panel; the per-stage CLI (`scellrun scrna annotate`) passes `True` only when `--panel` actually appeared on argv. The decision log row's `source` reflects the intent, not the string value. If you see `annotate.panel` `source="user"` in v1.1.0+, that genuinely means the user (or you, the agent) typed `--panel`. Quote the rationale back honestly. If you see `source="auto"`, this was scellrun's own pick (auto-pick at the orchestrator level, or a self-check `--auto-fix` retry); say "scellrun auto-picked..." not "you overrode...". ## Profile-specific glossaries Each profile that ships a marker panel adds its own glossary subsection here. The v1.2.0 cross-disease profiles (`tumor`, `brain`, `kidney`) ship `celltype_broad` panels but no fine-subtype glossaries yet — the panel labels for those profiles are self-explanatory English ("Tumor- associated macrophages", "Excitatory neurons", "Proximal tubule" etc.) and don't need a label-decode table. Add a glossary subsection here when a future profile ships an abbreviation panel like the Fan 2024 chondrocyte taxonomy below. ### `chondrocyte_markers` (Fan 2024) — joint-disease profile The `joint-disease` profile's `chondrocyte_markers` panel is the Fan 2024 human articular cartilage 11-subtype taxonomy. When the agent quotes one of these labels, here's what it means in plain language for the user: | label | full name | what it is in plain words | | ------- | -------------------------- | ------------------------- | | ProC | proliferating chondrocyte | actively dividing chondrocyte | | EC | effector chondrocyte | mature chondrocyte producing collagen II | | RegC | regulatory chondrocyte | matrix-modulating chondrocyte | | RepC | reparative chondrocyte | wound-repair-active chondrocyte | | HomC | homeostatic chondrocyte | stress-response / housekeeping chondrocyte | | preHTC | pre-hypertrophic | transitional toward hypertrophy | | HTC | hypertrophic | terminal hypertrophic chondrocyte | | preFC | pre-fibrochondrocyte | transitional toward fibro-cartilage | | FC | fibrochondrocyte | fibro-cartilage producing | | preInfC | pre-inflammatory | transitional toward inflammatory phenotype | | InfC | inflammatory chondrocyte | inflammation-driven chondrocyte | If the user is not a chondrocyte specialist, surface the plain-English column, not the abbreviation. The reference is Fan et al. 2024 — quote the PMID (38191537) when the user asks where the panel comes from. ## Finding the data The user often refers to files by name only ("my OA samples", "the cellranger output", "the synovium dataset"). Do not guess paths. - If the user gave you a path or hostname, use it. - If they did not, ask one clarifying question — "do you mean the run in `/path/to/X` or the new ones in `/path/to/Y`?" — rather than guess. - For multi-sample studies: scellrun does **not** auto-merge multiple cellranger outputs into one h5ad. Run `scellrun scrna convert` on each sample (or combine upstream with anndata.concat), produce a single merged h5ad with a `sample` / `orig.ident` / `batch` / `donor` obs column, and pass that one merged file to `analyze`. The integrate stage auto-detects the sample column and runs Harmony across it. This is a real gap; if you encounter it, do the merge in a small Python snippet and document the merge step in your reply to the user. ## Environment hygiene Always create a per-user / per-project conda env. Do not install scellrun into the user's system Python or a shared env. ```bash # Replace {userid} with something specific to the user/project, e.g. xiyu_oa. conda create -n scellrun-{userid} python=3.11 -y conda activate scellrun-{userid} pip install scellrun scellrun --version # confirm install ``` If the user already has a working env with scellrun, use it. If they are unsure, run `which scellrun` and `scellrun --version`; if either fails, create a fresh env as above. For LLM-enabled runs, pass `ANTHROPIC_API_KEY` through the env or `--ai` will fall back to off: ```bash export ANTHROPIC_API_KEY=... scellrun analyze data.h5ad --ai ``` ### Remote-server execution (ssh patterns) - When the user references "lab server" or "hospital" or hands you a path with no host, ask one clarifying question rather than guessing. Pick the wrong host and you're working on stale or absent data. - Run scellrun on the SAME machine where the data lives. Don't pull GBs of cellranger output across the network just so the CLI runs locally; the artifacts (and the report) end up on the wrong side of the link. - ssh pattern: ```bash ssh <host> 'bash -lc "conda activate scellrun-<userid> && scellrun analyze ..."' ``` The `bash -lc` matters — conda's shell hooks need a login shell to initialize, otherwise `conda activate` is a no-op and `scellrun` resolves to the wrong python. - If you (the agent) cannot ssh directly, write the commands the user should paste, and make them copy-pasteable. Don't assume the user knows ssh; spell out `ssh <host>` and the activation step. ### Multi-sample analysis (no auto-merge) scellrun does NOT auto-merge multiple cellranger output directories. v1.1.0 takes one h5ad per `analyze` call. For a multi-sample study (e.g. `BML_1..5 + NC_1..3`), the canonical pattern is convert-each-then-merge-externally: ```bash # convert each sample to its own h5ad for s in BML_1 BML_2 BML_3 BML_4 BML_5 NC_1 NC_2 NC_3; do scellrun scrna convert path/to/$s -o $s.h5ad done # then merge into one h5ad with anndata.concat python -c "import anndata as ad; \ samples = ['BML_1','BML_2','BML_3','BML_4','BML_5','NC_1','NC_2','NC_3']; \ adata = ad.concat({s: ad.read_h5ad(f'{s}.h5ad') for s in samples}, label='sample'); \ adata.write_h5ad('merged.h5ad')" scellrun analyze merged.h5ad --tissue "OA cartilage" ``` The `obs.sample` column from `ad.concat(label='sample')` is exactly the kind of column scellrun's auto-detect `sample_key` looks for (priority: `orig.ident` / `sample` / `batch` / `donor`). Harmony will fire and integrate batches. For a single sample (no merge needed), scellrun auto-degrades `--method harmony→none` per the v0.9.1 fix; you don't need to detect single-sample yourself. ### Auto-fix retry hygiene (cleanup) When `--auto-fix` runs and rescues a stage, you'll see two directories side by side: `<NN_stage>/` (the current good output) AND `<NN_stage>.failed-1/` (the original failed attempt). Both are kept. - Don't delete `.failed-N/` automatically. The user may want to compare the failed and the rescued state for debugging. - DO surface the existence of `.failed-N/` to the user. Say something like: "scellrun's QC self-check fired and we retried; the original failed attempt is at `01_qc.failed-1/`, the working result is at `01_qc/`." That way the user knows the retry happened and where to look. - If disk space is genuinely a concern, the user can `rm -rf` the `.failed-N/` directories themselves. Don't preemptively clean. ### Reporting back to the user (artifact handoff) The canonical handoff is `<run-dir>/05_report/index.html`. That's the file the user opens. - If the user is on Windows, iPad, or phone and can't open HTML on the remote server, copy the report locally: ```bash scp -r <host>:<run-dir>/05_report ./ ``` Then point the user at the local copy. - For long-running pipelines, mention the run-dir path EARLY ("scellrun is running, output will land in `/tmp/scellrun_out/run-...`; I'll surface the report when it's ready"). That gives the user something to reference if the connection drops or the agent times out before the run completes. ### Long-running remote jobs (tmux / nohup / scheduler) - A `scellrun analyze` on a real cohort takes 5-30 min on a workstation, longer on integration-heavy multi-sample. SSH disconnects mid-run kill the job. - Default: wrap the command in `tmux` or `screen`. Pattern: ```bash ssh <host> tmux new -s scellrun-<userid> conda activate scellrun-<userid> && scellrun analyze ... 2>&1 | tee run.log # detach: Ctrl-b d. Reattach: tmux attach -t scellrun-<userid> ``` - If the cluster has SLURM / PBS / LSF, prefer a job submission script over a foreground tmux. Surface to the user that this is the option; don't insist on tmux. - Capture stdout AND stderr to a log file (`2>&1 | tee run.log`) — scellrun's per-stage progress logs are how you debug a long run, and the run-dir's report.html only renders post-hoc. ### Disk space and run-dir naming - A typical run-dir for a 10k-cell sample is 50-200 MB; for a 100k-cell multi-sample integrate, 1-3 GB (the integrated.h5ad + UMAP grids dominate). - Default run-dir is `./scellrun_out/run-YYYYMMDD-HHMMSS/`. For multi-project users, override with `--run-dir my-project/2026-05-OA-pilot/` to keep things organized. - The `.failed-N/` directories from `--auto-fix` retries can double disk use. Surface to the user before they fill /tmp. ### Resume / inspect a half-finished run - If `analyze` died mid-pipeline, the run-dir has the partial stages. v0.9.1+ auto-resumes incomplete prior runs without `--force`. - To inspect: open `<run-dir>/00_run.json` (manifest) and `00_decisions.jsonl` (what scellrun decided so far). The last stage that completed has its report; failed-mid-stage will be empty or partial. - Don't `rm -rf` the run-dir to "start clean" without checking first — the user may want the failed artifacts for debugging. Use a fresh run-dir name instead. ### Secret handling (Anthropic API key + others) - scellrun's `--ai` paths read `ANTHROPIC_API_KEY` from environment. The default agent action of `export ANTHROPIC_API_KEY=...` in shell history leaks the key into `~/.bash_history`. - Patterns: - For interactive debugging: prefix the command, e.g. `ANTHROPIC_API_KEY=sk-... scellrun analyze ...`. Bash discards prefix-vars from history with `HISTCONTROL=ignoreboth`. - For automation: write the key to `~/.config/scellrun/.env` (mode 600) and `source` it before invocation. - Never commit a key, never paste a key into chat with the user. - If no key is set, `--no-ai` works. The deterministic pipeline runs fully; the LLM second opinions are skipped (which is fine for most analyses). ## Hard rules for the agent (don't break these) 1. **Don't write your own scanpy / scrublet code when scellrun covers the step.** scellrun encodes opinionated, reviewed defaults. Rolling your own with different thresholds silently undoes that. If the user's request maps to a scellrun command, call the command. 2. **Don't silently filter cells.** scellrun's policy is *flag, don't drop*. After QC, `qc.h5ad` contains every input cell with a `scellrun_qc_pass` boolean column. Hand the user the report and let them decide. Never run `adata = adata[adata.obs.scellrun_qc_pass]` on the user's behalf without telling them. 3. **Don't override profile defaults silently.** If you pass `--max-genes 4500` because the user mentioned osteoclasts (which are multinucleate), tell the user *why* you raised the cap and confirm the override appears as `source="user"` in the decision log. 4. **Don't assume raw counts.** scellrun warns if X looks log- normalized and skips scrublet in that case. If you see that warning, surface it to the user — they probably want to re-run on the raw matrix. 5. **(v0.7+) When a value differs from default, tell the user this was an override and quote the decision-log rationale.** The defaults ship with reasoned rationale strings; the user has the right to read them. 6. **(v0.8+) When self-check fires, surface the suggestion BEFORE answering "what does the data show".** A self-check trigger means the downstream finding is contingent on whether the user accepts the fix. Lead with the trigger, then the structured suggestion, then ask whether to re-run (or just re-run with `--auto-fix` if the user already said "go ahead"). ## Failure modes & how to handle them - **QC pass-rate < 60%.** Self-check fires `qc_low_pass_rate` (the trigger ceiling was raised from 30% in v0.8 to 60% in v0.9.1 to catch the degraded-prep band). Quote the suggestion ("raise `--max-pct-mt` to N"); ask the user; re-run with `--auto-fix` if they agree. - **Every resolution has ≤ 2 clusters.** Self-check fires `integrate_too_few_clusters`. Suggested fix is `--resolutions aio` (the 13-step AIO sweep from 0.01 to 2.0). The default sweep tops out at 1.0; some datasets need finer resolution. - **Largest cluster dominates > 50% at every resolution.** Often a cell-cycle artifact. Suggested fix is `--regress-cell-cycle`. - **Annotate has no panel for the profile, or the profile lacks an annotate panel entirely.** scellrun bails early with a clear message; suggest a different `--profile` (or use the per-stage `scrna annotate --panel <name>` if you know which panel to use). - **Annotate panel margins are all < 0.05.** Self-check fires `annotate_ambiguous_panel`. Suggested fix is to switch panel (typically chondrocyte → broad celltype). For synovium / joint- fluid datasets a separate `annotate_panel_tissue_mismatch` heuristic also fires. - **PyPI install times out / no internet.** Run on a server with a working channel, or `pip install --no-deps scellrun` and install the deps manually from a local mirror. - **No `ANTHROPIC_API_KEY`.** `--no-ai` works fine; tell the user that the resolution recommender and annotation second-opinion are skipped, but everything deterministic (panel match, markers, decision log, self-check) still works. ## End-to-end example dialogue This is the shape of interaction you should imitate. The agent's turns are what scellrun-aware behavior looks like. > **User:** "Can you take a look at my cartilage scRNA data? It's at `~/data/sample1/cellranger_outs/` on the lab server." > **Agent:** "I'll ssh in, set up a per-user conda env, convert the cellranger output to h5ad, and run the joint-disease pipeline end-to-end (cartilage tissue, so the Fan 2024 chondrocyte panel applies). Then I'll walk you through the report." > > ```bash > ssh lab-server > conda create -n scellrun-cart python=3.11 -y && conda activate scellrun-cart > pip install scellrun > scellrun scrna convert ~/data/sample1/cellranger_outs/ -o ~/data/sample1/data.h5ad > scellrun analyze ~/data/sample1/data.h5ad \ > --profile joint-disease \ > --tissue "OA cartilage" \ > --ai > ``` > > "Pipeline finished. Report is at `~/scellrun_out/run-20260430-094530/05_report/index.html`. Summary: > > - **QC:** 2,450 / 2,700 cells passed (90.7%). pct_mt median 6.1%, p95 17.4%; under the 20% joint-tissue cap. Doublets flagged: 38. No self-check triggers. > - **Integrate:** Single sample (no harmony), 30 PCs. Sweep produced 7 clusters at res 0.5, 11 at res 0.8. > - **Markers (res 0.5):** Strong COL2A1 / ACAN signatures across most clusters as expected for cartilage. > - **Annotate:** chondrocyte_markers panel matched. Top calls: HomC (cluster 0, 38% of cells), RegC (1, 22%), preFC (2, 14%), HTC (3, 9%), the rest minor. AI second-opinion agreed on HomC and RegC; flagged cluster 5 as 'unclear, possibly InfC'. > > Anything you want to drill into?" > **User:** "Why did you pick res 0.5?" > **Agent:** "From the decision log (`00_decisions.jsonl`): > > > stage=integrate, key=resolution_recommended, value=0.5, source=ai > > rationale: 'LLM picked 0.5 from the sweep — best separation of chondrocyte subtypes vs over-splitting at 0.8' > > The sweep tested 0.1 / 0.3 / 0.5 / 0.8 / 1.0; 0.1 and 0.3 collapsed RegC into HomC, 0.8 split HomC into two near-identical groups. 0.5 was the LLM's pick — call was AI, not deterministic. If you want the non-AI default, the integrate report also has cluster-counts and a UMAP grid for all five resolutions; pick whichever looks right to you." > **User:** "Cluster 5 looks weird — what does scellrun think it is?" > **Agent:** "Looking at `04_annotate/annotations.csv`: > > | cluster | panel_label | panel_score | panel_margin | top markers (res 0.5) | ai_label | > | --- | --- | --- | --- | --- | --- | > | 5 | InfC | 0.18 | 0.04 | IL6, CXCL8, MMP3, NFKBIA, IL1B | unclear, possibly InfC | > > The panel called InfC (inflammatory chondrocyte) at score 0.18 with margin 0.04 — close to the 0.05 ambiguity floor, so a self-check on the panel didn't fire but it's borderline. The top markers (IL6, CXCL8, MMP3) match the InfC signature in Fan 2024 (PMID 38191537) but also overlap with a stressed/senescent fibroblast profile. The AI second-opinion hedged for the same reason. I'd say lean InfC but flag for manual review; if you want PubMed evidence pulled per marker, I can re-run with `--pubmed` on." This dialogue is the template. Run the pipeline. Open the report. Quote the decision log. Surface self-check findings. Hedge when scellrun hedges. Defer to the user on overrides. ## Anti-patterns that look right but aren't - **"I'll just write a quick scanpy script."** No. The CLI exists so the defaults are reviewed and the artifacts are consistent across runs. Writing your own loses both. If a stage is genuinely missing (something scellrun does not cover yet), say so — don't paper over. - **"Let me filter out the low-quality cells before showing the user."** No. scellrun flags, never drops. The user sees the QC report and decides cuts. If you filter on their behalf you erase their ability to audit the threshold choice. - **"I'll set `--max-pct-mt 10` because that's what the textbook says."** No. The default 20% in joint tissues is intentional — joint and other stress-prone tissues lose real cells at 10%. If the user asks why not 10%, explain rather than cave. (Decision-log rationale: "joint-tissue-aware default; 10% loses real stressed cells".) - **"--ai is on by default, just leave it."** Only when `ANTHROPIC_API_KEY` is set. If the user does not want LLM calls in their pipeline (privacy, reproducibility, cost), pass `--no-ai` explicitly and tell them the deterministic outputs still work. - **"The self-check warned about a low pass-rate but the data still produced clusters, so it's fine."** No. The trigger means PI working practice flags this as suspect. Surface the warning even if downstream stages ran. The user decides whether to accept or re-run. - **"I'll add an `--auto-fix` because the user is busy."** Only if the user said "just fix it" or you've already shown them the suggestion and they agreed. Auto-fixing without consent silently changes the thresholds the analysis ran with. ## Source of truth - Repo: <https://github.com/drstrangerujn/scellrun> - ROADMAP: `<repo>/ROADMAP.md` — version map, AIO/Rmd reference points, shipped versions through v0.9. - Profiles: `<repo>/src/scellrun/profiles/` — read these to see current thresholds and panels. - Decision module: `<repo>/src/scellrun/decisions.py` — schema for the `00_decisions.jsonl` log. - Self-check module: `<repo>/src/scellrun/self_check.py` — trigger thresholds and structured fix shapes. ## Provenance Defaults trace to the in-house R AIO pipeline (Liu lab, Nanjing) and a clinical team's working practice for OARSI / MSK research. Where Python lacks a stable equivalent (e.g. decontX), the relevant R-only step is documented as "not in scope" rather than half-implemented.


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)