Pseudotime Differential Expression (Palantir + tradeSeq)
Domain: life_sciences — Genomics & Sequence Analysis

Summary: Discover genes with different expression patterns along pseudotime in a bone-marrow scRNA-seq dataset using Palantir for pseudotime estimation and tradeSeq for lineage-specific differential expression testing.

Task prompt:
You are a computational biologist performing pseudotime differential expression analysis on a bone-marrow scRNA-seq dataset on Linux.

Task directory:
- `base`

Input files:
- scRNA-seq counts: `base/input/marrow_sample_scseq_counts.h5ad`
- Detailed task brief: `base/input/task_description.txt`
- Python dependency manifest: `base/input/runtime_env/pyproject.toml`

Recommended setup:
- Work in `base`
- Install the Python analysis packages from the staged manifest, for example:
  `uv sync --project "base/input/runtime_env"`
  or `pip install anndata scanpy palantir pandas "numpy<2" scipy scikit-learn fa2-modified leidenalg`
- R packages `SingleCellExperiment` and `tradeSeq` must be available via `Rscript`.

Analysis pipeline:
1. Read `input/marrow_sample_scseq_counts.h5ad` (4142 cells x 16106 genes, raw counts).
2. Preprocess: normalize per cell, log-transform, select 1500 highly variable genes (flavor='cell_ranger'), PCA.
3. Run Palantir: diffusion maps (n_components=5), multiscale space, neighbors, UMAP, then `palantir.core.run_palantir` with start cell `Run5_164698952452459`, terminal states DC=`Run5_131097901611291`, Mono=`Run5_134936662236454`, Ery=`Run4_200562869397916`, num_waypoints=500.
4. Export expression matrix, pseudotime, fate probabilities, and UMAP from Python to CSV/MTX files.
5. In R, assemble a SingleCellExperiment from the exported files.
6. Keep DC (column 1) and Ery (column 3) lineages; drop cells with zero weight in both; renormalize weights row-wise.
7. Set R seed 27, run fitGAM(nknots=6), then patternTest(l2fc=log2(1.5)).
8. Apply BH correction; filter genes with padj < 0.05.

Required output:
- Save to: `base/output/de_genes.csv`
- Format: CSV with a single column `gene` containing HGNC symbols (one per row, deduplicated, order-independent).
- Success criterion: at least 80% symbol-set overlap with a hidden gold-standard gene list.

Do not modify input files or any non-output task directories.
Do not ask for confirmation. Execute directly.

The agent must:
Install Python analysis packages (scanpy, palantir, anndata, etc.) and R Bioconductor packages (SingleCellExperiment, tradeSeq).
Run Palantir pseudotime estimation with the specified start cell and terminal states.
Export Python outputs to disk and reassemble as a SingleCellExperiment in R.
Run tradeSeq fitGAM and patternTest on DC vs Ery lineages with specified parameters.
Write significant DE genes to output/de_genes.csv as a single-column CSV.

Software: Python
R
scanpy
palantir
anndata
tradeSeq
SingleCellExperiment

Input files:
name: marrow_sample_scseq_counts.h5ad
format: h5ad
description: Bone-marrow scRNA-seq raw counts (4142 cells x 16106 genes).
name: task_description.txt
format: txt
description: Detailed task brief with pipeline parameters and output contract.
name: runtime_env/pyproject.toml
format: toml
description: Python dependency manifest for agent installation.