TCGA BRCA Differential Expression Analysis
Domain: life_sciences — Genomics & Sequence Analysis

Summary: Analyze TCGA Breast Cancer expression and clinical matrices to identify genes differentially expressed between primary tumor and solid tissue normal samples.

Task prompt:
You are a bioinformatics analyst performing a TCGA Breast Cancer differential gene expression analysis on Linux.

Task directory:
- `base`

Visible input files:
- Expression matrix: `base/input/expression_matrix.tsv.gz`
- Clinical annotations: `base/input/clinical_matrix.tsv`
- Benchmark gene list: `base/input/truth_breast_degs.csv`
- Runtime dependency manifest: `base/input/runtime_env/pyproject.toml`
- Requirements file: `base/input/requirements.txt`
- Manifest template: `base/input/run_manifest_template.json`
- Detailed task brief: `base/input/task_description.txt`
- Output contract: `base/input/output_requirements.txt`

Recommended setup:
- Work in `base`
- Install the Python analysis packages into your own environment from the staged input, for example:
  `uv sync --project "base/input/runtime_env"`
  or `pip install -r "base/input/requirements.txt"`

Analysis requirements:
1. Match expression matrix sample columns with clinical rows.
2. Compare `Primary Tumor` against `Solid Tissue Normal` using the clinical `sample_type` field.
3. Filter genes where more than 80% of matched samples have expression equal to zero.
4. Run Welch's t-test for each remaining gene.
5. Compute `log2FC = mean(tumor) - mean(normal)`.
6. Apply Benjamini-Hochberg correction.
7. Mark significant DEGs using `abs(log2FC) > 1` and `padj < 0.05`.
8. Generate PCA and volcano plots.
9. Benchmark significant DEGs against `base/input/truth_breast_degs.csv`.

Required output directory:
- Write every deliverable under `base/output`

Required output files:
- `sample_summary.txt`
- `pca_plot.png`
- `deg_results_all.csv`
- `deg_results_significant.csv`
- `volcano_plot.png`
- `benchmark_summary.csv`
- `run_manifest.json`

Do not modify input files or any non-output task directories.
Do not ask for confirmation. Execute directly.

The agent must:
Match expression-matrix sample columns to clinical metadata rows.
Compare `Primary Tumor` samples against `Solid Tissue Normal` samples.
Filter genes with more than 80% zero expression across matched samples.
Run Welch's t-test for each retained gene.
Compute `log2FC = mean(tumor) - mean(normal)` and apply Benjamini-Hochberg correction.
Write complete and significant DEG CSV tables with the required columns.
Generate PCA and volcano PNG plots.
Benchmark significant DEGs against the provided gene list.
Write `sample_summary.txt`, `benchmark_summary.csv`, and `run_manifest.json` under `output/`.

Software: Python
pandas
scipy
statsmodels
matplotlib
scikit-learn

Input files:
name: expression_matrix.tsv.gz
format: TSV.GZ
description: Gene-by-sample TCGA BRCA expression matrix.
name: clinical_matrix.tsv
format: TSV
description: Clinical sample annotations including the `sample_type` field used for tumor-normal grouping.
name: truth_breast_degs.csv
format: CSV
description: Benchmark gene list used by the task workflow.
name: runtime_env/pyproject.toml
format: TOML
description: Python dependency manifest for the analysis environment.
name: run_manifest_template.json
format: JSON
description: Template for the required output manifest.
name: output_requirements.txt
format: TXT
description: Lists the required output artifact names and content expectations.