Skill: kuberwastaken/litmus (version 1.1.1)

--- name: litmus version: 1.1.1 description: "Parallel autonomous ML research agents with a Director, git worktrees for per-agent experiment branches, a Skills library for validated technique reuse, a Synthesizer that distills collective knowledge overnight, and circadian rhythm (leisure 03:00–06:00 for paper reading and creative thinking). Uses OpenClaw sessions_spawn, cron, and steer natively. Use when: (1) start or run ML research agents overnight, (2) check agent status or experiment results, (3) view leaderboard or morning digest, (4) steer or stop agents, (5) ask what agents discovered or are exploring, (6) set up Litmus for the first time. NOT for: general coding, non-ML tasks, or machines without a GPU." homepage: https://github.com/kuberwastaken/litmus source: https://github.com/kuberwastaken/litmus license: MIT-0 tags: [research, ml, machine-learning, training, autonomous-agents, overnight, experiments, gpu, llm, autoresearch] metadata: openclaw: emoji: "🔬" requires: bins: ["uv", "git", "python3"] recommends: bins: ["nvidia-smi"] os: [linux, darwin] configPaths: ["~/.litmus/"] optional: env: ["CLAWRXIV_API_KEY"] --- # Litmus — Parallel Autonomous ML Research Agents Litmus spawns multiple OpenClaw subagents that experiment on your GPU overnight. Each runs on its own **git branch** in a shared lab repository — every experiment is a commit, agents can read each other's code, cherry-pick breakthroughs, and build on the global best at any time. Validated techniques accumulate in a **Skills library** (`~/.litmus/shared/skills/`). A **Synthesizer** runs at 04:00 to distill collective knowledge into skills and write a research agenda for the next day. A **Director** runs every 2 hours to steer workers, trigger **Compass Resets** on stagnation, and orchestrate cross-agent knowledge transfer. **What makes it more than autoresearch**: - Git worktrees: agents share one repo, each on their own branch — full experiment history, cherry-pick, and cross-agent code inspection via `git -C ~/.litmus/repo log --all` - Skills library: validated techniques persist and compound — agents don't re-discover wins - Synthesizer: distills all overnight notes into reusable skills and a research agenda - Compass Reset: Director detects stagnation and forces structured pivots using the skills gap - Two-phase experiment budget: quick 90-second check before committing to a full run - Structured attempt records: JSON per experiment in `shared/attempts/` for rich analytics - Leisure mode (03:00–06:00): workers read papers, write moonshot hypotheses, identify gaps - Morning digest: research narrative delivered to your chat at 08:00 Everything is a native OpenClaw subagent. No external processes, no PID files. --- ## First-Time Setup **Recommended** — ask your OpenClaw agent (runs a guided onboarding conversation): > "Install https://clawhub.ai/kuberwastaken/litmus and set it up for my machine" Full onboarding instructions: `{baseDir}/references/onboarding.md` — read that file first. **Or manually:** ```bash git clone https://github.com/kuberwastaken/litmus ~/.litmus bash ~/.litmus/scripts/setup.sh ``` Clones Karpathy's training harness, builds the shared lab git repo at `~/.litmus/repo/`, installs Python deps via `uv`, downloads ~1 GB of training data. Wait for it to finish. --- ## Starting Research ### 1 — Prepare workspaces (creates git worktrees) ```bash bash {baseDir}/scripts/prepare-agents.sh --agents 4 --templates architecture,optimizer,general,general ``` Creates git worktrees under `~/.litmus/agents/`, each on its own branch in `~/.litmus/repo/`. The shared lab git repo means every agent's experiments are immediately visible to all others: ```bash git -C ~/.litmus/repo log --all --oneline --graph ``` ### 2 — Spawn research subagents ``` sessions_spawn task: "Read program.md in your current directory and run the research loop forever." runtime: "subagent" mode: "session" agentId: "litmus-worker-arch-1" cwd: "~/.litmus/agents/arch-1" ``` Repeat for each agent, then: ``` sessions_yield message: "Research agents running. I'll notify you on new discoveries." ``` **Templates**: `architecture` · `optimizer` · `regularization` · `general` Full template details: `{baseDir}/references/templates/` ### 3 — Start the Director Layer ```bash bash {baseDir}/scripts/setup-cron.sh --timezone "Your/Timezone" ``` Registers 6 cron jobs: | Cron | Default schedule | Role | |---|---|---| | `litmus-director` | Every 2h during research hours | Reviews results, steers workers, Compass Reset on stagnation | | `litmus-leisure` | 03:00 daily | Switches workers to paper-reading / creative thinking mode | | `litmus-synthesizer` | 04:00 daily | Distills notes into skills library, writes research agenda | | `litmus-dawn` | 06:00 daily | Wakes workers, queues synthesizer's priority experiments | | `litmus-watchdog` | Every 30 min | Liveness check, escape mode on zero improvements | | `litmus-digest` | 08:00 daily | Morning research narrative delivered to your chat | All times are configurable during onboarding — the setup agent pitches defaults and asks what you'd like to change. Common presets: night owl (01:00/02:00/04:00/07:00), early bird (23:00/00:30/02:00/05:30), intensive (1h director). Pass custom times to `scripts/setup-cron.sh` with `--leisure-start`, `--synthesizer-time`, `--dawn-time`, `--digest-time`, `--director-hours`, `--watchdog-minutes`. --- ## Managing Agents **Status** (experiment counts, best val_bpb, git tree): ```bash bash {baseDir}/scripts/status.sh ``` **Leaderboard** (cross-agent, from shared/attempts/ JSON): ```bash bash {baseDir}/scripts/results.sh --top 10 bash {baseDir}/scripts/results.sh --agent arch-1 # single agent ``` **Full lab git history** (all agents' experiments as a tree): ```bash git -C ~/.litmus/repo log --all --oneline --graph ``` **Inspect any experiment**: ```bash git -C ~/.litmus/repo show <commit-hash> # see what changed cat ~/.litmus/shared/attempts/<hash>.json # see the metrics ``` **Steer** (redirect mid-run, no restart): ``` subagents action: "steer" target: "litmus-worker-arch-1" message: "Stop refining depth. Checkout the best commit from opt-2 and combine their LR with DEPTH=10." ``` **Stop**: ``` subagents action: "kill" target: "all" ``` --- ## What Agents Write Overnight | Path | Contents | |---|---| | `~/.litmus/shared/attempts/<hash>.json` | Structured record for every experiment (agent, val_bpb, status, title) | | `~/.litmus/shared/skills/<name>.md` | Validated reusable techniques with YAML frontmatter | | `~/.litmus/shared/notes/discoveries/` | Per-improvement discovery notes | | `~/.litmus/shared/notes/anomalies/` | Unexpected result notes | | `~/.litmus/shared/notes/moonshots/` | Speculative hypotheses from leisure | | `~/.litmus/shared/notes/synthesis/` | Synthesizer's research agenda and combination matrix | | `~/.litmus/shared/discoveries.md` | Cross-agent knowledge base (flat, for quick reading) | | `~/.litmus/shared/midnight-reflections.md` | Leisure agent's nightly narrative | | `~/.litmus/repo/` (git) | All experiment commits across all agents on their branches | --- ## Reference Files - `{baseDir}/references/onboarding.md` — first-time setup conversation - `{baseDir}/references/program.md` — worker agent loop (git-aware, skills-reading, two-phase budget) - `{baseDir}/references/director.md` — Director cron (Compass Reset, cross-pollination) - `{baseDir}/references/leisure.md` — Leisure mode (paper reading, structured notes, skill extraction) - `{baseDir}/references/synthesizer.md` — Synthesizer cron (knowledge distillation, skills library) - `{baseDir}/references/dawn.md` — Dawn cron (wake workers, queue experiments) - `{baseDir}/references/watchdog.md` — Watchdog cron (liveness, escape mode) - `{baseDir}/references/digest.md` — Morning digest (research narrative) - `{baseDir}/references/templates/` — Research focus templates - `{baseDir}/references/clawrxiv.md` — ClawRxiv integration (optional auto-publishing)

path: INSTALL.md
content: # Installing Litmus

## Via ClawHub (recommended)

Ask your OpenClaw agent:
> "Install the litmus skill"

Or from the CLI:
```bash
npx clawhub install litmus
```

## Manual install

```bash
git clone https://github.com/kuberwastaken/litmus ~/.openclaw/skills/litmus
```

## Requirements

- **GPU**: NVIDIA GPU with CUDA (required for training runs)
- **`uv`**: Python package manager — `curl -LsSf https://astral.sh/uv/install.sh | sh`
- **`git`**: For experiment version control and git worktrees
- **`python3`**: For JSON attempt records, leaderboard scripts, and analytics
- **`nvidia-smi`**: Recommended for GPU detection (usually bundled with CUDA drivers)
- OS: Linux or macOS (Windows not supported)

## First-time data setup

After installing the skill, run setup once to download training data (~1 GB):

```bash
bash {baseDir}/scripts/setup.sh
```

This clones Karpathy's [autoresearch](https://github.com/karpathy/autoresearch) training harness,
installs Python deps via `uv`, prepares the dataset, and creates the shared lab git repository at
`~/.litmus/repo/`. Each agent will get its own branch in this shared repo via git worktrees.

Takes ~5 minutes on a good connection.

### What setup creates

```
~/.litmus/
  repo/          ← shared lab git repo (all agents' experiment branches live here)
  harness/       ← training code (cloned from autoresearch)
  shared/
    attempts/    ← one JSON file per experiment (source of truth for leaderboard)
    notes/
      discoveries/
      anomalies/
      moonshots/
      synthesis/
    skills/      ← validated reusable techniques (Skills Library)
  agents/        ← per-agent git worktrees (created by prepare-agents.sh)
```

## Uninstall

```bash
npx clawhub uninstall litmus
rm -rf ~/.litmus   # removes all runtime state, agent workspaces, and results
```


path: package.json
content: {
  "name": "litmus",
  "version": "1.1.1",
  "description": "Parallel autonomous ML research agents with Director, circadian rhythm, and morning digest for OpenClaw",
  "license": "MIT-0",
  "homepage": "https://github.com/kuberwastaken/litmus",
  "repository": {
    "type": "git",
    "url": "https://github.com/kuberwastaken/litmus.git"
  },
  "keywords": ["openclaw", "skill", "research", "ml", "training", "autonomous-agents", "gpu"],
  "openclaw": {
    "skills": ["./"]
  }
}


path: README.md
content: # Litmus — Run a Parallel Autonomous ML Research Organization on your OpenClaw instance.

<p align="center">
  <img src="assets/banner.png" alt="Litmus">
</p>

<p align="center">
  <a href="https://github.com/kuberwastaken/litmus/actions/workflows/deploy-site.yml"><img src="https://img.shields.io/github/actions/workflow/status/kuberwastaken/litmus/deploy-site.yml?branch=main&style=for-the-badge" alt="Deploy status"></a>
  <a href="package.json"><img src="https://img.shields.io/badge/version-1.1.1-ff4500?style=for-the-badge" alt="Version"></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT--0-blue?style=for-the-badge" alt="MIT-0 License"></a>
  <a href="https://clawhub.ai/kuberwastaken/litmus"><img src="https://img.shields.io/badge/OpenClaw-skill-8a2be2?style=for-the-badge" alt="ClawHub skill"></a>
</p>

**Litmus** is an [OpenClaw](https://github.com/openclaw/openclaw) skill that turns your always-on GPU machine into a self-directing ML research lab. Multiple native subagents run experiments overnight, each on their own git branch. A Director steers them every two hours. A Synthesizer distills collective knowledge into a reusable skills library at 04:00. A circadian rhythm shifts agents into creative thinking at 03:00. A research narrative lands in your chat by 08:00.

Built on [Karpathy's autoresearch](https://github.com/karpathy/autoresearch). Everything is a native `sessions_spawn` subagent — no external CLI processes, no PID files.

[Website](https://litmus.kuber.studio) · [Docs](https://litmus.kuber.studio/docs) · [ClawHub](https://clawhub.ai/kuberwastaken/litmus)

Install by asking your OpenClaw agent:
> **"Install https://clawhub.ai/kuberwastaken/litmus and set it up for my machine"**

Your agent checks your GPU, pitches a full schedule (with timing presets), spawns workers, and registers cron jobs — all in one conversation.

---

## What Litmus does that autoresearch doesn't

| Feature | autoresearch | Litmus |
|---|---|---|
| Parallel agents | 1 | 2–8 workers, each on their own git branch |
| Experiment history | Per-run log | Full git tree — every experiment a commit, cherry-pick across agents |
| Cross-agent learning | — | Shared `discoveries.md`, `anomalies.md`, structured `notes/` |
| Knowledge accumulation | — | **Skills library** — validated techniques agents build on, never re-discover |
| Knowledge distillation | — | **Synthesizer** — nightly agent that reads all results and writes reusable skills |
| Active direction | — | Director cron every 2h |
| Stagnation response | — | **Compass Reset** — Director reads skills gap + git history, steers to unexplored combos |
| Dead-end pruning | — | Two-phase budget — 90s quick check before full run, abandon early |
| Structured results | — | JSON attempt record per experiment in `shared/attempts/` |
| Creative thinking | — | Leisure mode 03:00–06:00, paper reading, moonshot hypotheses |
| Paper-grounded ideas | — | arxiv scan → experiment queue |
| Morning briefing | — | Digest to your chat at 08:00 |
| Setup | Manual | Conversational — pitches defaults, asks for changes |

---

## Architecture

```
YOU ──▶ OpenClaw agent
             │
             ├── sessions_spawn ──▶ worker-arch-1 ──┐  each on its own git branch
             ├── sessions_spawn ──▶ worker-opt-2  ──┤  in ~/.litmus/repo/
             ├── sessions_spawn ──▶ worker-gen-3  ──┘
             └── sessions_yield
                                        │
                               ~/.litmus/shared/
                                  attempts/       ← JSON record per experiment
                                  notes/          ← structured YAML-frontmatter notes
                                  skills/         ← validated reusable techniques
                                  discoveries.md
                                  anomalies.md

Director (cron · every 2h during research hours)
  └── reads shared/attempts/ → computes improvement rates
  └── Compass Reset on ≥6 experiments without improvement
  └── cross-pollinates discoveries across agents
  └── assigns anomaly investigation

03:00 ── litmus-leisure   ── arxiv scan · contradiction analysis · writes notes/moonshots/
04:00 ── litmus-synthesizer ── distills notes + attempts → skills/ + research agenda
06:00 ── litmus-dawn      ── reads synthesizer output · queues experiments · wakes workers
08:00 ── litmus-digest    ────────────────────────────────────────────────▶ YOUR CHAT
```

The shared git repo at `~/.litmus/repo/` holds every agent's branch. Browse the full experiment tree any time:
```bash
git -C ~/.litmus/repo log --all --oneline --graph
```

---

## Cron Layer

All times are defaults — configurable during onboarding.

| Job | Default schedule | Role |
|---|---|---|
| `litmus-director` | Every 2h during research hours | Reviews results, Compass Reset on stagnation, cross-pollination |
| `litmus-leisure` | 03:00 daily | Switches workers to thinking mode; reads arxiv; writes structured notes |
| `litmus-synthesizer` | 04:00 daily | Distills notes + attempts into skills library; writes research agenda |
| `litmus-dawn` | 06:00 daily | Reads synthesizer output; queues today's experiments; wakes workers |
| `litmus-watchdog` | Every 30 min | Liveness check, escape mode on zero improvements |
| `litmus-digest` | 08:00 daily | Morning research narrative → delivered to your chat |

---

## Configuration

Onboarding pitches all defaults and asks what you'd like to change. Common presets:

| Preset | Leisure | Synthesizer | Dawn | Digest |
|---|---|---|---|---|
| Standard (default) | 03:00 | 04:00 | 06:00 | 08:00 |
| Night owl | 01:00 | 02:00 | 04:00 | 07:00 |
| Early bird | 23:00 | 00:30 | 02:00 | 05:30 |

Full config options saved to `~/.litmus/config.json`:

| Category | Setting | Default |
|---|---|---|
| Timing | timezone | *ask* |
| Timing | leisure start | 03:00 |
| Timing | synthesizer time | 04:00 |
| Timing | dawn / research resume | 06:00 |
| Timing | digest delivery | 08:00 |
| Timing | director interval | every 2h |
| Timing | watchdog interval | every 30 min |
| Compute | agent count | GPU-based |
| Compute | experiment budget | 300s (5 min) |
| Compute | quick-check budget | 90s (cuts dead ends early) |
| Research | templates | architecture, general |
| Research | custom goal | none |
| Leisure | intensity | standard (3 searches · 5 papers · 5 moonshots) |
| Runtime | mode | subagents *(or claude-code)* |

Defaults in [`configs/default.json`](configs/default.json). Pass custom times to `setup-cron.sh` with `--leisure-start`, `--synthesizer-time`, `--dawn-time`, `--digest-time`, `--director-hours`, `--watchdog-minutes`.

---

## Managing Agents

```bash
# Status — per-agent experiment counts, best val_bpb, stagnation flags, git tree
bash {baseDir}/scripts/status.sh

# Leaderboard — all agents, from shared/attempts/ JSON
bash {baseDir}/scripts/results.sh --top 10
bash {baseDir}/scripts/results.sh --agent arch-1   # single agent

# Full experiment history as a git tree
git -C ~/.litmus/repo log --all --oneline --graph

# Inspect any specific experiment
git -C ~/.litmus/repo show <commit-hash>
cat ~/.litmus/shared/attempts/<hash>.json
```

```
# Steer a worker mid-run (no restart needed)
subagents action:"steer" target:"litmus-worker-arch-1"
  message:"Checkout opt-2's best commit and combine their LR with DEPTH=10."

# Stop everything
subagents action:"kill" target:"all"
```

Or just tell your agent: `"How are my Litmus agents doing?"` / `"Stop all Litmus agents"`.

---

## What Agents Write Overnight

| Path | Written by | Contents |
|---|---|---|
| `shared/attempts/<hash>.json` | Workers | Structured record per experiment — agent, val_bpb, status, title, commit, parent |
| `shared/skills/<name>.md` | Workers + Synthesizer | Validated reusable techniques with YAML frontmatter and evidence commits |
| `shared/notes/discoveries/` | Workers | Per-improvement discovery notes with frontmatter |
| `shared/notes/anomalies/` | Workers + Director | Unexpected result notes |
| `shared/notes/moonshots/` | Leisure + Workers | Speculative hypotheses from overnight thinking |
| `shared/notes/synthesis/` | Synthesizer | Research agenda, combination matrix, exhausted areas |
| `shared/discoveries.md` | Workers | Cross-agent best results (flat, for quick reading) |
| `shared/anomalies.md` | Workers + Director | Unexpected results (flat) |
| `shared/midnight-reflections.md` | Leisure | Nightly reflection narrative |
| `~/.litmus/repo/` (git) | Workers | All experiment commits on per-agent branches |

---

## Research Templates

| Template | Focus |
|---|---|
| `architecture` | Depth, aspect ratio, head dim, WINDOW_PATTERN (SLSL/LSLS/etc) |
| `optimizer` | Per-matrix learning rates, schedule shape, Muon vs AdamW |
| `regularization` | Softcap, gradient clipping, weight decay, residual scaling |
| `general` | Open-ended — combinatorial, tries anything, prefers unexplored combinations |

---

## Requirements

- **OS**: Linux or macOS (Windows not supported)
- **GPU**: NVIDIA with CUDA
- **`uv`**: `curl -LsSf https://astral.sh/uv/install.sh | sh`
- **`git`**: for experiment version control and worktrees
- **`python3`**: for JSON attempt records and leaderboard scripts

See [INSTALL.md](INSTALL.md) for full details.

---

## License

MIT-0 — do whatever you want with it.


path: sync.sh
content: #!/usr/bin/env bash
# Sync litmus-clawhub from the main litmus repo.
# Run from anywhere: bash path/to/litmus-clawhub/sync.sh

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SRC="$(cd "$SCRIPT_DIR/../litmus" && pwd)"
DST="$SCRIPT_DIR"

if [ ! -d "$SRC" ]; then
  echo "Error: litmus source not found at $SRC"
  exit 1
fi

cp "$SRC/SKILL.md"    "$DST/"
cp "$SRC/package.json" "$DST/"
cp "$SRC/INSTALL.md"  "$DST/"
cp "$SRC/README.md"   "$DST/"

cp "$SRC/configs/default.json" "$DST/configs/"

cp "$SRC/references/onboarding.md" \
   "$SRC/references/program.md" \
   "$SRC/references/director.md" \
   "$SRC/references/leisure.md" \
   "$SRC/references/dawn.md" \
   "$SRC/references/digest.md" \
   "$SRC/references/watchdog.md" \
   "$SRC/references/clawrxiv.md" \
   "$DST/references/"

cp "$SRC/references/templates/architecture.md" \
   "$SRC/references/templates/general.md" \
   "$SRC/references/templates/optimizer.md" \
   "$SRC/references/templates/regularization.md" \
   "$DST/references/templates/"

cp "$SRC/scripts/setup.sh" \
   "$SRC/scripts/prepare-agents.sh" \
   "$SRC/scripts/setup-cron.sh" \
   "$SRC/scripts/status.sh" \
   "$SRC/scripts/results.sh" \
   "$DST/scripts/"

echo "Synced from $SRC"


path: scripts/prepare-agents.sh
content: #!/usr/bin/env bash
# prepare-agents.sh — create git worktrees for research subagents from the shared lab repo.
# Each agent gets its own branch in ~/.litmus/repo/ and a worktree at ~/.litmus/agents/<id>/.
# Agents can browse all branches, cherry-pick changes, and build on each other's commits.
#
# Usage:
#   bash prepare-agents.sh --agents 4 --templates architecture,optimizer,general,general
#   bash prepare-agents.sh --agents 2 --templates general,general --time-budget 300
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_DIR="$(dirname "$SCRIPT_DIR")"
BASE_DIR="${LITMUS_BASE:-$HOME/.litmus}"
LAB_REPO="$BASE_DIR/repo"
SHARED_DIR="$BASE_DIR/shared"
TEMPLATES_DIR="$REPO_DIR/references/templates"

NUM_AGENTS=2
TEMPLATES_CSV="general,general"
TIME_BUDGET=300

while [[ $# -gt 0 ]]; do
  case "$1" in
    --agents)      NUM_AGENTS="$2";    shift 2 ;;
    --templates)   TEMPLATES_CSV="$2"; shift 2 ;;
    --time-budget) TIME_BUDGET="$2";   shift 2 ;;
    *) echo "Unknown arg: $1"; exit 1 ;;
  esac
done

IFS=',' read -ra TEMPLATES <<< "$TEMPLATES_CSV"

# Detect GPU count
if command -v nvidia-smi &>/dev/null; then
  GPU_COUNT=$(nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null | wc -l | tr -d ' ')
else
  GPU_COUNT=1
fi

# Verify the lab repo exists (setup.sh must run first)
if [ ! -d "$LAB_REPO/.git" ]; then
  echo "Error: lab repo not found at $LAB_REPO"
  echo "Run setup.sh first."
  exit 1
fi

echo "=== Preparing $NUM_AGENTS research agent worktree(s) ==="
echo "Templates:   ${TEMPLATES[*]}"
echo "Time budget: ${TIME_BUDGET}s per experiment"
echo "GPUs:        $GPU_COUNT"
echo "Lab repo:    $LAB_REPO"
echo ""

DATE_TAG=$(date +%Y%m%d)
AGENT_IDS=()

for i in $(seq 1 "$NUM_AGENTS"); do
  TEMPLATE_IDX=$(( (i - 1) % ${#TEMPLATES[@]} ))
  TEMPLATE="${TEMPLATES[$TEMPLATE_IDX]}"

  case "$TEMPLATE" in
    architecture)   SHORT="arch" ;;
    optimizer)      SHORT="opt"  ;;
    regularization) SHORT="reg"  ;;
    tokenization)   SHORT="tok"  ;;
    general)        SHORT="gen"  ;;
    *)              SHORT="$TEMPLATE" ;;
  esac

  AGENT_ID="${SHORT}-${i}"
  BRANCH="litmus/agent-${AGENT_ID}-${DATE_TAG}"
  AGENT_DIR="$BASE_DIR/agents/$AGENT_ID"
  GPU_ID=$(( (i - 1) % GPU_COUNT ))

  echo "Agent $i: id=$AGENT_ID  branch=$BRANCH  gpu=$GPU_ID"

  # Remove any existing worktree for this agent
  if [ -d "$AGENT_DIR" ]; then
    git -C "$LAB_REPO" worktree remove "$AGENT_DIR" --force 2>/dev/null || rm -rf "$AGENT_DIR"
    git -C "$LAB_REPO" branch -D "$BRANCH" 2>/dev/null || true
  fi

  # Create a fresh worktree on a new branch from main
  git -C "$LAB_REPO" worktree add "$AGENT_DIR" -b "$BRANCH" main -q
  git -C "$AGENT_DIR" config user.email "agent-${AGENT_ID}@litmus"
  git -C "$AGENT_DIR" config user.name "Agent ${AGENT_ID}"

  # Build program.md: inject config header + focus template + base program
  FOCUS_CONTENT=""
  if [ -f "$TEMPLATES_DIR/$TEMPLATE.md" ]; then
    FOCUS_CONTENT="$(cat "$TEMPLATES_DIR/$TEMPLATE.md")"
  fi

  CONFIG_HEADER="<!-- AGENT_ID: $i -->
<!-- TOTAL_AGENTS: $NUM_AGENTS -->
<!-- RESEARCH_FOCUS: $TEMPLATE -->
<!-- SHARED_DIR: $SHARED_DIR -->
<!-- LAB_REPO: $LAB_REPO -->
<!-- BRANCH: $BRANCH -->
<!-- GPU_ID: $GPU_ID -->
<!-- TIME_BUDGET: $TIME_BUDGET -->"

  PROGRAM_BASE="$(cat "$REPO_DIR/references/program.md")"

  # Substitute config block placeholder
  PROGRAM_BASE="${PROGRAM_BASE//<!-- AGENT_ID: 1 -->/$CONFIG_HEADER}"
  PROGRAM_BASE="${PROGRAM_BASE//<!-- TOTAL_AGENTS: 4 -->}"
  PROGRAM_BASE="${PROGRAM_BASE//<!-- RESEARCH_FOCUS: general -->}"
  PROGRAM_BASE="${PROGRAM_BASE//<!-- SHARED_DIR: ~/.litmus\/shared -->}"
  PROGRAM_BASE="${PROGRAM_BASE//<!-- LAB_REPO: ~/.litmus\/repo -->}"
  PROGRAM_BASE="${PROGRAM_BASE//<!-- BRANCH: litmus\/agent-gen-1-20260101 -->}"
  PROGRAM_BASE="${PROGRAM_BASE//<!-- GPU_ID: 0 -->}"
  PROGRAM_BASE="${PROGRAM_BASE//<!-- TIME_BUDGET: 300 -->}"

  # Replace literal placeholder strings in program body
  PROGRAM_BASE="${PROGRAM_BASE//AGENT_ID/$i}"
  PROGRAM_BASE="${PROGRAM_BASE//GPU_ID/$GPU_ID}"
  PROGRAM_BASE="${PROGRAM_BASE//SHARED_DIR/$SHARED_DIR}"
  PROGRAM_BASE="${PROGRAM_BASE//LAB_REPO/$LAB_REPO}"
  PROGRAM_BASE="${PROGRAM_BASE//MY_BRANCH/$BRANCH}"
  PROGRAM_BASE="${PROGRAM_BASE//TIME_BUDGET/$TIME_BUDGET}"

  # Inject focus template
  PROGRAM_FINAL="${PROGRAM_BASE//<!-- FOCUS_INSTRUCTIONS injected here by prepare-agents.sh -->/$FOCUS_CONTENT}"

  echo "$PROGRAM_FINAL" > "$AGENT_DIR/program.md"

  # results.tsv is local to each worktree — not committed, not shared
  # (shared/attempts/ JSON is the cross-agent source of truth)
  echo -e "commit\tval_bpb\tpeak_vram_gb\tstatus\tdescription" > "$AGENT_DIR/results.tsv"
  echo "results.tsv" >> "$AGENT_DIR/.git/info/exclude" 2>/dev/null || true
  echo "run.log"     >> "$AGENT_DIR/.git/info/exclude" 2>/dev/null || true

  AGENT_IDS+=("$AGENT_ID")
  echo "  Worktree ready: $AGENT_DIR  (branch: $BRANCH)"
done

# Record worker count for watchdog
echo "$NUM_AGENTS" > "$SHARED_DIR/worker-count.txt"

echo ""
echo "=== Worktrees created. Git structure ==="
git -C "$LAB_REPO" worktree list
echo ""
echo "All experiment history visible via:"
echo "  git -C $LAB_REPO log --all --oneline --graph"
echo ""

echo "=== Spawn commands (OpenClaw native subagents) ==="
echo ""
for AGENT_ID in "${AGENT_IDS[@]}"; do
  AGENT_DIR="$BASE_DIR/agents/$AGENT_ID"
  echo "sessions_spawn  task:\"Read program.md in your current directory and run the research loop forever.\"  runtime:\"subagent\"  mode:\"session\"  agentId:\"litmus-worker-$AGENT_ID\"  cwd:\"$AGENT_DIR\""
done
echo ""
echo "Then: sessions_yield  message:\"Research agents started. I'll notify you on discoveries.\""


path: scripts/results.sh
content: #!/usr/bin/env bash
# results.sh — cross-agent experiment leaderboard from shared/attempts/ JSON records.
set -euo pipefail

BASE_DIR="${LITMUS_BASE:-$HOME/.litmus}"
SHARED_DIR="$BASE_DIR/shared"
TOP_N=10
FILTER_AGENT=""
FILTER_FOCUS=""

while [[ $# -gt 0 ]]; do
  case "$1" in
    --top)    TOP_N="$2";       shift 2 ;;
    --agent)  FILTER_AGENT="$2"; shift 2 ;;
    --focus)  FILTER_FOCUS="$2"; shift 2 ;;
    *) echo "Unknown arg: $1"; exit 1 ;;
  esac
done

ATTEMPTS_DIR="$SHARED_DIR/attempts"

if [ ! -d "$ATTEMPTS_DIR" ] || [ -z "$(ls -A "$ATTEMPTS_DIR" 2>/dev/null)" ]; then
  echo "No attempt records yet. Agents write to $ATTEMPTS_DIR after each experiment."
  echo "(Tip: run setup.sh and prepare-agents.sh first)"
  exit 0
fi

python3 - "$ATTEMPTS_DIR" "$TOP_N" "$FILTER_AGENT" "$FILTER_FOCUS" << 'EOF'
import json, glob, sys, os
from datetime import datetime, timezone, timedelta

attempts_dir, top_n_str, filter_agent, filter_focus = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]
top_n = int(top_n_str)

attempts = []
for f in glob.glob(os.path.join(attempts_dir, '*.json')):
    try:
        d = json.load(open(f))
        if d.get('val_bpb', 0) > 0:
            if filter_agent and d.get('agent_id') != filter_agent:
                continue
            if filter_focus and d.get('focus_area') != filter_focus:
                continue
            attempts.append(d)
    except:
        pass

if not attempts:
    print("No completed experiments match the filter.")
    sys.exit(0)

# Sort by val_bpb ascending (lower is better)
attempts.sort(key=lambda x: x['val_bpb'])

print(f"=== Litmus Leaderboard (top {top_n}) ===")
print(f"Time: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
if filter_agent: print(f"Filter: agent={filter_agent}")
if filter_focus: print(f"Filter: focus={filter_focus}")
print()

print(f"{'RANK':<4}  {'AGENT':<12}  {'COMMIT':<8}  {'val_bpb':<10}  {'STATUS':<14}  {'MFU%':<5}  TITLE")
print(f"{'----':<4}  {'------------':<12}  {'--------':<8}  {'----------':<10}  {'--------------':<14}  {'-----':<5}  -----")

for i, a in enumerate(attempts[:top_n], 1):
    mfu = f"{a.get('mfu_percent', 0):.1f}" if a.get('mfu_percent') else "-"
    title = a.get('title', '')[:45]
    print(f"{i:<4}  {a['agent_id']:<12}  {a['commit']:<8}  {a['val_bpb']:<10.6f}  {a['status']:<14}  {mfu:<5}  {title}")

print()

# Statistics
now = datetime.now(timezone.utc)
last_24h = now - timedelta(hours=24)
recent = [a for a in attempts
          if datetime.fromisoformat(a['timestamp'].replace('Z','+00:00')) > last_24h]

total = len(attempts)
improved = sum(1 for a in attempts if a['status'] == 'improved')
agents = set(a['agent_id'] for a in attempts)

print(f"Total experiments: {total}  |  Improvements: {improved} ({improved/total*100:.1f}%)  |  Agents: {len(agents)}")
if recent:
    r_improved = sum(1 for a in recent if a['status'] == 'improved')
    print(f"Last 24h: {len(recent)} experiments, {r_improved} improvements ({r_improved/len(recent)*100:.1f}% rate)")

print()

# Per-agent summary
print("=== Per-Agent Summary ===")
from collections import defaultdict
by_agent = defaultdict(list)
for a in attempts:
    by_agent[a['agent_id']].append(a)

for agent_id in sorted(by_agent):
    exps = by_agent[agent_id]
    best = min(e['val_bpb'] for e in exps)
    imp = sum(1 for e in exps if e['status'] == 'improved')
    since_imp = 0
    for e in sorted(exps, key=lambda x: x['timestamp'], reverse=True):
        if e['status'] == 'improved':
            break
        since_imp += 1
    focus = exps[-1].get('focus_area', '?') if exps else '?'
    stagnant = ' [STAGNANT]' if since_imp >= 12 else ''
    print(f"  {agent_id:<12}  best={best:.6f}  {len(exps)} exps  {imp} wins  {since_imp} since last win  focus={focus}{stagnant}")

print()

# Skills library summary
skills_dir = os.path.join(os.path.dirname(attempts_dir), 'skills')
if os.path.exists(skills_dir):
    skill_files = [f for f in os.listdir(skills_dir) if f.endswith('.md') and f != 'INDEX.md']
    print(f"Skills library: {len(skill_files)} validated technique(s)")
    for sf in sorted(skill_files):
        print(f"  - {sf}")
EOF


path: scripts/setup-cron.sh
content: #!/usr/bin/env bash
# setup-cron.sh — register the Litmus cron jobs via the OpenClaw cron tool.
#
# All times are configurable. Defaults match the standard circadian schedule.
# The onboarding agent reads these from config.json and passes them here.
#
# Usage:
#   bash scripts/setup-cron.sh --timezone "America/New_York"
#   bash scripts/setup-cron.sh --timezone "Europe/London" \
#     --leisure-start 01:00 --synthesizer-time 02:30 --dawn-time 04:00 \
#     --digest-time 07:00 --director-hours 1 --watchdog-minutes 20
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_DIR="$(dirname "$SCRIPT_DIR")"
BASE_DIR="${LITMUS_BASE:-$HOME/.litmus}"
SHARED_DIR="$BASE_DIR/shared"
REFERENCES_DIR="$REPO_DIR/references"

# --- Defaults ---
TZ="UTC"
LEISURE_START="03:00"
SYNTHESIZER_TIME="04:00"
DAWN_TIME="06:00"
DIGEST_TIME="08:00"
DIRECTOR_HOURS=2
WATCHDOG_MINUTES=30

while [[ $# -gt 0 ]]; do
  case "$1" in
    --timezone)         TZ="$2";               shift 2 ;;
    --leisure-start)    LEISURE_START="$2";    shift 2 ;;
    --synthesizer-time) SYNTHESIZER_TIME="$2"; shift 2 ;;
    --dawn-time)        DAWN_TIME="$2";        shift 2 ;;
    --digest-time)      DIGEST_TIME="$2";      shift 2 ;;
    --director-hours)   DIRECTOR_HOURS="$2";   shift 2 ;;
    --watchdog-minutes) WATCHDOG_MINUTES="$2"; shift 2 ;;
    *) shift ;;
  esac
done

# --- Helper: parse HH:MM → hour and minute integers ---
parse_hhmm() {
  local t="$1"
  local h m
  h=$(echo "$t" | cut -d: -f1 | sed 's/^0*//' )
  m=$(echo "$t" | cut -d: -f2 | sed 's/^0*//' )
  H="${h:-0}"
  M="${m:-0}"
}

# Parse each time
parse_hhmm "$LEISURE_START";    L_H=$H; L_M=$M
parse_hhmm "$SYNTHESIZER_TIME"; S_H=$H; S_M=$M
parse_hhmm "$DAWN_TIME";        D_H=$H; D_M=$M
parse_hhmm "$DIGEST_TIME";      DG_H=$H; DG_M=$M

# --- Watchdog interval: convert minutes → milliseconds ---
WATCHDOG_MS=$(( WATCHDOG_MINUTES * 60000 ))

# --- Director research-hours range: dawn → leisure_start (wraps midnight) ---
# e.g. dawn=06:00, leisure=03:00 → "0 */2 6-2 * * *"
# We express the hour range as "DAWN_H through LEISURE_H (exclusive)"
# If LEISURE_H > DAWN_H (leisure starts after dawn, unusual) we warn.
if [ "$D_H" -lt "$L_H" ] || { [ "$D_H" -eq "$L_H" ] && [ "$D_M" -lt "$L_M" ]; }; then
  # Normal case: dawn before leisure (e.g. dawn 06:00, leisure 03:00 next night)
  # Director runs during hours D_H through L_H
  DIRECTOR_RANGE="${D_H}-${L_H}"
else
  # Unusual: leisure starts before dawn — director runs all day except leisure window
  DIRECTOR_RANGE="*"
fi

# Build cron expressions
LEISURE_CRON="0 $L_M $L_H * * *"
SYNTH_CRON="0 $S_M $S_H * * *"
DAWN_CRON="0 $D_M $D_H * * *"
DIGEST_CRON="0 $DG_M $DG_H * * *"
DIRECTOR_CRON="0 */${DIRECTOR_HOURS} ${DIRECTOR_RANGE} * * *"

echo "=== Litmus Cron Setup ==="
echo "Timezone:       $TZ"
echo "Leisure start:  $LEISURE_START  (workers enter creative mode)"
echo "Synthesizer:    $SYNTHESIZER_TIME  (distills knowledge into skills library)"
echo "Dawn:           $DAWN_TIME  (workers wake, experiment queue loaded)"
echo "Digest:         $DIGEST_TIME  (morning briefing delivered to you)"
echo "Director:       every ${DIRECTOR_HOURS}h during ${DIRECTOR_RANGE}:xx research hours"
echo "Watchdog:       every ${WATCHDOG_MINUTES} min"
echo ""
echo "Ask your OpenClaw agent to run these 6 cron tool calls:"
echo ""

cat << EOF
============================================================
CRON 1: Director (every ${DIRECTOR_HOURS}h during research hours)
Expr: "${DIRECTOR_CRON}"
============================================================
cron action:"add" job:{
  "name": "litmus-director",
  "enabled": true,
  "schedule": { "kind": "cron", "expr": "${DIRECTOR_CRON}", "tz": "$TZ" },
  "sessionTarget": "main",
  "wakeMode": "now",
  "payload": {
    "kind": "agentTurn",
    "message": "Run the Litmus Director cycle. Full instructions in $REFERENCES_DIR/director.md — read that file first. Read attempts from $SHARED_DIR/attempts/ for all metrics."
  },
  "delivery": { "mode": "none" }
}

============================================================
CRON 2: Leisure Mode ($LEISURE_START — enters creative thinking mode)
Expr: "${LEISURE_CRON}"
============================================================
cron action:"add" job:{
  "name": "litmus-leisure",
  "enabled": true,
  "schedule": { "kind": "cron", "expr": "${LEISURE_CRON}", "tz": "$TZ" },
  "sessionTarget": "main",
  "wakeMode": "now",
  "payload": {
    "kind": "agentTurn",
    "message": "Run the Litmus leisure session. Full instructions in $REFERENCES_DIR/leisure.md — read that file first. Write structured YAML-frontmatter notes to $SHARED_DIR/notes/. The Synthesizer reads these at $SYNTHESIZER_TIME."
  },
  "delivery": { "mode": "none" }
}

============================================================
CRON 3: Synthesizer ($SYNTHESIZER_TIME — distills knowledge into skills library)
Expr: "${SYNTH_CRON}"
============================================================
cron action:"add" job:{
  "name": "litmus-synthesizer",
  "enabled": true,
  "schedule": { "kind": "cron", "expr": "${SYNTH_CRON}", "tz": "$TZ" },
  "sessionTarget": "main",
  "wakeMode": "now",
  "payload": {
    "kind": "agentTurn",
    "message": "Run the Litmus Synthesizer. Full instructions in $REFERENCES_DIR/synthesizer.md — read that file first. Read all attempts from $SHARED_DIR/attempts/, all notes from $SHARED_DIR/notes/, extract validated skills to $SHARED_DIR/skills/, write research synthesis."
  },
  "delivery": { "mode": "none" }
}

============================================================
CRON 4: Dawn ($DAWN_TIME — wakes workers, queues experiments)
Expr: "${DAWN_CRON}"
============================================================
cron action:"add" job:{
  "name": "litmus-dawn",
  "enabled": true,
  "schedule": { "kind": "cron", "expr": "${DAWN_CRON}", "tz": "$TZ" },
  "sessionTarget": "main",
  "wakeMode": "now",
  "payload": {
    "kind": "agentTurn",
    "message": "Run the Litmus dawn session. Full instructions in $REFERENCES_DIR/dawn.md — read that file first. Wake workers from leisure mode and queue overnight ideas from the synthesizer output."
  },
  "delivery": { "mode": "none" }
}

============================================================
CRON 5: Watchdog (every ${WATCHDOG_MINUTES} minutes)
============================================================
cron action:"add" job:{
  "name": "litmus-watchdog",
  "enabled": true,
  "schedule": { "kind": "every", "everyMs": ${WATCHDOG_MS} },
  "sessionTarget": "main",
  "wakeMode": "now",
  "payload": {
    "kind": "agentTurn",
    "message": "Run the Litmus watchdog check. Full instructions in $REFERENCES_DIR/watchdog.md — read that file first. Read from $SHARED_DIR/attempts/ for metrics. Keep it under 3 minutes total."
  },
  "delivery": { "mode": "none" }
}

============================================================
CRON 6: Morning Digest ($DIGEST_TIME — delivered to you)
Expr: "${DIGEST_CRON}"
============================================================
cron action:"add" job:{
  "name": "litmus-digest",
  "enabled": true,
  "schedule": { "kind": "cron", "expr": "${DIGEST_CRON}", "tz": "$TZ" },
  "sessionTarget": "main",
  "wakeMode": "now",
  "payload": {
    "kind": "agentTurn",
    "message": "Write and deliver the Litmus morning research digest. Full instructions in $REFERENCES_DIR/digest.md — read that file first. Include the git tree snapshot from $BASE_DIR/repo. Write for a researcher, not a sysadmin."
  },
  "delivery": {
    "mode": "announce",
    "channel": "last",
    "bestEffort": true
  }
}
EOF

echo ""
echo "=== Schedule (${TZ}) ==="
echo "  $LEISURE_START  Leisure  — workers enter creative mode (papers, moonshots)"
echo "  $SYNTHESIZER_TIME  Synthesizer — distills notes into skills library"
echo "  $DAWN_TIME  Dawn     — workers wake, queue experiments"
echo "  $DIGEST_TIME  Digest   — morning research narrative delivered to you"
echo "  Every ${DIRECTOR_HOURS}h  Director  — steers workers, Compass Reset on stagnation"
echo "  Every ${WATCHDOG_MINUTES}m   Watchdog  — liveness check, escape mode"


path: scripts/status.sh
content: #!/usr/bin/env bash
# status.sh — show experiment counts, best val_bpb per agent, and lab git tree.
# Agent liveness is tracked by OpenClaw (use `subagents action:list`).
set -euo pipefail

BASE_DIR="${LITMUS_BASE:-$HOME/.litmus}"
SHARED_DIR="$BASE_DIR/shared"
LAB_REPO="$BASE_DIR/repo"

if [ ! -d "$BASE_DIR/agents" ]; then
  echo "No agent workspaces found. Run setup.sh and prepare-agents.sh first."
  exit 1
fi

echo "=== Litmus Status ==="
echo "Base: $BASE_DIR"
echo "Time: $(date)"
echo ""

# --- Per-agent status from attempt records ---
echo "--- Agent Status (from shared/attempts/) ---"

python3 - "$SHARED_DIR" << 'EOF'
import json, glob, sys, os
from datetime import datetime, timezone, timedelta
from collections import defaultdict

shared_dir = sys.argv[1]
attempts_dir = os.path.join(shared_dir, 'attempts')

attempts = []
for f in glob.glob(os.path.join(attempts_dir, '*.json')):
    try:
        d = json.load(open(f))
        if d.get('val_bpb', 0) > 0:
            attempts.append(d)
    except: pass

if not attempts:
    print("  No attempt records yet.")
    sys.exit(0)

by_agent = defaultdict(list)
for a in attempts:
    by_agent[a['agent_id']].append(a)

now = datetime.now(timezone.utc)
print(f"  {'AGENT':<14} {'EXPS':<6} {'BEST':<10} {'WINS':<5} {'SINCE_WIN':<10} {'LAST_EXP':<8} {'STATUS'}")
print(f"  {'------':<14} {'----':<6} {'----':<10} {'----':<5} {'---------':<10} {'--------':<8} {'------'}")

for agent_id in sorted(by_agent):
    exps = sorted(by_agent[agent_id], key=lambda x: x['timestamp'])
    best = min(e['val_bpb'] for e in exps)
    wins = sum(1 for e in exps if e['status'] == 'improved')
    since_win = 0
    for e in reversed(exps):
        if e['status'] == 'improved': break
        since_win += 1
    last = exps[-1]
    last_ts = datetime.fromisoformat(last['timestamp'].replace('Z','+00:00'))
    mins_ago = int((now - last_ts).total_seconds() / 60)
    last_str = f"{mins_ago}m ago"
    stagnant = "STAGNANT" if since_win >= 12 else "active"
    slow = " SLOW" if mins_ago > 25 else ""
    print(f"  {agent_id:<14} {len(exps):<6} {best:<10.6f} {wins:<5} {since_win:<10} {last_str:<8} {stagnant}{slow}")

print()
global_best = min(a['val_bpb'] for a in attempts)
global_best_a = min(attempts, key=lambda x: x['val_bpb'])
total = len(attempts)
wins = sum(1 for a in attempts if a['status'] == 'improved')
print(f"  Global best: {global_best:.6f} by {global_best_a['agent_id']} (commit {global_best_a['commit']})")
print(f"  Total: {total} experiments, {wins} improvements ({wins/total*100:.1f}% success rate)")
EOF

echo ""

# --- Mode ---
MODE=$(cat "$SHARED_DIR/mode.txt" 2>/dev/null | head -1 || echo "unknown")
COUNT=$(cat "$SHARED_DIR/global-experiment-count.txt" 2>/dev/null || echo "?")
LAST_SYNTH=$(cat "$SHARED_DIR/last-synthesis-count.txt" 2>/dev/null || echo "?")
echo "--- System State ---"
echo "  Mode: $MODE"
echo "  Global experiment count: $COUNT"
echo "  Last synthesis at experiment: $LAST_SYNTH"
echo "  Skills in library: $(ls "$SHARED_DIR/skills/"*.md 2>/dev/null | grep -v INDEX | wc -l | tr -d ' ')"
echo "  Open anomalies: $(ls "$SHARED_DIR/notes/anomalies/"*.md 2>/dev/null | wc -l | tr -d ' ')"
echo ""

# --- Git lab tree ---
if [ -d "$LAB_REPO/.git" ]; then
  echo "--- Lab Git Tree (all agent branches) ---"
  git -C "$LAB_REPO" log --all --oneline --graph | head -30
  echo ""
  echo "  All worktrees:"
  git -C "$LAB_REPO" worktree list
fi

echo ""
echo "Tips:"
echo "  bash results.sh --top 20         — full leaderboard"
echo "  bash results.sh --agent arch-1   — single agent history"
echo "  subagents action:list             — check which agents are running (in OpenClaw)"


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)