Skill: peterpcw/autoagent (version 0.8.9)

--- name: autoagent description: "Automatically improve agent guidance through iterative testing and scoring. Use when you want to optimize prompts, AGENTS.md entries, or skill definitions using a Karpathy-style training loop with OpenClaw cron." --- # Autoagent Skill Optimize any agent guidance through automated testing and iterative improvement. ## Quick Start ``` /autoagent ``` ## What It Does 1. **Setup Phase** - Asks where your guidance lives and what it should do 2. **Creates Sandbox** - Copies guidance to test folder with fixtures 3. **Runs Optimization Loop** - Every 5 minutes via cron: - Analyzes current guidance - Proposes improvement - Tests with subagent - Scores result - Keeps or discards change 4. **Logs Everything** - Check scores.md for history --- ## Setup Phase (Every Invocation Starts Fresh) Every invocation of `/autoagent` starts fresh with interactive setup questions. ### Step 1: Ask Sandbox Location Ask the user: > Where should I create the sandbox folder? Default: `../../autoagent-sandbox/` (resolves to `/clawd/autoagent-sandbox/`) You can respond with: - **Empty/default**: Press enter to use `../../autoagent-sandbox/` - **Just a name**: "news" creates `../../autoagent-news/` → `/clawd/autoagent-news/` - **Relative path**: "agentDev/optimize" creates `../../agentDev/optimize/` → `/clawd/agentDev/optimize/` - **Absolute path**: `/some/other/path/optimize/` → exact path Wait for their response (or empty for default). ### Step 2: Discuss Success Criteria Ask the user: > Let's define how we'll measure success. What does a "good" result look like for this task? Follow up one at a time based on their response: - What specific outputs are expected? - What format should they be in? - What's the minimum viable quality? - Any edge cases to consider? Once you have enough information, propose a draft scoring.md: ```markdown ## Proposed Scoring Criteria **Score Components:** - [Component 1]: [X] points - [description] - [Component 2]: [Y] points - [description] - ... **Total:** 100 points **[Any additional notes]** ``` Wait for user approval or modifications. ### Step 3: Ask About External Scripts/Tools Ask the user: > Does the guidance rely on any scripts, tools, or external software? > - If yes: Note each script/tool path and what functionality it provides > - The autoagent should analyze these to recommend improvements ### Step 4: Ask Cron Schedule Ask the user: > Run optimization every 5 minutes (default), or different interval? ### Step 5: Create Sandbox After all questions answered, create the sandbox folder at the user-specified path: ``` sandbox/ ├── guidance-under-test.md # Copy of original guidance ├── current-guidance.md # Same as guidance-under-test initially ├── fixtures/ │ └── test-cases.json # {"cases": [{"input": "...", "expected": "..."}]} ├── scoring.md # Scoring criteria document (user-approved) ├── scores.md # Score history table └── scripts/ # (optional) Copy of referenced scripts/tools ``` ### Step 6: Set Up Cron Use OpenClaw cron syntax to schedule the iteration agent: - Default: every 5 minutes (`*/5 * * * *`) - Command: invoke the iteration prompt with the sandbox path ### Step 7: Confirm Start Return confirmation message showing the resolved path: > "Optimization started at `/clawd/autoagent-news/`. I'll check back every 5 minutes. Monitor progress in `scores.md`." --- ## Iteration Phase (Runs Every Cron Interval) Each time the cron triggers, do the following: ### Step 1: Analyze Current State Read from the sandbox: - `current-guidance.md` - The guidance being optimized - `scores.md` - History of scores and changes - `scoring.md` - How to measure success - `fixtures/test-cases.json` - Test inputs (MUST read this to understand what the guidance is being tested against) Review score history (last 10 runs or all available runs if fewer than 10 exist), identify patterns, note current score. When fewer than 10 runs exist, treat all available scores as the set for plateau detection. **Important:** Load the test cases from `fixtures/test-cases.json` to understand what specific outputs/ behaviors are expected. The edit should address gaps revealed by test case failures or missing criteria. ### Step 1b: Analyze External Scripts/Tools (If Applicable) If the guidance references any scripts, tools, or external software: 1. **Locate each script/tool** - Find the actual script files or binary locations 2. **Analyze the functionality** - Read the code or documentation to understand what it does 3. **Identify improvement opportunities:** - **For open-source scripts:** Can the script be modified to improve functionality? - **For closed-source/compiled tools:** Can wrapper behavior be improved? Can you recommend API/interface changes? 4. **Note findings in the iteration** - If script improvements could help test scores, document them **Example outputs:** - "Script X does Y but could do Z - recommend modification to add feature W" - "Tool A is closed-source, recommend changing prompt to work around limitation B" - "Script C has bug in function D - fix would improve test outcomes" ### Step 2: Propose Edit Generate ONE specific edit to the guidance that might improve the score. **Analyze Score History First:** - Read scores.md to find the last 10 runs - Identify patterns: Which scoring criteria are consistently low? - Look for repeated failures - if the same criterion failed multiple times, that's your target - Check what changes were tried before (avoid repeating failed approaches) **Edit Selection Strategy (Priority Order):** 1. If scores exist: Target the lowest-scoring criteria from scoring.md 2. If all scores high (90+): Add missing detail to any criteria marked as partial 3. If only 1-2 runs: Assume baseline covered basics, add missing methodology 4. Prioritize edits that affect multiple scoring criteria at once **The edit should:** - Be specific and actionable (not vague like "improve clarity") - Address a weakness identified in scoring (target the lowest-scoring criteria) - Not be identical to recently tried changes (check scores.md for recent descriptions) - Include the exact text to add/remove/replace Format: ```markdown ## Proposed Edit **Rationale:** Why this change might help **Change:** ``` [Show exact diff or new text] ``` ``` ### Step 3: Apply Edit Write the edited guidance to `current-guidance.md` ### Step 4: Run Test Use a subagent to run the task with the new guidance: - Give the subagent `current-guidance.md` - Provide test inputs from `fixtures/test-cases.json` - Capture the output - **Subagent invocation:** Use `sessions_spawn` with `task` containing the full contents of current-guidance.md, include the test cases JSON inline in the task prompt, set `timeoutSeconds` to 120, and request the subagent to return the raw output (not just pass/fail) ### Step 5: Score Result Evaluate the output against `scoring.md` criteria. Generate a score 0-100. ### Step 6: Log Decision Append to `scores.md`: ```markdown | N | Description of change | SCORE | keep/discard | ``` Where N is the run number (increment from last). ### Step 7: Update Guidance - If score improved: Keep the edit (`current-guidance.md` is already updated) - If score declined: Revert `current-guidance.md` to previous version ### Step 8: Check Plateau If last 10 scores are within 5 points of each other: - Log "Plateau detected - pausing" - Notify user - Stop the cron (or pause and await user override) --- ## Files Created in Sandbox | File | Description | |------|-------------| | `guidance-under-test.md` | Original copy (read-only reference) | | `current-guidance.md` | Working version (edited each iteration) | | `fixtures/test-cases.json` | Input → expected output pairs | | `scoring.md` | Scoring methodology | | `scores.md` | Score history log | | `scripts/` | (optional) Copies of referenced scripts/tools for analysis | --- ## Usage 1. Invoke: `/autoagent` 2. Answer setup questions 3. Monitor `scores.md` for progress 4. Copy improvements to original when satisfied 5. Stop cron when done ## Stopping - User can stop cron anytime - Auto-stops if score plateaus for 10 runs - Check `scores.md` for progress --- ## Key Principles - **Non-destructive**: Original guidance stays in `guidance-under-test.md` - **Learn from history**: Don't repeat failed approaches - **Be specific**: Vague changes won't score well - **Human in the loop**: User defines success criteria, can override plateau detection

path: iteration-prompt.md
content: # Autoagent Iteration Prompt

You are running an optimization iteration for the autoagent skill.

## Context

Read these files from the sandbox:
- `current-guidance.md` - The guidance being optimized
- `scores.md` - History of scores and changes
- `scoring.md` - How to measure success
- `fixtures/test-cases.json` - Test inputs

## Your Task

### Step 1: Analyze Current State

- Read current guidance
- Review score history (last 10 runs)
- Identify patterns in what's been tried
- Note current score

### Step 2: Propose Edit

Generate ONE specific edit to the guidance that might improve the score.
The edit should:
- Be specific and actionable
- Address a weakness identified in scoring
- Not be identical to recently tried changes

Format:
```markdown
## Proposed Edit

**Rationale:** Why this change might help

**Change:**
```
[Show exact diff or new text]
```
```

### Step 3: Apply Edit

Write the edited guidance to current-guidance.md

### Step 4: Run Test

Use a subagent to run the task with the new guidance:
- Give the subagent current-guidance.md
- Provide test inputs from fixtures/test-cases.json
- Capture the output

### Step 5: Score Result

Evaluate the output against scoring.md criteria.
Generate a score 0-100.

### Step 6: Log Decision

Append to scores.md:

```markdown
| N   | Description of change | SCORE | keep/discard |
```

Where N is the run number (increment from last).

### Step 7: Update Guidance

- If score improved: Keep the edit (current-guidance.md is already updated)
- If score declined: Revert current-guidance.md to previous version

### Step 8: Check Plateau

If last 10 scores are identical:
- Log "Plateau detected - pausing"
- Notify user
- Stop the cron (or pause and await user override)

## Output Requirements

After completing:
1. Updated scores.md
2. Possibly updated current-guidance.md (if change kept)
3. Brief summary of what happened

## Remember

- Be specific in edits - vague changes won't score well
- Learn from score history - don't repeat failed approaches
- Non-destructive: original guidance stays in guidance-under-test.md


path: README.md
content: # Autoagent

> Automatically improve agent guidance through iterative testing and scoring.

Autoagent is an OpenClaw skill that optimizes any agent guidance (prompts, AGENTS.md entries, skill definitions) using a Karpathy-style training loop with cron-triggered optimization.

## What It Does

1. **Setup** - Asks where your guidance lives and what it should do
2. **Sandbox** - Copies guidance to a test folder with fixtures
3. **Optimize** - Runs every 5 minutes via cron:
   - Analyzes current guidance
   - Proposes improvement
   - Tests with subagent
   - Scores result
   - Keeps or discards change
4. **Logs** - Check scores.md for history

## Quick Start

```
/autoagent
```

## How It Works

### Setup Phase

Every invocation starts fresh with interactive setup:

1. **Sandbox Location** - Where should I create the folder? (absolute path)
2. **Success Criteria** - Discussion to define what "good" looks like, then propose draft scoring.md for approval
3. **External Scripts** - Any scripts/tools the guidance relies on?
4. **Cron Schedule** - How often to run (default: 5 minutes)

Then creates a sandbox at your specified path with:
- `guidance-under-test.md` - Original (read-only)
- `current-guidance.md` - Working version
- `fixtures/test-cases.json` - Test inputs
- `scoring.md` - User-approved success criteria
- `scores.md` - Score history
- `scripts/` - (optional) copies of referenced scripts

### Iteration Loop

Each cron run:
1. Analyze current guidance + score history
2. Propose ONE specific edit
3. Apply edit to current-guidance.md
4. Run subagent with new guidance
5. Score result (0-100)
6. Log decision (keep/discard)
7. Revert if score declined
8. Check for plateau (auto-pause after 10 identical scores)

## Example

```
User: /autoagent

Agent: Where should I create the sandbox folder? Default: `../../autoagent-sandbox/` (resolves to `/clawd/autoagent-sandbox/`)
> news

Agent: Let's define how we'll measure success. What does a "good" result look like for this task?
> It should summarize news into morning briefs with bullet points

Agent: What format should the output be in?
> Markdown with sections for Top Stories, Industry News, and Quick Hits

Agent: [proposes draft scoring.md]

[User approves scoring]

Agent: Does the guidance rely on any scripts, tools, or external software?
> No

Agent: Run optimization every 5 minutes (default), or different interval?
> Default is fine

[Sandbox created at `../../autoagent-news/` → `/clawd/autoagent-news/`, cron started]

Agent: Optimization started at `/clawd/autoagent-news/`. I'll check back every 5 minutes. Monitor progress in `scores.md`.

[5 minutes later...]
- Score: 45 → 52 (kept change)
- Score: 52 → 48 (discarded)
- Score: 48 → 61 (kept change)
...
```

## Files

| File | Description |
|------|-------------|
| `SKILL.md` | Full skill definition |
| `setup-prompt.md` | Setup phase questions |
| `iteration-prompt.md` | Iteration loop instructions |
| `templates/` | Score and fixture templates |
| `examples/` | Test examples |

## Requirements

- OpenClaw with cron support
- LLM for guidance analysis
- File system access

## License

MIT


path: setup-prompt.md
content: # Autoagent Setup Prompt

Every invocation of `/autoagent` starts fresh with these setup questions.

## Step 1: Sandbox Location

Ask the user:

> Where should I create the sandbox folder? Default: `../../autoagent-sandbox/` (resolves to `/clawd/autoagent-sandbox/`)

You can respond with:
- **Empty/default**: Press enter to use `../../autoagent-sandbox/`
- **Just a name**: "news" creates `../../autoagent-news/` → `/clawd/autoagent-news/`
- **Relative path**: "agentDev/optimize" creates `../../agentDev/optimize/` → `/clawd/agentDev/optimize/`
- **Absolute path**: `/some/other/path/optimize/` → exact path

Wait for their response (or empty for default).

## Step 2: Success Criteria Discussion

Ask the user:

> Let's define how we'll measure success. What does a "good" result look like for this task?

Follow up as needed:
- What specific outputs are expected?
- What format should they be in?
- What's the minimum viable quality?
- Any edge cases to consider?

## Step 3: Propose Draft Scoring

Once you've discussed enough to understand the task, propose a draft scoring.md:

```markdown
## Scoring Criteria

**Total:** 100 points

| Component | Points | Description |
|-----------|--------|-------------|
| [Name]    | [X]    | [What it measures] |
| ...       | ...    | ... |

**Notes:**
- [Any additional scoring rules]
```

Wait for user approval or request modifications.

## Step 4: External Scripts/Tools

Ask the user:

> Does the guidance rely on any scripts, tools, or external software?
> - If yes: Note each script/tool path and what functionality it provides
> - The autoagent should analyze these to recommend improvements

## Step 5: Cron Schedule

Ask the user:

> Run optimization every 5 minutes (default), or different interval?

## Step 6: Create Sandbox

After scoring is approved:
1. Create folder at user-specified path
2. Create files:
   - guidance-under-test.md
   - current-guidance.md
   - fixtures/test-cases.json
   - scoring.md (with approved criteria)
   - scores.md
   - scripts/ (optional)
3. Set up cron
4. Return confirmation with sandbox path


path: TESTING.md
content: # Testing the Autoagent Skill

## Manual Test

1. Create test sandbox:
```bash
mkdir -p /tmp/autoagent-test
cd /tmp/autoagent-test
```

2. Copy example:
```bash
cp /home/peterwsl/clawd/skills/autoagent/examples/simple-news-prompt.md ./test-guidance.md
```

3. Create fixtures manually to verify format

4. Run through iteration logic mentally

## Integration Test (OpenClaw)

Once OpenClaw is set up:
1. Invoke /autoagent
2. Answer questions with test paths
3. Verify sandbox creation
4. Verify cron setup
5. Check scores.md after iterations


path: examples/simple-news-prompt.md
content: # Simple News Prompt Example

## Original Guidance

You are a news summarizer. Summarize the following articles into a 3-bullet brief.

## Test Cases

Input: 3 article headlines about tech
Expected: 3 bullet points summarizing each

## Scoring

- All 3 articles covered: 60 points
- Concise bullets (under 20 words each): 20 points
- Relevant details included: 20 points
- Total: 100 points


path: templates/fixtures.json
content: {
  "cases": [
    {
      "id": "case-001",
      "input": "Description or content of test input",
      "expected": "Description of expected output"
    }
  ]
}


path: templates/scores.md
content: # Score History

| Run | Change | Score | Decision |
|-----|--------|-------|----------|
| 0   | Initial baseline | - | - |


path: references/openclaw-integration.md
content: # OpenClaw Integration

## Skill Invocation

Users invoke with: `/autoagent`

## Orchestration

On first invocation:
1. Read setup-prompt.md
2. Ask user questions
3. Create sandbox
4. Set up cron

## Cron Setup

The skill creates a cron job that runs every 5 minutes:
- Uses OpenClaw cron syntax
- Invokes iteration-prompt.md with sandbox path
- Passes context: sandbox directory, current iteration

## Subagent Execution

Each iteration spawns a subagent to:
- Execute the task with guidance
- Return output for scoring
- Isolate test runs from main agent

## File Locations

- Skill: /home/peterwsl/clawd/skills/autoagent/
- Sandbox: {user workspace}/autoagent/
- Cron: OpenClaw managed


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)