Skill: wd041216-bit/research-loop (version 1.0.0)

--- name: research-loop description: >- Claude Code compatibility mirror for the Codex-native 10000 Mentors Research Workflow. Use only when running from Claude Code and the user wants the same source-gated research loop contract. version: 1.0.0 metadata: openclaw: requires: bins: - python3 - git - gh emoji: "\U0001F52C" homepage: https://github.com/wd041216-bit/10000-mentors-research-workflow allowed-tools: - Read - Write - Edit - Bash - Grep - Glob - WebSearch model: inherit argument-hint: "<repo-url> [--max-cycles N] [--target-journal NAME] [--no-push]" --- # Research Loop Compatibility Skill This is the Claude Code compatibility mirror. The primary skill lives at: ```text .codex/skills/research-loop/ ``` Use this mirror only when the runtime is Claude Code. Keep the same safety contract: 1. Treat the current target repo HEAD as primary truth. 2. Treat MemPalace, prior handoffs, and expert memory as secondary context. 3. Use native web search only for short, attributed fresh context when needed. 4. Do not require Ollama API keys or Ollama native web search. 5. Run one bounded micro-step per cycle. 6. Write source changes into the workflow run's `source_changes/` repo-root mirror. 7. Write the executor manifest with `python3 -m autonomous_research_workflow.cli write-executor-manifest`. 8. Do not mark empirical progress unless runnable code or checkable result artifacts were produced. 9. Run the protocol hygiene checks in [protocol hygiene](references/protocol-hygiene.md) before choosing a micro-step. 10. Run the innovation-frontier checks in [innovation frontier](references/innovation-frontier.md) so current assets calibrate the loop without trapping it. 11. Stop only when Submission Advisor reports `score >= 100` and `level = submission_ready`, executor completed, and delivery completed. ## Phase Order Run the 15-phase loop described in [phases](references/phases.md): 1. `source_intake` 2. `mempalace_context` 3. `council_knowledge` 4. `expert_forum_coverage` 5. `literature_refresh` 6. `knowledge_publish` 7. `advisor_design` 8. `execution_packet` 9. `executor_run` 10. `advisor_reflection` 11. `openspace_retrospective` 12. `next_cycle_handoff` 13. `star_office_status` 14. `overwatcher_check` 15. `github_publish` ## Native Executor Phase 9 remains the key agent-native phase: - Read the execution packet. - Read the source clone. - Generate the smallest useful runnable research asset. - Write only to `source_changes/`. - Run lightweight checks when possible. - Write `executor_manifest.json`. Use [executor-template](references/executor-template.md) as the output contract. ## Protocol Hygiene Before the advisor or executor treats a repo as submission-ready, check for: - positioning drift between README claims and later submission/package locators - synthetic or structural claims being mislabeled as empirical results - tracked reproducibility artifacts that rewrite timestamps or other volatile fields - CI gates that omit confirm scripts or fail to check dirty diffs - premature dual-venue framing before empirical results exist - misleading names such as `result_quality_*` for input-quality analysis - weak contamination checks being promoted into strong evidence Use [protocol hygiene](references/protocol-hygiene.md) as the detailed rubric. ## Innovation Frontier Current assets are the evidence floor, not the imagination ceiling. Before selecting a micro-step, check whether the loop is overfitting to incomplete benchmarks, toy rows, inherited file names, or prior advisor backlog. Use [innovation frontier](references/innovation-frontier.md) to keep at least one evidence-linked but asset-transcending research route alive. ## Completion Gate Only stop when all are true: - Submission Advisor score is at least `100`. - Submission Advisor level is `submission_ready`. - Executor manifest status is `completed`. - Delivery status is `published` or explicitly `local_only` for dry runs.

path: references/advisor-rubric.md
content: # Submission Advisor Rubric

Score from `0` to `100` with `100` reserved for a repo that is ready to deliver as a submission package.

Axes:

- Problem definition and novelty
- Methodological rigor
- Result quality and analysis
- Manuscript or package completeness
- Reproducibility and asset hygiene
- Venue fit

The advisor must use current repo files and current literature as primary evidence. Prior memory and expert profiles can pressure-test conclusions but cannot support a claim alone.

## Protocol Hygiene Gates

Apply these gates before raising readiness or recommending a manuscript/submission package:

1. **Repo positioning gate**: if the README says the repo is only a theme, topic, benchmark, or asset repo, but later sections add venue submission package locators, classify this as positioning drift and recommend a cleanup micro-step.
2. **Empirical evidence gate**: synthetic toy rules, schema checks, reviewer locators, and structural validations are not empirical model results. They can support `structural_supported` or `synthetic_supported`, but not `empirical_supported`.
3. **Reproducibility gate**: any reproducibility/checkpoint script that rewrites tracked artifacts, timestamps, generated IDs, or volatile metadata must be fixed or moved out of tracked outputs before it is treated as deterministic.
4. **CI coverage gate**: if confirm scripts exist but CI runs only unit tests, asset audit, and one checkpoint command, mark CI as narrow and propose adding confirm scripts and a dirty-diff check.
5. **Venue timing gate**: ESWA/DMKD or any dual-venue framing is premature if the repo has no real empirical result table. Treat venue references as scouting notes, not packaging instructions.
6. **Naming gate**: names such as `result_quality_*` must not be used for input-quality analysis. Prefer `input_quality_*`, `dataset_quality_*`, or `annotation_quality_*`.
7. **Contamination-evidence gate**: a header-only CSV scan or narrow field-name scan can be a weak pollution smoke test, but cannot support a strong "no result contamination" claim.

Readiness cannot reach `submission_ready` while any gate above remains unresolved.

## Innovation Frontier Gate

The advisor must not let existing assets define the full research horizon. Treat incomplete task sets, toy benchmarks, prior reviewer feedback, inherited venue notes, and previous backlog items as calibration evidence, not as the only possible research path.

Before scoring or selecting the next action, produce or update:

1. `asset_bound_reading`: what the current repo genuinely supports.
2. `asset_fixation_risk`: where the loop may be overfitting to current files or toy rules.
3. `external_frontier_gap`: what current literature or expert-council pressure-testing suggests is missing.
4. `blank_slate_counterplan`: what the next research design would be if the current asset layout did not exist.
5. `next_exploration_probe`: the smallest safe probe that tests a new angle without corrupting current assets.

At least one backlog item should remain exploratory until the repo has empirical-supported results and a stable benchmark boundary. Exploratory does not mean speculative claims; it means a bounded artifact that expands the possible research design.

Completion requires:

- `overall_readiness.score >= 100`
- `overall_readiness.level == "submission_ready"`
- A completed executor manifest
- Published or explicitly local-only delivery


path: references/executor-template.md
content: # Agent-Native Executor Output Template

The executor must produce a bounded, runnable micro-step.

## Required Files

- At least one runnable Python file unless the micro-step is explicitly a config/test-only edit.
- `requirements.txt` only when new dependencies are required.
- Optional `docs/agentic_research/<cycle_id>/...` notes for protocol context.
- `executor_manifest.json` written via the CLI helper.

## Manifest Shape

```json
{
  "cycle_id": "20260411_120451",
  "generated_at": "2026-04-11T12:04:51Z",
  "status": "completed",
  "mode": "agent_native_executor",
  "source": "agent_native_exec",
  "selected_micro_step": {"title": "..."},
  "summary": "What changed and why this is the next smallest useful step.",
  "limitations": ["What this step still does not prove."],
  "next_probe": "The next smallest follow-up probe.",
  "protocol_hygiene": {
    "repo_positioning": "clean | drift_detected | not_checked",
    "empirical_evidence_level": "none | structural_supported | synthetic_supported | empirical_supported",
    "reproducibility_dirty_diff_checked": true,
    "ci_coverage_note": "What confirm scripts or dirty-diff checks were added or left missing.",
    "venue_framing_status": "none | scouting_only | packaging_ready | premature",
    "naming_boundary_note": "Any misleading names fixed or explicitly deferred.",
    "contamination_check_strength": "none | weak_smoke_test | strong_artifact_scan"
  },
  "innovation_frontier": {
    "asset_bound_reading": "What the current repo assets genuinely support.",
    "asset_fixation_risk": "Where the selected step may be trapped by current incomplete assets.",
    "frontier_gap": "What missing external idea, benchmark axis, or mechanism this step keeps visible.",
    "step_mode": "repair | exploit | explore | bridge",
    "blank_slate_counterplan_ref": "Path or note for the counterplan considered before this step.",
    "next_exploration_probe": "Smallest safe idea-expanding probe after this step."
  },
  "generated_files": [
    {
      "destination": "source_repo",
      "repo_path": "scripts/run_pilot.py",
      "local_path": "/absolute/path/under/source_changes/scripts/run_pilot.py",
      "purpose": "Minimal executable experiment script"
    }
  ],
  "claim_rule": "No empirical claim is supported unless tied to generated runnable evidence."
}
```

## Rules

1. Keep one micro-step per cycle.
2. Read the target repo before writing.
3. Write into the `source_changes/` repo-root mirror, not directly into the source clone.
4. Prefer deterministic offline checks over secret-dependent API calls.
5. Do not require `OLLAMA_API_KEY`.
6. If a model call is optional, make it explicit and document the fallback.
7. If the selected micro-step touches reproducibility, run the relevant script and then check for a dirty git diff in tracked artifacts.
8. If the repo has no empirical model-output results, do not create manuscript packaging or venue-specific submission assets; create topic, benchmark, or protocol assets instead.
9. If a check is only a weak smoke test, label it weak in the manifest and docs.
10. Do not only repair or package inherited assets for multiple consecutive cycles; if the repo is asset-poor or toy-only, create a bounded exploration probe or design bridge that can expand the research hypothesis space.
11. Never mark an exploratory probe as empirical support until it produces real model-output or checkable experimental artifacts.


path: references/innovation-frontier.md
content: # Innovation Frontier Rubric

Use this rubric when a target repo starts from incomplete assets such as a README, toy benchmark, partial question bank, reviewer notes, or inherited scripts. The goal is to prevent the loop from becoming an asset-polishing machine that never escapes the initial framing.

## Core Rule

Current assets are the **evidence floor**, not the **imagination ceiling**.

Use current repo files to decide what is supported, but do not assume they define the best research question, benchmark axis, intervention, or paper contribution.

## Asset Fixation Signals

Flag `asset_fixation_risk` when any of these appear:

- the loop repeatedly edits README, locators, or packaging files without adding a new experimental mechanism
- all next steps are derived only from the existing file tree
- toy rules or synthetic rows become the center of the research agenda
- venue packaging starts before the benchmark has real empirical results
- reviewer feedback is treated as the whole research roadmap
- the same task taxonomy is extended without asking whether it is the right taxonomy
- prior MemPalace memory is copied forward without current-head or literature challenge

## Required Three-Lane Reading

Before selecting a micro-step, maintain three lanes:

1. `repair`: fix false claims, reproducibility issues, CI gaps, naming drift, or README drift.
2. `exploit`: strengthen the best current asset with tests, data, or a runnable probe.
3. `explore`: open a bounded new angle suggested by literature, expert council critique, or blank-slate reasoning.

The selected micro-step can be only one lane, but the advisor should keep all three visible.

## Blank-Slate Counterplan

Every major advisor pass should ask:

```text
If this repo only contained the original research theme and no current assets, what would be the strongest first benchmark or experiment design?
```

Write a short counterplan that includes:

- alternative research question
- alternative benchmark axis
- alternative intervention or mechanism
- minimal experiment to test it
- why it may beat the inherited asset path
- why it is not yet supported by current evidence

Do not execute the counterplan wholesale. Use it to generate one small exploration probe.

## Exploration Probe Rules

Exploration probes must be bounded and safe:

- create a small design note, fixture, benchmark slice, or dry-run script
- do not overwrite current empirical assets
- do not rename the whole project unless the advisor explicitly chooses a repositioning step
- do not claim empirical support until real outputs exist
- keep the connection to the current repo evidence visible

Good probe examples:

- add `docs/frontier_questions.md` with three alternative mechanisms and falsifiable tests
- add a tiny fixture that tests a missing benchmark axis
- write a dry-run evaluator that checks whether the current task taxonomy can express a new failure mode
- add a literature-backed benchmark dimension as `candidate_axis`, not as a result

Bad probe examples:

- start a full manuscript package before results
- replace the project direction based only on expert vibes
- generate many speculative hypotheses with no runnable next step
- treat a toy fixture as a confirmed empirical finding

## External Challenge

Use current literature and expert council as challenge lenses:

- ask what strong reviewers would say the current asset cannot test
- ask what adjacent benchmark families measure that the current repo ignores
- ask what negative control or counterexample would break the current framing
- ask whether the current task set is measuring proxy quality instead of the intended phenomenon

Expert memory is not evidence. It only helps identify questions to test.

## Cycle Policy

If two consecutive cycles are `repair` or `exploit` only, the next advisor pass should propose at least one `explore` or `bridge` micro-step unless an unresolved safety/reproducibility gate blocks it.

`bridge` means a micro-step that connects current assets to a broader frontier without abandoning them, such as adding a missing axis, counterexample fixture, or alternative protocol note.


path: references/phases.md
content: # Research Loop Phases

All phases are small and inspectable. Python helper phases write deterministic artifacts; agent-native phases perform judgment and code generation.

| Phase | Owner | Output |
| --- | --- | --- |
| `source_intake` | Python + agent | source snapshot and inventory |
| `mempalace_context` | Python | current-head memory pack |
| `council_knowledge` | Python | expert council route |
| `expert_forum_coverage` | Python | coverage score and expansion queue |
| `literature_refresh` | OpenAlex + native web search | papers and web evidence |
| `knowledge_publish` | Python | cycle knowledge base |
| `advisor_design` | agent | advisor manifest, protocol hygiene findings, innovation-frontier reading, and action backlog |
| `execution_packet` | Python | one selected micro-step, preferring unresolved hygiene gates while preserving an exploration lane |
| `executor_run` | agent | runnable source changes, protocol hygiene notes, innovation-frontier notes, and executor manifest |
| `advisor_reflection` | Python | post-execution constraints and evidence-level demotions |
| `openspace_retrospective` | Python | loop improvement notes |
| `next_cycle_handoff` | Python | next-cycle handoff |
| `star_office_status` | Python | local dashboard status |
| `overwatcher_check` | Python | observe-only workflow health report |
| `github_publish` | Python | target repo delivery |

Do not rely on Ollama native web search. If fresh web evidence is needed inside an agent-native phase, use native web search and pass short, attributed snippets into the relevant helper stage.

Before choosing a new scientific claim or venue-packaging step, run the protocol hygiene rubric in [protocol hygiene](protocol-hygiene.md). Cleanup of positioning drift, deterministic artifact dirtiness, narrow CI gates, misleading result/input-quality names, and weak contamination checks takes priority over adding new claims.

Before choosing a new micro-step, also run the innovation frontier rubric in [innovation frontier](innovation-frontier.md). Current assets should bound supported claims, but they should not trap the research design or prevent bounded exploration probes.


path: references/protocol-hygiene.md
content: # Protocol Hygiene Rubric

Use this rubric whenever a target research repo contains benchmark, hallucination, evaluation, venue, or reproducibility assets. It is designed to prevent the loop from inheriting misleading repo state.

## 1. Positioning Drift

Check the README and top-level docs for contradictory stage signals.

Examples:

- early README says the repo is not a paper archive, not a result archive, and not a continuation of a current automated research line
- later README sections add ESWA, DMKD, or submission package locators

Action:

- classify this as `positioning_drift`
- make the next micro-step a README/stage cleanup if it can misroute future agents
- keep venue references as scouting notes unless empirical result assets exist

## 2. Evidence Level

Use explicit evidence levels:

- `none`: no runnable or checkable artifact supports the claim
- `structural_supported`: file layout, schema, locator, CI, or documentation supports the claim
- `synthetic_supported`: toy rows, synthetic slices, mock tables, or controlled fixtures support the claim
- `empirical_supported`: real model outputs, real experimental runs, or real result tables support the claim

Rules:

- synthetic toy rows are not empirical results
- reviewer-locator claims are not empirical results
- structural CI success is not an empirical result
- do not call a repo submission-ready without empirical-supported result assets

## 3. Reproducibility Dirtiness

If a script claims determinism or checkpoint reproducibility, run it and inspect whether tracked files changed.

Required command shape:

```bash
git status --short
<run checkpoint or confirm command>
git status --short
git diff -- <declared tracked artifacts>
```

If a generated timestamp, generated ID, or volatile metadata rewrites a tracked artifact, classify the checkpoint as `non_replay_stable` until fixed.

Preferred fixes:

- make timestamps stable when the input is stable
- write volatile run metadata to an untracked run directory
- compare normalized JSON that ignores allowed volatile fields
- add CI dirty-diff checks after reproducibility commands

## 4. CI Gate Width

CI is narrow if it only runs unit tests, asset audit, and one checkpoint while confirm scripts exist outside the gate.

Action:

- list un-gated confirm scripts
- add the smallest CI expansion that runs high-value confirm scripts
- add a post-command dirty-diff check for tracked artifacts
- do not mark reproducibility complete until CI covers the relevant confirmation path

## 5. Venue Timing

Venue framing must match evidence maturity.

Rules:

- `ESWA` / `DMKD` / dual-venue notes are allowed as scouting notes
- venue-specific submission package locators are premature without real empirical results
- dual venue framing should not drive file layout before the benchmark/result assets are stable
- if venue notes exist early, label them as `venue_scouting`, not `submission_package`

## 6. Naming Boundary

File names are prompts for future agents. Avoid names that imply stronger evidence than exists.

Examples:

- `result_quality_*` should not describe input-quality analysis
- prefer `input_quality_*`, `dataset_quality_*`, `annotation_quality_*`, or `candidate_result_schema_*`

Action:

- rename misleading files when safe
- otherwise write an explicit deprecation note and create a clean successor file

## 7. Contamination Check Strength

A scan that only checks CSV headers or a narrow set of field names is a weak smoke test.

Rules:

- label it `weak_smoke_test`
- do not use it as strong proof that the repo has no result contamination
- strengthen it by scanning data rows, metadata, docs, generated reports, and artifact directories
- keep limitations near the check output and in the advisor reflection

## Micro-Step Priority

When several problems exist, prefer this order:

1. fix tracked reproducibility dirtiness
2. fix README positioning drift that can misroute the next cycle
3. widen CI to cover existing confirm scripts and dirty diffs
4. rename misleading result/input-quality assets
5. demote premature venue packaging to scouting notes
6. strengthen weak contamination checks

The loop should prefer these cleanup steps before adding new scientific claims.


path: references/state-schema.md
content: # State Schema

Each run writes a durable local state tree and publishes sanitized artifacts to the target repo under `.autonomous-research-workflow/`.

Important paths:

```text
<run-dir>/
  events.jsonl
  manifest.json
  payload/.autonomous-research-workflow/
    cycles/<cycle_id>/cycle_manifest.json
    execution/<cycle_id>/execution_packet.json
    execution/<cycle_id>/executor_manifest.json
    execution/<cycle_id>/protocol_hygiene.json
    execution/<cycle_id>/innovation_frontier.json
    knowledge/<cycle_id>/knowledge_base.json
    memory/<cycle_id>/mempalace_context.json
    reflections/<cycle_id>/advisor_reflection.json
    retrospectives/<cycle_id>/openspace_retrospective.json
    handoffs/<cycle_id>/next_cycle_handoff.json
    office_status/<cycle_id>/star_office_status.json
    overwatcher/<cycle_id>/overwatcher_status.json
  source/<cycle_id>/<repo>/
  source_changes/
```

`source_changes/` mirrors the target repository root and is copied into the source clone during publish.

`protocol_hygiene.json` is optional but recommended when the repo contains benchmark, reproducibility, venue, or hallucination-evaluation assets. It should record positioning drift, evidence level, deterministic artifact dirtiness, CI gate width, venue timing, naming boundary issues, and contamination-check strength.

`innovation_frontier.json` is optional but recommended when the repo begins from incomplete assets. It should record asset-bound reading, asset fixation risk, repair/exploit/explore lanes, blank-slate counterplan, and the next bounded exploration probe.


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)