Skill: ivangdavila/prediction (version 1.0.0)

--- name: Prediction slug: prediction version: 1.0.0 homepage: https://clawic.com/skills/prediction description: Forecast uncertain outcomes with base rates, reference classes, calibration loops, and explicit scorekeeping. changelog: Initial release with question design, calibration, forecast review, and post-mortem workflows for probabilistic forecasting. metadata: {"clawdbot":{"emoji":"🔮","requires":{"bins":[]},"os":["linux","darwin","win32"],"configPaths":["~/prediction/"]}} --- ## When to Use User needs a defended forecast about what will happen, when it will happen, or how likely it is. Agent handles question design, base-rate search, reference-class selection, inside-vs-outside view balancing, explicit probability assignment, and after-action scoring. Use it for business, product, technical, operational, policy, sports, market, or personal planning questions whenever the task is to forecast an uncertain outcome rather than just explain the present. ## Architecture Memory lives in `~/prediction/`. If `~/prediction/` does not exist, run `setup.md`. See `memory-template.md` for structure. ```text ~/prediction/ ├── memory.md # Activation rules, forecasting defaults, and durable lessons ├── forecast-log.md # Open forecasts with probability, horizon, and next review date ├── scorecard.md # Resolved forecasts, Brier scores, and error patterns ├── reference-classes.md # Reusable base-rate cases by domain ├── assumptions.md # Active drivers, fragilities, and update triggers └── archive/ # Old resolved periods and retired forecasting themes ``` ## Quick Reference Use the smallest file that resolves the blocker. | Topic | File | Use it for | |-------|------|------------| | First-run activation | `setup.md` | Integration behavior, storage boundaries, and first local state | | Memory baseline | `memory-template.md` | Local templates for forecasts, scorecards, and assumptions | | BRACE forecast loop | `forecast-loop.md` | End-to-end process from question intake to review | | Forecastable question design | `question-design.md` | Turn vague prompts into resolvable prediction targets | | Calibration and confidence | `calibration.md` | Map evidence quality into probabilities and abstention rules | | Scoring and post-mortems | `scoring-and-review.md` | Score forecasts, inspect misses, and improve hit rate over time | ## Requirements - No credentials or external services are required by default. - Ask before storing sensitive personal forecasts, legal matters, health outcomes, or unreleased company information. - Prefer questions with a clear resolution rule and time horizon. If those are missing, define them before assigning a probability. ## Prediction Contract Every serious forecast should leave behind: 1. the exact question being forecast 2. the resolution rule and deadline 3. the base rate or reference class used 4. the main drivers that could move the answer 5. a numeric probability or ranked scenario split 6. the trigger that would cause an update before resolution 7. a later score or post-mortem once reality is known This is the minimum needed to improve accuracy instead of producing forgettable guesses. ## Core Rules ### 1. Turn the Prompt Into a Resolvable Question First - Use `question-design.md` before making any forecast that matters. - If the target, threshold, deadline, or resolution source is fuzzy, the forecast is not auditable and the hit rate cannot improve. ### 2. Start With the Outside View Before the Story - Pull a base rate or nearest reference class before building an inside-view narrative. - Humans overweight unique details and underweight how often similar situations actually happen. ### 3. Run the BRACE Forecast Loop on Every Non-Trivial Prediction - Use `forecast-loop.md`: Base rate, Resolution rule, Arguments both ways, Confidence assignment, Evaluation plan. - A loop beats intuition because it forces evidence on both sides and leaves a trail for later scoring. ### 4. Express Uncertainty Numerically and Defend It - Give a number, range, or explicit scenario split rather than words like "probably" or "maybe." - Use `calibration.md` to map evidence quality, sample size, and model disagreement into probability levels. ### 5. Separate Signal From Narrative Heat - Track what is actually predictive, what is merely interesting, and what is just recent or vivid. - Strong stories with weak base rates are noise, not edge. ### 6. Update Only on Information That Changes the Odds - Pre-commit to update triggers: deadline changes, threshold changes, a major driver flips, or new data changes the reference class. - Constant micro-updating on every headline produces churn without better accuracy. ### 7. Score Every Meaningful Forecast and Learn From Misses - Use `scoring-and-review.md` after resolution and store the result in the local scorecard. - Unscored forecasts feel smart in the moment and teach nothing later. ## Common Traps These are the failure modes that usually destroy forecast accuracy even when the reasoning sounds smart. | Trap | Why It Fails | Better Move | |------|--------------|-------------| | Predicting a vibe instead of an event | The forecast cannot be scored or falsified | Rewrite into one resolvable question with a deadline | | Going straight to inside-view storytelling | Unique details swamp the real base rate | Start with the nearest reference class and only then adjust | | Using words instead of numbers | "Likely" means different things to different people | Give a probability, range, or scenario table | | Refusing to abstain | Forced certainty creates fake precision | Say what is missing and hold a low-confidence or no-call position | | Treating new information as equally important | Noise looks like signal | Update only when a driver or resolution rule actually changes | | Forgetting to track misses | Accuracy never compounds | Score the forecast, log the error type, and update the reference class | | Confusing decision advice with certainty | A good decision can still have a bad outcome | Keep probability, recommendation, and risk management separate | ## Data Storage Local state in `~/prediction/` may include: - activation preferences and forecasting defaults - open forecasts with probabilities, scenarios, and review dates - resolved forecasts with scores and miss patterns - reusable reference classes and base-rate notes - assumptions and update triggers for active forecasting topics Store only the smallest durable note that improves the next forecast. ## Security & Privacy **Data that stays local:** - forecast logs, scorecards, assumptions, and reference-class notes in `~/prediction/` **Data that leaves your machine:** - none by default unless the current environment uses approved search or browsing tools for evidence collection **This skill does NOT:** - claim certainty where evidence is weak - guarantee accuracy or positive expected value - place bets, trades, or automatic decisions on the user's behalf - store credentials, account numbers, or private medical records - modify its own skill file ## Scope This skill ONLY: - turns ambiguous prediction prompts into auditable forecasting questions - uses base rates, reference classes, and explicit probability assignments - tracks forecast quality through scoring and post-mortems - stores lightweight local notes that improve later predictions This skill NEVER: - pretend that every question is forecastable with confidence - replace licensed legal, medical, or investment advice - confuse eloquent explanation with predictive power - skip scoring on forecasts that matter ## Related Skills Install with `clawhub install <slug>` if user confirms: - `analysis` - structure assumptions, causal chains, and trade-offs before forecasting. - `compare` - evaluate scenario branches and option differences after the forecast is framed. - `decide` - turn probabilities and uncertainty into explicit decision choices. - `statistics` - dig deeper into inference, distributions, and sampling logic behind the forecast. ## Feedback - If useful: `clawhub star prediction` - Stay updated: `clawhub sync`

path: calibration.md
content: # Calibration and Confidence

Use this file when choosing the actual probability or scenario weights.

## Probability ladder

| Probability | Meaning |
|-------------|---------|
| 5-15% | clear long shot |
| 20-35% | plausible but against the base rate |
| 40-60% | genuinely uncertain |
| 65-80% | favored but far from locked |
| 85-95% | strong edge with limited remaining uncertainty |

Avoid 0%, 100%, or any false precision unless the resolution is already effectively known.

## Calibration rules

- If the evidence is thin, stay near the base rate.
- If a key assumption is unverified, cap confidence.
- If multiple strong unknowns remain, widen the range instead of pretending a precise point estimate exists.
- If the forecast depends on one brittle driver, say that explicitly.

## When to abstain

Hold a low-confidence or no-call position when:
- the question is not resolvable
- the reference class is nonexistent or deeply mismatched
- the forecast horizon is undefined
- critical evidence is missing and cannot be estimated responsibly
- the user really needs domain advice rather than a forecast

## Update triggers

Update only when:
- the deadline or threshold changes
- new evidence changes the reference class
- a key driver flips from bullish to bearish or vice versa
- the prior forecast is now obviously stale

Noisy headlines alone are not enough.


path: forecast-loop.md
content: # BRACE Forecast Loop

Use this file when the forecast matters enough to deserve a repeatable process.

BRACE keeps the agent from drifting into pure storytelling:

## 1. Base Rate

- Find the nearest reference class before reading the current case as unique.
- Prefer observed frequencies over vibes, anecdotes, or one memorable example.
- If the base rate is weak, say so and keep confidence lower.

## 2. Resolution Rule

- Define exactly what counts as yes, no, or unresolved.
- Name the threshold, date, and evidence source that will settle the question.
- If the resolution rule is fuzzy, rewrite the question first.

## 3. Arguments Both Ways

- List the strongest three reasons for yes and the strongest three reasons for no.
- Separate leading indicators from lagging signals.
- Demote arguments that are only vivid, recent, or emotionally satisfying.

## 4. Confidence Assignment

- Start from the base rate, then adjust in small defended steps.
- Convert the final view into a number, range, or explicit scenario split.
- Explain what evidence would move the forecast materially.

## 5. Evaluation Plan

- Log the forecast in `forecast-log.md`.
- Set a review date or trigger, not endless reactive updates.
- When the question resolves, score it in `scorecard.md` and update any reusable lesson.

## Minimum Output Contract

Every BRACE forecast should include:
- question
- resolution rule
- probability or scenarios
- main supporting evidence
- main contrary evidence
- update trigger
- next review date


path: memory-template.md
content: # Memory Template - Prediction

Create these files inside `~/prediction/`.

## `memory.md`

```markdown
# Prediction Memory

## Status
status: ongoing
version: 1.0.0
last: YYYY-MM-DD
integration: pending | complete | paused | never_ask

## Forecasting Defaults
- Preferred domains:
- Preferred output format:
- Update cadence:
- Main optimization target:
- Known blind spots to watch:

## Guardrails
- Questions that should trigger abstention:
- Data types that should not be stored:
- Cases where this should stay advisory only:

## Notes
- Durable lessons from prior forecasts:
- Patterns behind good calls:
- Patterns behind misses:
```

## `forecast-log.md`

```markdown
# Forecast Log

| question | horizon | probability or scenarios | base rate | main drivers | next review | status | updated |
|----------|---------|--------------------------|-----------|--------------|-------------|--------|---------|
| Will launch ship in Q3? | 2026-Q3 | 42% | 3 of 10 similar launches | scope creep, hiring, blocker bug | 2026-06-01 | open | YYYY-MM-DD |
```

## `scorecard.md`

```markdown
# Prediction Scorecard

| date | question | forecast | outcome | brier | miss type | lesson |
|------|----------|----------|---------|-------|-----------|--------|
| YYYY-MM-DD | Will launch ship in Q3? | 42% yes | no | 0.1764 | planning optimism | stress staffing and slip history harder |
```

## `reference-classes.md`

```markdown
# Reference Classes

| domain | question type | base rate | sample | notes | updated |
|--------|---------------|-----------|--------|-------|---------|
| product | launch by announced quarter | 30% yes | 10 comparable launches | slips cluster around design + QA | YYYY-MM-DD |
```

## `assumptions.md`

```markdown
# Assumptions and Update Triggers

| forecast | assumption | direction | evidence needed to revise | owner | updated |
|----------|------------|-----------|---------------------------|-------|---------|
| Q3 launch | hiring closes in 30 days | bullish | role stays open >45 days | team | YYYY-MM-DD |
```

## Status Values

| Value | Meaning | Behavior |
|-------|---------|----------|
| `ongoing` | Forecasting style still forming | Keep gathering preferences naturally while answering |
| `complete` | Enough context exists | Work from stored defaults unless the user updates them |
| `paused` | User does not want setup questions right now | Forecast with minimal discovery and no setup push |
| `never_ask` | User does not want recurring profiling | Stop asking for more routing detail unless required |

## Key Principles

- Save durable forecasting defaults and lessons, not every passing opinion.
- Update `last` whenever an active forecast, scorecard, or guardrail changes materially.
- Open forecasts matter less than resolved ones if they are never scored; keep the scorecard alive.
- Reference classes should be compact, comparable, and easy to reuse across questions.


path: question-design.md
content: # Forecastable Question Design

Use this file to convert vague prompts into something that can actually be predicted and scored.

## A good prediction question is:

- singular: one event, not a bundle of outcomes
- time-bounded: it resolves by a named date or window
- thresholded: it says what counts as success or failure
- externally checkable: someone can later verify the answer
- decision-relevant: the result changes what the user should do

## Rewrite rules

- Replace "Will this go well?" with the concrete event that matters.
- Replace "soon" with a date or maximum delay.
- Replace "big growth" with a numeric threshold.
- Replace multi-part prompts with separate forecasts unless all parts must resolve together.

## Bad vs better

| Bad question | Why it fails | Better version |
|--------------|--------------|----------------|
| Will the launch be successful? | No resolution rule | Will the launch ship to paying users by 2026-09-30? |
| Will revenue improve? | No threshold | Will monthly recurring revenue exceed 120k by 2026-12-31? |
| Is the team on track? | Not forecastable as stated | Will the team close the milestone backlog by the end of May? |
| Will the market like this? | Vague subject | Will the feature reach 25% weekly active adoption within 8 weeks of release? |

## Question checklist

Before assigning odds, confirm:
- what exactly happens
- by when
- at what threshold
- who or what source resolves it
- whether there are multiple scenarios instead of one binary event

If any answer is missing, pause the forecast and fix the question.


path: scoring-and-review.md
content: # Scoring and Review

Use this file after the outcome resolves or when the review date arrives.

## Score the forecast

- For binary outcomes, use Brier score: `(forecast - outcome)^2` where outcome is 1 for yes and 0 for no.
- Lower is better: 0 is perfect, 0.25 is a coin-flip level at 50%, 1.0 is maximally wrong.
- For scenario forecasts, score the assigned probability to the realized scenario and note whether the scenario set itself was incomplete.

## Miss types to tag

| Miss type | What happened |
|-----------|---------------|
| base-rate neglect | The story overrode the historical rate |
| planning optimism | Timeline or execution was too rosy |
| driver miss | The wrong variable was treated as decisive |
| update failure | New evidence arrived and the forecast stayed stale |
| resolution error | The question was framed badly from the start |
| overconfidence | Probability was too extreme for the evidence quality |

## Post-mortem prompts

Ask:
- What did the forecast get right?
- What did it miss?
- Which evidence mattered most in hindsight?
- Was the reference class wrong, too broad, or too narrow?
- Should any lesson enter `memory.md` or `reference-classes.md`?

## Improvement loop

- Move durable miss patterns into `memory.md`.
- Update reusable rates in `reference-classes.md`.
- Archive dead forecasts and keep the live scorecard compact.
- Favor learning from repeated mistakes, not one-off bad luck.


path: setup.md
content: # Setup - Prediction

Use this file when `~/prediction/` is missing or empty, or when the user wants forecasting to become a recurring capability.

Answer the immediate prediction question first, then lock activation behavior early so future forecasting work starts with the right level of rigor instead of guessing from scratch.

## Immediate First-Run Actions

### 1. Lock integration behavior early

Within the first exchanges, clarify:
- should this activate whenever the user asks what is likely, what will happen, or what odds to assign
- should it jump in proactively for uncertain plans and bets, or only on explicit request
- whether rough estimation questions should route here, `analysis`, or `decide`

Keep this short. One clear routing answer is enough.

### 2. Learn the user's forecasting style

Understand the defaults that change prediction quality:
- domains they forecast most: business, product, engineering, markets, sports, politics, life planning, or mixed
- what output helps most: single-number probability, scenario ranges, confidence bands, or decision memo
- how aggressive they want updates to be: only on major evidence or after any material signal
- whether they care more about hit rate, calibration, downside avoidance, or speed

Start broad, then narrow only where the current prediction needs it.

### 3. Lock boundaries and storage scope

Clarify:
- what should be remembered across forecasts: preferred probability format, favored domains, known blind spots, update cadence, scoring habits
- what should stay out of memory: secrets, account details, health records, legal case specifics, or unreleased sensitive information
- whether examples may be stored in shorthand form or should stay fully ephemeral

### 4. Create local state after the routing contract is clear

```bash
mkdir -p ~/prediction/archive
touch ~/prediction/memory.md
touch ~/prediction/forecast-log.md
touch ~/prediction/scorecard.md
touch ~/prediction/reference-classes.md
touch ~/prediction/assumptions.md
chmod 700 ~/prediction ~/prediction/archive
chmod 600 ~/prediction/memory.md ~/prediction/forecast-log.md ~/prediction/scorecard.md ~/prediction/reference-classes.md ~/prediction/assumptions.md
```

If the files are empty, initialize them from `memory-template.md`.

### 5. What to save

Save only what improves the next forecast:
- activation behavior and routing preferences
- recurring domains, default probability style, and update preferences
- open forecasts that need revisiting
- resolved forecasts with scores and miss patterns
- reference classes and assumptions that recur often enough to matter

Do not store secrets, financial credentials, health details, or private case files.


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)