Skill: kingmadellc/xpulse (version 1.1.1)

--- name: Xpulse description: "Real-time X/Twitter social signal scanner for prediction market traders. Scans via DuckDuckGo (zero API cost) with two-stage local Qwen AI filtering: tradeable signal detection, then materiality gating to prevent alert fatigue. Position-aware — only alerts on signals matching your active Kalshi positions. Fail-closed design: silence over noise. Part of the OpenClaw Prediction Market Trading Stack — signals feed into Market Morning Brief and correlate with Prediction Market Arbiter divergences." --- # Xpulse — Real-Time Social Signal Scanner ## Overview Xpulse is a real-time X/Twitter signal scanner optimized for prediction market traders. It detects market-relevant signals on configurable topics using a three-stage intelligence pipeline: 1. **Stage 1: Signal Detection** — Scans X via DuckDuckGo (site:x.com queries), analyzes posts with local Qwen for tradeable signals 2. **Stage 2: Materiality Gate** — Compares against 48h signal history, filters out noise and repeat signals (fail-closed) 3. **Stage 3: Position Matching** — Only alerts on signals that match active Kalshi positions (keyword overlap, 2+ keywords required) **Key philosophy:** Zero API costs (DuckDuckGo + local Qwen), fail-closed design (silence over noise), position-aware filtering (no irrelevant alerts). ## When to Use This Skill - You're actively trading prediction markets on Kalshi - You want real-time social signals for your active positions - You prefer local LLM analysis over expensive API calls - You need to prevent notification fatigue from generic social signals - You want signals cached for morning brief consumption ## Requirements ### Python & Dependencies - Python 3.10 or higher - Required packages: ```bash pip install ddgs pyyaml ``` ### Local Infrastructure - **Ollama** (https://ollama.ai) with Qwen model - Install from https://ollama.ai (macOS, Linux, Windows) - Then: `ollama pull qwen3:latest` - Must be running: `ollama serve` (background process) ### Kalshi API (for position matching) - Free Kalshi account with API credentials (https://kalshi.com) - API key ID - Private key file (ES256 format) - Cost: Free (read-only access to positions) ## Configuration Create or update your `config.yaml`: ```yaml kalshi: enabled: true api_key_id: "your-key-id" private_key_file: "/path/to/private.key" ollama: enabled: true model: "qwen3:latest" timeout_seconds: 30 xpulse: enabled: true check_interval_minutes: 30 topics: - "tariff" - "Ukraine" - "inflation" - "Fed rate" - "Trump policy" min_confidence: 0.7 # Stage 1: signal confidence threshold materiality_gate: true # Stage 2: enable 48h deduplication position_gate: true # Stage 3: only alert on matched positions max_history_entries: 200 ``` ### Topics Configuration `topics` list should contain: - Market-relevant keywords you're interested in - DuckDuckGo will search: `site:x.com {topic}` - Examples: "Trump tariff", "inflation CPI", "Fed rate", "crypto SEC", "presidential election" ### Thresholds | Parameter | Default | Meaning | |-----------|---------|---------| | `min_confidence` | 0.7 | Stage 1: Only keep signals with ≥70% AI confidence of being tradeable | | `check_interval_minutes` | 30 | Minimum minutes between scans (prevent API hammering) | | `materiality_gate` | true | Stage 2: Enable/disable deduplication filter | | `position_gate` | true | Stage 3: Enable/disable Kalshi position matching | | `max_history_entries` | 200 | Max entries in 48h signal history (auto-trimmed) | ## The Three-Stage Pipeline ### Stage 1: Signal Detection **Input:** List of topics (e.g., ["tariff", "inflation", "Ukraine"]) **Process:** 1. For each topic, search X/Twitter via DuckDuckGo: `site:x.com {topic}` 2. Fetch up to 5 recent posts 3. Feed posts to local Qwen with this prompt: ``` You are a prediction market analyst. Given these recent X/Twitter posts about '{topic}', determine if there's a tradeable signal. {combined_posts} Respond in JSON: {"has_signal": true/false, "confidence": 0.0-1.0, "direction": "bullish/bearish/neutral", "summary": "one line"} ``` **Thresholding:** Only keep signals where: - `has_signal == true` - `confidence >= min_confidence` (default 0.7) **Output:** List of candidate signals with direction + confidence --- ### Stage 2: Materiality Gate (Fail-Closed) **Input:** Candidate signals from Stage 1 + 48h signal history **Purpose:** Prevent alert fatigue by filtering out: - Repeat signals about the same story (different wording) - Ongoing background noise (tariffs in news for days) - Speculation with no concrete new event - Minor updates to previously alerted developments **Process:** 1. Load recently sent signal history (last 20 alerts, 48h window) 2. Build prompt with: - Candidate new signals - What the user already knows (recent alerts) 3. Send to Qwen: ``` You are a personal alert filter for a prediction market trader. Your job is to PREVENT notification fatigue. RECENTLY SENT ALERTS (what the user already knows): - [3h ago] tariff: Trump announces new steel tariffs (confidence: 0.82) - [6h ago] inflation: Core CPI steady at 3.2% (confidence: 0.75) CANDIDATE NEW SIGNALS: - [tariff] Trump discusses additional tariffs (confidence: 0.68) - [inflation] Fed signals no near-term cuts (confidence: 0.79) RULES: - REJECT if same story as recent alert (even different wording) - REJECT if ongoing background noise - REJECT if no concrete new event - ACCEPT only if genuinely new development or significant escalation - When in doubt, REJECT. The user prefers silence over noise. Respond in JSON: {"keep": [list of topic strings to keep], "reasoning": "..."} ``` **Fail-Closed Design:** - If Qwen fails (timeout, error, unparseable): **drop all signals** - This prevents spam of uncertain signals when the filter breaks - Better to miss a signal than spam the user **Output:** Filtered list (subset of Stage 1 candidates) --- ### Stage 3: Position Matching Gate **Input:** Filtered signals from Stage 2 + active Kalshi positions **Purpose:** Only alert the user about signals that match their active positions **Process:** 1. Fetch active Kalshi positions (unsettled) 2. Extract keywords from ticker + market title (>2 char words) - Example: "POTUS-2028-DEM" → keywords: {potus, 2028, dem} 3. For each signal, extract keywords from topic + summary 4. Match if 2+ keyword overlap **Example:** ``` Signal: topic="Trump tariff", summary="Trump announces new steel tariffs" Signal words: {trump, tariff, steel, announces, new} Kalshi position: ticker="TRUMP-TARIFF-2025", title="Will Trump enact steel tariffs?" Position keywords: {trump, tariff, 2025, steel, enact} Overlap: {trump, tariff, steel} = 3 keywords ✓ MATCH → Signal passes to iMessage alert ``` **Suppressed Signals:** - Signals without matching positions are logged silently - Available in cache (`.x_signal_cache.json`) for morning brief - Not sent to user **Output:** Critical signals matched to positions → iMessage alert --- ## Signal Caching All signals (pre-filter) are cached in `.x_signal_cache.json`: ```json { "signals": [ { "topic": "tariff", "confidence": 0.82, "direction": "bearish", "summary": "Trump announces new steel tariffs", "post_count": 3 } ], "topics_scanned": 5, "timestamp": 1709990400.0 } ``` **Used for:** - Morning brief consumption (all signals, even suppressed ones) - On-demand signal review (what was detected but not alerted) - Debugging signal pipeline --- ## Running Xpulse ### Command Line ```bash # Single run python -m xpulse.xpulse # With full logging DEBUG=1 python -m xpulse.xpulse # Dry run (logs but doesn't send alerts) python -m xpulse.xpulse --dry-run # Force a run regardless of interval python -m xpulse.xpulse --force ``` ### Scheduled (Every 30 Minutes) **Via OpenClaw config:** ```yaml skills: xpulse: enabled: true schedule: "*/30 * * * *" timeout_seconds: 120 ``` **Via crontab:** ```bash # Add to crontab -e: */30 * * * * cd /path/to/xpulse && python -m xpulse.xpulse >> /tmp/xpulse.log 2>&1 ``` **Via launchd (macOS):** ```xml <!-- ~/.launchd/com.xpulse.scanner.plist --> <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>com.xpulse.scanner</string> <key>ProgramArguments</key> <array> <string>/usr/bin/python3</string> <string>/path/to/xpulse.py</string> </array> <key>StartInterval</key> <integer>1800</integer> <key>StandardOutPath</key> <string>/tmp/xpulse.log</string> <key>StandardErrorPath</key> <string>/tmp/xpulse.log</string> </dict> </plist> ``` --- ## Example Output ### Alert to iMessage (Stage 3 Pass) ``` ⚠️ X signal — affects your Kalshi positions: 📈 Trump tariff: Trump announces new 25% steel tariffs (82% conf) [TRUMP-TARIFF-2025] 📉 Inflation: Core CPI stays sticky, Fed unlikely to cut (75% conf) [FED-RATE-2026] ➡️ Ukraine: Peace talks accelerate, settlement probability rising (68% conf) [UKRAINE-2026] ``` ### Signal Cache (Stage 1 → Morning Brief) ```json { "signals": [ { "topic": "Trump tariff", "confidence": 0.82, "direction": "bearish", "summary": "Trump announces new 25% steel tariffs", "post_count": 3 }, { "topic": "inflation", "confidence": 0.75, "direction": "bearish", "summary": "Core CPI stays sticky, Fed unlikely to cut further", "post_count": 2 } ], "topics_scanned": 5, "timestamp": 1709990400.0 } ``` --- ## Materiality Gate — Technical Details See `references/materiality-gate.md` for detailed explanation of: - How the 48h rolling window works - Example filtering scenarios (signal vs noise) - Fail-closed behavior --- ## Position Matching — Technical Details See `references/position-matching.md` for: - Keyword extraction algorithm (>2 char threshold) - Overlap matching logic (2+ keyword requirement) - Example position-signal pairs --- ## Troubleshooting ### Ollama Not Running **Error:** `Connection refused` or `ollama: command not found` **Fix:** ```bash # Start Ollama in background ollama serve & # Or install if missing: # Install from https://ollama.ai, then: ollama pull qwen:latest ``` ### Qwen Model Not Found **Error:** `ollama run qwen:latest` fails **Fix:** ```bash ollama pull qwen:latest # Or use qwen3 if available: # ollama pull qwen3:latest ``` ### DuckDuckGo Returning No Results **Likely causes:** - Topic is too specific or niche - No recent posts on X about that topic - DuckDuckGo rate limiting (try again later) **Debug:** ```python from ddgs import DDGS d = DDGS() results = list(d.text("site:x.com Trump tariff", max_results=5)) print(results) ``` ### No Kalshi Positions (Stage 3 Suppresses All) **Expected behavior:** If you have no active Kalshi positions, all signals are suppressed (logged silently). This is intentional — position-aware alerting is the goal. **To test:** Either: 1. Create a test Kalshi position matching your topic 2. Disable Stage 3: set `position_gate: false` in config (alerts all non-materiality signals) ### Signal History Growing Too Large **Automatic:** History is capped at 200 entries (oldest trimmed first) **Manual reset:** ```bash rm ~/.openclaw/state/x_signal_history.json ``` --- ## Qwen Prompts (Core IP) ### Stage 1 Prompt: Signal Detection ``` You are a prediction market analyst. Given these recent X/Twitter posts about '{topic}', determine if there's a tradeable signal. {combined_posts} Respond in JSON: {"has_signal": true/false, "confidence": 0.0-1.0, "direction": "bullish/bearish/neutral", "summary": "one line"} ``` ### Stage 2 Prompt: Materiality Gate ``` You are a personal alert filter for a prediction market trader. Your job is to PREVENT notification fatigue. Only let through signals that are genuinely NEW and MATERIAL. RECENTLY SENT ALERTS (what the user already knows): {history_block} CANDIDATE NEW SIGNALS: {candidate_block} RULES: - REJECT if the signal covers the same story/development as a recent alert (even with different wording) - REJECT if it's ongoing background noise (e.g. 'Trump discusses tariffs' when tariffs have been in the news for days) - REJECT if there's no concrete new event, just commentary or speculation - ACCEPT only if: (a) a genuinely new development occurred (vote, announcement, emergency, data release, market move), OR (b) a significant escalation/reversal of something previously reported - When in doubt, REJECT. The user prefers silence over noise. Respond in JSON: {"keep": [list of topic strings to keep], "reasoning": "one line explaining why"} ``` --- ## OpenClaw Ecosystem Integration Social signal intelligence layer for the Prediction Market Trading Stack. | Connected Skill | How It Connects | |----------------|-----------------| | **Market Morning Brief** | X signals appear in your daily morning digest | | **Kalshalyst** | Social signals validate or challenge contrarian edges | | **Prediction Market Arbiter** | Correlate social sentiment with cross-platform divergences | | **Kalshi Command Center** | Act on position-matched signals via trade execution | **Install the complete stack:** ```bash clawhub install kalshalyst kalshi-command-center polymarket-command-center prediction-market-arbiter xpulse portfolio-drift-monitor market-morning-brief personality-engine ``` ## Implementation Notes Battle-tested in production trading environments. Design principles: 1. **Standalone implementation** — zero external dependencies beyond listed packages 2. **Direct alerts** — sends signals straight to you via your OpenClaw agent 3. **All thresholds, filters, and prompts** — refined through live social signal monitoring 4. **Fail-closed design** — no signal is better than a false positive 5. **Scripts are standalone** — works with any OpenClaw setup --- ## Performance & Cost ### Typical Run - **Runtime:** 1-2 minutes (5 topics, Qwen Stage 1+2) - **API calls:** - DuckDuckGo: 5 calls (1 per topic) - Kalshi: 1 call (fetch positions) - Ollama (local): 2 calls (Stage 1 + 2) - **Cost:** $0 (DuckDuckGo free, Ollama local, Kalshi free read-only) ### Scaling 10 runs per day = ~10-20 minutes total runtime + zero API cost. Minimal resource usage. --- ## Further Reading - `references/materiality-gate.md` — Signal deduplication + 48h window - `references/position-matching.md` — Keyword overlap logic --- ## API Reference ### Main Function ```python from xpulse import check_x_signals # Single run result = check_x_signals(state={}, dry_run=False, force=False) # Returns: bool (True if alert sent, False otherwise) ``` ### State Management ```python # State dict tracks: state = { "last_x_signal_check": 1709990400.0, # Unix timestamp of last scan "last_x_signals_silent": [...] # Suppressed signals (logged only) } ``` --- ## Support & Iteration Xpulse is designed for iteration: 1. **Topic Tuning:** Add/remove topics as your trading focus changes 2. **Confidence Threshold:** Lower `min_confidence` to catch more signals (noisier), raise to reduce false positives 3. **Materiality Gate:** Disable if you're missing important signals, enable if too noisy 4. **Position Gate:** Disable `position_gate: false` if you want ALL signals (not recommended) 5. **Qwen Prompts:** Customize Stage 1/2 prompts for your trading style --- ## License & Attribution **Author**: KingMadeLLC --- ## Feedback & Issues Found a bug? Have a feature request? Want to share results? - **GitHub Issues**: [github.com/kingmadellc/openclaw-prediction-stack/issues](https://github.com/kingmadellc/openclaw-prediction-stack/issues) - **X/Twitter**: [@KingMadeLLC](https://x.com/KingMadeLLC) Part of the **OpenClaw Prediction Stack** — the first prediction market skill suite on ClawHub.

path: README.md
content: # Xpulse

Real-time X/Twitter social signal scanner for prediction market traders. Zero API cost — scans via DuckDuckGo with two-stage local Qwen AI filtering.
<!-- CODEX: synchronized the documented Ollama model with the runtime default. -->

## Install

```bash
clawhub install xpulse
```

## Quick Start

1. Install Ollama from https://ollama.ai and pull Qwen: `ollama pull qwen3:latest`
2. Get a Kalshi API key at https://kalshi.com (Settings → API)
3. Configure topics and credentials in `~/.openclaw/config.yaml`
4. Run: `python scripts/xpulse.py`

## Pipeline

| Stage | What It Does |
|-------|-------------|
| **Signal Detection** | Scan X via DuckDuckGo, analyze with Qwen for tradeable signals |
| **Materiality Gate** | Compare against 48h history, filter noise and repeats (fail-closed) |
| **Position Matching** | Only alert on signals matching active Kalshi positions |

## Cost

$0. DuckDuckGo is free, Ollama runs locally, Kalshi read-only API is free.

## Full Documentation

See [SKILL.md](SKILL.md) for complete documentation including configuration, materiality gate design, position matching logic, and troubleshooting.

## Part of the OpenClaw Prediction Market Trading Stack

```bash
clawhub install kalshalyst kalshi-command-center polymarket-command-center prediction-market-arbiter xpulse portfolio-drift-monitor market-morning-brief personality-engine
```

**Author**: KingMadeLLC


path: example-config.yaml
content: # Example Xpulse Configuration
# Copy to ~/.openclaw/config.yaml and fill in your credentials
#
# CODEX: synchronized example config with the runtime default model.

kalshi:
  enabled: true
  api_key_id: "your-kalshi-api-key-id"
  private_key_file: "/path/to/kalshi/private.key"
  # Get these from https://kalshi.com → Settings → API

ollama:
  enabled: true
  model: "qwen3:latest"
  timeout_seconds: 30

xpulse:
  enabled: true
  check_interval_minutes: 30       # Run every 30 minutes

  # Topics to scan (site:x.com queries)
  topics:
    - "Trump tariff"
    - "Ukraine"
    - "inflation CPI"
    - "Fed rate decision"
    - "election 2028"
    - "crypto SEC"
    - "AI regulation"
    - "tech earnings"

  # Stage 1: Signal Detection
  min_confidence: 0.7             # Keep signals with ≥70% confidence

  # Stage 2: Materiality Gate
  materiality_gate: true          # Enable 48h deduplication

  # Stage 3: Position Matching
  position_gate: true             # Only alert on matched positions

  # History management
  max_history_entries: 200        # Auto-trim oldest signals


path: references/materiality-gate.md
content: # Materiality Gate — Signal Deduplication System

## Purpose

The materiality gate filters Stage 1 signals to prevent notification fatigue. It answers: **"Is this a genuinely NEW development, or just noise about something the user already knows?"**

## Design Philosophy

**Fail-Closed:** If the filter breaks (Ollama timeout, parse error), drop ALL signals rather than spam the user. A missed signal is acceptable; false positives are not.

---

## The 48-Hour Rolling Window

### Data Structure

Signal history is stored in `.openclaw/state/x_signal_history.json`:

```json
[
  {
    "topic": "tariff",
    "summary": "Trump announces 25% steel tariffs",
    "direction": "bearish",
    "confidence": 0.82,
    "matched_position": "TRUMP-TARIFF-2025",
    "timestamp": 1709987400.0
  },
  {
    "topic": "inflation",
    "summary": "Core CPI stays sticky at 3.2%",
    "direction": "neutral",
    "confidence": 0.75,
    "timestamp": 1709983800.0
  }
]
```

### Pruning

- **Max entries:** 200 (auto-trimmed, oldest first)
- **Time window:** 48 hours from current time
  ```python
  cutoff = time.time() - 48 * 3600
  history = [h for h in history if h["timestamp"] > cutoff]
  ```
- **Result:** History never grows unbounded, always reflects recent context

---

## Filter Algorithm

### Input

```
Stage 1 candidates (pre-filter):
[
  {"topic": "Trump tariff", "summary": "New 25% steel tariff announced", "confidence": 0.82},
  {"topic": "tariff", "summary": "Trump discusses additional tariffs", "confidence": 0.68},
  {"topic": "inflation", "summary": "Core CPI sticky, Fed won't cut", "confidence": 0.79},
]

History (last 48h alerts):
[
  {"topic": "tariff", "summary": "Trump announces 25% steel tariffs", "timestamp": 3h ago},
  {"topic": "Fed rate", "summary": "Fed signals no near-term cuts", "timestamp": 6h ago},
]
```

### Process

1. **Build context blocks** for Qwen
   - Recent alerts block (what user already knows)
   - Candidate signals block (what's new)

2. **Call Qwen Stage 2 prompt:**
   ```
   You are a personal alert filter for a prediction market trader.
   Your job is to PREVENT notification fatigue.

   RECENTLY SENT ALERTS:
   - [3h ago] tariff: Trump announces 25% steel tariffs
   - [6h ago] Fed rate: Fed signals no near-term cuts

   CANDIDATE NEW SIGNALS:
   - [Trump tariff] New 25% steel tariff announced (conf: 82%, bearish)
   - [tariff] Trump discusses additional tariffs (conf: 68%, bearish)
   - [inflation] Core CPI sticky, Fed won't cut (conf: 79%, neutral)

   RULES:
   - REJECT if same story as recent alert (even different wording)
   - REJECT if ongoing background noise
   - REJECT if no concrete new event
   - ACCEPT only if genuinely new development or significant escalation
   - When in doubt, REJECT.

   Respond: {"keep": [topic strings to keep], "reasoning": "..."}
   ```

3. **Parse response:**
   ```python
   parsed = json.loads(qwen_output)
   keep_topics = set(parsed.get("keep", []))
   # Result: {"keep": ["inflation"], "reasoning": "New CPI data unknown to user..."}
   ```

4. **Filter:**
   ```python
   filtered = [s for s in candidates if s["topic"] in keep_topics]
   # Returns: [{"topic": "inflation", ...}]
   ```

---

## Example Filtering Scenarios

### Scenario 1: Repeat Alert (REJECT)

**Candidate:** `{"topic": "tariff", "summary": "Trump discusses additional tariffs"}`

**History:** `{"topic": "tariff", "summary": "Trump announces 25% steel tariffs", 2h ago}`

**Qwen Decision:**
```
REJECT: Same story as recent alert. "Trump discusses tariffs" is ongoing commentary
about the steel tariff announcement from 2 hours ago. No new concrete development.
```

**Reasoning:** Noise — user already knows about tariff story

---

### Scenario 2: Genuine Escalation (ACCEPT)

**Candidate:** `{"topic": "tariff", "summary": "Trump extends tariffs to EU, Canada, Mexico"}`

**History:** `{"topic": "tariff", "summary": "Trump announces 25% steel tariffs", 5h ago}`

**Qwen Decision:**
```
ACCEPT: Significant escalation. Original alert was steel tariffs only.
This signal reports tariffs extended to additional trading partners and regions.
Concrete new development.
```

**Reasoning:** Material change — expansion beyond original announcement

---

### Scenario 3: Background Noise (REJECT)

**Candidate:** `{"topic": "inflation", "summary": "Inflation debate continues on Wall Street"}`

**History:**
- `{"topic": "inflation", "summary": "Core CPI released at 3.2%", 4h ago}`
- `{"topic": "inflation", "summary": "Fed signals rate hold", 6h ago}`

**Qwen Decision:**
```
REJECT: Background noise. "Inflation debate continues" is generic commentary.
User already knows about CPI release and Fed position. No new data, announcement,
or concrete development.
```

**Reasoning:** Ongoing background chatter, not material

---

### Scenario 4: New Data Release (ACCEPT)

**Candidate:** `{"topic": "inflation", "summary": "PCE inflation beats expectations, 2.8% vs 3.1% forecast"}`

**History:**
- `{"topic": "inflation", "summary": "Core CPI released at 3.2%", 4h ago}`
- `{"topic": "inflation", "summary": "Fed signals rate hold", 6h ago}`

**Qwen Decision:**
```
ACCEPT: Concrete new development. PCE data is different from CPI data released earlier.
Beats market expectations (2.8% vs 3.1% forecast). Material market-moving event.
```

**Reasoning:** New economic data, unexpected direction

---

### Scenario 5: Speculation (REJECT)

**Candidate:** `{"topic": "Ukraine", "summary": "Experts suggest peace talks could happen in Q2"}`

**History:** `{"topic": "Ukraine", "summary": "Peace negotiations underway", 3h ago}`

**Qwen Decision:**
```
REJECT: Pure speculation. "Experts suggest... could happen" is commentary and prediction,
not a concrete event. User already knows negotiations are underway.
No new announcement, decision, or development.
```

**Reasoning:** Speculation vs. hard news

---

## Fail-Closed Behavior

### When Qwen Fails

```python
if result.returncode != 0:
    _log("Stage 2 filter: Qwen failed, dropping all signals (fail-closed)")
    return []  # Fail closed — silence over noise when filter is broken
```

**Trigger points:**
- Ollama timeout (>45 seconds)
- Ollama not running
- Qwen crashes or returns unparseable JSON
- Network error

**Action:** Drop ALL candidate signals, log nothing

**Philosophy:** One missed signal is better than 5 false alerts when the system breaks

---

## Configuration

In `config.yaml`:

```yaml
xpulse:
  materiality_gate: true    # Enable/disable filtering
  max_history_entries: 200  # Cap on history size
```

### Disabling (For Testing/Debugging)

```yaml
xpulse:
  materiality_gate: false   # All Stage 1 candidates pass to Stage 3
```

**Warning:** This will increase alert frequency. Only for debugging.

---

## Performance Impact

- **Stage 2 runtime:** 30-45 seconds per run (Qwen LLM call)
- **Network:** 1 call to local Ollama
- **Cost:** $0 (local inference)

---

## Integration with Morning Brief

The full Stage 1 cache (pre-materiality) is stored separately:

```json
{
  "signals": [
    {"topic": "tariff", "confidence": 0.82, ...},    // sent to iMessage
    {"topic": "tariff", "confidence": 0.68, ...},    // filtered out
    {"topic": "inflation", "confidence": 0.79, ...}  // sent to iMessage
  ],
  "timestamp": 1709990400.0
}
```

Morning brief can consume this cache to show:
- What was alerted
- What was filtered (and why)
- Full context of X scanning activity


path: references/position-matching.md
content: # Position Matching — Signal-to-Kalshi Position Alignment

## Purpose

Stage 3 filters signals to match only active Kalshi positions. This prevents alerts about market-moving events that don't affect your portfolio.

**Philosophy:** Only alert about signals that matter to YOUR trading.

---

## Keyword Extraction

### From Kalshi Position

Each active position contributes keywords extracted from:
- **Ticker** (e.g., `TRUMP-TARIFF-2025`)
- **Market title** (e.g., `Will Trump enact 25% tariffs on steel by 2025?`)
- **Event ticker** (if present)

### Algorithm

```python
keywords = set()
for field in [ticker, title, event_ticker]:
    for word in field.lower().replace("-", " ").replace("_", " ").split():
        if len(word) > 2:  # Minimum 3 characters
            keywords.add(word)
```

### Example

**Position 1:**
- Ticker: `TRUMP-TARIFF-2025`
- Title: `Will Trump enact steel tariffs?`
- Keywords: {trump, tariff, 2025, steel, enact, will}

**Position 2:**
- Ticker: `FED-RATE-HOLD-JUN`
- Title: `Will the Fed hold rates in June?`
- Keywords: {fed, rate, hold, jun, will, the} → {fed, rate, hold, jun}

(Stop words "will", "the" typically have >2 char threshold, so excluded)

---

## Signal-Position Matching

### From Signal

Keywords extracted from:
- **Topic** (e.g., `Trump tariff`)
- **Summary** (e.g., `Trump announces 25% steel tariffs`) — only words >3 chars

```python
signal_topic = signal.get("topic", "").lower()
signal_summary = signal.get("summary", "").lower()
signal_words = set(signal_topic.replace("-", " ").replace("_", " ").split())
signal_words |= set(w for w in signal_summary.split() if len(w) > 3)
```

### Match Logic

```python
def _signal_matches_position(signal: dict, positions: list) -> dict:
    """Check if signal matches any position. Returns matched position or None."""
    signal_words = extract_signal_keywords(signal)

    best_match = None
    best_overlap = 0

    for pos in positions:
        overlap = len(signal_words & pos["keywords"])
        if overlap >= 2 and overlap > best_overlap:  # 2+ keyword overlap required
            best_overlap = overlap
            best_match = pos

    return best_match
```

**Requirements:**
- Minimum 2 keywords must overlap
- Returns best match (highest overlap count)
- Returns None if no match

---

## Example Matchings

### Match 1: Direct Topic Alignment ✓

**Signal:**
- Topic: `Trump tariff`
- Summary: `Trump announces 25% steel tariffs`
- Words: {trump, tariff, announces, steel, tariffs}

**Position:**
- Ticker: `TRUMP-TARIFF-2025`
- Title: `Will Trump enact steel tariffs?`
- Keywords: {trump, tariff, 2025, steel, enact}

**Overlap:** {trump, tariff, steel} = 3 words ✓ **MATCH**

---

### Match 2: Indirect Alignment ✓

**Signal:**
- Topic: `Fed decision`
- Summary: `Fed holds rates steady as inflation cools`
- Words: {holds, rates, steady, inflation, cools}

**Position:**
- Ticker: `FED-RATE-HOLD-JUN`
- Title: `Will the Fed hold rates in June?`
- Keywords: {fed, rate, hold, jun}

**Overlap:** {hold} = 1 word ✗ **NO MATCH** (needs 2+)

---

### Match 3: Weak Signal, Strong Position Context ✓

**Signal:**
- Topic: `tariff`
- Summary: `Trump extends tariff policy`
- Words: {trump, extends, tariff, policy}

**Position:**
- Ticker: `TRUMP-TARIFF-2025`
- Title: `Will Trump enact steel tariffs by 2025?`
- Keywords: {trump, tariff, 2025, steel, enact}

**Overlap:** {trump, tariff} = 2 words ✓ **MATCH**

---

### No Match: Unrelated Event ✗

**Signal:**
- Topic: `UK election`
- Summary: `UK Labour party gains in polls`
- Words: {labour, party, gains, polls}

**Position:**
- Ticker: `TRUMP-TARIFF-2025`
- Keywords: {trump, tariff, 2025, steel}

**Overlap:** {} = 0 words ✗ **NO MATCH**

---

### Match 4: Semantic Alignment with Synonyms

**Signal:**
- Topic: `inflation data`
- Summary: `CPI rises faster than expected`
- Words: {inflation, data, cpi, rises, faster, expected}

**Position 1:**
- Ticker: `CPI-Q1-2026`
- Keywords: {cpi, q1, 2026}

**Overlap:** {cpi} = 1 word ✗ **NO MATCH**

**Position 2:**
- Ticker: `INFLATION-STICKY-2026`
- Title: `Will inflation stay above 3% through 2026?`
- Keywords: {inflation, sticky, 2026, stay, will, above}

**Overlap:** {inflation} = 1 word ✗ **NO MATCH**

**Note:** This signal wouldn't match either position (both only 1-word overlap).
This is intentional — it prevents false positives on related-but-not-directly-relevant signals.

---

## Position State

### Fetching Active Positions

```python
def _get_active_kalshi_topics() -> list:
    """Fetch unsettled Kalshi positions and extract keywords."""
    client = _get_client()  # Kalshi API
    resp = client.call_api(
        "GET",
        "https://api.elections.kalshi.com/trade-api/v2/portfolio/positions?limit=100&settlement_status=unsettled"
    )
    data = json.loads(resp.read())
    positions = data.get("market_positions", [])

    topics = []
    for pos in positions:
        ticker = pos.get("ticker", "")
        title = pos.get("market_title", "")
        event_ticker = pos.get("event_ticker", "")

        keywords = extract_keywords(ticker, title, event_ticker)

        topics.append({
            "ticker": ticker,
            "title": title,
            "keywords": keywords,
            "side": pos.get("side", ""),  # "yes" or "no"
            "quantity": pos.get("total_traded", 0),
        })

    return topics
```

### Caching

Positions are fetched fresh on every run (no caching). This ensures:
- Recently closed positions don't trigger alerts
- New positions immediately start matching signals
- No stale state

---

## Silent Suppression

Signals that fail to match any position are logged silently:

```python
for s in signals:
    match = _signal_matches_position(s, kalshi_positions)
    if not match:
        silent_signals.append(s)

state["last_x_signals_silent"] = [
    {"topic": s["topic"], "summary": s["summary"][:80], "confidence": s["confidence"]}
    for s in silent_signals
]
```

These are:
1. **Logged to console** (DEBUG output)
2. **Stored in state** (accessible programmatically)
3. **Cached** (available in `.x_signal_cache.json` for morning brief)
4. **Not sent to iMessage** (silent)

---

## Alert Output

Signals passing all 3 stages are formatted for iMessage:

```
⚠️ X signal — affects your Kalshi positions:

  📈 Trump tariff: Trump announces 25% steel tariffs (82% conf) [TRUMP-TARIFF-2025]
  📉 Inflation: Core CPI sticky, Fed unlikely to cut (75% conf) [FED-RATE-2026]
  ➡️  Ukraine: Peace talks accelerate, settlement rising (68% conf) [UKRAINE-2026]
```

**Format per signal:**
- Icon (📈/📉/➡️ based on direction)
- Topic + summary (first 80 chars)
- Confidence (percentage)
- Matched position ticker

Max 3 signals per message (sorted by confidence, highest first).

---

## Configuration

### Enable/Disable

```yaml
xpulse:
  position_gate: true   # Default: enable matching
```

### Testing Without Positions

To test signal detection without active Kalshi positions:

```yaml
xpulse:
  position_gate: false   # Skip Stage 3, alert all materiality-filtered signals
```

**Warning:** This will alert you to all X signals about your topics, even ones not related to your positions. Recommended for debugging only.

---

## Performance

- **Runtime:** ~1 second (fetch positions + keyword matching)
- **API call:** 1 Kalshi API call (positions list)
- **Cost:** $0 (free Kalshi read-only)

---

## Failure Modes

### No Active Positions

```python
if not kalshi_positions:
    _log("X scanner: no active Kalshi positions — all signals suppressed (logged only)")
    return False
```

All signals are silently suppressed. Logged in state, cached, but not alerted.

**Behavior:** Safe silence when portfolio is empty.

### Kalshi API Error

```python
except Exception as e:
    _log(f"Kalshi position gate error: {e}")
    return []  # Fail closed — don't send if we can't verify positions
```

If position fetch fails (API down, credentials invalid), all signals are suppressed.

**Behavior:** Better to miss signals than send alerts without position context.

---

## Integration with Morning Brief

Position matches are cached with metadata:

```json
{
  "signals": [
    {
      "topic": "Trump tariff",
      "confidence": 0.82,
      "direction": "bearish",
      "summary": "Trump announces 25% steel tariffs",
      "matched_position": "TRUMP-TARIFF-2025",
      "position_side": "yes"
    },
    {
      "topic": "tariff",
      "confidence": 0.68,
      "summary": "Trump discusses additional tariffs",
      "matched_position": null,
      "suppressed_reason": "no matching position"
    }
  ]
}
```

Morning brief can show both:
- Signals that alerted (matched positions)
- Signals that were suppressed (no match)


path: requirements.txt
content: ddgs==0.3.0
pyyaml==6.0.3
kalshi-python==1.0.0


path: scripts/__init__.py
content: """Xpulse scripts package."""


path: scripts/json_utils.py
content: """Robust JSON parsing utilities for Qwen/Ollama responses.

Handles multiple output formats:
1. Raw JSON: {"key": "value"}
2. Markdown code blocks: ```json\n{"key": "value"}\n```
3. Wrapped JSON: Some text... {"key": "value"} ...more text
4. Fallback: Extract key-value pairs manually

This prevents silent failures when Qwen outputs JSON in unexpected formats.
"""

from __future__ import annotations

import json
import logging
import re
from typing import Any, Optional

_log = logging.getLogger(__name__)

# Sentinel value to distinguish "no fallback provided" from "fallback is None"
_SENTINEL = object()


def _extract_json_by_braces(text: str) -> Optional[str]:
    """Extract JSON string by matching braces, handling nested structures properly.

    Finds the first '{' and matches it with the corresponding '}', accounting for
    escaped quotes and nested braces.

    Args:
        text: Text potentially containing JSON

    Returns:
        Extracted JSON string, or None if no valid JSON object found
    """
    start = text.find("{")
    if start < 0:
        return None

    depth = 0
    in_string = False
    escape_next = False

    for i in range(start, len(text)):
        char = text[i]

        if escape_next:
            escape_next = False
            continue

        if char == "\\" and in_string:
            escape_next = True
            continue

        if char == '"':
            in_string = not in_string
            continue

        if not in_string:
            if char == "{":
                depth += 1
            elif char == "}":
                depth -= 1
                if depth == 0:
                    return text[start : i + 1]

    return None


def safe_parse_json(
    text: str,
    fallback: Any = _SENTINEL,
    logger_prefix: str = "",
) -> Optional[dict]:
    """Parse JSON from text with multiple fallback strategies.

    Attempts in order:
    1. Direct json.loads()
    2. Extract from markdown code blocks (```json ... ```)
    3. Extract JSON object using regex (finds {...})
    4. Manual key-value pair extraction (last resort)

    Args:
        text: Text potentially containing JSON
        fallback: Default value if all strategies fail.
                  If not provided, defaults to empty dict {}.
                  Can be explicitly set to None to get None on failure.
        logger_prefix: Prefix for debug logs to identify the caller

    Returns:
        Parsed dict, or fallback value if all strategies fail
    """
    # Handle fallback: use empty dict if not provided
    if fallback is _SENTINEL:
        fallback = {}

    if not text or not isinstance(text, str):
        _log.debug(f"{logger_prefix} safe_parse_json: input is empty or not string")
        return fallback

    text = text.strip()

    # Strategy 1: Direct json.loads()
    try:
        result = json.loads(text)
        if isinstance(result, dict):
            _log.debug(f"{logger_prefix} safe_parse_json: success via direct json.loads()")
            return result
    except json.JSONDecodeError:
        _log.debug(f"{logger_prefix} safe_parse_json: direct json.loads() failed")
    except Exception as e:
        _log.debug(f"{logger_prefix} safe_parse_json: direct json.loads() error: {e}")

    # Strategy 2: Extract from markdown code blocks
    # Patterns: ```json ... ```, ```JSON ... ```, or generic ``` ... ```
    json_block_match = re.search(
        r'```(?:json|JSON)?\s*\n?(.*?)\n?```',
        text,
        re.DOTALL
    )
    if json_block_match:
        json_text = json_block_match.group(1).strip()
        try:
            result = json.loads(json_text)
            if isinstance(result, dict):
                _log.debug(f"{logger_prefix} safe_parse_json: success via markdown code block extraction")
                return result
        except json.JSONDecodeError as e:
            _log.debug(f"{logger_prefix} safe_parse_json: markdown block extraction failed: {e}")

    # Strategy 3: Use brace-matching to find JSON object
    # This handles nested structures by counting braces
    json_text = _extract_json_by_braces(text)
    if json_text:
        try:
            result = json.loads(json_text)
            if isinstance(result, dict):
                _log.debug(f"{logger_prefix} safe_parse_json: success via brace-matching extraction")
                return result
        except json.JSONDecodeError as e:
            _log.debug(f"{logger_prefix} safe_parse_json: brace-matching extraction failed: {e}")

    # Strategy 4: Manual key-value pair extraction (last resort)
    # Try to find quoted key-value pairs: "key": value or "key": "value"
    manual_result = _extract_key_values(text)
    if manual_result:
        _log.debug(f"{logger_prefix} safe_parse_json: success via manual key-value extraction")
        return manual_result

    # All strategies failed
    _log.warning(
        f"{logger_prefix} safe_parse_json: all strategies failed, raw text:\n{text[:200]}"
    )
    return fallback


def _extract_key_values(text: str) -> Optional[dict]:
    """Manually extract key-value pairs from text (last-resort parser).

    Looks for patterns like:
    - "key": "value"
    - "key": true/false
    - "key": 123
    - "key": [...]

    Returns:
        Dict with extracted pairs, or None if nothing found
    """
    result = {}

    # Pattern: "key": value (with quotes around key, any value)
    # This handles strings, booleans, numbers, arrays, objects
    pattern = r'"([^"]+)"\s*:\s*([^,}]+)'

    matches = re.findall(pattern, text)
    for key, value in matches:
        key = key.strip()
        value = value.strip()

        # Try to parse the value
        parsed_value: Any = value
        if value.lower() == "true":
            parsed_value = True
        elif value.lower() == "false":
            parsed_value = False
        elif value.lower() == "null":
            parsed_value = None
        elif value.startswith('"') and value.endswith('"'):
            # String value
            parsed_value = value[1:-1]
        elif value.startswith('[') or value.startswith('{'):
            # Array or object — try to parse
            try:
                parsed_value = json.loads(value)
            except (json.JSONDecodeError, ValueError):
                parsed_value = value
        else:
            # Try to parse as number
            try:
                if '.' in value:
                    parsed_value = float(value)
                else:
                    parsed_value = int(value)
            except (ValueError, TypeError):
                parsed_value = value

        result[key] = parsed_value

    return result if result else None


path: scripts/xpulse.py
content: """Xpulse — X/Twitter signal scanner for prediction markets.

Real-time social signal detection via DuckDuckGo + local Qwen LLM.
Three-stage pipeline: Signal Detection → Materiality Gate → Position Matching
"""

import json
import os
import subprocess
import sys
import time
import logging
from pathlib import Path
from typing import Optional

try:
    import yaml
except ImportError:
    yaml = None

# ──────────────────────────────────────────────────────────────────────────────
# Logging
# ──────────────────────────────────────────────────────────────────────────────

logging.basicConfig(
    level=logging.DEBUG if os.environ.get("DEBUG") else logging.INFO,
    format="%(asctime)s | %(levelname)s | %(message)s"
)
_log = logging.getLogger(__name__).debug


# ──────────────────────────────────────────────────────────────────────────────
# Configuration
# ──────────────────────────────────────────────────────────────────────────────

CONFIG_PATHS = [
    Path.home() / ".xpulse" / "config.yaml",
    Path.home() / ".openclaw" / "config.yaml",
    Path("/etc/xpulse/config.yaml"),
]

STATE_DIR = Path.home() / ".openclaw" / "state"
STATE_DIR.mkdir(parents=True, exist_ok=True)


def _load_config() -> dict:
    """Load config from first available path."""
    if yaml is None:
        logging.debug("PyYAML not installed — running with defaults")
        return {}
    for path in CONFIG_PATHS:
        if path.exists():
            with open(path) as f:
                return yaml.safe_load(f) or {}
    return {}


CONFIG = _load_config()
XPULSE_CFG = CONFIG.get("xpulse", {})
KALSHI_CFG = CONFIG.get("kalshi", {})
OLLAMA_CFG = CONFIG.get("ollama", {"enabled": True, "model": "qwen3:latest", "timeout_seconds": 15})

# qwen3 has a thinking/reasoning mode on by default that massively inflates latency.
# Prepend /no_think to all prompts to disable it and get fast JSON responses.
_QWEN3_NO_THINK = OLLAMA_CFG.get("model", "").startswith("qwen3")


# ──────────────────────────────────────────────────────────────────────────────
# Stage 1: Signal Detection
# ──────────────────────────────────────────────────────────────────────────────

def _search_x_posts(topic: str) -> list:
    """Search X/Twitter via DuckDuckGo for recent posts."""
    try:
        from ddgs import DDGS
        d = DDGS()
        results = list(d.text(f"site:x.com {topic}", max_results=5))
        return [r.get("body", "") for r in results if r.get("body")]
    except ImportError:
        pass
    except Exception:
        pass

    # Fallback to deprecated package
    try:
        from duckduckgo_search import DDGS as DDGS_OLD
        with DDGS_OLD() as ddgs:
            results = list(ddgs.text(f"site:x.com {topic}", max_results=5))
        return [r.get("body", "") for r in results if r.get("body")]
    except ImportError:
        pass
    except Exception:
        pass

    return []


def _check_ollama_model() -> bool:
    """Check if configured Ollama model is available. Warn if not."""
    model = OLLAMA_CFG.get("model", "qwen3:latest")
    try:
        import urllib.request
        req = urllib.request.Request("http://localhost:11434/api/tags")
        with urllib.request.urlopen(req, timeout=5) as resp:
            data = json.loads(resp.read().decode("utf-8"))
            models = [m.get("name", "") for m in data.get("models", [])]
            if not any(model in m or m.startswith(model.split(":")[0]) for m in models):
                logging.warning(
                    f"Ollama model '{model}' not found. Available: {', '.join(models[:5])}. "
                    f"Try: ollama pull {model} — or set ollama.model to qwen2.5:7b in config."
                )
                return False
            return True
    except Exception as e:
        logging.warning(f"Ollama not reachable at localhost:11434: {e}")
        return False


# Ollama check is deferred to first use — avoids confusing warnings on unconfigured installs


def _call_ollama(prompt: str, timeout: int = 15) -> Optional[dict]:
    """Call Ollama via HTTP API (localhost:11434) for JSON responses.

    Uses the /api/generate endpoint which supports format: "json".
    Falls back to CLI if HTTP fails.
    """
    model = OLLAMA_CFG.get("model", "qwen3:latest")

    # Disable qwen3 thinking mode for fast responses
    if _QWEN3_NO_THINK:
        prompt = "/no_think\n" + prompt

    # Primary: HTTP API (supports JSON format natively)
    try:
        import urllib.request
        import urllib.error

        payload = json.dumps({
            "model": model,
            "prompt": prompt,
            "format": "json",
            "stream": False,
            "options": {"num_predict": 256},  # Cap output tokens — we only need short JSON
        }).encode("utf-8")

        req = urllib.request.Request(
            "http://localhost:11434/api/generate",
            data=payload,
            headers={"Content-Type": "application/json"},
            method="POST",
        )

        with urllib.request.urlopen(req, timeout=timeout) as resp:
            data = json.loads(resp.read().decode("utf-8"))
            response_text = data.get("response", "").strip()
            return json.loads(response_text)

    except Exception as e:
        _log(f"Ollama HTTP API failed: {e}")

    # Fallback: CLI with stdin pipe
    try:
        result = subprocess.run(
            ["ollama", "run", model],
            input=prompt,
            capture_output=True, timeout=timeout, text=True
        )
        if result.returncode == 0 and result.stdout.strip():
            # Try to extract JSON from response
            text = result.stdout.strip()
            # Find first { and last } to extract JSON
            start = text.find("{")
            end = text.rfind("}") + 1
            if start >= 0 and end > start:
                return json.loads(text[start:end])
        _log(f"Ollama CLI fallback failed: returncode={result.returncode}")
    except Exception as e:
        _log(f"Ollama CLI fallback error: {e}")

    return None


def _analyze_signals_local(topic: str, posts: list) -> dict:
    """Stage 1: Analyze posts for tradeable signals using local Qwen."""
    if not posts:
        return {"confidence": 0, "signal": None}

    try:
        combined = "\n".join(f"- {p}" for p in posts[:3])
        prompt = (
            f"You are a prediction market analyst. Given these recent X/Twitter posts about '{topic}', "
            f"determine if there's a tradeable signal.\n\n{combined}\n\n"
            f"Respond in JSON: {{\"has_signal\": true/false, \"confidence\": 0.0-1.0, "
            f"\"direction\": \"bullish/bearish/neutral\", \"summary\": \"one line\"}}"
        )

        timeout = min(OLLAMA_CFG.get("timeout_seconds", 15), 20)  # Cap at 20s per topic
        parsed = _call_ollama(prompt, timeout=timeout)

        if not parsed:
            _log(f"Qwen Stage 1 failed for {topic}")
            return {"confidence": 0, "signal": None}

        return {
            "confidence": float(parsed.get("confidence", 0)),
            "has_signal": parsed.get("has_signal", False),
            "direction": parsed.get("direction", "neutral"),
            "summary": parsed.get("summary", ""),
        }
    except Exception as e:
        _log(f"Stage 1 analysis error: {e}")
        return {"confidence": 0, "signal": None}


# ──────────────────────────────────────────────────────────────────────────────
# Stage 2: Materiality Gate
# ──────────────────────────────────────────────────────────────────────────────

def _load_signal_history() -> list:
    """Load history of previously sent X signal alerts."""
    history_path = STATE_DIR / "x_signal_history.json"
    try:
        with open(history_path) as f:
            data = json.load(f)
            # Only keep last 48h of history
            cutoff = time.time() - 48 * 3600
            return [h for h in data if h.get("timestamp", 0) > cutoff]
    except (OSError, json.JSONDecodeError):
        return []


def _save_signal_history(history: list):
    """Persist sent signal history."""
    history_path = STATE_DIR / "x_signal_history.json"
    try:
        # Keep max 200 entries
        with open(history_path, "w") as f:
            json.dump(history[-200:], f, indent=2)
    except OSError:
        pass


def _filter_novel_signals(signals: list, history: list) -> list:
    """Stage 2: Materiality gate — filter out noise and repeat signals."""
    if not signals:
        return []

    # Build history context for Qwen
    recent_summaries = []
    for h in history[-20:]:  # Last 20 sent signals
        ts = h.get("timestamp", 0)
        age_h = (time.time() - ts) / 3600
        recent_summaries.append(f"- [{age_h:.0f}h ago] {h.get('topic', '?')}: {h.get('summary', '')}")

    history_block = "\n".join(recent_summaries) if recent_summaries else "(No previous alerts sent)"

    # Build candidate signals block
    candidate_block = "\n".join(
        f"- [{s['topic']}] {s['summary']} (confidence: {s['confidence']:.0%}, direction: {s['direction']})"
        for s in signals
    )

    try:
        prompt = (
            "You are a personal alert filter for a prediction market trader. "
            "Your job is to PREVENT notification fatigue. Only let through signals that are genuinely NEW and MATERIAL.\n\n"
            "RECENTLY SENT ALERTS (what the user already knows):\n"
            f"{history_block}\n\n"
            "CANDIDATE NEW SIGNALS:\n"
            f"{candidate_block}\n\n"
            "RULES:\n"
            "- REJECT if the signal covers the same story/development as a recent alert (even with different wording)\n"
            "- REJECT if it's ongoing background noise (e.g. 'Trump discusses tariffs' when tariffs have been in the news for days)\n"
            "- REJECT if there's no concrete new event, just commentary or speculation\n"
            "- ACCEPT only if: (a) a genuinely new development occurred (vote, announcement, emergency, data release, market move), "
            "OR (b) a significant escalation/reversal of something previously reported\n"
            "- When in doubt, REJECT. The user prefers silence over noise.\n\n"
            "Respond in JSON: {\"keep\": [list of topic strings to keep], \"reasoning\": \"one line explaining why\"}"
        )

        timeout = min(OLLAMA_CFG.get("timeout_seconds", 15), 20)  # Cap at 20s
        parsed = _call_ollama(prompt, timeout=timeout)

        if not parsed:
            _log("Stage 2 filter: Qwen failed, dropping all signals (fail-closed)")
            return []  # Fail closed

        keep_topics = set(parsed.get("keep", []))
        reasoning = parsed.get("reasoning", "")

        filtered = [s for s in signals if s["topic"] in keep_topics]
        _log(f"Stage 2 filter: {len(signals)} candidates → {len(filtered)} kept. Reason: {reasoning}")
        return filtered

    except Exception as e:
        _log(f"Stage 2 filter error: {e}, dropping all signals (fail-closed)")
        return []  # Fail closed


# ──────────────────────────────────────────────────────────────────────────────
# Stage 3: Kalshi Position Matching
# ──────────────────────────────────────────────────────────────────────────────

def _get_active_kalshi_topics() -> list:
    """Fetch current Kalshi positions and extract keywords."""
    try:
        try:
            from kalshi_python_sync import Configuration, KalshiClient
        except ImportError:
            from kalshi_python import Configuration, KalshiClient

        if not KALSHI_CFG.get("enabled"):
            _log("Kalshi disabled in config")
            return []

        api_key_id = KALSHI_CFG.get("api_key_id")
        private_key_file = KALSHI_CFG.get("private_key_file")

        if not api_key_id or not private_key_file or api_key_id == "REPLACE_WITH_YOUR_KEY_ID":
            _log("Kalshi credentials missing or not configured")
            return []

        with open(private_key_file) as f:
            private_key = f.read()

        config = Configuration(
            host="https://api.elections.kalshi.com/trade-api/v2"
        )
        config.api_key_id = api_key_id
        config.private_key_pem = private_key

        client = KalshiClient(config)

        # Raw API call — avoids SDK v3 pydantic deserialization bug (Issue #9)
        resp = client._portfolio_api.get_positions_without_preload_content(limit=100)
        raw_data = json.loads(resp.read())
        _log(f"Raw positions API keys: {sorted(raw_data.keys())}")

        positions = (
            raw_data.get("event_positions")
            or raw_data.get("positions")
            or raw_data.get("market_positions")
            or []
        )

        # Filter to unsettled, non-zero positions locally
        filtered = []
        for p in positions:
            status = p.get("settlement_status")
            if status is not None and status != "unsettled":
                continue
            qty = int(float(p.get("position_fp") or p.get("position", 0) or 0))
            if qty != 0:
                filtered.append(p)
        positions = filtered

        if not positions:
            _log("Kalshi position gate: no active positions — suppressing all X signals")
            return []

        topics = []
        for pos in positions:
            # Handle both SDK objects (attributes) and dicts
            if isinstance(pos, dict):
                ticker = pos.get("ticker", "")
                title = pos.get("market_title", "") or pos.get("title", "")
                event_ticker = pos.get("event_ticker", "")
                side = pos.get("side", "")
                quantity = pos.get("total_traded", 0)
            else:
                ticker = getattr(pos, "ticker", "") or ""
                title = getattr(pos, "market_title", "") or getattr(pos, "title", "") or ""
                event_ticker = getattr(pos, "event_ticker", "") or ""
                side = getattr(pos, "side", "") or ""
                quantity = getattr(pos, "total_traded", 0) or 0

            keywords = set()
            for field in [ticker, title, event_ticker]:
                for word in field.lower().replace("-", " ").replace("_", " ").split():
                    if len(word) > 2:
                        keywords.add(word)

            topics.append({
                "ticker": ticker,
                "title": title,
                "keywords": keywords,
                "side": side,
                "quantity": quantity,
            })

        _log(f"Kalshi position gate: {len(topics)} active positions loaded")
        return topics

    except Exception as e:
        _log(f"Kalshi position gate error: {e}")
        return []


def _signal_matches_position(signal: dict, positions: list) -> Optional[dict]:
    """Check if signal matches any active Kalshi position (2+ keywords required)."""
    signal_topic = signal.get("topic", "").lower()
    signal_summary = signal.get("summary", "").lower()
    signal_words = set(signal_topic.replace("-", " ").replace("_", " ").split())
    signal_words |= set(w for w in signal_summary.split() if len(w) > 3)

    best_match = None
    best_overlap = 0

    for pos in positions:
        overlap = len(signal_words & pos["keywords"])
        if overlap >= 2 and overlap > best_overlap:
            best_overlap = overlap
            best_match = pos

    return best_match


# ──────────────────────────────────────────────────────────────────────────────
# Main Trigger
# ──────────────────────────────────────────────────────────────────────────────

def check_x_signals(state: dict, dry_run: bool = False, force: bool = False) -> bool:
    """Scan X/Twitter for market-relevant signals via DuckDuckGo + local Qwen."""
    if not XPULSE_CFG.get("enabled", True):
        return False

    interval = XPULSE_CFG.get("check_interval_minutes", 30)
    last_check = state.get("last_x_signal_check", 0)
    if not force and time.time() - last_check < interval * 60:
        return False

    state["last_x_signal_check"] = time.time()
    topics = XPULSE_CFG.get("topics", [])
    min_confidence = XPULSE_CFG.get("min_confidence", 0.7)

    if not topics:
        logging.info(
            "⚠️  Xpulse: no topics configured — nothing to scan.\n"
            "\n"
            "   Add topics to ~/.openclaw/config.yaml:\n"
            "     xpulse:\n"
            "       enabled: true\n"
            "       topics:\n"
            "         - tariffs\n"
            "         - federal reserve\n"
            "         - bitcoin\n"
            "\n"
            "   Then run again: python xpulse.py --dry-run --force"
        )
        return False

    # Check Ollama availability only when we have topics to scan
    _check_ollama_model()

    _log(f"X signal scanner starting... ({len(topics)} topics)")

    # ── Stage 1: Signal Detection (parallelized) ──────────────────────────────
    from concurrent.futures import ThreadPoolExecutor, as_completed

    def _scan_topic(topic):
        """Scan a single topic — called in parallel."""
        posts = _search_x_posts(topic)
        if not posts:
            return None
        _log(f"  {topic}: {len(posts)} posts found")
        analysis = _analyze_signals_local(topic, posts)
        if analysis.get("has_signal") and analysis.get("confidence", 0) >= min_confidence:
            return {
                "topic": topic,
                "confidence": analysis["confidence"],
                "direction": analysis.get("direction", "?"),
                "summary": analysis.get("summary", ""),
                "post_count": len(posts),
            }
        return None

    signals = []
    with ThreadPoolExecutor(max_workers=min(len(topics), 2)) as executor:  # 2 workers max on Mac Mini
        futures = {executor.submit(_scan_topic, t): t for t in topics}
        for future in as_completed(futures, timeout=30):  # 30s total for all topics
            try:
                result = future.result(timeout=10)
                if result:
                    signals.append(result)
            except Exception as e:
                _log(f"  Topic scan error: {e}")

    # Cache all Stage 1 signals for morning brief
    cache_path = STATE_DIR / "x_signal_cache.json"
    try:
        from datetime import datetime as _dt, timezone as _tz
        with open(cache_path, "w") as f:
            json.dump({
                "signals": signals,
                "topics_scanned": len(topics),
                "cached_at": _dt.now(_tz.utc).isoformat(),
                "timestamp": time.time(),
            }, f, indent=2)
    except OSError:
        pass

    _log(f"X scanner: {len(signals)} high-confidence signals from {len(topics)} topics")

    if not signals:
        return False

    signals_found = len(signals)

    # ── Stage 2: Materiality Gate ──────────────────────────────────────────────
    signal_history = _load_signal_history()
    use_materiality = XPULSE_CFG.get("materiality_gate", True)

    if use_materiality:
        signals = _filter_novel_signals(signals, signal_history)
        if not signals:
            _log("X scanner: all signals filtered by materiality gate (nothing new)")
            return False

    # ── Stage 3: Kalshi Position Gate ──────────────────────────────────────────
    kalshi_positions = _get_active_kalshi_topics()

    if not kalshi_positions:
        _log("X scanner: no active Kalshi positions — all signals suppressed (logged only)")
        state["last_x_signals_silent"] = [
            {"topic": s["topic"], "summary": s["summary"][:80], "confidence": s["confidence"]}
            for s in signals
        ]
        return False

    critical_signals = []
    silent_signals = []

    for s in signals:
        match = _signal_matches_position(s, kalshi_positions)
        if match:
            s["matched_position"] = match["ticker"]
            s["position_side"] = match["side"]
            critical_signals.append(s)
        else:
            silent_signals.append(s)

    if silent_signals:
        _log(f"X scanner: {len(silent_signals)} signals suppressed (no matching Kalshi position)")
        state["last_x_signals_silent"] = [
            {"topic": s["topic"], "summary": s["summary"][:80], "confidence": s["confidence"]}
            for s in silent_signals
        ]

    if not critical_signals:
        _log("X scanner: signals found but none match active Kalshi positions — silent")
        return False

    # ── Format Alert Message ───────────────────────────────────────────────────
    critical_signals.sort(key=lambda x: x["confidence"], reverse=True)
    parts = ["⚠️ X signal — affects your Kalshi positions:"]
    for s in critical_signals[:3]:
        icon = "📈" if s["direction"] == "bullish" else "📉" if s["direction"] == "bearish" else "➡️ "
        pos_info = f"[{s.get('matched_position', '?')}]"
        parts.append(f"  {icon} {s['topic']}: {s['summary'][:80]} ({int(s['confidence']*100)}% conf) {pos_info}")

    message = "\n".join(parts)

    # ── Send Alert (or dry-run) ────────────────────────────────────────────────
    if dry_run:
        _log(f"[DRY RUN] Would send: {message}")
        return False

    # In production, the agent/caller handles message routing
    # For standalone testing, just print
    print(message)
    _log(f"X signal CRITICAL alert: {len(critical_signals)} signals matching Kalshi positions")

    # ── Update History ─────────────────────────────────────────────────────────
    for s in critical_signals[:3]:
        signal_history.append({
            "topic": s["topic"],
            "summary": s["summary"],
            "direction": s["direction"],
            "confidence": s["confidence"],
            "matched_position": s.get("matched_position"),
            "timestamp": time.time(),
        })
    _save_signal_history(signal_history)

    return True


if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser(description="Xpulse — X/Twitter signal scanner")
    parser.add_argument("--dry-run", action="store_true", help="Log but don't send alerts")
    parser.add_argument("--force", action="store_true", help="Force run regardless of interval")
    args = parser.parse_args()

    state = {}
    try:
        result = check_x_signals(state, dry_run=args.dry_run, force=args.force)
        sys.exit(0 if result else 1)
    except KeyboardInterrupt:
        _log("Interrupted")
        sys.exit(130)
    except Exception as e:
        _log(f"Fatal error: {e}", exc_info=True)
        sys.exit(1)


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)