Skill: hzlvv/openclaw-new-digest (version 0.1.0)

--- name: news-digest description: "User-configurable multi-slot news aggregation and push system. Schedule, topics, and keywords are defined by the user via config.json. Aggregates from Twitter, Hacker News, Tavily, filters by influence, summarizes, stores locally, supports feedback iteration." metadata: {"clawdbot":{"emoji":"📰","requires":{"bins":["node"],"env":["TAVILY_API_KEY","XPOZ_API_KEY"]},"primaryEnv":"TAVILY_API_KEY"}} --- # News Digest AI-assisted news aggregation and push system with **user-defined schedule and topics**. The agent fetches content from Twitter (Xpoz MCP), Hacker News, and Tavily Search, then filters, summarizes, and delivers via Feishu/Telegram. This is an **agent-driven workflow** — the agent is the executor, triggered manually or by cron. ## First-Time Setup (IMPORTANT) Before executing any push, **check if a configuration exists**: ```bash node {baseDir}/scripts/manage-config.mjs show ``` If the output says "No Configuration Found", you **MUST** ask the user to set up their schedule before proceeding. Do NOT use hardcoded defaults without user consent. ### Setup Conversation Flow Ask the user these questions one by one: 1. **"你希望每天推送几次？"** — Determine the number of daily push slots (typical: 2-4). 2. **For each slot, ask:** - **"第 N 个推送的时间是？"** (e.g. 08:00, 12:00, 18:00) - **"这个时段的主题/方向是什么？"** (e.g. 金融, AI/Agent, 综合热点) - **"这个主题的标签名称？"** (e.g. 金融早报, AI午报, 晚间热点) - **"关键词有哪些？"** (comma-separated) - **"优先级规则？"** (e.g. crypto > 金融事件 > 美股) 3. **"需要调整默认设置吗？"** — Summary length (default: 100-300 chars), language (default: zh-CN), items per push (default: 3-10), time window (default: 24h). ### Save Each Slot After collecting user preferences, save each slot: ```bash node {baseDir}/scripts/manage-config.mjs set-slot \ --name <slot_name> \ --time <HH:MM> \ --topic "<topic>" \ --label "<display_label>" \ --keywords "<kw1,kw2,kw3>" \ --priority "<priority_rules>" \ --sources "twitter,tavily,hackernews" ``` Example: ```bash node {baseDir}/scripts/manage-config.mjs set-slot --name morning --time 08:00 --topic "Finance" --label "金融早报" --keywords "crypto,bitcoin,ethereum,finance,stock" --priority "crypto > financial events > US stocks" --sources "twitter,tavily,hackernews" ``` ### Update Defaults (Optional) ```bash node {baseDir}/scripts/manage-config.mjs set-defaults --summary-language en --items-per-push "5-15" ``` ### Quick-Init with Defaults If the user is okay with the standard 3-slot setup (08:00 Finance, 12:00 AI, 18:00 General): ```bash node {baseDir}/scripts/manage-config.mjs init ``` ### Verify ```bash node {baseDir}/scripts/manage-config.mjs show ``` ## Quick Start (After Config Exists) **Manual push** — just tell the agent: > "Execute the news digest push for [slot_name]" **Scheduled push** — set up cron entries matching your config (see `references/setup-guide.md`): ```bash # Example — adjust times and slot names to match YOUR config 0 8 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot morning 0 12 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot noon 0 18 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot evening ``` The `cron-trigger.sh` script reads `config.json` for slot definitions, loads `.env`, and spawns an OpenClaw session. **Check environment**: ```bash node {baseDir}/scripts/env-check.mjs ``` ## Configuration Management ```bash # View current config node {baseDir}/scripts/manage-config.mjs show # Add or update a slot node {baseDir}/scripts/manage-config.mjs set-slot --name <name> --time <HH:MM> --topic <topic> --label <label> --keywords <kw> # Remove a slot node {baseDir}/scripts/manage-config.mjs remove-slot --name <name> # Update defaults node {baseDir}/scripts/manage-config.mjs set-defaults --summary-length "100-300" # Reset to factory defaults node {baseDir}/scripts/manage-config.mjs reset ``` The user can change their schedule at any time. Just re-run `set-slot` or `remove-slot` and update cron accordingly. ## Workflow For each push slot, follow these steps **in order**. Read the slot's configuration first. ### Step 0: Load Slot Configuration ```bash node {baseDir}/scripts/manage-config.mjs show ``` Identify the target slot from the output: its `topic`, `keywords`, `priority`, `sources`, and `influence_thresholds`. All subsequent steps use these values — do NOT use hardcoded topics or keywords. ### Step 1: Check Previous Feedback ```bash node {baseDir}/scripts/query.mjs feedback --days 3 ``` Review recent feedback and adjust your filtering/summarization strategy accordingly. ### Step 2: Fetch Data from All Sources Use the slot's **keywords** and **sources** from config. Run in parallel: **Hacker News** (if `hackernews` in sources): ```bash node {baseDir}/scripts/fetch-hackernews.mjs "<keywords joined with OR>" --min-score <hackernews_score from config> --hours <time_window_hours from defaults> -n 20 ``` **Tavily Search** (if `tavily` in sources): ```bash node {baseDir}/scripts/fetch-tavily.mjs "<keywords as search query>" --topic news --days 1 -n 10 ``` Run multiple Tavily queries if the slot has diverse keywords — e.g. one for high-priority keywords, one for secondary. **Twitter** via Xpoz MCP (if `twitter` in sources and `XPOZ_API_KEY` is set): Use the Xpoz MCP tool `xpoz_search_tweets` with the slot's keywords. Filter by `min_likes > <twitter_likes from config>`. If `XPOZ_API_KEY` is not configured, skip and rely on other sources. ### Step 3: Deep-Read Key Articles (Optional) For articles with thin summaries, extract full content: ```bash node {baseDir}/scripts/extract-tavily.mjs "https://example.com/article1" "https://example.com/article2" ``` ### Step 4: Filter & Select Apply these rules, using thresholds from the slot's config: 1. **Time window**: Only content within the configured `time_window_hours` (default: 24h). 2. **Influence threshold**: Use `influence_thresholds` from the slot config. 3. **Topic match**: Content must match the slot's topic and keywords. Use `priority` to rank. 4. **Deduplication**: Same event from different outlets counts as one. 5. **Count**: Select items within the `items_per_push` range from config defaults. ### Step 5: Summarize Each Item For each selected item: - **Do NOT copy the original text verbatim.** Read and distill. - **Length**: Use `summary_length` from config defaults (default: 100-300 chars). - **Language**: Use `summary_language` from config defaults (default: zh-CN). - **Structure**: Event overview → Key data/viewpoints → Potential impact. ### Step 6: Format the Push ```markdown 【{slot.label}】{time} 推送 1. [{category}] {title}（来源：{source}） {summary} 🔗 {url} 2. [{category}] {title}（来源：{source}） {summary} 🔗 {url} ... --- 📊 本次用量：Tavily Search ×{n} | Tavily Extract ×{n} | HN ×{n} | Xpoz ×{n} 💬 如有反馈，请直接回复。 ``` Use the `label` field from the slot config as the topic label. **Last slot of the day extra**: For the last scheduled slot in the config, append a monthly API usage forecast: ```bash node {baseDir}/scripts/track-usage.mjs forecast ``` ### Step 7: Track Usage & Store the Push **Count your API calls** during steps 2-3: - `fetch-tavily.mjs` = +1 `tavily_search` - `extract-tavily.mjs` = +1 `tavily_extract` per URL batch - `fetch-hackernews.mjs` = +1 `hackernews` - Xpoz MCP tool call = +1 `xpoz` Include the counts in the `usage` field when storing: ```bash echo '{ "slot": "<slot_name>", "topic": "<slot_topic>", "keywords": ["<from config>"], "items": [ { "source": "twitter", "category": "crypto", "title": "Bitcoin hits new high", "url": "https://...", "author": "@example", "influence_score": 1520, "raw_excerpt": "Original excerpt...", "summary": "Your summary...", "metadata": {"likes": 1520, "retweets": 340} } ], "usage": { "tavily_search": 2, "tavily_extract": 1, "hackernews": 1, "xpoz": 1 } }' | node {baseDir}/scripts/store-push.mjs ``` This automatically persists usage data to `data/usage/YYYY-MM.json`. You can also record usage separately: ```bash node {baseDir}/scripts/track-usage.mjs record --slot <name> --tavily-search 2 --tavily-extract 1 --hackernews 1 --xpoz 1 ``` ### Step 8: Deliver the Push Send the formatted push content through OpenClaw's built-in messaging (Feishu or Telegram, as configured by the user). ### Step 9: Handle Feedback When a user provides feedback (free text), record it: ```bash node {baseDir}/scripts/add-feedback.mjs --date 2026-03-06 --slot <name> "feedback text" # Feedback on a specific item node {baseDir}/scripts/add-feedback.mjs --date 2026-03-06 --slot <name> --item-id item-001 "this summary is too brief" ``` When a user asks to **modify the schedule or topics**, use `manage-config.mjs`: ```bash # Change a slot's time node {baseDir}/scripts/manage-config.mjs set-slot --name morning --time 07:30 # Change keywords node {baseDir}/scripts/manage-config.mjs set-slot --name morning --keywords "crypto,defi,bitcoin,ethereum" # Add a new slot node {baseDir}/scripts/manage-config.mjs set-slot --name afternoon --time 15:00 --topic "Markets" --label "盘后速递" --keywords "stocks,earnings,markets" # Remove a slot node {baseDir}/scripts/manage-config.mjs remove-slot --name evening ``` ## Query History ```bash # View a specific push node {baseDir}/scripts/query.mjs pushes --date 2026-03-06 --slot <name> # View all pushes for a date node {baseDir}/scripts/query.mjs pushes --date 2026-03-06 # View feedback node {baseDir}/scripts/query.mjs feedback --date 2026-03-06 --slot <name> # Search by keyword across recent pushes node {baseDir}/scripts/query.mjs search --keyword "bitcoin" --days 7 ``` ## API Usage Tracking ```bash # Today's usage breakdown by slot node {baseDir}/scripts/track-usage.mjs today # Current month's total usage node {baseDir}/scripts/track-usage.mjs monthly # Monthly forecast node {baseDir}/scripts/track-usage.mjs forecast ``` Usage is recorded automatically when `store-push.mjs` receives a `usage` field. ## Feedback Iteration Strategy Before each push, review feedback from the past 3 days. Adjust behavior based on patterns: | Feedback Pattern | Adjustment | |------------------|------------| | "Too brief" on summaries | Increase summary detail, aim for upper end of summary_length | | "More about X topic" | Add X-related keywords via `manage-config.mjs set-slot` | | "Less about Y" | Remove or deprioritize Y keywords | | "Timing too early/late" | Update slot time via `manage-config.mjs set-slot --name <name> --time <HH:MM>` | | "Wrong category" | Refine topic matching rules | | "Liked this item" | Boost similar content sources/authors | ## Environment Variables | Variable | Required | Description | |----------|----------|-------------| | `TAVILY_API_KEY` | Yes | Tavily Search API key. Get from https://tavily.com | | `XPOZ_API_KEY` | Recommended | Xpoz MCP service key. If missing, Twitter source is skipped gracefully. | | `NEWS_DIGEST_DATA_DIR` | No | Data storage directory. Default: `{baseDir}/data/` | ## Notes - All scripts output Markdown for easy parsing. - Scripts use only Node.js built-in APIs (no npm dependencies). - Tavily search and extract are built into this skill — no need to install `tavily-search` separately. - If Xpoz MCP is unavailable, degrade gracefully by relying on Tavily + HN. - Store every push for traceability. Never skip the storage step. - Feedback is the primary mechanism for quality improvement. Always check before pushing. - **Schedule and topics are user-defined.** Always read config before pushing. Never assume slots exist.

path: _meta.json
content: ownerId: kn70cjr952qdec1nx70zs6wefn7ynq2t
slug: news-digest
version: 1.0.0
publishedAt: 1772870400000

path: hooks/openclaw/HOOK.md
content: ---
name: news-digest
description: "Injects time-aware push schedule reminder during agent bootstrap. Checks current hour to determine active slot (morning/noon/evening) and reminds the agent to execute the push workflow."
metadata: {"openclaw":{"emoji":"📰","events":["agent:bootstrap"]}}
---

# News Digest Hook

Injects a push schedule reminder during agent bootstrap based on the current time.

## What It Does

- Fires on `agent:bootstrap` (before workspace files are injected)
- Checks the current hour against the push schedule:
  - 07:00-09:00 → Reminds to execute **金融早报** (morning, finance/crypto)
  - 11:00-13:00 → Reminds to execute **AI午报** (noon, AI/Agent)
  - 17:00-19:00 → Reminds to execute **晚间热点** (evening, general tech)
- Outside push windows, shows next scheduled push and available query commands
- Skips sub-agent sessions to avoid redundant reminders

## Configuration

No configuration needed. Enable with:

```bash
openclaw hooks enable news-digest
```


path: hooks/openclaw/handler.js
content: /**
 * News Digest Bootstrap Hook for OpenClaw
 *
 * Reads slot configuration from data/config.json (user-defined schedule).
 * If no config exists, reminds the agent to run first-time setup.
 */

const path = require("path");
const fs = require("fs");

function loadConfig() {
  const skillRoot = path.join(__dirname, "..", "..");
  const dataDir =
    (process.env.NEWS_DIGEST_DATA_DIR || "").trim() ||
    path.join(skillRoot, "data");
  const configFile = path.join(dataDir, "config.json");

  if (!fs.existsSync(configFile)) return null;
  try {
    return JSON.parse(fs.readFileSync(configFile, "utf-8"));
  } catch {
    return null;
  }
}

function buildReminder() {
  const config = loadConfig();
  const now = new Date();
  const hour = now.getHours();
  const timeStr = now.toTimeString().slice(0, 5);

  const lines = ["## News Digest Reminder\n"];

  if (!config || !config.slots || config.slots.length === 0) {
    lines.push(
      `**Current time**: ${timeStr}`,
      "",
      "**No schedule configured yet.** This is likely first-time use.",
      "",
      "Please ask the user to set up their preferred push schedule:",
      "1. Ask how many daily pushes they want (e.g. 2-4)",
      "2. For each push, ask: time, topic/theme, keywords, priority rules",
      "3. Save using: `node {baseDir}/scripts/manage-config.mjs set-slot --name <name> --time <HH:MM> --topic <topic> --label <label> --keywords <kw1,kw2> --sources twitter,tavily,hackernews`",
      "",
      "Or create defaults first: `node {baseDir}/scripts/manage-config.mjs init`",
      "Then show config: `node {baseDir}/scripts/manage-config.mjs show`"
    );
    return lines.join("\n");
  }

  const activeSlot = config.slots.find((s) => {
    const w = s.window ?? [Math.max(0, (s.hour ?? 0) - 1), Math.min(23, (s.hour ?? 0) + 1)];
    return hour >= w[0] && hour <= w[1];
  });

  if (activeSlot) {
    lines.push(
      `**Current time**: ${timeStr}`,
      `**Active slot**: ${activeSlot.label} (${activeSlot.name})`,
      `**Topic**: ${activeSlot.topic}`,
      `**Keywords**: ${(activeSlot.keywords ?? []).join(", ")}`,
      "",
      "You should execute the news digest push workflow now:",
      "1. Read config: `node {baseDir}/scripts/manage-config.mjs show`",
      "2. Check recent feedback: `node {baseDir}/scripts/query.mjs feedback --days 3`",
      "3. Fetch data from configured sources, filter, summarize, format, store, and deliver",
      "",
      "Refer to the News Digest skill (SKILL.md) for the full workflow."
    );
  } else {
    const nextSlot = config.slots.find((s) => (s.hour ?? 0) > hour) || config.slots[0];
    lines.push(
      `**Current time**: ${timeStr}`,
      `**Next push**: ${nextSlot ? `${nextSlot.label} at ${nextSlot.time}` : "none scheduled"}`,
      "",
      "No push is due right now. The News Digest skill is available for:",
      "- View schedule: `node {baseDir}/scripts/manage-config.mjs show`",
      "- Query push history: `node {baseDir}/scripts/query.mjs pushes --days 3`",
      "- Review feedback: `node {baseDir}/scripts/query.mjs feedback --days 3`",
      "- Search past content: `node {baseDir}/scripts/query.mjs search --keyword \"keyword\" --days 7`",
      "- Manual on-demand push if requested by the user"
    );
  }

  return lines.join("\n");
}

const handler = async (event) => {
  if (!event || typeof event !== "object") return;
  if (event.type !== "agent" || event.action !== "bootstrap") return;
  if (!event.context || typeof event.context !== "object") return;

  const sessionKey = event.context.sessionKey ?? "";
  if (sessionKey.includes(":subagent:")) return;

  if (Array.isArray(event.context.bootstrapFiles)) {
    event.context.bootstrapFiles.push({
      path: "NEWS_DIGEST_REMINDER.md",
      content: buildReminder(),
      virtual: true,
    });
  }
};

module.exports = handler;
module.exports.default = handler;


path: references/setup-guide.md
content: # News Digest Setup Guide

## Prerequisites

- **Node.js** >= 18 (for built-in `fetch` support)
- **OpenClaw** installed and configured

## Installation

### Via ClawdHub (recommended)

```bash
clawdhub install news-digest
```

### Manual

```bash
git clone <repo-url> ~/.openclaw/skills/news-digest
```

Or copy the `news-digest-1.0.0/` directory to `~/.openclaw/skills/news-digest/`.

## Environment Variables

### Create a `.env` File

Create a `.env` file in your OpenClaw workspace (this is the preferred location for cron compatibility):

```bash
# ~/.openclaw/workspace/.env
TAVILY_API_KEY=tvly-xxxxxxxxxxxxx
XPOZ_API_KEY=your-xpoz-key
```

The `cron-trigger.sh` script automatically searches for `.env` in this order:
1. `~/.openclaw/workspace/.env` (recommended)
2. `{skill-root}/.env`
3. `~/.env`

### Required

#### `TAVILY_API_KEY`

Get your API key from [https://tavily.com](https://tavily.com). Required for Tavily Search and Extract.

### Recommended

#### `XPOZ_API_KEY`

Configure the Xpoz MCP service in OpenClaw for Twitter access. If not configured, the skill degrades gracefully — Twitter data is skipped and the agent relies on Tavily + Hacker News.

### Optional

#### `NEWS_DIGEST_DATA_DIR`

Override the default data storage directory. Default: `{skill-root}/data/`.

## Quick Environment Check

Verify everything is configured correctly:

```bash
node scripts/env-check.mjs
```

## First-Time Configuration

After installing, the agent will automatically detect that no `config.json` exists and prompt the user to set up their push schedule and topics.

You can also configure manually:

### Quick Setup (Defaults)

Create the standard 3-slot schedule (08:00 Finance, 12:00 AI, 18:00 General):

```bash
node scripts/manage-config.mjs init
```

### Custom Setup

Add slots one by one:

```bash
node scripts/manage-config.mjs set-slot \
  --name morning \
  --time 08:00 \
  --topic "Finance" \
  --label "金融早报" \
  --keywords "crypto,bitcoin,ethereum,finance,stock" \
  --priority "crypto > financial events > US stocks" \
  --sources "twitter,tavily,hackernews"

node scripts/manage-config.mjs set-slot \
  --name noon \
  --time 12:00 \
  --topic "AI / Agent" \
  --label "AI午报" \
  --keywords "AI,LLM,GPT,agent,machine learning" \
  --sources "twitter,tavily,hackernews"
```

### Verify Config

```bash
node scripts/manage-config.mjs show
```

### Modify Later

```bash
# Change time
node scripts/manage-config.mjs set-slot --name morning --time 07:30

# Change keywords
node scripts/manage-config.mjs set-slot --name morning --keywords "crypto,defi,bitcoin"

# Add a new slot
node scripts/manage-config.mjs set-slot --name afternoon --time 15:00 --topic "Markets" --label "盘后速递" --keywords "stocks,earnings"

# Remove a slot
node scripts/manage-config.mjs remove-slot --name evening

# Reset everything
node scripts/manage-config.mjs reset
```

## Enable the Bootstrap Hook

The hook reads `config.json` to inject a time-aware reminder at session start. If no config exists, it reminds the agent to run first-time setup.

```bash
# Step 1: Copy hook files to OpenClaw hooks directory
cp -r hooks/openclaw ~/.openclaw/hooks/news-digest

# Step 2: Enable the hook
openclaw hooks enable news-digest

# Step 3: Verify it's registered
openclaw hooks list
```

If `news-digest` appears in the list, the hook is active. It will fire on every `agent:bootstrap` event.

## Scheduling (Cron)

This skill is an **AI-assisted workflow** — the agent executes the push when triggered. Use cron to trigger it automatically at scheduled times.

### Important: Configure Before Scheduling

Cron entries should match the slots in your `config.json`. After changing your schedule via `manage-config.mjs`, update your crontab accordingly.

### Setup

The `cron-trigger.sh` script reads `config.json` for slot definitions, loads `.env`, and spawns an OpenClaw session.

**Step 1: Test the script first**

```bash
# Dry run — shows slot info from config without triggering
bash /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot morning --dry-run
```

Expected output:
```
[2026-03-06T10:00:00+08:00] slot=morning env=/home/user/.openclaw/workspace/.env [DRY RUN]
  Config: /path/to/news-digest-1.0.0/data/config.json
  Slot: morning (金融早报)
  Topic: Finance
  Would run: openclaw run --message "Execute news digest push for slot ..."
  TAVILY_API_KEY: set
  XPOZ_API_KEY: set
```

**Step 2: Add cron entries**

```bash
crontab -e
```

Add entries for each slot in your config. The times should match:

```cron
# Example for default 3-slot config — adjust to your config.json!
0 8 * * *  /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot morning >> /tmp/news-digest-cron.log 2>&1
0 12 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot noon >> /tmp/news-digest-cron.log 2>&1
0 18 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot evening >> /tmp/news-digest-cron.log 2>&1
```

If you added a custom slot (e.g. `afternoon` at 15:00), add:
```cron
0 15 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot afternoon >> /tmp/news-digest-cron.log 2>&1
```

**Step 3: Verify cron is running**

```bash
crontab -l
```

### Auto-Detection

If you omit `--slot`, the script reads `config.json` and auto-detects the active slot based on the current hour and each slot's `window` field:

```bash
# Will auto-detect based on config.json
bash /path/to/news-digest-1.0.0/scripts/cron-trigger.sh
```

### Why cron-trigger.sh?

Cron runs in a minimal shell environment — it does **not** load `.bashrc`, `.zshrc`, or `.env` files. The `cron-trigger.sh` script solves this by:
1. Sourcing the `.env` file before doing anything
2. Reading slot definitions from `config.json`
3. Logging every trigger to `data/logs/cron.log` for traceability
4. Providing `--dry-run` for safe testing

### Logs

Cron execution logs are written to:
- `{dataDir}/logs/cron.log` — internal trigger log (written by the script)
- `/tmp/news-digest-cron.log` — stdout/stderr capture (from the crontab redirect)

## Verify Installation

Test each script individually:

```bash
# Check environment
node scripts/env-check.mjs

# View config
node scripts/manage-config.mjs show

# Test Hacker News fetch
node scripts/fetch-hackernews.mjs "AI" --min-score 30 --hours 48 -n 5

# Test Tavily search (requires TAVILY_API_KEY)
node scripts/fetch-tavily.mjs "AI news today" --topic news --days 1 -n 3

# Test storage
echo '{"slot":"morning","topic":"test","items":[{"title":"Test","source":"manual"}],"usage":{"tavily_search":1,"hackernews":1}}' | node scripts/store-push.mjs

# Test query
node scripts/query.mjs pushes --date $(date +%Y-%m-%d)

# Test feedback
node scripts/add-feedback.mjs --slot morning "测试反馈"

# Test usage tracking
node scripts/track-usage.mjs today
node scripts/track-usage.mjs forecast
```

## Data Directory Structure

After running, the data directory will look like:

```
data/
├── config.json          ← User-defined schedule & topics
├── pushes/
│   └── 2026-03-06/
│       ├── morning.json
│       ├── noon.json
│       └── evening.json
├── feedback/
│   └── 2026-03-06/
│       ├── morning.json
│       └── noon.json
├── usage/
│   └── 2026-03.json
└── logs/
    └── cron.log
```

## Troubleshooting

### "Config file not found"

The schedule hasn't been configured yet. Either:
- Tell the agent to help you set up: "Help me configure my news digest schedule"
- Or run: `node scripts/manage-config.mjs init` for defaults

### "Missing TAVILY_API_KEY"

1. Check the variable is set: `echo $TAVILY_API_KEY`
2. Check your `.env` file exists: `cat ~/.openclaw/workspace/.env`
3. If running via cron, verify `cron-trigger.sh --dry-run` shows the key as "set"

### Cron not triggering

1. Verify cron is running: `crontab -l`
2. Check cron daemon: `pgrep cron` (Linux) or `launchctl list | grep cron` (macOS)
3. Check logs: `cat /tmp/news-digest-cron.log`
4. Check internal logs: `cat data/logs/cron.log`
5. Test manually: `bash scripts/cron-trigger.sh --slot morning --dry-run`

### Cron triggers but slot not found

The slot name in your crontab must match a slot name in `config.json`. Run:
```bash
node scripts/manage-config.mjs show
```
to see available slot names, then update your crontab.

### Cron triggers but API keys are missing

The `.env` file must be in one of these locations:
- `~/.openclaw/workspace/.env` (preferred)
- `{skill-root}/.env`
- `~/.env`

Format (no `export`, no quotes around values):
```
TAVILY_API_KEY=tvly-xxxxxxxxxxxxx
XPOZ_API_KEY=your-key-here
```

### No Hacker News results

- Try broader keywords or lower `--min-score`
- HN API is public but rate-limited; if you see errors, wait and retry
- Use `--hours 48` to expand the time window

### Empty push results

- Check that sources are returning data by running fetch scripts individually
- Lower the influence thresholds in your config
- Use a broader topic slot for testing

### Hook not firing

1. Verify hook is enabled: `openclaw hooks list`
2. Check hook file location: `ls ~/.openclaw/hooks/news-digest/`
3. Both `HOOK.md` and `handler.js` must be present
4. The hook only fires on `agent:bootstrap` — it won't fire for subagent sessions


path: scripts/add-feedback.mjs
content: #!/usr/bin/env node

import {
  readJSON,
  writeJSON,
  todayStr,
  nowISO,
  feedbackPath,
  generateId,
} from "./lib/storage.mjs";

function usage() {
  console.error(
    `Usage: add-feedback.mjs [--date YYYY-MM-DD] --slot morning|noon|evening [--item-id <id>] "feedback text"

Options:
  --date <date>     Target push date (default: today)
  --slot <slot>     Push slot: morning, noon, or evening (required)
  --item-id <id>    Optional: ID of a specific item to attach feedback to

Example:
  node add-feedback.mjs --slot morning "希望多一些DeFi相关的新闻"
  node add-feedback.mjs --date 2026-03-06 --slot noon --item-id item-002 "这条总结太简略了"`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

let date = todayStr();
let slot = null;
let itemId = null;
let feedbackText = null;

for (let i = 0; i < args.length; i++) {
  const a = args[i];
  if (a === "--date") {
    date = args[++i];
    continue;
  }
  if (a === "--slot") {
    slot = args[++i];
    continue;
  }
  if (a === "--item-id") {
    itemId = args[++i];
    continue;
  }
  if (a.startsWith("-")) {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
  feedbackText = a;
}

if (!slot || !["morning", "noon", "evening"].includes(slot)) {
  console.error(
    `Error: --slot is required and must be one of: morning, noon, evening. Got: "${slot}"`
  );
  process.exit(1);
}

if (!feedbackText) {
  console.error("Error: feedback text is required as the last argument.");
  process.exit(1);
}

const filePath = feedbackPath(date, slot);
const dateCompact = date.replace(/-/g, "");
const pushId = `push-${dateCompact}-${slot}`;

let record = readJSON(filePath);
if (!record) {
  record = {
    push_id: pushId,
    feedbacks: [],
  };
}

const fb = {
  id: generateId("fb"),
  item_id: itemId || null,
  text: feedbackText,
  created_at: nowISO(),
};

record.feedbacks.push(fb);
writeJSON(filePath, record);

console.log(`## Feedback Recorded\n`);
console.log(`- **ID**: ${fb.id}`);
console.log(`- **Push**: ${pushId}`);
if (itemId) {
  console.log(`- **Item**: ${itemId}`);
}
console.log(`- **Text**: ${feedbackText}`);
console.log(`- **Path**: ${filePath}`);


path: scripts/cron-trigger.sh
content: #!/usr/bin/env bash
#
# Cron trigger for News Digest skill.
# Loads .env, reads config.json for dynamic slot definitions, and spawns
# an OpenClaw session.
#
# Usage:
#   bash cron-trigger.sh [--slot <name>] [--dry-run]
#
# Slots are read from data/config.json. If --slot is omitted, the script
# auto-detects the active slot based on the current hour.
#
# Crontab example (times should match your config.json):
#   0 8 * * *  /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot morning
#   0 12 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot noon
#   0 18 * * * /path/to/news-digest-1.0.0/scripts/cron-trigger.sh --slot evening

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
SKILL_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
DATA_DIR="${NEWS_DIGEST_DATA_DIR:-$SKILL_ROOT/data}"
CONFIG_FILE="$DATA_DIR/config.json"
LOG_DIR="$DATA_DIR/logs"

SLOT=""
DRY_RUN=false

while [[ $# -gt 0 ]]; do
  case "$1" in
    --slot)   SLOT="$2"; shift 2 ;;
    --dry-run) DRY_RUN=true; shift ;;
    -h|--help)
      echo "Usage: cron-trigger.sh [--slot <name>] [--dry-run]"
      echo "Slots are defined in $CONFIG_FILE"
      exit 0 ;;
    *) echo "Unknown arg: $1"; exit 2 ;;
  esac
done

# --- Load .env ---
ENV_CANDIDATES=(
  "$HOME/.openclaw/workspace/.env"
  "$SKILL_ROOT/.env"
  "$HOME/.env"
)

ENV_LOADED=""
for candidate in "${ENV_CANDIDATES[@]}"; do
  if [[ -f "$candidate" ]]; then
    set -a
    source "$candidate"
    set +a
    ENV_LOADED="$candidate"
    break
  fi
done

# --- Read config.json ---
if [[ ! -f "$CONFIG_FILE" ]]; then
  echo "[$(date -Iseconds)] ERROR: Config file not found: $CONFIG_FILE"
  echo "Run 'node $SCRIPT_DIR/manage-config.mjs init' to create default config."
  exit 1
fi

# Use node to extract slot info from config (avoids jq dependency)
read_slot_info() {
  node -e "
    const config = JSON.parse(require('fs').readFileSync('$CONFIG_FILE', 'utf-8'));
    const slot = config.slots.find(s => s.name === '$1');
    if (!slot) { process.exit(1); }
    console.log(JSON.stringify(slot));
  " 2>/dev/null
}

list_slots() {
  node -e "
    const config = JSON.parse(require('fs').readFileSync('$CONFIG_FILE', 'utf-8'));
    config.slots.forEach(s => console.log(s.name + '|' + (s.hour ?? 0) + '|' + (s.window ? s.window[0] : 0) + '|' + (s.window ? s.window[1] : 23) + '|' + (s.label ?? s.name) + '|' + (s.topic ?? '')));
  " 2>/dev/null
}

# --- Auto-detect slot from current hour if not specified ---
if [[ -z "$SLOT" ]]; then
  HOUR=$(date +%H)
  FOUND=""
  while IFS='|' read -r sname shour swin_start swin_end slabel stopic; do
    if (( HOUR >= swin_start && HOUR <= swin_end )); then
      FOUND="$sname"
      break
    fi
  done < <(list_slots)

  if [[ -z "$FOUND" ]]; then
    echo "[$(date -Iseconds)] No active slot for hour $HOUR. Exiting."
    exit 0
  fi
  SLOT="$FOUND"
fi

# --- Get slot details from config ---
SLOT_JSON=$(read_slot_info "$SLOT" 2>/dev/null) || {
  echo "Error: slot '$SLOT' not found in config."
  echo "Available slots:"
  list_slots | while IFS='|' read -r sname shour sws swe slabel stopic; do
    echo "  $sname ($slabel) at hour $shour"
  done
  exit 1
}

TOPIC_LABEL=$(node -e "console.log(JSON.parse(process.argv[1]).label ?? '')" "$SLOT_JSON" 2>/dev/null)
TOPIC=$(node -e "console.log(JSON.parse(process.argv[1]).topic ?? '')" "$SLOT_JSON" 2>/dev/null)

MESSAGE="Execute news digest push for slot \"$SLOT\" ($TOPIC_LABEL — $TOPIC). Read config first with: node $SCRIPT_DIR/manage-config.mjs show"

# --- Ensure log directory ---
mkdir -p "$LOG_DIR"

TIMESTAMP=$(date -Iseconds)
LOG_ENTRY="[$TIMESTAMP] slot=$SLOT env=$ENV_LOADED"

if $DRY_RUN; then
  echo "$LOG_ENTRY [DRY RUN]"
  echo "  Config: $CONFIG_FILE"
  echo "  Slot: $SLOT ($TOPIC_LABEL)"
  echo "  Topic: $TOPIC"
  echo "  Would run: openclaw run --message \"$MESSAGE\""
  echo "  TAVILY_API_KEY: ${TAVILY_API_KEY:+set}"
  echo "  XPOZ_API_KEY: ${XPOZ_API_KEY:+set}"
  exit 0
fi

echo "$LOG_ENTRY" >> "$LOG_DIR/cron.log"

# --- Trigger OpenClaw ---
if command -v openclaw &>/dev/null; then
  openclaw run --message "$MESSAGE" 2>&1 | tee -a "$LOG_DIR/cron.log" || {
    echo "[$TIMESTAMP] ERROR: openclaw run failed (exit $?)" >> "$LOG_DIR/cron.log"
    exit 1
  }
else
  echo "[$TIMESTAMP] ERROR: openclaw not found in PATH" >> "$LOG_DIR/cron.log"
  echo "Error: openclaw command not found. Ensure OpenClaw is installed and in PATH."
  exit 1
fi

echo "[$TIMESTAMP] Completed: $SLOT ($TOPIC_LABEL)" >> "$LOG_DIR/cron.log"


path: scripts/env-check.mjs
content: #!/usr/bin/env node

import { existsSync } from "node:fs";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";
import { getDataDir, loadConfig, configPath } from "./lib/storage.mjs";

const __dirname = dirname(fileURLToPath(import.meta.url));
const SKILL_ROOT = join(__dirname, "..");

const ENV_CANDIDATES = [
  join(process.env.HOME ?? "~", ".openclaw", "workspace", ".env"),
  join(SKILL_ROOT, ".env"),
  join(process.env.HOME ?? "~", ".env"),
];

function checkEnvVar(name, required) {
  const val = (process.env[name] ?? "").trim();
  if (val) return `OK (set, ${val.slice(0, 4)}...)`;
  return required ? "MISSING" : "not set (optional)";
}

console.log("## Environment Check\n");

console.log(`- **TAVILY_API_KEY**: ${checkEnvVar("TAVILY_API_KEY", true)}`);
console.log(`- **XPOZ_API_KEY**: ${checkEnvVar("XPOZ_API_KEY", false)}`);
console.log(`- **NEWS_DIGEST_DATA_DIR**: ${(process.env.NEWS_DIGEST_DATA_DIR ?? "").trim() || "(default)"}`);

console.log();
console.log(`- **Node.js**: ${process.version}`);
console.log(`- **Skill root**: ${SKILL_ROOT}`);
console.log(`- **Data dir**: ${getDataDir()} (${existsSync(getDataDir()) ? "exists" : "will be created on first use"})`);

const envFile = ENV_CANDIDATES.find((p) => existsSync(p));
console.log(`- **.env file**: ${envFile ? `found at ${envFile}` : "not found"}`);

const config = loadConfig();
if (config && config.slots) {
  const slotSummary = config.slots.map((s) => `${s.name} (${s.time})`).join(", ");
  console.log(`- **Config**: ${configPath()} — ${config.slots.length} slot(s): ${slotSummary}`);
} else {
  console.log(`- **Config**: not found — run \`manage-config.mjs init\` or ask the agent to set up`);
}

let issues = 0;
if (!(process.env.TAVILY_API_KEY ?? "").trim()) {
  console.log("\n**Warning**: TAVILY_API_KEY is not set. Tavily search/extract will fail.");
  issues++;
}
if (!(process.env.XPOZ_API_KEY ?? "").trim()) {
  console.log("\n**Note**: XPOZ_API_KEY is not set. Twitter source will be skipped (graceful degradation).");
}
if (!config) {
  console.log("\n**Warning**: No schedule config found. The agent needs to set up push schedule with the user.");
  issues++;
}

if (issues === 0) {
  console.log("\nAll checks passed.");
} else {
  console.log(`\n${issues} issue(s) found. See above.`);
}


path: scripts/extract-tavily.mjs
content: #!/usr/bin/env node

function usage() {
  console.error(`Usage: extract-tavily.mjs "url1" ["url2" ...]`);
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const urls = args.filter((a) => !a.startsWith("-"));

if (urls.length === 0) {
  console.error("No URLs provided");
  usage();
}

const [REDACTED_SECRET] ?? "").trim();
if (!apiKey) {
  console.error("Missing TAVILY_API_KEY. Get one from https://tavily.com");
  process.exit(1);
}

const resp = await fetch("https://api.tavily.com/extract", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ [REDACTED_SECRET], urls }),
});

if (!resp.ok) {
  const text = await resp.text().catch(() => "");
  console.error(`Tavily Extract failed (${resp.status}): ${text}`);
  process.exit(1);
}

const data = await resp.json();
const results = data.results ?? [];
const failed = data.failed_results ?? [];

for (const r of results) {
  const url = String(r?.url ?? "").trim();
  const content = String(r?.raw_content ?? "").trim();

  console.log(`# ${url}\n`);
  console.log(content || "(no content extracted)");
  console.log("\n---\n");
}

if (failed.length > 0) {
  console.log("## Failed URLs\n");
  for (const f of failed) {
    console.log(`- ${f.url}: ${f.error}`);
  }
}


path: scripts/fetch-hackernews.mjs
content: #!/usr/bin/env node

const HN_API = "https://hacker-news.firebaseio.com/v0";

function usage() {
  console.error(
    `Usage: fetch-hackernews.mjs ["query"] [-n 20] [--min-score 50] [--hours 24]

Arguments:
  "query"         Keywords to filter by (case-insensitive, OR-separated). Empty string = no filter.
  -n <count>      Max items to return (default: 20, fetches up to 200 candidates)
  --min-score <n> Minimum HN score threshold (default: 10)
  --hours <h>     Only include stories from the last h hours (default: 24)`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args[0] === "-h" || args[0] === "--help") usage();

let query = "";
let maxResults = 20;
let minScore = 10;
let hours = 24;

let positionalDone = false;
for (let i = 0; i < args.length; i++) {
  const a = args[i];
  if (a === "-n") {
    maxResults = Number.parseInt(args[++i] ?? "20", 10);
    positionalDone = true;
    continue;
  }
  if (a === "--min-score") {
    minScore = Number.parseInt(args[++i] ?? "10", 10);
    positionalDone = true;
    continue;
  }
  if (a === "--hours") {
    hours = Number.parseInt(args[++i] ?? "24", 10);
    positionalDone = true;
    continue;
  }
  if (a.startsWith("-")) {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
  if (!positionalDone) {
    query = a;
    positionalDone = true;
  }
}

const cutoff = Math.floor(Date.now() / 1000) - hours * 3600;

const keywords = query
  .split(/\s+OR\s+/i)
  .map((k) => k.trim().toLowerCase())
  .filter(Boolean);

function matchesQuery(title) {
  if (keywords.length === 0) return true;
  const lower = title.toLowerCase();
  return keywords.some((kw) => lower.includes(kw));
}

async function fetchJSON(url) {
  const resp = await fetch(url);
  if (!resp.ok) throw new Error(`HN API error: ${resp.status}`);
  return resp.json();
}

const topIds = await fetchJSON(`${HN_API}/topstories.json`);
const candidateIds = topIds.slice(0, 200);

const BATCH = 20;
const items = [];

for (let i = 0; i < candidateIds.length; i += BATCH) {
  const batch = candidateIds.slice(i, i + BATCH);
  const fetched = await Promise.all(
    batch.map((id) => fetchJSON(`${HN_API}/item/${id}.json`).catch(() => null))
  );

  for (const item of fetched) {
    if (!item || item.type !== "story" || item.dead || item.deleted) continue;
    if ((item.score ?? 0) < minScore) continue;
    if ((item.time ?? 0) < cutoff) continue;
    if (!matchesQuery(item.title ?? "")) continue;
    items.push(item);
  }

  if (items.length >= maxResults * 2) break;
}

items.sort((a, b) => (b.score ?? 0) - (a.score ?? 0));
const selected = items.slice(0, maxResults);

if (selected.length === 0) {
  console.log(
    `## Hacker News Results\n\nNo stories found matching criteria (query="${query}", min_score=${minScore}, hours=${hours}).`
  );
  process.exit(0);
}

console.log(
  `## Hacker News Results (${selected.length} stories)\n`
);

for (const s of selected) {
  const title = s.title ?? "(untitled)";
  const url = s.url || `https://news.ycombinator.com/item?id=${s.id}`;
  const hnLink = `https://news.ycombinator.com/item?id=${s.id}`;
  const score = s.score ?? 0;
  const comments = s.descendants ?? 0;
  const age = new Date((s.time ?? 0) * 1000).toISOString();
  const by = s.by ?? "unknown";

  console.log(`- **${title}** (score: ${score}, comments: ${comments})`);
  console.log(`  By: ${by} | ${age}`);
  console.log(`  ${url}`);
  if (url !== hnLink) {
    console.log(`  HN: ${hnLink}`);
  }
  console.log();
}


path: scripts/fetch-tavily.mjs
content: #!/usr/bin/env node

function usage() {
  console.error(
    `Usage: fetch-tavily.mjs "query" [-n 5] [--deep] [--topic general|news] [--days 7]`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const query = args[0];
let n = 5;
let searchDepth = "basic";
let topic = "general";
let days = null;

for (let i = 1; i < args.length; i++) {
  const a = args[i];
  if (a === "-n") {
    n = Number.parseInt(args[i + 1] ?? "5", 10);
    i++;
    continue;
  }
  if (a === "--deep") {
    searchDepth = "advanced";
    continue;
  }
  if (a === "--topic") {
    topic = args[i + 1] ?? "general";
    i++;
    continue;
  }
  if (a === "--days") {
    days = Number.parseInt(args[i + 1] ?? "7", 10);
    i++;
    continue;
  }
  console.error(`Unknown arg: ${a}`);
  usage();
}

const [REDACTED_SECRET] ?? "").trim();
if (!apiKey) {
  console.error("Missing TAVILY_API_KEY. Get one from https://tavily.com");
  process.exit(1);
}

const body = {
  [REDACTED_SECRET],
  query,
  search_depth: searchDepth,
  topic,
  max_results: Math.max(1, Math.min(n, 20)),
  include_answer: true,
  include_raw_content: false,
};

if (topic === "news" && days) {
  body.days = days;
}

const resp = await fetch("https://api.tavily.com/search", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify(body),
});

if (!resp.ok) {
  const text = await resp.text().catch(() => "");
  console.error(`Tavily Search failed (${resp.status}): ${text}`);
  process.exit(1);
}

const data = await resp.json();

if (data.answer) {
  console.log("## Answer\n");
  console.log(data.answer);
  console.log("\n---\n");
}

const results = (data.results ?? []).slice(0, n);
console.log("## Sources\n");

for (const r of results) {
  const title = String(r?.title ?? "").trim();
  const url = String(r?.url ?? "").trim();
  const content = String(r?.content ?? "").trim();
  const score = r?.score
    ? ` (relevance: ${(r.score * 100).toFixed(0)}%)`
    : "";
  const published = r?.published_date ? ` | ${r.published_date}` : "";

  if (!title || !url) continue;
  console.log(`- **${title}**${score}${published}`);
  console.log(`  ${url}`);
  if (content) {
    console.log(
      `  ${content.slice(0, 400)}${content.length > 400 ? "..." : ""}`
    );
  }
  console.log();
}


path: scripts/keyword-analyzer.mjs
content: #!/usr/bin/env node

/**
 * Keyword Analyzer
 * Analyzes recent push results to extract content for AI analysis.
 * Helps identify potential new keywords based on what content has been captured.
 *
 * Usage:
 *   node keyword-analyzer.mjs --slot morning --days 7
 *   node keyword-analyzer.mjs --date 2026-03-06 --slot morning
 */

import {
  readJSON,
  pushPath,
  listDates,
  loadConfig,
} from "./lib/storage.mjs";

function usage() {
  console.error(
    `Usage: keyword-analyzer.mjs [options]

Options:
  --slot <slot>     Slot name: morning, noon, evening (required)
  --date <date>     Target date (YYYY-MM-DD), defaults to today
  --days <n>        Look back n days (default: 7)

Examples:
  node keyword-analyzer.mjs --slot morning --days 7
  node keyword-analyzer.mjs --date 2026-03-06 --slot morning
`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

let slot = null;
let date = null;
let days = 7;

for (let i = 0; i < args.length; i++) {
  const a = args[i];
  if (a === "--slot") {
    slot = args[++i];
    continue;
  }
  if (a === "--date") {
    date = args[++i];
    continue;
  }
  if (a === "--days") {
    days = Number.parseInt(args[++i] ?? "7", 10);
    continue;
  }
  if (a.startsWith("-")) {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
}

if (!slot) {
  console.error("Error: --slot is required.");
  usage();
}

// Load current config to get existing keywords
const config = loadConfig();
const slotConfig = config?.slots?.find((s) => s.name === slot);
const currentKeywords = slotConfig?.keywords ?? [];

function dateRange(numDays, targetDate = null) {
  const dates = [];
  const now = targetDate ? new Date(targetDate) : new Date();
  for (let i = 0; i < numDays; i++) {
    const d = new Date(now);
    d.setDate(d.getDate() - i);
    const y = d.getFullYear();
    const m = String(d.getMonth() + 1).padStart(2, "0");
    const day = String(d.getDate()).padStart(2, "0");
    dates.push(`${y}-${m}-${day}`);
  }
  return dates;
}

function analyzeContent(records) {
  const allItems = [];

  for (const record of records) {
    if (!record?.items) continue;

    for (const item of record.items) {
      allItems.push({
        date: record.date,
        slot: record.slot,
        source: item.category || item.source || "unknown",
        title: item.title || "",
        summary: item.summary || "",
        url: item.url || "",
        // Extract meaningful text content
        content: [
          item.title,
          item.summary,
          item.raw_excerpt,
          item.category,
          item.author,
        ]
          .filter(Boolean)
          .join(" | "),
      });
    }
  }

  return allItems;
}

function extractTopics(items) {
  // Simple keyword/topic extraction based on content
  const topics = new Map();

  for (const item of items) {
    const text = (item.title + " " + (item.summary || "")).toLowerCase();

    // Common tech/finance keywords to look for
    const patterns = [
      /\b(ai|artificial intelligence|machine learning|llm|large language model|gpt|claude|openai|anthropic)\b/gi,
      /\b(crypto|bitcoin|btc|ethereum|eth|defi|blockchain|web3|nft)\b/gi,
      /\b(stock|market|trading|invest|finance|economy|fed|interest rate)\b/gi,
      /\b(tech|technology|software|startup|venture|funding)\b/gi,
      /\b(agent|agentic|auto gpt|autonomous|automation)\b/gi,
    ];

    for (const pattern of patterns) {
      const matches = text.match(pattern);
      if (matches) {
        for (const match of matches) {
          const kw = match.toLowerCase();
          const count = topics.get(kw) || { count: 0, sources: new Set(), dates: new Set() };
          count.count++;
          count.sources.add(item.source);
          count.dates.add(item.date);
          topics.set(kw, count);
        }
      }
    }
  }

  // Convert to sorted array
  const sorted = Array.from(topics.entries())
    .map(([keyword, data]) => ({
      keyword,
      count: data.count,
      sources: Array.from(data.sources),
      dates: Array.from(data.dates),
    }))
    .sort((a, b) => b.count - a.count);

  return sorted;
}

// Main execution
const dates = date ? [date] : dateRange(days);
const records = [];

console.log(`## Keyword Analysis for "${slot}"\n`);
console.log(`Date range: ${dates[dates.length - 1]} to ${dates[0]}`);
console.log(`Current keywords: ${currentKeywords.join(", ") || "(none)"}\n`);

// Collect push records
for (const d of dates) {
  const path = pushPath(d, slot);
  const record = readJSON(path);
  if (record) {
    records.push({ ...record, date: d, slot });
  }
}

if (records.length === 0) {
  console.log("No push records found for the specified criteria.");
  process.exit(0);
}

console.log(`Found ${records.length} push records.\n`);

// Analyze content
const allItems = analyzeContent(records);

console.log(`Total items analyzed: ${allItems.length}\n`);

// Output structured data for AI consumption
const output = {
  meta: {
    slot,
    date_range: { start: dates[dates.length - 1], end: dates[0] },
    current_keywords: currentKeywords,
    push_count: records.length,
    total_items: allItems.length,
  },
  // Top items by source for AI to review
  sample_items: allItems.slice(0, 50).map((item) => ({
    date: item.date,
    source: item.source,
    title: item.title,
    summary: item.summary?.slice(0, 200),
    url: item.url,
  })),
  // All items for deeper analysis
  all_items: allItems.map((item) => ({
    date: item.date,
    source: item.source,
    content: item.content,
  })),
};

console.log("### Structured Output (for AI analysis)\n");
console.log(JSON.stringify(output, null, 2));


path: scripts/keyword-suggester.mjs
content: #!/usr/bin/env node

/**
 * Keyword Suggester
 * Analyzes user feedback to generate keyword adjustment suggestions.
 *
 * Usage:
 *   node keyword-suggester.mjs --slot morning --days 7
 *   node keyword-suggester.mjs --date 2026-03-06 --slot morning
 */

import {
  readJSON,
  feedbackPath,
  listDates,
  loadConfig,
} from "./lib/storage.mjs";

function usage() {
  console.error(
    `Usage: keyword-suggester.mjs [options]

Options:
  --slot <slot>     Slot name: morning, noon, evening (required)
  --date <date>     Target date (YYYY-MM-DD), defaults to today
  --days <n>        Look back n days (default: 7)

Examples:
  node keyword-suggester.mjs --slot morning --days 7
  node keyword-suggester.mjs --date 2026-03-06 --slot morning
`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

let slot = null;
let date = null;
let days = 7;

for (let i = 0; i < args.length; i++) {
  const a = args[i];
  if (a === "--slot") {
    slot = args[++i];
    continue;
  }
  if (a === "--date") {
    date = args[++i];
    continue;
  }
  if (a === "--days") {
    days = Number.parseInt(args[++i] ?? "7", 10);
    continue;
  }
  if (a.startsWith("-")) {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
}

if (!slot) {
  console.error("Error: --slot is required.");
  usage();
}

// Load current config to get existing keywords
const config = loadConfig();
const slotConfig = config?.slots?.find((s) => s.name === slot);
const currentKeywords = slotConfig?.keywords ?? [];

function dateRange(numDays, targetDate = null) {
  const dates = [];
  const now = targetDate ? new Date(targetDate) : new Date();
  for (let i = 0; i < numDays; i++) {
    const d = new Date(now);
    d.setDate(d.getDate() - i);
    const y = d.getFullYear();
    const m = String(d.getMonth() + 1).padStart(2, "0");
    const day = String(d.getDate()).padStart(2, "0");
    dates.push(`${y}-${m}-${day}`);
  }
  return dates;
}

function analyzeFeedback(feedbacks) {
  const positiveSignals = [];
  const negativeSignals = [];
  const neutralSignals = [];

  // Keywords commonly associated with positive/negative feedback
  // Order matters: check negative patterns that override positive ones first
  const negativePatterns = [
    // Chinese negative patterns
    /不太|不很|不怎|不喜|不感|不想要|不感兴趣|无聊|没用|太短|太长|不够|太少|不想|不要|删掉|移除|没意思|没兴趣|腻了|烦了/,
    // English negative patterns
    /not (interested|like|want|need)|dislike|boring|useless|too short|too long|not enough|don'?t want|remove|tired of|stop/,
  ];

  const positivePatterns = [
    // Chinese positive patterns
    /喜欢|感兴趣|不错|很好|棒|感谢|有用|有帮助|正是|正是我想|正是我需|及时|实用|希望|想要|多要/,
    // English positive patterns
    /good|great|love|like|interested|useful|helpful|thanks|thank|exactly|what I need|good job|more|want|need|please add/,
  ];

  for (const fb of feedbacks) {
    const text = fb.text;

    let isPositive = false;
    let isNegative = false;

    // Check negative patterns first (they override positive)
    for (const pattern of negativePatterns) {
      if (pattern.test(text)) {
        isNegative = true;
        break;
      }
    }

    // Only check positive if not negative
    if (!isNegative) {
      for (const pattern of positivePatterns) {
        if (pattern.test(text)) {
          isPositive = true;
          break;
        }
      }
    }

    const signal = {
      id: fb.id,
      text: fb.text,
      item_id: fb.item_id,
      date: fb.created_at,
    };

    if (isPositive) {
      positiveSignals.push(signal);
    } else if (isNegative) {
      negativeSignals.push(signal);
    } else {
      neutralSignals.push(signal);
    }
  }

  return { positiveSignals, negativeSignals, neutralSignals };
}

// Main execution
const dates = date ? [date] : dateRange(days);
const allFeedbacks = [];

console.log(`## Keyword Suggestion Analysis for "${slot}"\n`);
console.log(`Date range: ${dates[dates.length - 1]} to ${dates[0]}`);
console.log(`Current keywords: ${currentKeywords.join(", ") || "(none)"}\n`);

// Collect feedback records
for (const d of dates) {
  const path = feedbackPath(d, slot);
  const record = readJSON(path);
  if (record?.feedbacks) {
    for (const fb of record.feedbacks) {
      allFeedbacks.push({ ...fb, date: d });
    }
  }
}

if (allFeedbacks.length === 0) {
  console.log("No feedback found for the specified criteria.");
  console.log("\n### AI Prompt Template\n");
  console.log(
`Analyze recent push content and suggest new keywords:

Current keywords: ${currentKeywords.join(", ")}

Analyze the push content and identify 3-5 potential new keywords that could help capture more relevant content. Exclude keywords already in the current list.

Output format (JSON):
{
  "suggestions": [
    { "keyword": "defi", "reason": "Frequently appearing in recent news", "priority": "high" }
  ]
}
`
  );
  process.exit(0);
}

// Analyze feedback
const analysis = analyzeFeedback(allFeedbacks);

console.log(`### Feedback Summary\n`);
console.log(`Total feedback: ${allFeedbacks.length}`);
console.log(`- Positive signals: ${analysis.positiveSignals.length}`);
console.log(`- Negative signals: ${analysis.negativeSignals.length}`);
console.log(`- Neutral: ${analysis.neutralSignals.length}\n`);

if (analysis.positiveSignals.length > 0) {
  console.log("#### Positive Feedback (user likes content):\n");
  for (const fb of analysis.positiveSignals) {
    console.log(`- [${fb.date}] ${fb.text}`);
  }
  console.log();
}

if (analysis.negativeSignals.length > 0) {
  console.log("#### Negative Feedback (user dislikes content):\n");
  for (const fb of analysis.negativeSignals) {
    console.log(`- [${fb.date}] ${fb.text}`);
  }
  console.log();
}

// Generate structured output for AI
const output = {
  meta: {
    slot,
    date_range: { start: dates[dates.length - 1], end: dates[0] },
    current_keywords: currentKeywords,
    total_feedback: allFeedbacks.length,
  },
  analysis: {
    positive_count: analysis.positiveSignals.length,
    negative_count: analysis.negativeSignals.length,
    neutral_count: analysis.neutralSignals.length,
  },
  positive_signals: analysis.positiveSignals.map((fb) => ({
    text: fb.text,
    item_id: fb.item_id,
    date: fb.date,
  })),
  negative_signals: analysis.negativeSignals.map((fb) => ({
    text: fb.text,
    item_id: fb.item_id,
    date: fb.date,
  })),
  all_feedback: allFeedbacks.map((fb) => ({
    text: fb.text,
    item_id: fb.item_id,
    date: fb.date,
  })),
};

console.log("### Structured Output (for AI analysis)\n");
console.log(JSON.stringify(output, null, 2));

// AI prompt template
console.log("\n### AI Prompt Template\n");
console.log(
`Analyze user feedback and suggest keyword adjustments:

Current keywords: ${currentKeywords.join(", ")}

Feedback analysis:
- Positive signals (${analysis.positiveSignals.length}): ${analysis.positiveSignals.map(f => f.text).join("; ")}
- Negative signals (${analysis.negativeSignals.length}): ${analysis.negativeSignals.map(f => f.text).join("; ")}

Based on this feedback, generate keyword adjustment suggestions.

Output format (JSON):
{
  "suggestions": [
    {
      "action": "add|remove|increase_weight|decrease_weight",
      "keyword": "defi",
      "reason": "User requested more DeFi content",
      "priority": "high|medium|low"
    }
  ],
  "summary": "Brief explanation of the recommended changes"
}
`
);


path: scripts/lib/storage.mjs
content: import { readFileSync, writeFileSync, mkdirSync, existsSync, readdirSync } from "node:fs";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";

const __dirname = dirname(fileURLToPath(import.meta.url));
const SKILL_ROOT = join(__dirname, "..", "..");

export function getDataDir() {
  return (process.env.NEWS_DIGEST_DATA_DIR ?? "").trim() || join(SKILL_ROOT, "data");
}

export function ensureDir(dirPath) {
  if (!existsSync(dirPath)) {
    mkdirSync(dirPath, { recursive: true });
  }
}

export function readJSON(filePath) {
  if (!existsSync(filePath)) return null;
  try {
    return JSON.parse(readFileSync(filePath, "utf-8"));
  } catch {
    return null;
  }
}

export function writeJSON(filePath, data) {
  ensureDir(dirname(filePath));
  writeFileSync(filePath, JSON.stringify(data, null, 2) + "\n", "utf-8");
}

export function todayStr() {
  const now = new Date();
  const y = now.getFullYear();
  const m = String(now.getMonth() + 1).padStart(2, "0");
  const d = String(now.getDate()).padStart(2, "0");
  return `${y}-${m}-${d}`;
}

export function nowISO() {
  return new Date().toISOString();
}

export function pushDir(date) {
  return join(getDataDir(), "pushes", date);
}

export function feedbackDir(date) {
  return join(getDataDir(), "feedback", date);
}

export function pushPath(date, slot) {
  return join(pushDir(date), `${slot}.json`);
}

export function feedbackPath(date, slot) {
  return join(feedbackDir(date), `${slot}.json`);
}

/**
 * List date directories under a given subdirectory of data (e.g. "pushes" or "feedback").
 * Returns sorted date strings descending (newest first).
 */
export function listDates(subDir) {
  const base = join(getDataDir(), subDir);
  if (!existsSync(base)) return [];
  return readdirSync(base, { withFileTypes: true })
    .filter((d) => d.isDirectory() && /^\d{4}-\d{2}-\d{2}$/.test(d.name))
    .map((d) => d.name)
    .sort()
    .reverse();
}

export function configPath() {
  return join(getDataDir(), "config.json");
}

export function loadConfig() {
  return readJSON(configPath());
}

export function saveConfig(config) {
  config.updated_at = nowISO();
  writeJSON(configPath(), config);
}

export function currentMonth() {
  const now = new Date();
  const y = now.getFullYear();
  const m = String(now.getMonth() + 1).padStart(2, "0");
  return `${y}-${m}`;
}

export function usagePath(yearMonth) {
  return join(getDataDir(), "usage", `${yearMonth}.json`);
}

/**
 * Generate a short ID: prefix-YYYYMMDD-NNN
 */
export function generateId(prefix) {
  const now = new Date();
  const dateStr =
    String(now.getFullYear()) +
    String(now.getMonth() + 1).padStart(2, "0") +
    String(now.getDate()).padStart(2, "0");
  const rand = String(Math.floor(Math.random() * 1000)).padStart(3, "0");
  return `${prefix}-${dateStr}-${rand}`;
}


path: scripts/manage-config.mjs
content: #!/usr/bin/env node

import {
  loadConfig,
  saveConfig,
  configPath,
  nowISO,
} from "./lib/storage.mjs";

function usage() {
  console.error(
    `Usage: manage-config.mjs <command> [options]

Commands:
  show                 Display current configuration
  init                 Create default configuration (will not overwrite existing)
  set-slot             Add or update a slot
  remove-slot          Remove a slot by name
  set-defaults         Update default settings
  reset                Reset to factory defaults (overwrites existing!)

set-slot options (all required for new slot, optional for update):
  --name <name>        Slot identifier (e.g. morning, noon, evening, custom-name)
  --time <HH:MM>       Push time (e.g. 08:00)
  --topic <topic>      Topic label (e.g. "Finance", "AI/Agent")
  --label <label>      Display label (e.g. "金融早报")
  --keywords <kw>      Comma-separated keywords
  --priority <desc>    Priority description (free text)
  --sources <src>      Comma-separated sources: twitter,tavily,hackernews

remove-slot options:
  --name <name>        Slot name to remove

set-defaults options:
  --summary-length <range>    e.g. "100-300"
  --summary-language <lang>   e.g. "zh-CN"
  --items-per-push <range>    e.g. "3-10"
  --time-window-hours <n>     e.g. 24

Examples:
  node manage-config.mjs show
  node manage-config.mjs init
  node manage-config.mjs set-slot --name morning --time 08:30 --topic Finance --label 金融早报 --keywords "crypto,bitcoin,stock" --sources "twitter,tavily,hackernews"
  node manage-config.mjs remove-slot --name evening
  node manage-config.mjs set-defaults --summary-language en
  node manage-config.mjs reset`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const command = args[0];

function parseArgs() {
  const opts = {};
  for (let i = 1; i < args.length; i++) {
    const a = args[i];
    if (a.startsWith("--")) {
      const key = a.slice(2);
      const val = args[++i];
      if (val === undefined) { console.error(`Missing value for ${a}`); process.exit(2); }
      opts[key] = val;
    }
  }
  return opts;
}

function defaultConfig() {
  return {
    slots: [
      {
        name: "morning",
        time: "08:00",
        hour: 8,
        window: [7, 9],
        topic: "Finance",
        label: "金融早报",
        keywords: ["crypto", "bitcoin", "ethereum", "finance", "stock"],
        priority: "crypto > financial events > US stocks = HK stocks > A-shares",
        sources: ["twitter", "tavily", "hackernews"],
        influence_thresholds: { twitter_likes: 100, hackernews_score: 50, tavily_relevance: 60 },
      },
      {
        name: "noon",
        time: "12:00",
        hour: 12,
        window: [11, 13],
        topic: "AI / Agent",
        label: "AI午报",
        keywords: ["AI", "LLM", "GPT", "agent", "machine learning"],
        priority: "LLM progress > AI applications > agent frameworks > AI debates",
        sources: ["twitter", "tavily", "hackernews"],
        influence_thresholds: { twitter_likes: 100, hackernews_score: 50, tavily_relevance: 60 },
      },
      {
        name: "evening",
        time: "18:00",
        hour: 18,
        window: [17, 19],
        topic: "General (divergent)",
        label: "晚间热点",
        keywords: ["technology", "science", "trending"],
        priority: "finance + AI extended to tech, science, society hotspots",
        sources: ["twitter", "tavily", "hackernews"],
        influence_thresholds: { twitter_likes: 200, hackernews_score: 80, tavily_relevance: 60 },
      },
    ],
    defaults: {
      summary_length: "100-300",
      summary_language: "zh-CN",
      items_per_push: "3-10",
      time_window_hours: 24,
    },
    created_at: nowISO(),
    updated_at: nowISO(),
  };
}

function parseTime(timeStr) {
  const m = timeStr.match(/^(\d{1,2}):(\d{2})$/);
  if (!m) return null;
  return Number.parseInt(m[1], 10);
}

function printConfig(config) {
  console.log("## News Digest Configuration\n");

  console.log("### Slots\n");
  if (!config.slots || config.slots.length === 0) {
    console.log("No slots configured. Run `manage-config.mjs init` or `manage-config.mjs set-slot` to add slots.\n");
  } else {
    console.log("| Name | Time | Topic | Label | Keywords |");
    console.log("|------|------|-------|-------|----------|");
    for (const s of config.slots) {
      const kw = (s.keywords ?? []).join(", ");
      console.log(`| ${s.name} | ${s.time} | ${s.topic} | ${s.label} | ${kw} |`);
    }
    console.log();
    for (const s of config.slots) {
      console.log(`**${s.name}** (${s.label})`);
      console.log(`- Time: ${s.time} (window: ${s.window?.[0] ?? "?"}:00-${s.window?.[1] ?? "?"}:00)`);
      console.log(`- Priority: ${s.priority}`);
      console.log(`- Sources: ${(s.sources ?? []).join(", ")}`);
      console.log(`- Thresholds: Twitter likes > ${s.influence_thresholds?.twitter_likes ?? "?"}, HN score > ${s.influence_thresholds?.hackernews_score ?? "?"}, Tavily relevance > ${s.influence_thresholds?.tavily_relevance ?? "?"}%`);
      console.log();
    }
  }

  console.log("### Defaults\n");
  const d = config.defaults ?? {};
  console.log(`- Summary length: ${d.summary_length ?? "100-300"} chars`);
  console.log(`- Summary language: ${d.summary_language ?? "zh-CN"}`);
  console.log(`- Items per push: ${d.items_per_push ?? "3-10"}`);
  console.log(`- Time window: ${d.time_window_hours ?? 24} hours`);
  console.log();
  console.log(`Config path: ${configPath()}`);
  console.log(`Last updated: ${config.updated_at ?? "unknown"}`);
}

if (command === "show") {
  const config = loadConfig();
  if (!config) {
    console.log("## No Configuration Found\n");
    console.log("No config file exists yet. The agent should ask the user to set up their preferred schedule and topics.\n");
    console.log("To create default config: `node manage-config.mjs init`");
    console.log("To set up a custom slot: `node manage-config.mjs set-slot --name morning --time 08:00 --topic Finance --label 金融早报 --keywords \"crypto,bitcoin\" --sources \"twitter,tavily,hackernews\"`");
  } else {
    printConfig(config);
  }

} else if (command === "init") {
  const existing = loadConfig();
  if (existing) {
    console.log("## Configuration Already Exists\n");
    console.log("Use `show` to view, `set-slot` to modify, or `reset` to overwrite.");
    process.exit(0);
  }
  const config = defaultConfig();
  saveConfig(config);
  console.log("## Default Configuration Created\n");
  printConfig(config);

} else if (command === "set-slot") {
  const opts = parseArgs();
  if (!opts.name) {
    console.error("Error: --name is required for set-slot.");
    process.exit(1);
  }

  let config = loadConfig();
  if (!config) {
    config = { slots: [], defaults: defaultConfig().defaults, created_at: nowISO(), updated_at: nowISO() };
  }

  const idx = config.slots.findIndex((s) => s.name === opts.name);
  const existing = idx >= 0 ? config.slots[idx] : {};

  const hour = opts.time ? parseTime(opts.time) : existing.hour;
  if (opts.time && hour === null) {
    console.error(`Error: invalid time format "${opts.time}". Use HH:MM.`);
    process.exit(1);
  }

  const slot = {
    name: opts.name,
    time: opts.time ?? existing.time ?? "08:00",
    hour: hour ?? existing.hour ?? 8,
    window: hour != null ? [Math.max(0, hour - 1), Math.min(23, hour + 1)] : (existing.window ?? [7, 9]),
    topic: opts.topic ?? existing.topic ?? opts.name,
    label: opts.label ?? existing.label ?? opts.name,
    keywords: opts.keywords ? opts.keywords.split(",").map((k) => k.trim()) : (existing.keywords ?? []),
    priority: opts.priority ?? existing.priority ?? "",
    sources: opts.sources ? opts.sources.split(",").map((s) => s.trim()) : (existing.sources ?? ["twitter", "tavily", "hackernews"]),
    influence_thresholds: existing.influence_thresholds ?? { twitter_likes: 100, hackernews_score: 50, tavily_relevance: 60 },
  };

  if (idx >= 0) {
    config.slots[idx] = slot;
  } else {
    config.slots.push(slot);
  }

  config.slots.sort((a, b) => (a.hour ?? 0) - (b.hour ?? 0));
  saveConfig(config);

  console.log(`## Slot "${opts.name}" ${idx >= 0 ? "Updated" : "Added"}\n`);
  console.log(`- Time: ${slot.time}`);
  console.log(`- Topic: ${slot.topic}`);
  console.log(`- Label: ${slot.label}`);
  console.log(`- Keywords: ${slot.keywords.join(", ")}`);
  console.log(`- Sources: ${slot.sources.join(", ")}`);

} else if (command === "remove-slot") {
  const opts = parseArgs();
  if (!opts.name) {
    console.error("Error: --name is required for remove-slot.");
    process.exit(1);
  }

  const config = loadConfig();
  if (!config) {
    console.error("Error: no config file exists.");
    process.exit(1);
  }

  const before = config.slots.length;
  config.slots = config.slots.filter((s) => s.name !== opts.name);

  if (config.slots.length === before) {
    console.log(`Slot "${opts.name}" not found. Available: ${config.slots.map((s) => s.name).join(", ")}`);
    process.exit(0);
  }

  saveConfig(config);
  console.log(`## Slot "${opts.name}" Removed\n`);
  console.log(`Remaining slots: ${config.slots.map((s) => `${s.name} (${s.time})`).join(", ") || "none"}`);

} else if (command === "set-defaults") {
  const opts = parseArgs();
  let config = loadConfig();
  if (!config) {
    config = defaultConfig();
  }

  if (opts["summary-length"]) config.defaults.summary_length = opts["summary-length"];
  if (opts["summary-language"]) config.defaults.summary_language = opts["summary-language"];
  if (opts["items-per-push"]) config.defaults.items_per_push = opts["items-per-push"];
  if (opts["time-window-hours"]) config.defaults.time_window_hours = Number.parseInt(opts["time-window-hours"], 10);

  saveConfig(config);
  console.log("## Defaults Updated\n");
  console.log(`- Summary length: ${config.defaults.summary_length}`);
  console.log(`- Summary language: ${config.defaults.summary_language}`);
  console.log(`- Items per push: ${config.defaults.items_per_push}`);
  console.log(`- Time window: ${config.defaults.time_window_hours} hours`);

} else if (command === "reset") {
  const config = defaultConfig();
  saveConfig(config);
  console.log("## Configuration Reset to Defaults\n");
  printConfig(config);

} else {
  console.error(`Unknown command: ${command}`);
  usage();
}


path: scripts/query.mjs
content: #!/usr/bin/env node

import { existsSync, readdirSync } from "node:fs";
import { join } from "node:path";
import {
  readJSON,
  todayStr,
  pushPath,
  pushDir,
  feedbackPath,
  feedbackDir,
  listDates,
  getDataDir,
} from "./lib/storage.mjs";

function usage() {
  console.error(
    `Usage: query.mjs <command> [options]

Commands:
  pushes    View push records
  feedback  View feedback records
  search    Search across pushes by keyword

Options:
  --date <YYYY-MM-DD>   Target date (default: today)
  --slot <slot>         Filter by slot: morning, noon, evening
  --days <n>            Look back n days (default: 1, for search default: 7)
  --keyword <kw>        Keyword to search for (search command only)

Examples:
  node query.mjs pushes --date 2026-03-06 --slot morning
  node query.mjs pushes --days 3
  node query.mjs feedback --date 2026-03-06
  node query.mjs feedback --days 3
  node query.mjs search --keyword "bitcoin" --days 7`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const command = args[0];
let date = null;
let slot = null;
let days = null;
let keyword = null;

for (let i = 1; i < args.length; i++) {
  const a = args[i];
  if (a === "--date") { date = args[++i]; continue; }
  if (a === "--slot") { slot = args[++i]; continue; }
  if (a === "--days") { days = Number.parseInt(args[++i] ?? "1", 10); continue; }
  if (a === "--keyword") { keyword = args[++i]; continue; }
  if (a.startsWith("-")) {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
}

const SLOTS = ["morning", "noon", "evening"];

function dateRange(numDays) {
  const dates = [];
  const now = new Date();
  for (let i = 0; i < numDays; i++) {
    const d = new Date(now);
    d.setDate(d.getDate() - i);
    const y = d.getFullYear();
    const m = String(d.getMonth() + 1).padStart(2, "0");
    const day = String(d.getDate()).padStart(2, "0");
    dates.push(`${y}-${m}-${day}`);
  }
  return dates;
}

function resolveDates() {
  if (date) return [date];
  const n = days ?? 1;
  return dateRange(n);
}

function printPush(record) {
  if (!record) return;
  console.log(`### ${record.id} — ${record.topic} (${record.slot})`);
  console.log(`- **Time**: ${record.push_time}`);
  console.log(`- **Items**: ${record.item_count ?? record.items?.length ?? 0}`);
  if (record.keywords?.length) {
    console.log(`- **Keywords**: ${record.keywords.join(", ")}`);
  }
  console.log();
  for (const item of record.items ?? []) {
    console.log(
      `  ${item.id}. [${item.category || item.source}] **${item.title}**`
    );
    if (item.summary) {
      console.log(`  ${item.summary.slice(0, 200)}${item.summary.length > 200 ? "..." : ""}`);
    }
    if (item.url) {
      console.log(`  ${item.url}`);
    }
    console.log();
  }
  console.log("---\n");
}

function printFeedback(record, dateLabel) {
  if (!record || !record.feedbacks?.length) return;
  console.log(`### Feedback for ${record.push_id}`);
  for (const fb of record.feedbacks) {
    const itemTag = fb.item_id ? ` [${fb.item_id}]` : "";
    console.log(`- ${fb.created_at}${itemTag}: ${fb.text}`);
  }
  console.log();
}

if (command === "pushes") {
  const dates = resolveDates();
  const slots = slot ? [slot] : SLOTS;
  let found = false;

  console.log(`## Push Records\n`);
  for (const d of dates) {
    for (const s of slots) {
      const record = readJSON(pushPath(d, s));
      if (record) {
        printPush(record);
        found = true;
      }
    }
  }
  if (!found) {
    console.log("No push records found for the specified criteria.");
  }
} else if (command === "feedback") {
  const defaultDays = days ?? (date ? 1 : 3);
  const dates = date ? [date] : dateRange(defaultDays);
  const slots = slot ? [slot] : SLOTS;
  let found = false;

  console.log(`## Feedback Records\n`);
  for (const d of dates) {
    for (const s of slots) {
      const record = readJSON(feedbackPath(d, s));
      if (record && record.feedbacks?.length) {
        printFeedback(record, d);
        found = true;
      }
    }
  }
  if (!found) {
    console.log("No feedback found for the specified criteria.");
  }
} else if (command === "search") {
  if (!keyword) {
    console.error("Error: --keyword is required for search.");
    process.exit(1);
  }

  const searchDays = days ?? 7;
  const dates = dateRange(searchDays);
  const kw = keyword.toLowerCase();
  let matchCount = 0;

  console.log(`## Search Results for "${keyword}" (last ${searchDays} days)\n`);

  for (const d of dates) {
    for (const s of SLOTS) {
      const record = readJSON(pushPath(d, s));
      if (!record) continue;

      const matchingItems = (record.items ?? []).filter((item) => {
        const haystack = [
          item.title,
          item.summary,
          item.category,
          item.author,
          item.raw_excerpt,
        ]
          .join(" ")
          .toLowerCase();
        return haystack.includes(kw);
      });

      if (matchingItems.length > 0) {
        console.log(`### ${d} / ${s} (${record.topic})\n`);
        for (const item of matchingItems) {
          console.log(
            `- [${item.category || item.source}] **${item.title}**`
          );
          if (item.summary) {
            console.log(`  ${item.summary.slice(0, 200)}...`);
          }
          if (item.url) console.log(`  ${item.url}`);
          console.log();
          matchCount++;
        }
      }
    }
  }

  if (matchCount === 0) {
    console.log(`No items matching "${keyword}" found.`);
  } else {
    console.log(`---\n\nTotal: ${matchCount} matching items.`);
  }
} else {
  console.error(`Unknown command: ${command}`);
  usage();
}


path: scripts/store-push.mjs
content: #!/usr/bin/env node

import { readFileSync } from "node:fs";
import {
  readJSON,
  writeJSON,
  todayStr,
  nowISO,
  pushPath,
  currentMonth,
  usagePath,
} from "./lib/storage.mjs";

function usage() {
  console.error(
    `Usage: store-push.mjs [--slot morning|noon|evening] [--input file.json]

Reads push data from stdin (default) or a file, then persists it.

The JSON input must have: slot, topic, items[].
If --slot is given, it overrides the slot in the JSON.

Example:
  echo '{"slot":"morning","topic":"finance","items":[...]}' | node store-push.mjs
  node store-push.mjs --slot morning --input push-data.json`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args[0] === "-h" || args[0] === "--help") usage();

let slotOverride = null;
let inputFile = null;

for (let i = 0; i < args.length; i++) {
  const a = args[i];
  if (a === "--slot") {
    slotOverride = args[++i];
    continue;
  }
  if (a === "--input") {
    inputFile = args[++i];
    continue;
  }
  console.error(`Unknown arg: ${a}`);
  usage();
}

let rawJSON;
if (inputFile) {
  rawJSON = readFileSync(inputFile, "utf-8");
} else {
  const chunks = [];
  for await (const chunk of process.stdin) {
    chunks.push(chunk);
  }
  rawJSON = Buffer.concat(chunks).toString("utf-8");
}

if (!rawJSON.trim()) {
  console.error("Error: empty input. Provide JSON via stdin or --input.");
  process.exit(1);
}

let pushData;
try {
  pushData = JSON.parse(rawJSON);
} catch (e) {
  console.error(`Error: invalid JSON — ${e.message}`);
  process.exit(1);
}

const slot = slotOverride || pushData.slot;
if (!slot || !["morning", "noon", "evening"].includes(slot)) {
  console.error(
    `Error: slot must be one of: morning, noon, evening. Got: "${slot}"`
  );
  process.exit(1);
}

const today = todayStr();
const dateCompact = today.replace(/-/g, "");
const pushId = `push-${dateCompact}-${slot}`;

const items = (pushData.items ?? []).map((item, idx) => ({
  id: item.id || `item-${String(idx + 1).padStart(3, "0")}`,
  source: item.source ?? "unknown",
  category: item.category ?? "",
  title: item.title ?? "",
  url: item.url ?? "",
  author: item.author ?? "",
  influence_score: item.influence_score ?? 0,
  raw_excerpt: item.raw_excerpt ?? "",
  summary: item.summary ?? "",
  metadata: item.metadata ?? {},
}));

const usageData = pushData.usage ?? null;

const record = {
  id: pushId,
  slot,
  topic: pushData.topic ?? slot,
  keywords: pushData.keywords ?? [],
  push_time: nowISO(),
  items,
  item_count: items.length,
  usage: usageData,
  created_at: nowISO(),
};

const filePath = pushPath(today, slot);

const existing = readJSON(filePath);
if (existing) {
  record.created_at = existing.created_at;
  record.updated_at = nowISO();
}

writeJSON(filePath, record);

if (usageData) {
  const API_KEYS = ["tavily_search", "tavily_extract", "hackernews", "xpoz"];
  const ym = currentMonth();
  const monthFile = usagePath(ym);
  const monthData = readJSON(monthFile) ?? { month: ym, daily: {}, totals: {} };

  if (!monthData.daily[today]) monthData.daily[today] = {};
  const existing = monthData.daily[today][slot] ?? Object.fromEntries(API_KEYS.map((k) => [k, 0]));
  for (const key of API_KEYS) {
    existing[key] = (existing[key] ?? 0) + (usageData[key] ?? 0);
  }
  monthData.daily[today][slot] = existing;

  monthData.totals = {};
  for (const day of Object.values(monthData.daily)) {
    for (const slotData of Object.values(day)) {
      if (slotData) {
        for (const key of API_KEYS) {
          monthData.totals[key] = (monthData.totals[key] ?? 0) + (slotData[key] ?? 0);
        }
      }
    }
  }

  writeJSON(monthFile, monthData);
}

console.log(`## Push Stored\n`);
console.log(`- **ID**: ${pushId}`);
console.log(`- **Slot**: ${slot}`);
console.log(`- **Topic**: ${record.topic}`);
console.log(`- **Items**: ${items.length}`);
if (usageData) {
  const parts = Object.entries(usageData).filter(([, v]) => v > 0).map(([k, v]) => `${k} ×${v}`);
  if (parts.length > 0) console.log(`- **Usage**: ${parts.join(" | ")}`);
}
console.log(`- **Path**: ${filePath}`);


path: scripts/track-usage.mjs
content: #!/usr/bin/env node

import {
  readJSON,
  writeJSON,
  todayStr,
  currentMonth,
  usagePath,
} from "./lib/storage.mjs";

const API_KEYS = ["tavily_search", "tavily_extract", "hackernews", "xpoz"];

function usage() {
  console.error(
    `Usage: track-usage.mjs <command> [options]

Commands:
  record   Record API usage for a push slot
  today    Show today's usage
  monthly  Show current month's total usage
  forecast Predict end-of-month usage based on daily average

Record options:
  --slot <slot>              Push slot (required for record)
  --tavily-search <n>        Tavily Search API calls
  --tavily-extract <n>       Tavily Extract API calls
  --hackernews <n>           Hacker News API calls
  --xpoz <n>                 Xpoz MCP calls

Examples:
  node track-usage.mjs record --slot morning --tavily-search 3 --tavily-extract 2 --hackernews 1 --xpoz 2
  node track-usage.mjs today
  node track-usage.mjs monthly
  node track-usage.mjs forecast`
  );
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const command = args[0];

function parseRecordArgs() {
  let slot = null;
  const counts = { tavily_search: 0, tavily_extract: 0, hackernews: 0, xpoz: 0 };

  for (let i = 1; i < args.length; i++) {
    const a = args[i];
    if (a === "--slot") { slot = args[++i]; continue; }
    if (a === "--tavily-search") { counts.tavily_search = Number.parseInt(args[++i] ?? "0", 10); continue; }
    if (a === "--tavily-extract") { counts.tavily_extract = Number.parseInt(args[++i] ?? "0", 10); continue; }
    if (a === "--hackernews") { counts.hackernews = Number.parseInt(args[++i] ?? "0", 10); continue; }
    if (a === "--xpoz") { counts.xpoz = Number.parseInt(args[++i] ?? "0", 10); continue; }
    console.error(`Unknown arg: ${a}`);
    usage();
  }

  if (!slot || !["morning", "noon", "evening"].includes(slot)) {
    console.error(`Error: --slot is required (morning|noon|evening). Got: "${slot}"`);
    process.exit(1);
  }

  return { slot, counts };
}

function loadMonth(ym) {
  const file = usagePath(ym);
  return readJSON(file) ?? { month: ym, daily: {}, totals: {} };
}

function saveMonth(data) {
  writeJSON(usagePath(data.month), data);
}

function addToTotals(totals, counts) {
  for (const key of API_KEYS) {
    totals[key] = (totals[key] ?? 0) + (counts[key] ?? 0);
  }
}

function zeroUsage() {
  return Object.fromEntries(API_KEYS.map((k) => [k, 0]));
}

function daysInMonth(ym) {
  const [y, m] = ym.split("-").map(Number);
  return new Date(y, m, 0).getDate();
}

function dayOfMonth() {
  return new Date().getDate();
}

if (command === "record") {
  const { slot, counts } = parseRecordArgs();
  const ym = currentMonth();
  const today = todayStr();
  const data = loadMonth(ym);

  if (!data.daily[today]) data.daily[today] = {};
  const existing = data.daily[today][slot] ?? zeroUsage();
  for (const key of API_KEYS) {
    existing[key] = (existing[key] ?? 0) + (counts[key] ?? 0);
  }
  data.daily[today][slot] = existing;

  data.totals = {};
  for (const day of Object.values(data.daily)) {
    for (const slotData of Object.values(day)) {
      if (slotData) addToTotals(data.totals, slotData);
    }
  }

  saveMonth(data);

  console.log("## Usage Recorded\n");
  console.log(`- **Date**: ${today}`);
  console.log(`- **Slot**: ${slot}`);
  for (const key of API_KEYS) {
    if (counts[key] > 0) console.log(`- **${key}**: +${counts[key]}`);
  }
  console.log(`- **Path**: ${usagePath(ym)}`);

} else if (command === "today") {
  const ym = currentMonth();
  const today = todayStr();
  const data = loadMonth(ym);
  const dayData = data.daily[today];

  console.log(`## API Usage — ${today}\n`);

  if (!dayData || Object.keys(dayData).length === 0) {
    console.log("No usage recorded today.");
  } else {
    console.log("| Slot | Tavily Search | Tavily Extract | HN | Xpoz |");
    console.log("|------|--------------|---------------|-----|------|");
    const dayTotals = zeroUsage();
    for (const slot of ["morning", "noon", "evening"]) {
      const s = dayData[slot];
      if (s) {
        console.log(`| ${slot} | ${s.tavily_search ?? 0} | ${s.tavily_extract ?? 0} | ${s.hackernews ?? 0} | ${s.xpoz ?? 0} |`);
        addToTotals(dayTotals, s);
      } else {
        console.log(`| ${slot} | - | - | - | - |`);
      }
    }
    console.log(`| **Total** | **${dayTotals.tavily_search}** | **${dayTotals.tavily_extract}** | **${dayTotals.hackernews}** | **${dayTotals.xpoz}** |`);
  }

} else if (command === "monthly") {
  const ym = currentMonth();
  const data = loadMonth(ym);
  const t = data.totals ?? {};
  const activeDays = Object.keys(data.daily).length;

  console.log(`## API Monthly Usage — ${ym}\n`);
  console.log(`Active days: ${activeDays}\n`);
  console.log("| API | Total |");
  console.log("|-----|-------|");
  for (const key of API_KEYS) {
    console.log(`| ${key} | ${t[key] ?? 0} |`);
  }

} else if (command === "forecast") {
  const ym = currentMonth();
  const data = loadMonth(ym);
  const t = data.totals ?? {};
  const activeDays = Object.keys(data.daily).length;
  const totalDays = daysInMonth(ym);
  const currentDay = dayOfMonth();
  const remainingDays = totalDays - currentDay;

  console.log(`## API 月度用量预测 (${ym})\n`);

  if (activeDays === 0) {
    console.log("No usage data yet this month. Run some pushes first.");
  } else {
    console.log(`统计天数: ${activeDays} | 本月总天数: ${totalDays} | 剩余: ${remainingDays}\n`);
    console.log("| API | 本月已用 | 日均 | 预计月末 |");
    console.log("|-----|---------|------|---------|");
    for (const key of API_KEYS) {
      const used = t[key] ?? 0;
      const avg = activeDays > 0 ? (used / activeDays).toFixed(1) : "0";
      const projected = activeDays > 0 ? Math.round((used / activeDays) * totalDays) : 0;
      console.log(`| ${key} | ${used} | ${avg} | ~${projected} |`);
    }
  }

} else {
  console.error(`Unknown command: ${command}`);
  usage();
}


Security scan verdicts — ClawScan: suspicious | static analysis: suspicious | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)