Skill: octo-patch/morning-ai (version 1.2.5)

--- name: morning-ai version: "1.2.5" description: "Daily-scheduled AI news tracker. Collects updates from 80+ AI entities across 6 sources every 24 hours (default 08:00 UTC+8). Generates scored, deduplicated Markdown reports. Supports unattended cron/scheduled execution with date-stamped idempotent output." argument-hint: 'morning-ai, morning-ai --exclude Funding, morning-ai --depth deep, morning-ai --lang zh, morning-ai --schedule "0 9 * * *", morning-ai --lang zh (with MESSAGE_ENABLED=true for message digest)' allowed-tools: Bash, Read, Write, Edit, WebSearch homepage: https://github.com/octo-patch/MorningAI repository: https://github.com/octo-patch/MorningAI author: octo-patch license: MIT user-invocable: true schedule: frequency: daily default-cron: "0 8 * * *" timezone: "Asia/Shanghai" idempotent: true unattended: true estimated-duration: "2-5min" metadata: openclaw: emoji: "📰" requires: optionalEnv: - GITHUB_TOKEN bins: - python3 files: - "skills/*/scripts/collect.py" - "skills/*/scripts/gen_infographic.py" - "lib/*" homepage: https://github.com/octo-patch/MorningAI tags: - ai - news - tracking - daily-report - model - product - benchmark - funding - multi-source - scheduled - cron - automated - unattended - ai-skill - clawhub --- # morning-ai: AI News Daily Report Generator > **Permissions overview:** Collects public data from Reddit, Hacker News, GitHub, HuggingFace, arXiv, and X/Twitter. Requires optional API keys configured in `.env` or `~/.config/morning-ai/.env`. Writes report files to the current working directory. See [Configuration](#configuration) for details. Track 80+ AI entities across 6 sources. Collect updates from the past 24 hours, score and deduplicate them, and generate a structured Markdown daily report. Covers 4 types: **Product** (feature launches, version releases), **Model** (new models, open-source weights), **Benchmark** (leaderboard changes, papers), **Funding** (rounds, acquisitions, milestones). --- ## Step 0: Configuration Gate (REQUIRED — must complete before any other step) **Run this command FIRST before doing anything else:** ```bash if [ -f "$HOME/.config/morning-ai/.env" ] || [ -f ".claude/morning-ai.env" ] || [ -f ".env" ]; then echo "CONFIG_STATUS=READY"; else echo "CONFIG_STATUS=MISSING"; fi ``` **Branch on the output:** - **If output is `CONFIG_STATUS=READY`** — read the config file, report which sources are active, then proceed to Step 1. - **If output is `CONFIG_STATUS=MISSING`** — **STOP. You MUST complete the First-Time Onboarding below before proceeding to Step 1.** ### First-Time Onboarding (when `MISSING`) > **CRITICAL: STOP HERE.** > You MUST complete all onboarding steps below interactively with the user. > Do NOT run Step 1 (data collection) until a config file exists and the gate check returns `READY`. > Running data collection without configuration will produce incomplete results. Walk the user through setup interactively, waiting for their response at each step: 1. **Welcome** — briefly explain what morning-ai does: tracks 80+ AI entities across 6 sources (Reddit, Hacker News, GitHub, HuggingFace, arXiv, X/Twitter), generates scored daily reports 2. **Show what works for free** — 6 sources (5 need no API keys, 1 optional): - Reddit (public JSON), Hacker News (Algolia API), HuggingFace (public API), arXiv (public API), X/Twitter (web search) - GitHub (public API, optional `GITHUB_TOKEN` for higher rate limits) 3. **Ask the user** if they want to enable GitHub with higher rate limits: | Key | Source | Get it at | |-----|--------|-----------| | `GITHUB_TOKEN` | GitHub releases & repos (higher rate limit) | https://github.com/settings/tokens | 4. **Ask about infographics** (optional): | Key | Description | |-----|-------------| | `IMAGE_GEN_PROVIDER` | Provider: `gemini` \| `minimax` \| `none` (default: none) | | `IMAGE_STYLE` | Style: `classic` \| `dark` \| `glassmorphism` \| `newspaper` \| `tech` | | `GEMINI_API_KEY` | Google Gemini/Imagen (https://aistudio.google.com/apikey) | | `MINIMAX_API_KEY` | MiniMax global(https://www.minimax.io) | | `MINIMAX_API_KEY` | MiniMax cn (https://platform.minimaxi.com) | 5. **Ask about social content distribution** (optional): - Enable social content generation? Set `SOCIAL_ENABLED=true` - Which platforms? X (Twitter), Xiaohongshu (Little Red Book), or both - For advanced multi-account/multi-style setup, create `~/.config/morning-ai/social_channels.json` (see `skills/gen-social/SKILL.md` for schema). For quick single-channel setup, just set `SOCIAL_PLATFORM`, `SOCIAL_STYLE`, and `SOCIAL_LANG` env vars. 6. **Ask about message digest** (optional): - Enable concise message digest for sharing on messaging platforms (WeChat, Telegram, Slack)? - If yes: set `MESSAGE_ENABLED=true` - Optional settings: `MESSAGE_MIN_SCORE` (default 5), `MESSAGE_MAX_ITEMS` (default 10), `MESSAGE_LINKS` (`bottom` or `inline`) 7. **Create the config file** — collect the keys the user provides and write them to `~/.config/morning-ai/.env` in `KEY=value` format (one per line). Create the directory if needed: `mkdir -p ~/.config/morning-ai` 8. **Confirm** — show how many sources are now active (N/9) 9. **Verify** — re-run the gate check to confirm `CONFIG_STATUS=READY`: ```bash if [ -f "$HOME/.config/morning-ai/.env" ] || [ -f ".claude/morning-ai.env" ] || [ -f ".env" ]; then echo "CONFIG_STATUS=READY"; else echo "CONFIG_STATUS=MISSING"; fi ``` Only proceed to Step 1 if the output is `READY`. 10. If the user wants to skip API key setup and use only free sources, create a minimal config file first, then proceed to Step 1: ```bash mkdir -p ~/.config/morning-ai && echo "# morning-ai config — free sources only" > ~/.config/morning-ai/.env ``` --- ## Language | Parameter | Default | Example | |-----------|---------|---------| | `--lang` | `en` (English) | `--lang zh` (Chinese), `--lang ja` (Japanese) | **Rules:** - **Default is English. Unless `--lang` is explicitly specified, the report MUST be written entirely in English.** All report text — titles, summaries, section headers, table labels, bullet points, "Why It Matters" analysis, and all other human-readable content — must be in English. - If `--lang` is specified, use that language for all human-readable content instead. - **Entity names are proper nouns** (OpenAI, DeepSeek, Midjourney, etc.) — keep them as-is regardless of language. - When source data is in a different language than the target (e.g. Chinese source → English report), **translate it** into the target language during report generation. Do NOT leave untranslated fragments. - The `--lang` setting also applies to infographic prompt content (see Step 4). --- > **Prerequisite:** Step 0 must have returned `CONFIG_STATUS=READY`. If you have not completed Step 0, go back and run it now. ## Step 1: Data Collection Run the Python collector to gather data from automated sources: ```bash cd {SKILL_DIR} && python3 skills/tracking-list/scripts/collect.py --date {YYYY-MM-DD} --depth default -o {CWD}/data_{YYYY-MM-DD}.json ``` **Parameters:** - `--date`: Target date, default today (YYYY-MM-DD) - `--depth`: Collection depth — `quick` (fast, fewer results), `default`, or `deep` (comprehensive) - `--sources`: Specific sources only, e.g. `--sources reddit hackernews github` - `-o`: Output JSON file path **What it does:** - Runs 5 collectors concurrently (Reddit, HN, GitHub, HuggingFace, arXiv) - Time window: `[Yesterday 08:00, Today 08:00) UTC+8` - Pipeline: collect → score (1-10) → deduplicate → cross-source link → verification bonus - Returns structured JSON with all items, stats, and collection metadata **Timeout:** Allow up to 3 minutes for default depth, 5 minutes for deep. If the user provides `--exclude` types (e.g. `--exclude Funding`), note which types to filter out in Step 3. ### X/Twitter Search After the automated collection completes, use **web search** to discover recent X/Twitter updates from tracked entities. The tracked X handles are listed in `{SKILL_DIR}/lib/entities.py` under `X_HANDLES`. #### Search Strategy: Multi-Layer Account Checking Search X/Twitter in **three layers**, in priority order: **Layer 1 — Official Accounts** (highest priority): Search for recent posts from official company/product accounts. Handles are listed in entity files under `{SKILL_DIR}/entities/`. - Focus: Model releases, product launches, API updates, pricing changes **Layer 2 — CEO / Core Personnel Accounts**: Check key people's accounts for announcements, previews, and context that official accounts may not cover. Listed as "Key People" in each entity file. - Focus: Early previews, strategic context, technical details, competitive commentary **Layer 3 — KOLs & Benchmark Institutions**: Check AI opinion leaders and evaluation accounts for independent analysis, benchmark results, and trending discoveries. See `{SKILL_DIR}/entities/kol.md` and `{SKILL_DIR}/entities/benchmarks-academic.md`. - Focus: Paper highlights, benchmark rankings, community trends, independent testing #### Search Execution For each search depth: | Depth | Layer 1 (Official) | Layer 2 (Personnel) | Layer 3 (KOLs) | |-------|-------------------|--------------------|-----------------| | `quick` | Top 5 entities by priority | Skip | Skip | | `default` | All major entities (~20) | Top CEO accounts (~10) | Top KOLs (~5) | | `deep` | All entities with X handles | All personnel accounts | All KOLs + benchmark accounts | Use web search queries like: - `site:x.com @{handle} since:{yesterday}` — for specific account posts - `site:x.com "{entity name}" AI announcement` — for broader discovery - `site:x.com AI model release OR benchmark OR open-source {date}` — for trending AI news #### RT/Quote Tweet Handling When a discovered post is a retweet (RT) or quote tweet: 1. **Trace to the original post** — the event timestamp is the **original post time**, not the RT/quote time 2. Validate the **original post time** falls within the 24-hour collection window `[Yesterday 08:00, Today 08:00) UTC+8` 3. Use the **original post URL** as the `source_url`, not the RT/quote URL 4. Credit the original author in `source_label` (e.g., `"@AnthropicAI on X (via @karpathy RT)"`) #### Timeliness Validation - All X post times must be converted to **UTC+8** for window validation - The collection window is `[Yesterday 08:00, Today 08:00) UTC+8` - Posts outside this window are **rejected** — do NOT include them regardless of relevance - For RT/quote tweets, validate the **original post time**, not the retweet time #### Source Priority for X-Sourced Items | Priority | Source Type | Credibility | |----------|-----------|-------------| | 1 | Official blog / changelog | Highest | | 2 | Official X/Twitter account | High | | 3 | API changelog / docs | High | | 4 | Official GitHub release | High | | 5 | CEO / core personnel X account | Medium-High | | 6 | Benchmark institution X account | Medium | | 7 | KOL X account | Reference only — requires cross-verification | Items sourced **only** from KOL accounts (Priority 7) should be scored conservatively and flagged for cross-verification with an official source. #### Incorporating X Findings For each verified X/Twitter update: 1. Create a TrackerItem with `source: "x"`, `source_url` pointing to the original tweet, and `source_label` as `"@{handle} on X"` 2. If the same event was already found by an automated collector (Reddit, HN, GitHub, etc.), merge it as a `cross_ref` rather than creating a duplicate — this strengthens the verification score 3. If it's a genuinely new finding not in the automated data, add it as a new item with appropriate scoring --- ## Step 2: Read Specifications After data collection completes, read the tracking specification to understand scoring criteria, record format, and timeliness rules: ``` Read {SKILL_DIR}/skills/tracking-list/SKILL.md ``` This specification defines: - 4 tracking types (Product / Model / Benchmark / Funding) with include/exclude criteria - Source priority rankings - Scoring criteria (1-10 scale with 5 dimensions) - Timeliness validation rules (event date ≠ page date) - Cross-verification requirements (7+ scores need 2+ independent sources) - Record format for the report **Internalize the specification before writing the report.** Pay special attention to the scoring reference tables and type classification guide. --- ## Step 3: Generate Report 1. Read the JSON output from Step 1 2. Read the report template: `Read {SKILL_DIR}/templates/report.md` 3. Generate `report_{YYYY-MM-DD}.md` in the working directory **Report generation rules:** - **Language**: Default is **English**. Write ALL content in English unless `--lang` is explicitly specified. If source data is in a different language, translate it. Entity names (proper nouns) stay as-is. - **Source links**: Every item MUST include a clickable source link `[Source Name](URL)` pointing to the original content. This applies to all sections: TLDR, detailed entries, and compact table rows. - **Detail quality**: Summary bullet points must include specific details — version numbers, percentage improvements, parameter counts, pricing, availability dates, benchmark scores. Avoid vague descriptions without concrete numbers. - **Factual verification**: See `skills/tracking-list/SKILL.md` → "Factual Detail Verification" for the full protocol. Never write a number from memory or inference — omit unverifiable details. - Filter out any excluded types (if `--exclude` was specified) - Sort items by score within each type section - **TLDR section**: Only items with score 7+ (across all types), sorted high to low. Each item includes a one-line summary with specifics, plus an _Impact_ sentence explaining why it matters. Must include a source link `[[Source](URL)]` at the end. - **Type sections**: Group by score range (9-10 / 7-8 / 5-6 / 3-4) - For items with score 7+, include multi-source verification if available - **Item format**: Follow the record format defined in the tracking specification (read in Step 2), including detail level requirements, "Why It Matters", and "Key Data" sections. For mid-score (5-6) and lower-score (3-4) items, use the compact formats defined there. - Fill in the statistics summary table --- ## Step 4: Generate Infographics (Optional) **This step is optional. Skip if no image generation capability is available or configured.** 1. Read the infographic specification: ``` Read {SKILL_DIR}/skills/gen-infographic/SKILL.md ``` 2. **Generate cover + per-type sections + stitch** (see **Image Strategy** in `skills/gen-infographic/SKILL.md`): - Always generate cover (9:16 portrait) + per-type section images (9:16 portrait), then stitch into one long image 3. **Cover image**: Sort by score and select the **top 4-5** updates (across all types). Build prompt using the Cover Prompt Template (9:16 portrait). 4. **Per-type section images**: For each type (Model/Product/Benchmark/Funding) with 7+ score items, build a prompt using the Per-Type Prompt Template (9:16 portrait). - Default (`IMAGE_GEN_TYPES=auto`): only types with 7+ score items - Set `IMAGE_GEN_TYPES=all` for all types, `none` for cover only 5. Generate images and stitch: **Option A** — Native tool (Claude Code or other tools with built-in image generation): Use your tool's built-in image generation capability, one call per image. Then stitch sections together. **Option B** — Python script batch mode (any environment, requires `IMAGE_GEN_PROVIDER` configured): Build a manifest JSON with all prompts and outputs, then run: ```bash cd {SKILL_DIR} && python3 skills/gen-infographic/scripts/gen_infographic.py --batch {CWD}/manifest.json --stitch ``` Supported providers: `gemini`, `minimax`. See [Configuration](#configuration) for API keys. Requires `pip install Pillow`. The final output is **`news_infographic_YYYY-MM-DD_combined.png`** — a single long image containing cover + all section images. 6. Insert images into the report: - Combined long image at the beginning - Individual per-type images optionally at the top of each type section --- ## Step 5: Generate Social Content (Optional) **Skip this step if `SOCIAL_ENABLED` is not `true` or no social channels are configured.** Generate platform-optimized copy and images for social media distribution (X, Xiaohongshu, etc.). 1. Read the social content specification: ``` Read {SKILL_DIR}/skills/gen-social/SKILL.md ``` 2. Load channel configuration: - If `SOCIAL_CHANNELS_FILE` exists → read the JSON channel list - Else if `SOCIAL_PLATFORM` env var is set → build a single channel from `SOCIAL_PLATFORM` + `SOCIAL_STYLE` + `SOCIAL_LANG` - Else → skip this step 3. For each channel: a. Read the channel's template: `{SKILL_DIR}/skills/gen-social/templates/{platform}/{style}.md` b. Select top items from the report data (filter by `min_score`, limit by `items`, translate if `lang` differs from source) c. Generate copy following the template's format rules, tone, and character limits d. **Validate character counts** — each tweet ≤ 280 chars, Xiaohongshu title ≤ 20 chars, body ≤ 1000 chars e. Write copy to `{CWD}/social/social_{YYYY-MM-DD}_{channel_id}.md` f. If channel has `image: true` — generate platform-adapted images using the same providers as Step 4 - X: 16:9 or 1:1 aspect ratio - Xiaohongshu: 3:4 portrait, carousel multi-image supported g. Write images to `{CWD}/social/social_{YYYY-MM-DD}_{channel_id}_{N}.png` 4. Write manifest to `{CWD}/social/social_{YYYY-MM-DD}_manifest.json` listing all generated files **Channel config examples**: See `skills/gen-social/SKILL.md` for the full JSON schema and quick-setup env vars. --- ## Step 6: Generate Message Digest (Optional) **Skip this step if `MESSAGE_ENABLED` is not `true`.** Generate a concise, share-friendly message digest suitable for messaging platforms (WeChat, Telegram, Slack, etc.). The digest provides bold titles with one-line summaries and reference links — optimized for copy-paste sharing. 1. Read the message specification: ``` Read {SKILL_DIR}/skills/gen-message/SKILL.md ``` 2. Read the digest template: ``` Read {SKILL_DIR}/skills/gen-message/templates/digest.md ``` 3. Select items from the report data (`data_{YYYY-MM-DD}.json`): - Filter by score >= `MESSAGE_MIN_SCORE` (default: 5) - Sort by importance score descending - Limit to `MESSAGE_MAX_ITEMS` (default: 10) - Use `MESSAGE_LANG` for language (default: from `--lang`) 4. Generate text digest following the template format: - Write to `{CWD}/message_{YYYY-MM-DD}.md` - Each item: emoji marker + **bold title** + one-line summary + source link - Source link (`🔗 URL`) after each item by default (or grouped at bottom if `MESSAGE_LINKS=bottom`) - Language-specific header and footer 5. If image generation is available (`IMAGE_GEN_PROVIDER` is configured): - Build a 9:16 portrait image prompt from the specification's image prompt template - Generate `{CWD}/message_{YYYY-MM-DD}.png` using the same image generation method as Step 4 (native tool or Python script) **Output files:** - `message_{YYYY-MM-DD}.md` — copy-paste text for messaging - `message_{YYYY-MM-DD}.png` — accompanying image (only if image generation is configured) --- ## Entity Reference The `entities/` directory contains detailed entity registries organized by tracking group: | File | Scope | Entities | |------|-------|----------| | `entities/ai-labs.md` | Frontier AI Labs + China AI | OpenAI, Anthropic, Google, Meta AI, xAI, Microsoft, Qwen, DeepSeek, + 11 more | | `entities/model-infra.md` | Model Infrastructure | NVIDIA, Mistral, Cohere, Perplexity, AWS, Together, Groq, Apple | | `entities/coding-agent.md` | Coding Agent | Cursor, Cline, OpenCode, Droid, OpenClaw, Windsurf, + 5 more | | `entities/ai-apps.md` | AI Applications | v0, bolt.new, Lovable, Replit, Lovart, Manus, + 2 more | | `entities/vision-media.md` | Vision & Media | Midjourney, Runway, Pika, FLUX, ElevenLabs, + 7 more | | `entities/benchmarks-academic.md` | Benchmarks & Academic | LMSYS, HuggingFace, arXiv channels, industry media | | `entities/kol.md` | Key Opinion Leaders | Andrej Karpathy, AK, Andrew Ng, Swyx, Simon Willison, + 3 more | | `entities/trending-discovery.md` | Trending Discovery | GitHub Trending, Product Hunt, Hacker News, Reddit | Each file lists X/Twitter accounts, key people, official blogs, changelogs, GitHub repos, and other source URLs for every tracked entity. Read these files when you need to verify or supplement the automated collection. ### Custom Entities Users can add their own tracked entities by placing markdown files in `entities/custom/` (or `~/.config/morning-ai/entities/`, or a path set via `CUSTOM_ENTITIES_DIR`). Custom entity files use a simplified format — see `entities/custom-example.md` for the template. Custom entities are automatically merged into the built-in registries at runtime and collected alongside the default 80+ entities. --- ## Scheduling Morning-AI is designed for daily automated execution. Each run produces date-stamped files (`report_YYYY-MM-DD.md`, `data_YYYY-MM-DD.json`), making it safe to run on a recurring schedule. ### Schedule Configuration Use `--schedule` to set a custom cron expression (default: `0 8 * * *`): | Parameter | Format | Default | Example | |-----------|--------|---------|---------| | `--schedule` | Cron expression (5-field) | `0 8 * * *` (daily 8am) | `0 9 * * 1-5` (weekdays 9am) | The schedule is passed to the agent's native scheduler (CronCreate, /loop, system cron, etc.). Morning-AI itself does not run a scheduler — it relies on the host agent or system to trigger runs. ### Unattended Behavior - **Idempotent**: Re-running on the same date overwrites previous output — no duplicate accumulation - **No interactive prompts**: All steps run without user input when API keys are configured - **Partial success**: If some sources fail, the report generates with available data and logs warnings - **Timeout**: Allow 3 min (default depth) or 5 min (deep) ### Agent Integration Examples **Claude Code** (CronCreate / loop): ``` /loop 24h /morning-ai ``` With custom schedule: ``` /morning-ai --schedule "0 9 * * 1-5" ``` **System cron** (manual setup): ```bash 0 8 * * * cd /path/to/workspace && claude -p "/morning-ai" ``` **OpenClaw / always-on bot**: ```yaml schedule: "0 8 * * *" skill: morning-ai ``` --- ## Configuration ### Config File Locations (priority order) 1. **Environment variables** (highest priority) 2. **Project config**: `.env` in skill directory 3. **Global config**: `~/.config/morning-ai/.env` ### Config File Format ```bash # ~/.config/morning-ai/.env GITHUB_TOKEN=ghp_xxx ``` ### Free Sources (no API key needed) | Source | API | Rate Limit | |--------|-----|-----------| | Reddit | Public JSON | Generous | | Hacker News | Algolia API | Generous | | GitHub | Public API (optional token for higher limits) | 60 req/hr (unauthenticated) | | HuggingFace | Public API | Generous | | arXiv | Public API | Generous | | X/Twitter | Web search | Generous | ### Message Digest Configuration See `skills/gen-message/SKILL.md` for message digest configuration variables (`MESSAGE_ENABLED`, `MESSAGE_MIN_SCORE`, `MESSAGE_MAX_ITEMS`, etc.). --- ## Security & Permissions - **Data access**: Reads public web/platform data only. No private or authenticated content is accessed. - **API keys**: Stored locally in `.env` files. Never transmitted except to their respective APIs. - **File writes**: Only writes report files (`report_*.md`, `data_*.json`), message digest files (`message_*.md`, `message_*.png`), and cache files to the skill/working directory. - **Network**: Outbound HTTP/HTTPS requests to public APIs (Reddit, GitHub, etc.). No inbound connections. - **No telemetry**: No usage data is collected or sent anywhere.

path: AGENTS.md
content: # MorningAI

Daily-scheduled AI news tracker. Collects updates from 80+ AI entities across 6 sources every 24 hours. Generates scored, deduplicated Markdown reports with optional cover infographics.

## Skills

This repository provides a single skill: **morning-ai**.

To use it, read `SKILL.md` at the repository root — it contains full step-by-step instructions for data collection, report generation, and optional infographic creation.

Invoke with: `/morning-ai` or `morning-ai`

## Quick Reference

- **Entry point**: `SKILL.md` (step-by-step workflow)
- **Collector script**: `skills/tracking-list/scripts/collect.py`
- **Entity registries**: `entities/*.md` (80+ tracked entities)
- **Report template**: `templates/report.md`
- **Config**: `~/.config/morning-ai/.env` or project `.env`

## Configuration

Optional keys can be configured in `~/.config/morning-ai/.env`:

```
GITHUB_TOKEN=...   # GitHub (optional, higher rate limits)
```

All sources work out of the box without API keys: Reddit, Hacker News, GitHub, HuggingFace, arXiv, and X/Twitter.

## Scheduling

Default schedule: daily at 08:00 UTC+8. Each run produces date-stamped files (`report_YYYY-MM-DD.md`, `data_YYYY-MM-DD.json`), making it safe to run unattended on a recurring schedule.

## Permissions

- Reads public web/API data only
- Writes report and data files to the working directory
- Outbound HTTPS requests to public APIs (Reddit, GitHub, HuggingFace, arXiv, etc.)
- No telemetry, no inbound connections


path: CHANGELOG.md
content: # Changelog

## [1.2.5] - 2026-04-14

### Improvements
- **Claude Code skill discovery via symlink**: Add `skills/morning-ai/SKILL.md` symlink pointing to root `SKILL.md`, enabling Claude Code to discover the main skill through its `skills/` directory scan while keeping root `SKILL.md` intact for Codex, OpenCode, and OpenClaw

## [1.2.4] - 2026-04-14

### Fixes
- **Reddit community links fixed**: Replace dead/private subreddit links — r/Qwen → r/Qwen_AI (original was private), r/HailuoAI → r/HailuoAiOfficial (original didn't exist), removed r/TencentAI (doesn't exist)
- **Message digest: prioritize substance over vanity metrics**: Summary now requires answering "what is it" + "why it matters" instead of leading with download/star counts. GitHub Trending items lead with project description, star count as supporting context

## [1.2.3] - 2026-04-14

### Improvements
- **Reddit: entity-specific subreddit support**: Each entity can now define dedicated subreddits via `Reddit Community` field (e.g. `r/DeepSeek`, `r/ClaudeAI`). The collector fetches hot posts directly from these communities — no keyword search needed since the entire subreddit is about that entity
- **Two-phase Reddit collection**: Phase 1 fetches hot posts from entity-specific subreddits; Phase 2 keyword-searches general subreddits (`MachineLearning`, `LocalLLaMA`, `artificial`, `singularity`) as before, with cross-phase deduplication
- **25 entity-specific subreddits added**: Covers AI labs (r/OpenAI, r/ChatGPT, r/ClaudeAI, r/Gemini, r/DeepSeek, r/Grok, etc.), coding agents (r/cursor, r/Windsurf, r/ClineAI), model infra (r/MistralAI, r/perplexity_ai), vision/media (r/midjourney, r/StableDiffusion, r/SunoAI), and apps (r/CharacterAI)
- **Custom entity template updated**: `Reddit Community` added as a supported platform field in `custom-example.md`
- **Cache TTL default reduced to 1 hour**: `--cache-ttl` now defaults to 1h; added `--no-cache` and `--clear-cache` flags
- **Cover image switched to 9:16 portrait**: Cover infographic now uses 9:16 portrait (same as per-type sections) for seamless vertical stitching into combined long image

## [1.2.2] - 2026-04-14

### Improvements
- **Message digest: all items require traceable source**: Every item must have a valid `source_url` to an authoritative primary source, regardless of score. 7+ score items additionally require cross-source verification (`verified == true`, 2+ independent sources)
- **Message digest: inline source links by default**: Each item ends with a `🔗 URL` source link. Changed `MESSAGE_LINKS` default from `bottom` to `inline`
- **Examples switched to English**: All examples in gen-message SKILL.md and digest template are now in English for consistency
- **Enhanced X/Twitter search**: Multi-layer search strategy (official accounts → CEO/personnel → KOLs), RT/quote tweet tracing to original post time, structured per-category search with source priority hierarchy

## [1.2.0] - 2026-04-14

### New Features
- **Message digest mode** (`MESSAGE_ENABLED=true`): Generate concise, copy-paste-friendly message digests for sharing on messaging platforms (WeChat, Telegram, Slack). Each item gets a bold title + one-line summary + emoji marker (🔥/⭐/🔷 by score). Optional 9:16 portrait image for visual sharing. Configurable via `MESSAGE_MIN_SCORE`, `MESSAGE_MAX_ITEMS`, `MESSAGE_LINKS` (bottom/inline)

## [1.1.8] - 2026-04-14

### Fixes
- **Fix plugin manifest validation**: Remove `skills` and `hooks` fields from `plugin.json` (Claude Code auto-discovers root `SKILL.md`); remove `$schema` and move `description` into `metadata` in `marketplace.json` to match current Claude Code schema

## [1.1.7] - 2026-04-14

### Fixes
- **Fix skills field format**: Revert to `["./"]` (array) for `.claude-plugin/plugin.json` — matches working plugins (e.g. last30days). Previous changes to `"SKILL.md"` caused "skills: Invalid input" validation error
- **Remove redundant `.agents/plugins/`**: Deleted `marketplace.json` that used `../../` path escaping; OpenCode/Hermes discover skills via `AGENTS.md` at repo root

## [1.1.4] - 2026-04-13

### New Features
- **OpenCode & Hermes Agent support**: Added installation guides for OpenCode (native + compatible paths) and Hermes Agent (`hermes skills install`); updated sync.sh deploy targets

### Improvements
- **Factual detail verification rules**: All specific numbers (parameter counts, benchmark scores, pricing, context lengths, etc.) must now be verified from authoritative primary sources before inclusion — omit rather than guess
- **Date display restricted to cover image**: Per-type section infographic images (Model/Product/Benchmark/Funding) no longer show the date in the header; only the cover image displays the date

## [1.1.3] - 2026-04-13

### Breaking Changes
- **Removed YouTube, Discord, and ScrapeCreators data sources**: Streamlined to 5 automated sources (Reddit, HN, GitHub, HuggingFace, arXiv) + agent-driven X/Twitter search
- **X/Twitter switched to agent web search**: No API key needed — X updates are now discovered via agent web search instead of dedicated collectors

### Improvements
- **Default to long image mode**: Infographics now generate as stitched long images by default
- **Cover image switched to 16:9 landscape**: Cover infographic now uses 16:9 aspect ratio; per-type sections remain 9:16 portrait
- **Removed sparse image strategy**: Always generate cover + per-type sections + stitch into long image, regardless of item count
- **English as default report language**: Reports are now written in English by default unless `--lang` is specified
- **Cleaned up manifest examples**: Removed redundant `aspect_ratio` field from image generation manifests
- **Simplified tool compatibility docs**: Removed references to Amp and Jules; updated GitHub token prefix example

## [1.1.2] - 2026-04-13

### Improvements
- **Richer report detail for high-score items (7+)**: Added required "Why It Matters" analysis section and optional "Key Data" metrics table to record format
- **Enhanced mid-score items (5-6)**: Upgraded from one-line summaries to two-line format with concrete details, numbers, and source links
- **More informative TLDR**: Each TLDR entry now includes an Impact line explaining industry significance
- **Detail quality rule**: Summary bullet points now explicitly require specific numbers (versions, benchmarks, pricing, parameters) instead of vague descriptions

## [1.1.1] - 2026-04-10

### New Features
- **Style presets for infographics**: 5 built-in visual styles (classic, dark, glassmorphism, newspaper, tech) selectable via `IMAGE_STYLE` config
- **Style-aware image stitching**: Background color matches selected style when stitching multi-section infographics
- **Content density enforcement**: Automatic injection of content rendering rules to maximize information display in generated images
- **Section continuity rules**: Seamless visual flow between stitched image sections with per-style overrides

## [1.1.0] - 2026-04-10

### New Features
- **Onboarding flow**: First-time interactive setup guide with Step 0 gate to prevent agents skipping configuration
- **Custom entity watchlists**: Users can add personal entities to track beyond the built-in registries
- **Language control**: Default English output with `--lang` override for other languages
- **Source links**: All report items now include source URLs for reader click-through
- **Adaptive infographic generation**: Long-image layout with per-image aspect ratio support
- **MiniMax region support**: Separate `intl` and `cn` API endpoints for image generation
- **SQLite leaderboard snapshots**: Track benchmark ranking changes over time
- **Multi-image infographic**: Pluggable multi-image stitching for cover generation
- **Cron/scheduled execution**: Scheduling metadata and documentation for unattended daily runs

### Multi-Platform Support
- **Codex plugin**: Added `.codex-plugin/` with interface metadata for OpenAI Codex CLI
- **AGENTS.md**: Cross-agent skill discovery for Codex and other agent platforms
- **Gemini CLI extension**: `gemini-extension.json` with environment variable settings

### Improvements
- Refactored from Claude Code-only plugin to universal skill format
- Renamed project identity from `ai-tracker` to `morning-ai`
- Restructured `agents/` directory to `entities/` for clarity
- Colocated scripts with their skill definitions; promoted `lib/` to top level
- Output generated files to caller's working directory instead of skill directory
- Expanded tracked entities from 76+ to 80+
- Extracted KOL entities from benchmarks-academic into standalone registry
- Added new leaderboards: Vending-Bench, SimpleBench, Repo Bench
- Added KTransformers and Hermes Agent to coding-agent registry
- Merged `frontier-labs` and `china-ai` into unified `ai-labs` entity file

### Fixes
- Fixed `sys.path` resolution: use `parents[3]` to reach project root from nested scripts
- Fixed Claude Code install command: use `marketplace add` without `/plugin` prefix

## [1.0.0] - 2026-04-07

Initial release.

- 9 data sources: X/Twitter, Reddit, Hacker News, GitHub, HuggingFace, arXiv, YouTube, Discord, web search
- 76+ tracked AI entities across labs, models, benchmarks, apps, and KOLs
- Scoring and deduplication engine
- Markdown report generation with configurable templates
- Claude Code plugin with marketplace listing


path: README.md
content: # MorningAI

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python 3.9+](https://img.shields.io/badge/Python-3.9%2B-3776AB.svg)](https://python.org)
[![Entities](https://img.shields.io/badge/Tracked_Entities-80%2B-orange.svg)](#tracked-entities-80)
[![Sources](https://img.shields.io/badge/Data_Sources-6-green.svg)](#how-it-works)
[![Platforms](https://img.shields.io/badge/Platforms-6%2B-purple.svg)](#install)

**What happened in AI today?** — An AI news tracking skill that runs inside your coding agent. No Docker, no servers — just invoke `/morning-ai` in Claude Code, Codex, OpenCode, Hermes, Cursor, or any SKILL.md-compatible tool. Monitors 80+ entities across 6 sources, generates scored daily reports with optional infographics.

## Features

- **Skill-Native** — Runs inside your AI coding tool (Claude Code, Codex, OpenCode, Hermes, Cursor). No Docker, no servers, no extra infra — just `/morning-ai`
- **Entity-Centric Tracking** — 80+ curated entities across AI labs, model infra, coding agents, apps, vision/media, benchmarks, KOLs. Per-entity cross-platform handles (X, GitHub, HF, arXiv), not keyword search
- **6 Sources** — Reddit, HN, GitHub, HuggingFace, arXiv, X/Twitter. All sources free — no API keys required
- **Smart Scoring** — Two-stage scoring: automated initial scoring from metadata (relevance, engagement, source reliability, recency), then agent evaluation across 5 qualitative dimensions (Impact, Differentiation, Breakthrough, Coverage, Timeliness). Score 7+ items auto-verified across multiple independent sources
- **Custom Watchlists** — Add your own entities via simple markdown files — no code changes needed
- **5 Infographic Styles** — `classic`, `dark`, `glassmorphism`, `newspaper`, `tech` — ready for social sharing
- **Message Digest** — Concise, copy-paste-friendly digest with one-line summaries + links, optimized for WeChat, Telegram, Slack and other messaging platforms
- **Scheduled & Unattended** — Idempotent daily runs, no interactive prompts, partial success support

## Sample Output

Here's what a MorningAI daily report looks like — 26 updates scored, deduplicated, and verified across multiple sources in under 50 seconds:

<details>
<summary><b>TLDR — Today's 7+ Score Updates (click to expand)</b></summary>

- **[Model] Anthropic - Claude 4.5 Sonnet released** (9.2): New mid-tier model with +18% SWE-Bench, 200K context, 40% faster output
- **[Model] Google DeepMind - Gemini 2.5 Flash public preview** (8.8): 1M context, native multimodal, free tier on AI Studio
- **[Product] Cursor - Background Agents GA** (8.5): Autonomous agents, cloud sandbox, multi-file refactoring
- **[Model] DeepSeek - V3-0407 open-weight release** (8.3): 671B MoE, MIT license, weights on HuggingFace
- **[Product] OpenAI - Codex CLI open-sourced** (8.0): Terminal agent, suggest/auto-edit/full-auto modes
- **[Benchmark] LMSYS - Chatbot Arena April rankings** (7.5): Claude 4.5 Sonnet #2 overall, Gemini 2.5 Pro #1 coding
- **[Product] GitHub - Copilot Coding Agent preview** (7.3): Autonomous agent on issues, creates PRs in sandbox
- **[Funding] Windsurf - $200M Series C at $3B** (7.1): Largest round in coding tools space

</details>

> Full sample report: [samples/report_2026-04-08.md](samples/report_2026-04-08.md)
> Raw data JSON: [samples/data_2026-04-08.json](samples/data_2026-04-08.json)
> Infographic style previews: [docs/styles.md](docs/styles.md)

## How It Works

```
SKILL.md (loaded by any AI tool)
    |
    +- Step 1: python3 skills/tracking-list/scripts/collect.py  ->  data_{date}.json
    |           (5 sources, concurrent, score + dedupe)
    |           + Agent searches X/Twitter via web search
    |
    +- Step 2: Read skills/tracking-list/SKILL.md  ->  scoring & format spec
    |
    +- Step 3: Write report_{date}.md  ->  structured daily report
    |
    +- Step 4: (optional) Read skills/gen-infographic/SKILL.md  ->  cover image
    |
    +- Step 6: (optional) Read skills/gen-message/SKILL.md  ->  message digest + image
```

The Python collector runs 5 sources concurrently (Reddit, HN, GitHub, HuggingFace, arXiv), then scores, deduplicates, and cross-links results. The AI tool also searches X/Twitter via web search for tracked entities, then reads the JSON output and generates a formatted Markdown report.

## Install

### Claude Code

```bash
# Step 1: Add marketplace source
marketplace add octo-patch/MorningAI

# Step 2: Install the plugin
/plugin install morning-ai

# Step 3: Restart Claude Code, then use
/morning-ai
```

Or manual install:

```bash
git clone https://github.com/octo-patch/MorningAI.git ~/.claude/skills/morning-ai
```

### ClawHub

```bash
clawhub install morning-ai
```

### OpenCode

OpenCode auto-discovers skills from `.agents/skills/` and `.claude/skills/` directories — the same paths MorningAI already supports. Install as a skill:

```bash
git clone https://github.com/octo-patch/MorningAI.git ~/.config/opencode/skills/morning-ai
```

Or use any of the compatible paths:

```bash
# Any of these work — OpenCode scans all three
git clone https://github.com/octo-patch/MorningAI.git ~/.config/opencode/skills/morning-ai
git clone https://github.com/octo-patch/MorningAI.git ~/.agents/skills/morning-ai
git clone https://github.com/octo-patch/MorningAI.git ~/.claude/skills/morning-ai
```

Then invoke in OpenCode: type `morning-ai` or let the agent load the skill on demand.

> **Tip**: OpenCode also reads `AGENTS.md` at the repo root. If you clone the repo into any project directory, OpenCode will auto-discover it.

### Hermes Agent

Install directly from GitHub:

```bash
hermes skills install octo-patch/MorningAI
```

Or clone manually:

```bash
git clone https://github.com/octo-patch/MorningAI.git ~/.agents/skills/morning-ai
```

Then invoke in Hermes: type `morning-ai` or use `/skills` to browse installed skills.

> **Tip**: Hermes also supports `AGENTS.md` auto-discovery. Clone the repo into your project and Hermes will find it.

### OpenAI Codex

If cloned or forked, Codex auto-discovers the plugin via `.codex-plugin/plugin.json` and `AGENTS.md` — no manual setup needed.

Or install as a skill:

```bash
git clone https://github.com/octo-patch/MorningAI.git ~/.agents/skills/morning-ai
```

### Other Tools (Cursor, etc.)

`AGENTS.md` at the repo root is an open standard recognized by Codex, Cursor, OpenCode, Hermes and more. Clone the repo and the tool will auto-discover it.

### Manual (any tool)

```bash
git clone https://github.com/octo-patch/MorningAI.git
cd MorningAI
```

## Setup

Create a config file at `~/.config/morning-ai/.env`:

```env
GITHUB_TOKEN=gh_xxx               # GitHub (optional, higher rate limits)
MESSAGE_ENABLED=true               # Message digest (optional, for messaging platforms)
```

Without any API keys, all sources work out of the box: **Reddit**, **Hacker News**, **GitHub**, **HuggingFace**, **arXiv**, and **X/Twitter**.

## Custom Entity Watchlist

Track your own entities beyond the built-in 80+ by creating a markdown file:

```bash
cp entities/custom-example.md entities/custom/my-watchlist.md
```

Edit the file with your entities:

```markdown
## My Startup

| Platform | Value |
|----------|-------|
| X | @my_startup, @founder |
| GitHub | my-startup-org |
| HuggingFace | my-startup |
| Reddit | MyStartup |
```

Each entity starts with a `## Name` heading followed by a `| Platform | Value |` table. Not all platforms are required — add only what you need. Supported platforms: `X`, `GitHub`, `HuggingFace`, `arXiv`, `Web`, `Reddit`, `HN`.

Custom entity files are loaded from (in priority order):

1. `CUSTOM_ENTITIES_DIR` env var
2. `~/.config/morning-ai/entities/`
3. `entities/custom/` in the project directory

Files in `entities/custom/` are gitignored, so your watchlists stay local.

## Usage

Invoke the skill in your AI tool:

```
/morning-ai
```

Or run the collector standalone:

```bash
python3 skills/tracking-list/scripts/collect.py --date 2026-04-08 --output report.json
```

## Infographic Styles

Set `IMAGE_STYLE` in your `.env` to choose a visual style for generated infographics:

| Style | Description |
|-------|-------------|
| `classic` | Clean editorial magazine — off-white background, navy/coral/teal accents **(default)** |
| `dark` | Dark mode — charcoal background, electric blue/violet accents |
| `glassmorphism` | Frosted glass cards on soft gradient background — modern SaaS feel |
| `newspaper` | Classic newsprint — serif typography, cream/black/crimson, broadsheet layout |
| `tech` | Terminal aesthetic — dark background, monospace, cyan/green/amber accents |

```env
IMAGE_STYLE=dark
IMAGE_GEN_PROVIDER=gemini
```

## Tracked Entities (80+)

| Group | Entities | Count |
|-------|----------|-------|
| **ai-labs** | OpenAI, Anthropic, Google, Meta AI, xAI, Microsoft, Qwen, DeepSeek, Doubao, GLM, Kimi, MiniMax, Kling, InternLM, LongCat, Yi, Baichuan, StepFun, Hunyuan | 19 |
| **model-infra** | NVIDIA, Mistral AI, Cohere, Perplexity AI, AWS, Together AI, Groq, Apple/MLX, vLLM, SGLang, KTransformers | 11 |
| **coding-agent** | Cursor, Cline, OpenCode, Droid, OpenClaw, Windsurf, Augment, Aider, Devin, browser-use, Hermes Agent | 11 |
| **ai-apps** | v0, bolt.new, Lovable, Replit, Lovart, Manus, Genspark, Character.ai | 8 |
| **vision-media** | Midjourney, FLUX, Ideogram, Adobe Firefly, Leonardo AI, Stability AI, Lightricks, Runway, Pika, Luma AI, ElevenLabs, Udio/Suno | 12 |
| **benchmarks-academic** | LMSYS, LMArena, Artificial Analysis, HuggingFace, Scale AI SEAL, OpenCompass, LiveBench, WildBench, Terminal-Bench, Vals AI, Design Arena, Vending-Bench, SimpleBench, Repo Bench, Replicate, 5 paper channels, 2 Reddit communities, 4 industry media | 20+ |
| **kol** | Andrej Karpathy, AK, Andrew Ng, Rowan Cheung, Ben Tossell, Elie Bakouch, Swyx, Simon Willison | 8 |
| **trending-discovery** | GitHub Trending, Product Hunt, Hacker News, Reddit | 4 sources |

All items are classified into 4 types: **Product**, **Model**, **Benchmark**, **Funding**.

## Deploy to Multiple Tools

```bash
bash scripts/sync.sh
```

Distributes the skill to `~/.claude/skills/`, `~/.agents/skills/`, `~/.codex/skills/`, and `~/.config/opencode/skills/`.

## License

MIT


path: agents/openai.yaml
content: interface:
  display_name: "Morning AI"
  short_description: "Track 80+ AI entities across 6 sources with daily scored reports"
  default_prompt: "Use morning-ai to collect today's AI news from 6 sources, score and deduplicate updates, and generate a structured daily report."


path: docs/styles.md
content: # Infographic Styles

MorningAI generates social-ready infographics in 5 visual styles. Set `IMAGE_STYLE` in your `.env` to choose.

## classic (default)

Clean editorial magazine aesthetic.

- **Background**: Off-white (#FAFAF8)
- **Typography**: Inter / DM Sans — modern sans-serif
- **Accents**: Navy (#1B2A4A), coral (#E8634A), teal (#2A9D8F)
- **Layout**: Card-based with subtle shadows, generous whitespace
- **Best for**: Professional reports, team sharing, documentation

```
┌─────────────────────────────────────────────┐
│  AI News Daily · 2026-04-08                 │
│─────────────────────────────────────────────│
│                                             │
│  ┌─────────────────────────────────────┐    │
│  │ ★ 9.2  Anthropic                    │    │
│  │ Claude 4.5 Sonnet released          │    │
│  │ +18% SWE-Bench, 200K context        │    │
│  └─────────────────────────────────────┘    │
│                                             │
│  ┌─────────────────────────────────────┐    │
│  │ ★ 8.8  Google DeepMind              │    │
│  │ Gemini 2.5 Flash public preview     │    │
│  │ 1M context, native multimodal       │    │
│  └─────────────────────────────────────┘    │
│                                             │
│  ┌─────────────────────────────────────┐    │
│  │ ★ 8.5  Cursor                       │    │
│  │ Background Agents GA                │    │
│  │ Autonomous multi-file refactoring   │    │
│  └─────────────────────────────────────┘    │
│                                             │
│  8 verified updates · 6 sources · 80+ entities │
└─────────────────────────────────────────────┘
```

## dark

Dark mode with electric accents — ideal for developer audiences.

- **Background**: Charcoal (#0D1117) to dark grey (#161B22)
- **Typography**: Inter — clean and readable on dark
- **Accents**: Electric blue (#58A6FF), violet (#BC8CFF), emerald (#3FB950)
- **Layout**: Cards with subtle borders, glow effects on high-score items
- **Best for**: Developer communities, X/Twitter posts, Discord sharing

```
┌─────────────────────────────────────────────┐
│  ░░ AI News Daily · 2026-04-08 ░░           │
│─────────────────────────────────────────────│
│                                             │
│  ┌─ ◆ 9.2 ─────────────────────────────┐   │
│  │  Anthropic · Claude 4.5 Sonnet      │   │
│  │  +18% SWE-Bench · 200K ctx · 40% ↑  │   │
│  └──────────────────────────────────────┘   │
│                                             │
│  ┌─ ◆ 8.8 ─────────────────────────────┐   │
│  │  Google · Gemini 2.5 Flash          │   │
│  │  1M context · multimodal · free tier │   │
│  └──────────────────────────────────────┘   │
│                                             │
│  ┌─ ◆ 8.5 ─────────────────────────────┐   │
│  │  Cursor · Background Agents GA      │   │
│  │  Cloud sandbox · 10 concurrent      │   │
│  └──────────────────────────────────────┘   │
│                                             │
│  ▓ 8 verified · 6 sources · 47.3s          │
└─────────────────────────────────────────────┘
```

## glassmorphism

Frosted glass cards on soft gradient — modern SaaS aesthetic.

- **Background**: Soft gradient (lavender → peach → sky blue)
- **Typography**: Plus Jakarta Sans — rounded and friendly
- **Accents**: White glass cards with blur backdrop, pastel highlights
- **Layout**: Overlapping translucent cards with rounded corners
- **Best for**: Product Hunt posts, LinkedIn sharing, design-forward audiences

```
┌─────────────────────────────────────────────┐
│  ·····  AI News Daily  ·····  2026-04-08    │
│  ░░░░░░░░░░░░ gradient bg ░░░░░░░░░░░░░░░  │
│                                             │
│    ╭╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╮    │
│    ┊  ★ 9.2  Anthropic                ┊    │
│    ┊  Claude 4.5 Sonnet released      ┊    │
│    ╰╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╯    │
│      ╭╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╮    │
│      ┊  ★ 8.8  Google DeepMind       ┊    │
│      ┊  Gemini 2.5 Flash preview     ┊    │
│      ╰╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╯    │
│        ╭╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╮    │
│        ┊  ★ 8.5  Cursor              ┊    │
│        ┊  Background Agents GA       ┊    │
│        ╰╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╯    │
│                                             │
└─────────────────────────────────────────────┘
```

## newspaper

Classic newsprint with serif typography — traditional editorial gravitas.

- **Background**: Cream (#FFF8F0)
- **Typography**: Playfair Display (headlines) + Source Serif Pro (body)
- **Accents**: Black (#1A1A1A), crimson (#8B0000), gold (#B8860B) for scores
- **Layout**: Multi-column broadsheet, horizontal rules, dateline headers
- **Best for**: Newsletter embeds, email digests, executive briefings

```
┌─────────────────────────────────────────────┐
│  THE AI DAILY          Tuesday, April 8 2026│
│═════════════════════════════════════════════│
│                                             │
│  CLAUDE 4.5 SONNET     │ CURSOR AGENTS GA   │
│  RELEASED              │ NOW AVAILABLE       │
│  ─────────────────     │ ──────────────────  │
│  Anthropic unveils     │ Background agents   │
│  mid-tier model with   │ reach general       │
│  18% coding gains,     │ availability with   │
│  200K context.  [9.2]  │ cloud sandbox. [8.5]│
│                        │                     │
│  GEMINI 2.5 FLASH      │ WINDSURF $200M      │
│  PUBLIC PREVIEW        │ SERIES C            │
│  ─────────────────     │ ──────────────────  │
│  1M context, free      │ AI IDE valued at    │
│  tier on AI Studio,    │ $3B. Largest round  │
│  multimodal.    [8.8]  │ in coding tools.[7.1]│
│                                             │
│  ── 26 updates · 8 verified · 80+ entities ─│
└─────────────────────────────────────────────┘
```

## tech

Terminal aesthetic with monospace — for the hacker crowd.

- **Background**: Near-black (#0A0A0A)
- **Typography**: JetBrains Mono / Fira Code — monospace throughout
- **Accents**: Cyan (#00D4FF), green (#00FF41), amber (#FFB000)
- **Layout**: CLI-style with `>` prompts, box-drawing characters, scan-line effects
- **Best for**: Developer forums, GitHub README embeds, terminal screenshot aesthetic

```
┌─────────────────────────────────────────────┐
│  > morning-ai --date 2026-04-08             │
│  > scanning 6 sources... done (47.3s)       │
│  > 186 raw → 26 deduped → 8 verified        │
│                                             │
│  [9.2] anthropic/claude-4.5-sonnet          │
│        +18% swe-bench | 200k ctx | api live │
│                                             │
│  [8.8] google/gemini-2.5-flash              │
│        1M ctx | multimodal | free tier      │
│                                             │
│  [8.5] cursor/background-agents             │
│        GA | cloud sandbox | 10 concurrent   │
│                                             │
│  [8.3] deepseek/v3-0407                     │
│        671B MoE | MIT license | HF weights  │
│                                             │
│  [8.0] openai/codex-cli                     │
│        open-source | MIT | full-auto mode   │
│                                             │
│  > 8 verified across 3+ sources             │
│  > next run: 2026-04-09 08:00 UTC+8         │
└─────────────────────────────────────────────┘
```

## Configuration

Set your preferred style in `.env`:

```env
IMAGE_STYLE=dark
IMAGE_GEN_PROVIDER=gemini
```

Or pass it inline:

```bash
/morning-ai --style glassmorphism
```

## Long Image Mode

All styles support vertical long-image stitching for mobile-friendly scrolling (e.g., WeChat Moments, Instagram Stories):

```env
IMAGE_STITCH=true
IMAGE_STITCH_ORIENTATION=vertical
```

This generates per-type images (model, product, benchmark, funding) and stitches them into a single 9:16 portrait image.


path: entities/ai-apps.md
content: # AI Apps & Platform Entity Registry

Entity registry for AI app-building platforms and consumer AI products.


---

## Assigned Entity List

### 1. v0 (Vercel)

| Attribute | Info |
|------|------|
| **X Official Account** | [@v0](https://x.com/v0) |
| **Key People** | [@rauchg](https://x.com/rauchg) - Guillermo Rauch, Vercel CEO |
| **Vercel Changelog** | https://vercel.com/changelog |

### 2. bolt.new

| Attribute | Info |
|------|------|
| **X Official Account** | [@boltdotnew](https://x.com/boltdotnew) |
| **Key People** | [@EricSimons](https://x.com/EricSimons) - Eric Simons, StackBlitz CEO |
| **Release Notes** | https://support.bolt.new/release-notes |

### 3. Lovable

| Attribute | Info |
|------|------|
| **X Official Account** | [@Lovable](https://x.com/Lovable) |
| **Key People** | [@antonosika](https://x.com/antonosika) - Anton Osika, CEO<br>[@felixhhaas](https://x.com/felixhhaas) - Felix Haas, Co-founder |
| **Changelog** | https://lovable.dev/changelog |

### 4. Replit

| Attribute | Info |
|------|------|
| **X Official Account** | [@Replit](https://x.com/Replit) |
| **Key People** | [@amasad](https://x.com/amasad) - Amjad Masad, CEO |
| **Changelog** | https://docs.replit.com/updates |

### 5. Lovart

| Attribute | Info |
|------|------|
| **X Official Account** | [@lovart_ai](https://x.com/lovart_ai) |
| **Key People** | [@Elena_Leung_29](https://x.com/Elena_Leung_29) - Elena Leung, COO |
| **Changelog** | https://www.lovart.ai/changelog |

### 6. Manus

| Attribute | Info |
|------|------|
| **X Official Account** | [@ManusAI](https://x.com/ManusAI) |
| **Key People** | [@Red_Xiao_](https://x.com/Red_Xiao_) - Red Xiao, CEO<br>[@pelotonben](https://x.com/pelotonben) - Yichao Peak Ji, Co-founder |

### 7. Genspark

| Attribute | Info |
|------|------|
| **X Official Account** | [@genspark_ai](https://x.com/genspark_ai) |
| **Key People** | [@ericjing_ai](https://x.com/ericjing_ai) - Eric Jing, CEO<br>[@sang_wen](https://x.com/sang_wen) - Wen Sang, COO |
| **Changelog** | https://www.genspark.ai/docs/ai_slides_changelog |

### 8. Character.ai

| Attribute | Info |
|------|------|
| **X Official Account** | [@character_ai](https://x.com/character_ai) |
| **Official Website** | https://character.ai |
| **Reddit Community** | [r/CharacterAI](https://www.reddit.com/r/CharacterAI/) |

---



path: entities/ai-labs.md
content: # AI Labs Entity Registry

Entity registry for major AI labs worldwide, including model releases, product updates, visual models, and academic research.

---

## Assigned Entity List

### Frontier Labs

#### 1. OpenAI (Full Line)

| Attribute | Info |
|------|------|
| **X Official Account** | [@OpenAI](https://x.com/OpenAI), [@ChatGPTapp](https://x.com/ChatGPTapp) |
| **Key People** | [@sama](https://x.com/sama) - Sam Altman, CEO<br>[@markchen90](https://x.com/markchen90) - Mark Chen, SVP Research<br>[@gdb](https://x.com/gdb) - Greg Brockman, Co-founder<br>[@nickturley](https://x.com/nickturley) - Nick Turley, ChatGPT Product Lead |
| **Official Blog** | https://openai.com/blog |
| **Research** | https://openai.com/research |
| **API Changelog** | https://platform.openai.com/docs/changelog |
| **ChatGPT Release Notes** | https://help.openai.com/en/articles/6825453-chatgpt-release-notes |
| **Sora Page** | https://openai.com/sora |
| **GitHub** | https://github.com/openai |
| **arXiv** | https://arxiv.org/search/?query=OpenAI |
| **Covered Products** | GPT series models, ChatGPT product, DALL-E, Sora, Codex CLI |
| **Reddit Keywords** | OpenAI, ChatGPT, GPT-5 |
| **Reddit Community** | [r/OpenAI](https://www.reddit.com/r/OpenAI/), [r/ChatGPT](https://www.reddit.com/r/ChatGPT/) |
| **HN Keywords** | OpenAI |

#### 2. Anthropic (Full Line)

| Attribute | Info |
|------|------|
| **X Official Account** | [@AnthropicAI](https://x.com/AnthropicAI), [@claudeai](https://x.com/claudeai), [@claude_code](https://x.com/claude_code) |
| **Key People** | [@DarioAmodei](https://x.com/DarioAmodei) - Dario Amodei, CEO<br>[@alexalbert__](https://x.com/alexalbert__) - Alex Albert, Claude Relations<br>[@bcherny](https://x.com/bcherny) - Boris Cherny, Claude Code core<br>[@trq212](https://x.com/trq212) - Thariq, Claude Code core |
| **Official Blog** | https://www.anthropic.com/news |
| **Research** | https://www.anthropic.com/research |
| **Release Notes** | https://docs.anthropic.com/en/release-notes/overview, https://support.claude.com/en/articles/12138966-release-notes |
| **Claude Code Changelog** | https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md |
| **NPM Package** | https://www.npmjs.com/package/@anthropic-ai/claude-code |
| **GitHub** | https://github.com/anthropics |
| **arXiv** | https://arxiv.org/search/?query=Anthropic |
| **Covered Products** | Claude series models, Claude Code, Claude Desktop/Web |
| **Reddit Keywords** | Anthropic, Claude |
| **Reddit Community** | [r/ClaudeAI](https://www.reddit.com/r/ClaudeAI/) |
| **HN Keywords** | Anthropic, Claude |

#### 3. Google (Full Line)

| Attribute | Info |
|------|------|
| **X Official Account** | [@GoogleDeepMind](https://x.com/GoogleDeepMind), [@GoogleAI](https://x.com/GoogleAI), [@GoogleAIStudio](https://x.com/GoogleAIStudio), [@googleaidevs](https://x.com/googleaidevs), [@GeminiApp](https://x.com/GeminiApp) |
| **Key People** | [@JeffDean](https://x.com/JeffDean) - Jeff Dean, Chief Scientist<br>[@OfficialLoganK](https://x.com/OfficialLoganK) - Logan Kilpatrick, Product Lead |
| **DeepMind Blog** | https://deepmind.google/discover/blog |
| **Research Blog** | https://research.google/blog |
| **Publications** | https://deepmind.google/research/publications |
| **Gemini API Changelog** | https://ai.google.dev/gemini-api/docs/changelog |
| **Gemini Release Notes** | https://gemini.google/release-notes/ |
| **Gemini Updates Blog** | https://blog.google/products/gemini/ |
| **Antigravity Changelog** | https://antigravity.google/changelog |
| **Developer Blog** | https://developers.googleblog.com |
| **GitHub** | https://github.com/google-deepmind |
| **HuggingFace** | https://huggingface.co/google |
| **arXiv** | https://arxiv.org/search/?query=DeepMind |
| **Covered Products** | Gemini series models, Gemini App, AI Studio, Antigravity, Veo |
| **Reddit Keywords** | Gemini, DeepMind |
| **Reddit Community** | [r/Gemini](https://www.reddit.com/r/Gemini/), [r/GoogleGeminiAI](https://www.reddit.com/r/GoogleGeminiAI/) |
| **HN Keywords** | Gemini, DeepMind |

#### 4. Meta AI (Full Line)

| Attribute | Info |
|------|------|
| **X Official Account** | [@AIatMeta](https://x.com/AIatMeta) |
| **Key People** | [@ylecun](https://x.com/ylecun) - Yann LeCun, Chief AI Scientist |
| **Official Blog** | https://ai.meta.com/blog |
| **GitHub** | https://github.com/meta-llama |
| **HuggingFace** | https://huggingface.co/meta-llama |
| **arXiv** | https://arxiv.org/search/?query=Meta+AI+FAIR |
| **Covered Products** | Llama series models, Meta AI Studio |
| **Reddit Keywords** | Llama, Meta AI |
| **Reddit Community** | [r/LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/) |
| **HN Keywords** | Llama, Meta AI |

#### 5. xAI (Full Line)

| Attribute | Info |
|------|------|
| **X Official Account** | [@xai](https://x.com/xai) |
| **Key People** | [@elonmusk](https://x.com/elonmusk) - Elon Musk, Founder |
| **Official Blog** | https://x.ai/blog |
| **GitHub** | https://github.com/xai-org |
| **HuggingFace** | https://huggingface.co/xai-org |
| **Covered Products** | Grok series models |
| **Reddit Keywords** | Grok, xAI |
| **Reddit Community** | [r/Grok](https://www.reddit.com/r/Grok/) |

#### 6. Microsoft (Full Line)

| Attribute | Info |
|------|------|
| **X Official Account** | [@Microsoft](https://x.com/Microsoft), [@MSFTCopilot](https://x.com/MSFTCopilot), [@OpenAtMicrosoft](https://x.com/OpenAtMicrosoft) |
| **AI Blog** | https://blogs.microsoft.com/ai/, https://www.microsoft.com/en-us/ai/blog |
| **GitHub** | https://github.com/microsoft |
| **HuggingFace** | https://huggingface.co/microsoft |
| **arXiv** | https://arxiv.org/search/?query=Microsoft+AI |
| **Covered Products** | Copilot, Phi series models, Azure OpenAI, GitHub Copilot |
| **Reddit Keywords** | Copilot, Phi model |
| **Reddit Community** | [r/MicrosoftCopilot](https://www.reddit.com/r/MicrosoftCopilot/) |
| **HN Keywords** | Copilot, Microsoft AI |

### China AI Labs

#### 7. Alibaba (Qwen)

| Attribute | Info |
|------|------|
| **X Official Account** | [@Alibaba_Qwen](https://x.com/Alibaba_Qwen) |
| **Official Blog** | https://qwenlm.github.io/blog |
| **GitHub** | https://github.com/QwenLM |
| **HuggingFace** | https://huggingface.co/Qwen |
| **arXiv** | https://arxiv.org/search/?query=Qwen |
| **Reddit Keywords** | Qwen |
| **Reddit Community** | [r/Qwen_AI](https://www.reddit.com/r/Qwen_AI/) |

#### 8. DeepSeek

| Attribute | Info |
|------|------|
| **X Official Account** | [@deepseek_ai](https://x.com/deepseek_ai) |
| **Official Website** | https://www.deepseek.com |
| **Changelog** | https://api-docs.deepseek.com/updates |
| **GitHub** | https://github.com/deepseek-ai |
| **HuggingFace** | https://huggingface.co/deepseek-ai |
| **arXiv** | https://arxiv.org/search/?query=DeepSeek |
| **Reddit Keywords** | DeepSeek |
| **Reddit Community** | [r/DeepSeek](https://www.reddit.com/r/DeepSeek/) |
| **HN Keywords** | DeepSeek |

#### 9. ByteDance (Doubao)

| Attribute | Info |
|------|------|
| **X Official Account** | [@doubaoAi](https://x.com/doubaoAi) |
| **Volcengine** | https://www.volcengine.com/product/doubao |
| **GitHub** | https://github.com/bytedance |
| **HuggingFace** | https://huggingface.co/bytedance |
| **Covered Products** | Doubao, Coze |

#### 10. Zhipu AI (GLM)

| Attribute | Info |
|------|------|
| **X Official Account** | [@Zai_org](https://x.com/Zai_org) |
| **Official Website** | https://www.zhipuai.cn |
| **Changelog** | https://docs.bigmodel.cn/cn/update/new-releases |
| **GitHub** | https://github.com/THUDM |
| **HuggingFace** | https://huggingface.co/THUDM |
| **arXiv** | https://arxiv.org/search/?query=GLM |

#### 11. Moonshot AI (Kimi)

| Attribute | Info |
|------|------|
| **X Official Account** | [@Kimi_Moonshot](https://x.com/Kimi_Moonshot) |
| **Official Website** | https://www.moonshot.cn |
| **Changelog** | https://platform.moonshot.cn/blog/posts/changelog |
| **Kimi Product** | https://kimi.ai |
| **GitHub** | https://github.com/MoonshotAI |

#### 12. MiniMax / Hailuo

| Attribute | Info |
|------|------|
| **X Official Account** | [@MiniMax_AI](https://x.com/MiniMax_AI), [@MiniMaxAI](https://x.com/MiniMaxAI) |
| **Changelog** | https://platform.minimaxi.com/docs/release-notes/models |
| **GitHub** | https://github.com/MiniMax-AI |
| **HuggingFace** | https://huggingface.co/MiniMaxAI |
| **Hailuo Product** | https://hailuoai.com |
| **Covered Products** | MiniMax Models, Hailuo Video |
| **Reddit Keywords** | MiniMax, Hailuo |
| **Reddit Community** | [r/HailuoAiOfficial](https://www.reddit.com/r/HailuoAiOfficial/) |

#### 13. Kuaishou (Kling)

| Attribute | Info |
|------|------|
| **X Official Account** | [@Kling_ai](https://x.com/Kling_ai), [@AiKling](https://x.com/AiKling) |
| **Official Website** | https://klingai.com |

#### 14. Shanghai AI Lab (InternLM)

| Attribute | Info |
|------|------|
| **X Official Account** | [@Shanghai_AI_Lab](https://x.com/Shanghai_AI_Lab) |
| **GitHub** | https://github.com/InternLM |
| **HuggingFace** | https://huggingface.co/internlm |
| **arXiv** | arXiv InternLM/InternVL related papers |

#### 15. Meituan (LongCat)

| Attribute | Info |
|------|------|
| **X Official Account** | [@Meituan_LongCat](https://x.com/Meituan_LongCat) |
| **Tech Blog** | https://tech.meituan.com/tags/longcat.html |
| **GitHub** | https://github.com/orgs/meituan-longcat/repositories |
| **HuggingFace** | https://huggingface.co/meituan-longcat |

#### 16. 01.AI (Yi Series)

| Attribute | Info |
|------|------|
| **X Official Account** | [@01ai_yi](https://x.com/01ai_yi) |
| **GitHub** | https://github.com/01-ai |
| **HuggingFace** | https://huggingface.co/01-ai |

#### 17. Baichuan AI

| Attribute | Info |
|------|------|
| **X Official Account** | [@Baichuan_AI](https://x.com/Baichuan_AI) |
| **GitHub** | https://github.com/baichuan-inc |
| **HuggingFace** | https://huggingface.co/baichuan-inc |

#### 18. StepFun

| Attribute | Info |
|------|------|
| **X Official Account** | [@StepFunAI](https://x.com/StepFunAI) |
| **Official Website** | https://www.stepfun.com |

#### 19. Tencent Hunyuan

| Attribute | Info |
|------|------|
| **X Official Account** | [@TencentHunyuan](https://x.com/TencentHunyuan), [@TXhunyuan](https://x.com/TXhunyuan) |
| **Official Website** | https://hunyuan.tencent.com |
| **Research** | https://hunyuan.tencent.com/research |
| **GitHub** | https://github.com/Tencent-Hunyuan |
| **HuggingFace** | https://huggingface.co/tencent, https://huggingface.co/Tencent-Hunyuan |
| **arXiv** | https://arxiv.org/search/?query=Tencent+Hunyuan |
| **Covered Products** | Hunyuan LLM, HunyuanVideo, HunyuanImage |
| **Reddit Keywords** | Hunyuan, Tencent AI |


---



path: entities/benchmarks-academic.md
content: # Benchmarks & Academic Entity Registry

Entity registry for benchmark organizations, academic research channels, and industry media.


---

## Assigned Entity List

### Benchmark Organizations

| Name | X Account | Website | Modality | Description |
|------|--------|------|------|------|
| LMArena | [@arena](https://x.com/arena) | - | Text, Vision | Model battle ranking |
| Artificial Analysis | [@ArtificialAnlys](https://x.com/ArtificialAnlys) | https://artificialanalysis.ai | Text, Image, Video, Audio | Model speed/quality comparison |
| HuggingFace Open LLM | [@huggingface](https://x.com/huggingface) | https://huggingface.co/spaces/open-llm-leaderboard | Text | Open-source model leaderboard |
| Scale AI SEAL | [@scale_AI](https://x.com/scale_AI) | https://scale.com/leaderboard | Text | Scale Benchmark |
| LMSYS Chatbot Arena | [@laboramsys](https://x.com/laboramsys) | https://lmsys.org | Text, Vision | Chatbot Arena / VLM Arena |
| OpenCompass | - | https://opencompass.org.cn | Text, Multimodal | Multimodal benchmark |
| LiveBench | - | https://livebench.ai | Text | Real-time benchmark |
| WildBench | - | https://huggingface.co/spaces/allenai/WildBench | Text | Allen AI Benchmark |
| Terminal-Bench | - | https://www.tbench.ai | Code | Terminal agent benchmark |
| Vals AI | [@ValsAI](https://x.com/ValsAI) | https://www.vals.ai | Text | Domain-specific benchmark (law, finance, code) |
| Design Arena | [@Designarena](https://x.com/Designarena) | https://www.designarena.ai/leaderboard | Code, Image, Video | Crowdsourced AI design benchmark |
| Vending-Bench | [@andonlabs](https://x.com/andonlabs) | https://andonlabs.com/evals/vending-bench | Text | Long-term agent coherence benchmark |
| SimpleBench | [@AIExplainedYT](https://x.com/AIExplainedYT) | https://simple-bench.com | Text | Commonsense reasoning benchmark |
| Repo Bench | [@RepoPrompt](https://x.com/RepoPrompt) | https://repoprompt.com/bench | Code | Large context file editing benchmark |

### Additional Benchmark Sources

| Name | X Account | Description |
|------|--------|------|
| HuggingFace | [@huggingface](https://x.com/huggingface) | Model releases, trends |
| Replicate | [@replicate](https://x.com/replicate) | Model deployment, trends |

### Paper Channels

| Name | URL | Type |
|------|-----|------|
| Papers with Code | https://paperswithcode.com | Papers + Code |
| HuggingFace Papers | https://huggingface.co/papers | Daily trending papers |
| arXiv cs.LG | https://arxiv.org/list/cs.LG/recent | Machine Learning |
| arXiv cs.CL | https://arxiv.org/list/cs.CL/recent | Natural Language Processing |
| arXiv cs.CV | https://arxiv.org/list/cs.CV/recent | Computer Vision |

### Community Discussions

| Name | URL | Type |
|------|-----|------|
| Reddit r/MachineLearning | https://reddit.com/r/MachineLearning | ML academic discussion |
| Reddit r/LocalLLaMA | https://reddit.com/r/LocalLLaMA | Open-source LLM community |
| Reddit r/artificial | https://reddit.com/r/artificial | AI general discussion |
| Reddit r/singularity | https://reddit.com/r/singularity | AI frontier discussion |

### Industry Media (for lead discovery only, must trace to source)

| Name | URL |
|------|-----|
| VentureBeat AI | https://venturebeat.com/ai |
| The Decoder | https://the-decoder.com |
| Ars Technica AI | https://arstechnica.com/ai |
| TechCrunch AI | https://techcrunch.com/category/artificial-intelligence/ |

---

## Deduplication Notes

This Agent focuses on **cross-vendor** benchmarks/papers/trends. The following content is covered by other Agents, this Agent skips:

- Specific vendor model/product updates -> covered by corresponding vendor Agent
- Vendor-specific technical papers -> if papers are from OpenAI/Anthropic etc., covered by ai-labs etc. Agent
- This Agent only focuses on: independent benchmark results, cross-vendor comparisons, general academic research, open-source community updates, industry trends

---



path: entities/coding-agent.md
content: # Coding Agent Entity Registry

Entity registry for AI coding agents and developer assistant platforms (agents under major labs such as Claude Code, Codex CLI, GitHub Copilot are handled by the ai-labs group).

---

## Assigned Entity List

### 1. Cursor

| Attribute | Info |
|------|------|
| **X Official Account** | [@cursor_ai](https://x.com/cursor_ai) |
| **Key People** | [@mntruell](https://x.com/mntruell) - Michael Truell, CEO<br>[@ericzakariasson](https://x.com/ericzakariasson) - Eric Zakariasson |
| **Changelog** | https://cursor.com/changelog |
| **Reddit Keywords** | Cursor AI |
| **Reddit Community** | [r/cursor](https://www.reddit.com/r/cursor/) |
| **HN Keywords** | Cursor |

### 2. Cline

| Attribute | Info |
|------|------|
| **X Official Account** | [@cline](https://x.com/cline) |
| **Key People** | [@sdrzn](https://x.com/sdrzn) - Saoud Rizwan, CEO<br>[@nickbaumann_](https://x.com/nickbaumann_) - Nick Baumann |
| **GitHub Releases** | https://github.com/cline/cline/releases |
| **Reddit Keywords** | Cline AI |
| **Reddit Community** | [r/ClineAI](https://www.reddit.com/r/ClineAI/) |

### 3. OpenCode

| Attribute | Info |
|------|------|
| **Key People** | [@thdxr](https://x.com/thdxr) - Dax Raad, Founder<br>[@fanjiewang](https://x.com/fanjiewang) - Fanjie Wang |
| **GitHub Releases** | https://github.com/sst/opencode/releases |

### 4. Droid (Factory AI)

| Attribute | Info |
|------|------|
| **X Official Account** | [@FactoryAI](https://x.com/FactoryAI) |
| **CLI Changelog** | https://docs.factory.ai/changelog/cli-updates |

### 5. OpenClaw (formerly Clawd/Moltbot)

| Attribute | Info |
|------|------|
| **X Official Account** | [@openclaw](https://x.com/openclaw) |
| **Key People** | [@steipete](https://x.com/steipete) - Peter Steinberger, Founder |
| **Official Website** | https://openclaw.ai |
| **Docs** | https://docs.openclaw.ai |
| **Blog** | https://openclaws.io/blog |
| **Changelog** | https://openclawai.io/changelog |
| **GitHub** | https://github.com/openclaw/openclaw |
| **GitHub Releases** | https://github.com/openclaw/openclaw/releases |
| **Reddit Keywords** | OpenClaw |
| **Reddit Community** | [r/OpenClaw](https://www.reddit.com/r/OpenClaw/) |

### 6. Windsurf (Codeium)

| Attribute | Info |
|------|------|
| **X Official Account** | [@windsurf](https://x.com/windsurf) |
| **Official Blog** | https://windsurf.com/blog |
| **Changelog** | https://windsurf.com/changelog |
| **Reddit Keywords** | Windsurf, Codeium |
| **Reddit Community** | [r/Windsurf](https://www.reddit.com/r/Windsurf/) |

### 7. Augment Code

| Attribute | Info |
|------|------|
| **X Official Account** | [@AugmentCode](https://x.com/AugmentCode) |
| **Official Blog** | https://www.augmentcode.com/blog |

### 8. Aider

| Attribute | Info |
|------|------|
| **Key People** | Paul Gauthier, Creator |
| **Official Website** | https://aider.chat |
| **GitHub** | https://github.com/Aider-AI/aider |
| **GitHub Releases** | https://github.com/Aider-AI/aider/releases |

### 9. Devin (Cognition)

| Attribute | Info |
|------|------|
| **X Official Account** | [@cognition](https://x.com/cognition) |
| **Official Blog** | https://cognition.ai/blog |
| **Reddit Keywords** | Devin AI |
| **Reddit Community** | [r/DevinAI](https://www.reddit.com/r/DevinAI/) |

### 10. browser-use

| Attribute | Info |
|------|------|
| **X Official Account** | [@browser_use](https://x.com/browser_use) |
| **Key People** | [@gregpr07](https://x.com/gregpr07), [@larsencc](https://x.com/larsencc) |
| **Official Website** | https://browser-use.com |
| **Changelog** | https://browser-use.com/changelog |
| **GitHub** | https://github.com/browser-use/browser-use |
| **GitHub Releases** | https://github.com/browser-use/browser-use/releases |

### 11. Hermes Agent (Nous Research)

| Attribute | Info |
|------|------|
| **X Official Account** | [@NousResearch](https://x.com/NousResearch) |
| **Official Website** | https://hermes-agent.nousresearch.com |
| **Docs** | https://hermes-agent.nousresearch.com/docs |
| **GitHub** | https://github.com/NousResearch/hermes-agent |

---



path: entities/custom-example.md
content: # Custom Entity Watchlist — Example

<!-- This is a template. Copy it to create your own watchlist:
     cp entities/custom-example.md entities/custom/my-watchlist.md
     Then edit the copy. Files in entities/custom/ are gitignored. -->

<!-- FORMAT:
     - Each entity starts with a ## heading
     - Add a table with | Platform | Value | rows
     - Not all platforms are required — only add what you need
     - Multiple values per field: separate with commas

     SUPPORTED PLATFORMS:
     | Platform         | Description                | Example Value                         |
     |----------------- |--------------------------- |---------------------------------------|
     | X                | X/Twitter handles          | @handle1, @handle2                    |
     | GitHub           | Orgs or user/repo          | org-name, user/repo                   |
     | HuggingFace      | HF author/org names        | author1, author2                      |
     | arXiv            | arXiv search terms         | search term 1, search term 2          |
     | Web              | Blog/changelog URLs        | site:example.com/blog                 |
     | Reddit           | Reddit search keywords     | keyword1, keyword2                    |
     | Reddit Community | Dedicated subreddits       | r/ExampleAI, r/ExampleProduct         |
     | HN               | Hacker News keywords       | keyword1                              |
-->

## Example Corp

| Platform | Value |
|----------|-------|
| X | @example_ai, @example_ceo |
| GitHub | example-ai, example-ai/flagship-model |
| HuggingFace | example-ai |
| arXiv | Example Corp |
| Reddit | Example AI, ExampleModel |
| Reddit Community | r/ExampleAI |
| HN | Example Corp |

## Another Project

| Platform | Value |
|----------|-------|
| X | @another_project |
| GitHub | another/project-repo |
| Web | site:another-project.dev/blog |


path: entities/kol.md
content: # Key Opinion Leaders (KOL) Entity Registry

Entity registry for AI industry key opinion leaders and influential voices.


---

## Assigned Entity List

### Key Opinion Leaders (KOL)

| Name | X Account | Description |
|------|--------|------|
| Andrej Karpathy | [@karpathy](https://x.com/karpathy) | Former Tesla AI / OpenAI founding member |
| AK (Paper digest) | [@_akhaliq](https://x.com/_akhaliq) | Real-time AI paper releases, HuggingFace member |
| Andrew Ng | [@AndrewYNg](https://x.com/AndrewYNg) | AI pioneer |
| Rowan Cheung | [@rowancheung](https://x.com/rowancheung) | The Rundown AI Founder |
| Ben Tossell | [@bentossell](https://x.com/bentossell) | Ben's Bites Founder |
| Elie Bakouch | [@eliebakouch](https://x.com/eliebakouch) | AI news blogger |
| Swyx | [@swyx](https://x.com/swyx) | Latent Space Host, AI engineering advocate |
| Simon Willison | [@simonw](https://x.com/simonw) | Django Co-creator, in-depth AI tool reviews |

> **Note**: KOL information must be traced back to official channels for confirmation, cannot be used directly as a primary source

---

## Deduplication Notes

This entity group tracks **individual opinion leaders**. The following content is covered by other entities, this group skips:

- Official company announcements -> covered by corresponding vendor entity
- Benchmark results/papers -> covered by benchmarks-academic entity
- This group only focuses on: KOL commentary, trend analysis, independent reviews, community discussions initiated by KOLs

---


path: entities/model-infra.md
content: # Model & Infrastructure Entity Registry

Entity registry for model providers and inference infrastructure platforms.


---

## Assigned Entity List

### 1. NVIDIA

| Attribute | Info |
|------|------|
| **X Official Account** | [@nvidia](https://x.com/nvidia), [@NVIDIAAIDev](https://x.com/NVIDIAAIDev) |
| **Key People** | [@DrJimFan](https://x.com/DrJimFan) - Jim Fan, Robotics Director |
| **Official Blog** | https://blogs.nvidia.com |
| **GitHub** | https://github.com/NVIDIA |
| **HuggingFace** | https://huggingface.co/nvidia |
| **arXiv** | https://arxiv.org/search/?query=NVIDIA+AI |
| **Reddit Keywords** | NVIDIA AI, CUDA |
| **Reddit Community** | [r/nvidia](https://www.reddit.com/r/nvidia/) |

### 2. Mistral AI

| Attribute | Info |
|------|------|
| **X Official Account** | [@MistralAI](https://x.com/MistralAI) |
| **Key People** | [@arthurmensch](https://x.com/arthurmensch) - Arthur Mensch, CEO |
| **Official Blog** | https://mistral.ai/news |
| **Changelog** | https://docs.mistral.ai/getting-started/changelog |
| **GitHub** | https://github.com/mistralai |
| **HuggingFace** | https://huggingface.co/mistralai |
| **arXiv** | https://arxiv.org/search/?query=Mistral+AI |
| **Reddit Keywords** | Mistral |
| **Reddit Community** | [r/MistralAI](https://www.reddit.com/r/MistralAI/) |
| **HN Keywords** | Mistral |

### 3. Cohere

| Attribute | Info |
|------|------|
| **X Official Account** | [@cohere](https://x.com/cohere) |
| **Official Blog** | https://cohere.com/blog |
| **GitHub** | https://github.com/cohere-ai |
| **HuggingFace** | https://huggingface.co/CohereForAI |
| **Reddit Keywords** | Cohere, Command R |
| **Reddit Community** | [r/CohereAI](https://www.reddit.com/r/CohereAI/) |

### 4. Perplexity AI

| Attribute | Info |
|------|------|
| **X Official Account** | [@perplexity_ai](https://x.com/perplexity_ai) |
| **Key People** | [@AravSrinivas](https://x.com/AravSrinivas) - Aravind Srinivas, CEO<br>[@DenisYarats](https://x.com/DenisYarats) - Denis Yarats, CTO |
| **Official Blog** | https://www.perplexity.ai/hub/blog |
| **Reddit Keywords** | Perplexity |
| **Reddit Community** | [r/perplexity_ai](https://www.reddit.com/r/perplexity_ai/) |
| **HN Keywords** | Perplexity |

### 5. Amazon/AWS (AI)

| Attribute | Info |
|------|------|
| **X Official Account** | [@awscloud](https://x.com/awscloud) |
| **AI Blog** | https://aws.amazon.com/blogs/machine-learning/ |
| **Bedrock** | https://aws.amazon.com/bedrock/ |

### 6. Together AI

| Attribute | Info |
|------|------|
| **X Official Account** | [@togethercomputer](https://x.com/togethercomputer) |
| **Official Blog** | https://together.ai/blog |
| **GitHub** | https://github.com/togethercomputer |

### 7. Groq

| Attribute | Info |
|------|------|
| **X Official Account** | [@GroqInc](https://x.com/GroqInc) |
| **Official Blog** | https://wow.groq.com/blog/ |

### 8. Apple (MLX/AI)

| Attribute | Info |
|------|------|
| **ML Research Blog** | https://machinelearning.apple.com |
| **GitHub (MLX)** | https://github.com/ml-explore |
| **HuggingFace** | https://huggingface.co/apple |

### Inference Frameworks

| Name | X Account | GitHub | Description |
|------|--------|--------|------|
| vLLM | [@vllm_project](https://x.com/vllm_project) | https://github.com/vllm-project/vllm | High-throughput LLM inference engine |
| SGLang | [@sgl_project](https://x.com/sgl_project) | https://github.com/sgl-project/sglang | High-performance LLM/VLM serving framework |
| KTransformers | - | https://github.com/kvcache-ai/ktransformers | Heterogeneous CPU/GPU inference framework |

---



path: entities/trending-discovery.md
content: # Trending Discovery Entity Registry

Entity registry for trending platform scanning — discovers emerging AI tools, projects, and important updates beyond known entities.

---

## Scan Sources

### 1. GitHub Trending (AI/ML)

| Attribute | Info |
|------|------|
| **Trending Page** | https://github.com/trending?since=daily |
| **Focus Area** | AI/ML related trending repos |
| **Filter Criteria** | AI/ML related, significant star growth today |

### 2. Product Hunt

| Attribute | Info |
|------|------|
| **Daily Page** | https://www.producthunt.com |
| **Focus Area** | Newly launched products in AI category |
| **Filter Criteria** | AI/ML tags, top voted |

### 3. Hacker News Front Page

| Attribute | Info |
|------|------|
| **API** | https://hacker-news.firebaseio.com/v0/topstories |
| **Algolia Search** | https://hn.algolia.com/api/v1/search |
| **Focus Area** | AI/ML/LLM related top stories |
| **Filter Criteria** | Title or content contains AI/ML/LLM keywords, score > 50 |

### 4. Reddit Frontier Discussions

| Attribute | Info |
|------|------|
| **r/singularity** | https://reddit.com/r/singularity | AI frontier discussion |
| **r/artificial** | https://reddit.com/r/artificial | AI general discussion |
| **r/MachineLearning** | https://reddit.com/r/MachineLearning | ML academic discussion |
| **r/LocalLLaMA** | https://reddit.com/r/LocalLLaMA | Open-source LLM community |
| **Filter Criteria** | Top upvoted posts within 24h |
| **Note** | Entity-specific subreddits (e.g. r/DeepSeek, r/ClaudeAI) are fetched per-entity via `Reddit Community` field |

---

## Working Rules

1. **Exclude known entities**: If discovered content belongs to another Agent's entity (e.g., OpenAI, Cursor, etc.), record but mark as "already covered by XX Agent"
2. **Focus on discovery**: Prioritize reporting the following types of discoveries:
   - Brand new AI tools/products debut
   - Open-source projects exploding (star count surge)
   - Important industry events (policy, partnerships, milestones)
   - Influential community discussions or controversies
3. **Scoring criteria**: Follow shared specification, but new tools/projects typically score 3-5, unless widespread attention

---



path: entities/vision-media.md
content: # Vision & Media Entity Registry

Entity registry for independent visual/audio/video generation vendors (major lab vision models like Sora, Veo, DALL-E, Kling, Hailuo are covered by ai-labs).


---

## Assigned Entity List

### Image Generation

#### 1. Midjourney

| Attribute | Info |
|------|------|
| **X Official Account** | [@midjourney](https://x.com/midjourney) |
| **Key People** | [@DavidHolz](https://x.com/DavidHolz) - David Holz, CEO |
| **Official Website** | https://www.midjourney.com |
| **Discord** | https://discord.gg/midjourney |
| **Reddit Community** | [r/midjourney](https://www.reddit.com/r/midjourney/) |

#### 2. FLUX / Black Forest Labs

| Attribute | Info |
|------|------|
| **X Official Account** | [@bfl_ml](https://x.com/bfl_ml) |
| **GitHub** | https://github.com/black-forest-labs |
| **HuggingFace** | https://huggingface.co/black-forest-labs |

#### 3. Ideogram

| Attribute | Info |
|------|------|
| **X Official Account** | [@ideaboramAI](https://x.com/ideaboramAI) |
| **Official Website** | https://ideogram.ai |

#### 4. Adobe Firefly

| Attribute | Info |
|------|------|
| **X Official Account** | [@AdobeFirefly](https://x.com/AdobeFirefly) |
| **Official Blog** | https://blog.adobe.com/en/topics/creativity |

#### 5. Leonardo AI

| Attribute | Info |
|------|------|
| **X Official Account** | [@LeonardoAi_](https://x.com/LeonardoAi_) |
| **Official Website** | https://leonardo.ai |

#### 6. Stability AI

| Attribute | Info |
|------|------|
| **X Official Account** | [@StabilityAI](https://x.com/StabilityAI) |
| **Key People** | [@robrombach](https://x.com/robrombach) - Robin Rombach |
| **Official Blog** | https://stability.ai/news |
| **GitHub** | https://github.com/Stability-AI |
| **HuggingFace** | https://huggingface.co/stabilityai |
| **Discord** | https://discord.gg/stablediffusion |
| **Reddit Keywords** | Stable Diffusion, Stability AI |
| **Reddit Community** | [r/StableDiffusion](https://www.reddit.com/r/StableDiffusion/) |

#### 7. Lightricks

| Attribute | Info |
|------|------|
| **X Official Account** | [@Lightricks](https://x.com/Lightricks) |
| **Official Blog** | https://www.lightricks.com/blog |

### Video Generation

#### 8. Runway

| Attribute | Info |
|------|------|
| **X Official Account** | [@runwayml](https://x.com/runwayml) |
| **Key People** | [@cpvalenzuela](https://x.com/cpvalenzuela) - Cristobal Valenzuela, CEO |
| **Official Blog** | https://runwayml.com/blog |
| **Changelog** | https://runwayml.com/changelog |

#### 9. Pika Labs

| Attribute | Info |
|------|------|
| **X Official Account** | [@pika_labs](https://x.com/pika_labs) |
| **Key People** | [@demi_chen](https://x.com/demi_chen) - Demi Chen, Co-founder |
| **Official Website** | https://pika.art |

#### 10. Luma AI

| Attribute | Info |
|------|------|
| **X Official Account** | [@LumaLabsAI](https://x.com/LumaLabsAI) |
| **Key People** | [@abolishingme](https://x.com/abolishingme) - Amit Jain, CEO |
| **Official Website** | https://lumalabs.ai |

### Audio/Speech/Music

#### 11. ElevenLabs

| Attribute | Info |
|------|------|
| **X Official Account** | [@elevenlabsio](https://x.com/elevenlabsio) |
| **Official Blog** | https://elevenlabs.io/blog |
| **Reddit Community** | [r/ElevenLabs](https://www.reddit.com/r/ElevenLabs/) |

#### 12. Udio / Suno (Music AI)

| Attribute | Info |
|------|------|
| **X Official Account (Udio)** | [@udiomusic](https://x.com/udiomusic) |
| **X Official Account (Suno)** | [@SunoAIMusic](https://x.com/SunoAIMusic) |
| **Udio Website** | https://www.udio.com |
| **Suno Website** | https://suno.com |
| **Reddit Community** | [r/SunoAI](https://www.reddit.com/r/SunoAI/), [r/udio](https://www.reddit.com/r/udio/) |

---



path: hooks/hooks.json
content: {
  "hooks": {
    "SessionStart": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "bash ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/check-config.sh",
            "timeout": 5
          }
        ]
      }
    ]
  }
}


path: hooks/scripts/check-config.sh
content: #!/usr/bin/env bash
# morning-ai configuration check — runs on session start
set -euo pipefail

PROJECT_ENV=".claude/morning-ai.env"
GLOBAL_ENV="$HOME/.config/morning-ai/.env"

# Find active config
ACTIVE_ENV=""
if [[ -f "$PROJECT_ENV" ]]; then
    ACTIVE_ENV="$PROJECT_ENV"
elif [[ -f ".env" ]]; then
    ACTIVE_ENV=".env"
elif [[ -f "$GLOBAL_ENV" ]]; then
    ACTIVE_ENV="$GLOBAL_ENV"
fi

# Load env if found
if [[ -n "$ACTIVE_ENV" ]]; then
    set -a
    # shellcheck disable=SC1090
    source "$ACTIVE_ENV" 2>/dev/null || true
    set +a
fi

# ── No config found: first-time onboarding ──────────────────────────────
if [[ -z "$ACTIVE_ENV" ]]; then
    cat <<'WELCOME'
morning-ai: First-time setup

What is morning-ai?
  AI news daily report generator. Tracks 80+ entities (OpenAI, Anthropic,
  Google, Meta, xAI, DeepSeek, Cursor, Midjourney, etc.) across 5 automated
  sources + agent-driven X/Twitter search. Generates scored, deduplicated
  Markdown reports with optional infographics.

FREE sources (work immediately, no keys needed):
  Reddit            public JSON API
  Hacker News       Algolia API
  GitHub            public API (optional token for higher rate limits)
  HuggingFace       public API
  arXiv             public API
  X/Twitter         agent web search (no API key needed)

API keys (optional):
  GITHUB_TOKEN             GitHub higher rate limits   https://github.com/settings/tokens

Image generation (optional):
  IMAGE_GEN_PROVIDER       gemini | minimax | none (default: none)
  IMAGE_STYLE              classic | dark | glassmorphism | newspaper | tech
  GEMINI_API_KEY           Google Gemini/Imagen    https://aistudio.google.com/apikey
  MINIMAX_API_KEY          MiniMax global          https://www.minimax.io
  MINIMAX_API_KEY          MiniMax cn              https://platform.minimaxi.com

Setup:
  Create ~/.config/morning-ai/.env with KEY=value format (one per line).
  All sources work without API keys. Add GITHUB_TOKEN for higher rate limits.

Next: run /morning-ai
WELCOME
    exit 0
fi

# ── Config exists: show source status ────────────────────────────────────
SOURCES=5  # Reddit, HN, GitHub, HuggingFace, arXiv are always available
DETAILS="reddit,hackernews,github,huggingface,arxiv,x(agent-search)"

if [[ -n "${GITHUB_TOKEN:-}" ]]; then
    DETAILS="$DETAILS,github-token(enhanced)"
fi

IMG=""
if [[ -n "${IMAGE_GEN_PROVIDER:-}" && "${IMAGE_GEN_PROVIDER:-}" != "none" ]]; then
    IMG=" | image: ${IMAGE_GEN_PROVIDER}"
fi

echo "morning-ai: $SOURCES sources + X agent search [$DETAILS]${IMG} (config: $ACTIVE_ENV)"


path: lib/__init__.py
content: # morning-ai data collection library


path: lib/arxiv.py
content: """arXiv collector for morning-ai.

Searches arXiv papers via the Atom/XML API (free, no auth).
New collector — no last30days equivalent.
"""

import re
import sys
import xml.etree.ElementTree as ET
from typing import Any, Dict, List, Optional
from urllib.parse import urlencode, quote

from . import http
from .schema import TrackerItem, Engagement, CollectionResult, SOURCE_ARXIV

ARXIV_API = "http://export.arxiv.org/api/query"

DEPTH_CONFIG = {"quick": 10, "default": 25, "deep": 50}

# arXiv categories for AI
AI_CATEGORIES = ["cs.AI", "cs.LG", "cs.CL", "cs.CV", "cs.MA"]

ATOM_NS = "{http://www.w3.org/2005/Atom}"


def _log(msg: str):
    if sys.stderr.isatty():
        sys.stderr.write(f"[arXiv] {msg}\n")
        sys.stderr.flush()


def _parse_date(date_str: Optional[str]) -> Optional[str]:
    if not date_str:
        return None
    if len(date_str) >= 10:
        return date_str[:10]
    return None


def _clean_text(text: str) -> str:
    """Clean arXiv abstract text."""
    text = re.sub(r"\s+", " ", text)
    return text.strip()


def search_papers(
    query: str,
    from_date: str,
    to_date: str,
    categories: Optional[List[str]] = None,
    max_results: int = 25,
) -> List[Dict[str, Any]]:
    """Search arXiv papers via Atom API.

    Args:
        query: Search query (author, title, abstract keywords)
        from_date: Start date YYYY-MM-DD
        to_date: End date YYYY-MM-DD
        categories: arXiv categories to filter (e.g. ["cs.CL", "cs.LG"])
        max_results: Max papers to return

    Returns:
        List of paper dicts
    """
    # Build arXiv search query
    search_parts = []
    if query:
        search_parts.append(f"all:{query}")
    if categories:
        cat_query = " OR ".join(f"cat:{cat}" for cat in categories)
        search_parts.append(f"({cat_query})")

    search_query = " AND ".join(search_parts) if search_parts else query

    params = urlencode({
        "search_query": search_query,
        "start": "0",
        "max_results": str(max_results),
        "sortBy": "submittedDate",
        "sortOrder": "descending",
    })
    url = f"{ARXIV_API}?{params}"

    try:
        xml_text = http.get(url, timeout=30, raw=True)
    except Exception as e:
        _log(f"arXiv search '{query}' failed: {e}")
        return []

    # Parse XML
    try:
        root = ET.fromstring(xml_text)
    except ET.ParseError as e:
        _log(f"Failed to parse arXiv XML: {e}")
        return []

    papers = []
    for entry in root.findall(f"{ATOM_NS}entry"):
        title_el = entry.find(f"{ATOM_NS}title")
        summary_el = entry.find(f"{ATOM_NS}summary")
        published_el = entry.find(f"{ATOM_NS}published")
        updated_el = entry.find(f"{ATOM_NS}updated")

        title = _clean_text(title_el.text) if title_el is not None and title_el.text else ""
        abstract = _clean_text(summary_el.text) if summary_el is not None and summary_el.text else ""
        published = _parse_date(published_el.text if published_el is not None else None)
        updated = _parse_date(updated_el.text if updated_el is not None else None)

        date = published or updated
        if date and (date < from_date or date > to_date):
            continue

        # Get paper URL
        paper_url = ""
        pdf_url = ""
        for link in entry.findall(f"{ATOM_NS}link"):
            href = link.get("href", "")
            link_type = link.get("type", "")
            title_attr = link.get("title", "")
            if title_attr == "pdf" or "pdf" in href:
                pdf_url = href
            elif link_type == "text/html" or (href and "abs" in href):
                paper_url = href
            elif not paper_url and href:
                paper_url = href

        # Get authors
        authors = []
        for author_el in entry.findall(f"{ATOM_NS}author"):
            name_el = author_el.find(f"{ATOM_NS}name")
            if name_el is not None and name_el.text:
                authors.append(name_el.text.strip())

        # Get categories
        cats = []
        for cat_el in entry.findall("{http://arxiv.org/schemas/atom}primary_category"):
            term = cat_el.get("term", "")
            if term:
                cats.append(term)
        for cat_el in entry.findall(f"{ATOM_NS}category"):
            term = cat_el.get("term", "")
            if term and term not in cats:
                cats.append(term)

        # Extract arXiv ID from URL
        arxiv_id = ""
        if paper_url:
            match = re.search(r"(\d{4}\.\d{4,5})(v\d+)?$", paper_url)
            if match:
                arxiv_id = match.group(1)

        papers.append({
            "arxiv_id": arxiv_id,
            "title": title,
            "abstract": abstract,
            "authors": authors,
            "date": date,
            "published": published,
            "updated": updated,
            "url": paper_url,
            "pdf_url": pdf_url,
            "categories": cats,
        })

    _log(f"arXiv '{query}': {len(papers)} papers in date range")
    return papers


def collect(
    entities: Dict[str, List[str]],
    from_date: str,
    to_date: str,
    depth: str = "default",
) -> CollectionResult:
    """Collect arXiv papers for tracked entities.

    Args:
        entities: Dict mapping entity name -> list of search queries
            e.g. {"OpenAI": ["OpenAI"], "Anthropic": ["Anthropic"]}
        from_date: Start date YYYY-MM-DD
        to_date: End date YYYY-MM-DD
        depth: Search depth

    Returns:
        CollectionResult
    """
    result = CollectionResult(source=SOURCE_ARXIV)
    all_items = []
    max_results = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"])

    for entity_name, queries in entities.items():
        result.entities_checked += 1
        entity_found = False

        for query in queries:
            papers = search_papers(
                query, from_date, to_date,
                categories=AI_CATEGORIES,
                max_results=max_results,
            )

            for paper in papers:
                authors = paper.get("authors", [])
                author_str = ", ".join(authors[:3])
                if len(authors) > 3:
                    author_str += f" et al. ({len(authors)} authors)"

                cats = paper.get("categories", [])
                cat_str = ", ".join(cats[:3])

                abstract = paper.get("abstract", "")
                summary = f"By {author_str}. {abstract[:200]}" if author_str else abstract[:300]

                all_items.append(TrackerItem(
                    id=f"ARXIV-{paper.get('arxiv_id', '')}",
                    title=paper.get("title", ""),
                    summary=summary,
                    entity=entity_name,
                    source=SOURCE_ARXIV,
                    source_url=paper.get("url", ""),
                    source_label=f"arXiv [{cat_str}]" if cat_str else "arXiv",
                    date=paper.get("date"),
                    date_confidence="high" if paper.get("date") else "low",
                    raw_text=abstract,
                    relevance=0.7,
                ))
                entity_found = True

        if entity_found:
            result.entities_with_updates += 1

    result.items = all_items
    _log(f"Collected {len(all_items)} arXiv papers from {result.entities_checked} entities")
    return result


path: lib/cache.py
content: """Caching utilities for morning-ai (adapted from last30days)."""

import hashlib
import json
import os
import tempfile
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Optional

CACHE_DIR = Path.home() / ".cache" / "morning-ai"
DEFAULT_TTL_HOURS = 1  # short TTL for daily tracking — prevents stale cross-day cache


def ensure_cache_dir():
    global CACHE_DIR
    env_dir = os.environ.get("MORNING_AI_CACHE_DIR")
    if env_dir:
        CACHE_DIR = Path(env_dir)
    try:
        CACHE_DIR.mkdir(parents=True, exist_ok=True)
    except PermissionError:
        CACHE_DIR = Path(tempfile.gettempdir()) / "morning-ai" / "cache"
        CACHE_DIR.mkdir(parents=True, exist_ok=True)


def get_cache_key(source: str, entity: str, date: str) -> str:
    key_data = f"{source}|{entity}|{date}"
    return hashlib.sha256(key_data.encode()).hexdigest()[:16]


def get_cache_path(cache_key: str) -> Path:
    return CACHE_DIR / f"{cache_key}.json"


def is_cache_valid(cache_path: Path, ttl_hours: int = DEFAULT_TTL_HOURS) -> bool:
    if not cache_path.exists():
        return False
    try:
        stat = cache_path.stat()
        mtime = datetime.fromtimestamp(stat.st_mtime, tz=timezone.utc)
        now = datetime.now(timezone.utc)
        age_hours = (now - mtime).total_seconds() / 3600
        return age_hours < ttl_hours
    except OSError:
        return False


def load_cache(cache_key: str, ttl_hours: int = DEFAULT_TTL_HOURS) -> Optional[dict]:
    cache_path = get_cache_path(cache_key)
    if not is_cache_valid(cache_path, ttl_hours):
        return None
    try:
        with open(cache_path, "r") as f:
            return json.load(f)
    except (json.JSONDecodeError, OSError):
        return None


def save_cache(cache_key: str, data: dict):
    ensure_cache_dir()
    cache_path = get_cache_path(cache_key)
    try:
        with open(cache_path, "w") as f:
            json.dump(data, f, ensure_ascii=False)
    except OSError:
        pass


def clear_cache():
    if CACHE_DIR.exists():
        for f in CACHE_DIR.glob("*.json"):
            try:
                f.unlink()
            except OSError:
                pass


path: lib/custom_entities.py
content: """Entity loader for morning-ai.

Parses entity markdown files from entities/ and merges them into
platform-specific registries at runtime.

Supports three markdown formats:
  1. Standard — ``#### N. Entity Name`` headers with ``| Attribute | Info |`` tables
  2. Tabular — ``| Name | X Account | ... |`` row-per-entity tables (KOL, benchmarks)
  3. Custom  — ``## Entity Name`` headers with ``| Platform | Value |`` tables

Search paths for custom entities (first match wins):
    1. CUSTOM_ENTITIES_DIR env var
    2. ~/.config/morning-ai/entities/
    3. {project_root}/entities/custom/
"""

import os
import re
import sys
from pathlib import Path
from typing import Dict, List, Any, Tuple
from urllib.parse import unquote_plus


# Platform field → target dict key mapping (for custom entity format)
PLATFORM_KEYS = {
    "x": "x_handles",
    "github": "github_sources",
    "huggingface": "huggingface_authors",
    "arxiv": "arxiv_queries",
    "web": "web_queries",
    "reddit": "reddit_keywords",
    "reddit_community": "reddit_subreddits",
    "hn": "hn_keywords",
}

# Attribute names that map to X handles
_X_ATTRS = {"x official account", "x account"}

# Attribute names that map to GitHub sources
_GITHUB_ATTRS = {"github", "github releases"}

# Attribute keywords that map to web_queries (blogs, changelogs, docs, etc.)
_WEB_KEYWORDS = [
    "blog", "changelog", "release notes", "research", "docs",
    "official website", "page", "volcengine", "npm package",
]

# Files to skip when loading built-in entities
_SKIP_FILES = {"custom-example.md", "trending-discovery.md"}


# ---------------------------------------------------------------------------
# Value extraction helpers
# ---------------------------------------------------------------------------

def _extract_x_handles(value: str) -> List[str]:
    """Extract X handles from markdown like ``[@OpenAI](https://x.com/OpenAI)``."""
    handles = re.findall(r'@(\w+)', value)
    return handles


def _extract_github(value: str) -> Dict[str, List[str]]:
    """Extract GitHub orgs and repos from URLs."""
    orgs: List[str] = []
    repos: List[str] = []
    for match in re.finditer(r'github\.com/([^/\s\)]+)(?:/([^/\s\)]+))?', value):
        org = match.group(1)
        repo_name = match.group(2)
        if repo_name and repo_name not in ("releases", "blob", "tree", "wiki", "issues", "pulls"):
            repo = f"{org}/{repo_name}"
            if repo not in repos:
                repos.append(repo)
        else:
            if org not in orgs:
                orgs.append(org)
    return {"orgs": orgs, "repos": repos}


def _extract_hf_authors(value: str) -> List[str]:
    """Extract HuggingFace author/org from URLs like ``huggingface.co/google``."""
    authors = []
    for match in re.finditer(r'huggingface\.co/([^/\s\)]+)', value):
        name = match.group(1)
        if name not in ("spaces", "papers", "datasets", "models"):
            authors.append(name)
    return authors


def _extract_arxiv_queries(value: str) -> List[str]:
    """Extract arXiv search queries from URLs or plain text."""
    queries = []
    for match in re.finditer(r'query=([^&\s\)]+)', value):
        queries.append(unquote_plus(match.group(1)))
    return queries


def _urls_to_site_queries(value: str) -> List[str]:
    """Convert URLs to ``site:`` search queries."""
    queries = []
    for match in re.finditer(r'https?://(?:www\.)?([^\s\)]+)', value):
        url = match.group(1).rstrip("/,.")
        # Skip repo-level GitHub/HF/arXiv URLs (handled by dedicated extractors)
        # But keep deep-path GitHub URLs (blob/tree/wiki — these are web content)
        if url.startswith("github.com"):
            parts = url.split("/")
            if len(parts) <= 3:  # github.com/org or github.com/org/repo
                continue
        if url.startswith("huggingface.co") or url.startswith("arxiv.org"):
            continue
        queries.append(f"site:{url}")
    return queries


def _parse_multi(value: str) -> List[str]:
    """Split comma-separated value and strip whitespace."""
    return [v.strip() for v in value.split(",") if v.strip()]


def _extract_subreddits(value: str) -> List[str]:
    """Extract subreddit names from Reddit Community field.

    Handles formats like:
      - [r/DeepSeek](https://www.reddit.com/r/DeepSeek/)
      - r/DeepSeek
      - https://www.reddit.com/r/DeepSeek/
    """
    subs = []
    for match in re.finditer(r'r/(\w+)', value):
        sub = match.group(1)
        if sub not in subs:
            subs.append(sub)
    return subs


def _parse_x_handles_custom(value: str) -> List[str]:
    """Parse X handles from custom format, stripping @ prefix."""
    return [h.lstrip("@") for h in _parse_multi(value)]


def _parse_github_custom(value: str) -> Dict[str, List[str]]:
    """Parse GitHub value from custom format into orgs and repos."""
    orgs = []
    repos = []
    for item in _parse_multi(value):
        if "/" in item:
            repos.append(item)
        else:
            orgs.append(item)
    return {"orgs": orgs, "repos": repos}


# ---------------------------------------------------------------------------
# Built-in entity format parser (Attribute | Info tables)
# ---------------------------------------------------------------------------

def _classify_attr(attr: str) -> str:
    """Map an attribute name to a platform key or 'skip'."""
    lower = attr.lower().strip("* ")

    # Exact matches first
    if lower in _X_ATTRS:
        return "x"
    if lower.startswith("x official account"):  # handles "(Suno)", "(Udio)" suffixes
        return "x"
    if lower in _GITHUB_ATTRS or lower.startswith("github"):
        return "github"
    if lower == "huggingface":
        return "huggingface"
    if lower == "arxiv":
        return "arxiv"
    if lower == "key people":
        return "key_people"
    if lower == "reddit keywords":
        return "reddit"
    if lower == "reddit community":
        return "reddit_community"
    if lower == "hn keywords":
        return "hn"

    # Web query keywords
    for kw in _WEB_KEYWORDS:
        if kw in lower:
            return "web"

    return "skip"


def parse_builtin_file(path: Path) -> Dict[str, Dict[str, Any]]:
    """Parse a built-in entity file with ``| Attribute | Info |`` format.

    Also handles tabular sections (``| Name | X Account | ... |``).
    """
    result = {key: {} for key in PLATFORM_KEYS.values()}
    text = path.read_text(encoding="utf-8")
    lines = text.split("\n")

    i = 0
    while i < len(lines):
        line = lines[i]

        # Detect entity header: ### N. Name or #### N. Name
        header_match = re.match(r'^#{2,4}\s+(?:\d+\.\s+)?(.+)', line)
        if header_match:
            raw_name = header_match.group(1).strip()
            # Strip suffixes like "(Full Line)"
            raw_name = re.sub(r'\s*\(Full Line\)\s*$', '', raw_name)
            entity_name = raw_name.strip()

            # Check if next non-empty lines contain a table
            i += 1
            # Look ahead for table
            while i < len(lines) and not lines[i].strip():
                i += 1

            if i < len(lines) and lines[i].strip().startswith("|"):
                # Detect table type by header row
                header_row = lines[i].strip()
                cols = [c.strip() for c in header_row.split("|") if c.strip()]

                if len(cols) >= 2 and cols[0].lower() in ("attribute", "info"):
                    # Standard Attribute | Info table
                    i += 1  # skip header
                    # Skip separator row
                    if i < len(lines) and lines[i].strip().startswith("|--"):
                        i += 1
                    # Parse rows
                    while i < len(lines) and lines[i].strip().startswith("|"):
                        row = lines[i].strip()
                        parts = [p.strip() for p in row.split("|")]
                        parts = [p for p in parts if p]
                        if len(parts) >= 2 and not parts[0].startswith("--"):
                            attr_name = parts[0].strip("* ")
                            attr_value = parts[1]
                            _store_attr(result, entity_name, attr_name, attr_value)
                        i += 1

                elif len(cols) >= 2 and cols[0].lower() == "name":
                    # Tabular format: Name | X Account | ... (multiple entities)
                    col_headers = [c.lower().strip("* ") for c in cols]
                    i += 1
                    if i < len(lines) and lines[i].strip().startswith("|--"):
                        i += 1
                    while i < len(lines) and lines[i].strip().startswith("|"):
                        row = lines[i].strip()
                        parts = [p.strip() for p in row.split("|")]
                        parts = [p for p in parts if p]
                        if len(parts) >= 2 and not parts[0].startswith("--"):
                            row_name = parts[0].strip()
                            _store_tabular_row(result, row_name, col_headers, parts)
                        i += 1
                else:
                    i += 1
            continue

        # Also detect standalone tabular sections (no entity header, like benchmarks)
        if line.strip().startswith("|"):
            cols = [c.strip() for c in line.split("|") if c.strip()]
            if len(cols) >= 2 and cols[0].lower() == "name":
                col_headers = [c.lower().strip("* ") for c in cols]
                i += 1
                if i < len(lines) and lines[i].strip().startswith("|--"):
                    i += 1
                while i < len(lines) and lines[i].strip().startswith("|"):
                    row = lines[i].strip()
                    parts = [p.strip() for p in row.split("|")]
                    parts = [p for p in parts if p]
                    if len(parts) >= 2 and not parts[0].startswith("--"):
                        row_name = parts[0].strip()
                        _store_tabular_row(result, row_name, col_headers, parts)
                    i += 1
                continue

        i += 1

    return result


def _store_attr(result: dict, entity: str, attr_name: str, value: str):
    """Store a single attribute value into the appropriate registry."""
    platform = _classify_attr(attr_name)

    if platform == "x":
        handles = _extract_x_handles(value)
        if handles:
            result["x_handles"].setdefault(entity, []).extend(handles)
    elif platform == "key_people":
        # Extract X handles from Key People field
        handles = _extract_x_handles(value)
        if handles:
            result["x_handles"].setdefault(entity, []).extend(handles)
    elif platform == "github":
        gh = _extract_github(value)
        existing = result["github_sources"].get(entity, {"orgs": [], "repos": []})
        existing["orgs"].extend(o for o in gh["orgs"] if o not in existing["orgs"])
        existing["repos"].extend(r for r in gh["repos"] if r not in existing["repos"])
        result["github_sources"][entity] = existing
    elif platform == "huggingface":
        authors = _extract_hf_authors(value)
        if authors:
            result["huggingface_authors"].setdefault(entity, []).extend(authors)
    elif platform == "arxiv":
        queries = _extract_arxiv_queries(value)
        if queries:
            result["arxiv_queries"].setdefault(entity, []).extend(queries)
    elif platform == "web":
        queries = _urls_to_site_queries(value)
        if queries:
            result["web_queries"].setdefault(entity, []).extend(queries)
    elif platform == "reddit":
        keywords = _parse_multi(value)
        if keywords:
            result["reddit_keywords"][entity] = keywords
    elif platform == "reddit_community":
        subs = _extract_subreddits(value)
        if subs:
            result["reddit_subreddits"].setdefault(entity, []).extend(
                s for s in subs if s not in result["reddit_subreddits"].get(entity, [])
            )
    elif platform == "hn":
        keywords = _parse_multi(value)
        if keywords:
            result["hn_keywords"][entity] = keywords


def _store_tabular_row(result: dict, name: str, col_headers: List[str], parts: List[str]):
    """Store a row from a tabular entity table (Name | X Account | ...)."""
    for idx, header in enumerate(col_headers):
        if idx >= len(parts):
            break
        value = parts[idx]
        if not value or value == "-":
            continue

        if header in ("x account",):
            handles = _extract_x_handles(value)
            if handles:
                result["x_handles"].setdefault(name, []).extend(handles)
        elif header == "github":
            gh = _extract_github(value)
            existing = result["github_sources"].get(name, {"orgs": [], "repos": []})
            existing["orgs"].extend(o for o in gh["orgs"] if o not in existing["orgs"])
            existing["repos"].extend(r for r in gh["repos"] if r not in existing["repos"])
            result["github_sources"][name] = existing
        elif header == "website":
            queries = _urls_to_site_queries(value)
            if queries:
                result["web_queries"].setdefault(name, []).extend(queries)


# ---------------------------------------------------------------------------
# Built-in entity loader
# ---------------------------------------------------------------------------

def load_builtin_entities() -> Dict[str, Dict[str, Any]]:
    """Load all built-in entity files from ``entities/`` directory.

    Returns:
        Merged dictionary with keys matching PLATFORM_KEYS values.
    """
    merged = {key: {} for key in PLATFORM_KEYS.values()}
    entities_dir = Path(__file__).resolve().parent.parent / "entities"

    if not entities_dir.is_dir():
        return merged

    for md_file in sorted(entities_dir.glob("*.md")):
        if md_file.name in _SKIP_FILES:
            continue
        if md_file.name.startswith("custom"):
            continue
        try:
            parsed = parse_builtin_file(md_file)
            for dict_key in PLATFORM_KEYS.values():
                for entity, values in parsed[dict_key].items():
                    if dict_key in ("github_sources",):
                        existing = merged[dict_key].get(entity, {"orgs": [], "repos": []})
                        existing["orgs"].extend(o for o in values["orgs"] if o not in existing["orgs"])
                        existing["repos"].extend(r for r in values["repos"] if r not in existing["repos"])
                        merged[dict_key][entity] = existing
                    elif isinstance(values, list):
                        merged[dict_key].setdefault(entity, []).extend(
                            v for v in values if v not in merged[dict_key].get(entity, [])
                        )
                    else:
                        merged[dict_key][entity] = values
        except Exception as e:
            print(f"[morning-ai] Warning: failed to parse {md_file}: {e}", file=sys.stderr)

    return merged


# ---------------------------------------------------------------------------
# Custom entity format parser (Platform | Value tables)
# ---------------------------------------------------------------------------

def parse_custom_file(path: Path) -> Dict[str, Dict[str, Any]]:
    """Parse a single custom entity markdown file.

    Returns:
        Dict with keys matching PLATFORM_KEYS values, each mapping
        entity names to their platform-specific values.
    """
    result = {key: {} for key in PLATFORM_KEYS.values()}

    text = path.read_text(encoding="utf-8")
    sections = re.split(r"^## +", text, flags=re.MULTILINE)

    for section in sections[1:]:
        lines = section.strip().split("\n")
        entity_name = lines[0].strip()
        if not entity_name:
            continue

        for line in lines[1:]:
            line = line.strip()
            if not line.startswith("|") or line.startswith("|--") or line.startswith("| Platform"):
                continue
            parts = [p.strip() for p in line.split("|")]
            parts = [p for p in parts if p]
            if len(parts) < 2:
                continue

            platform = parts[0].lower().strip("*")
            value = parts[1]

            if platform == "x":
                result["x_handles"][entity_name] = _parse_x_handles_custom(value)
            elif platform == "github":
                result["github_sources"][entity_name] = _parse_github_custom(value)
            elif platform == "huggingface":
                result["huggingface_authors"][entity_name] = _parse_multi(value)
            elif platform == "arxiv":
                result["arxiv_queries"][entity_name] = _parse_multi(value)
            elif platform == "web":
                result["web_queries"][entity_name] = _parse_multi(value)
            elif platform == "reddit":
                result["reddit_keywords"][entity_name] = _parse_multi(value)
            elif platform in ("reddit community", "reddit_community"):
                subs = _extract_subreddits(value)
                if subs:
                    result["reddit_subreddits"][entity_name] = subs
            elif platform == "hn":
                result["hn_keywords"][entity_name] = _parse_multi(value)

    return result


def _get_search_dirs() -> List[Path]:
    """Get custom entity directories in priority order."""
    dirs = []
    env_dir = os.environ.get("CUSTOM_ENTITIES_DIR")
    if env_dir:
        dirs.append(Path(env_dir))
    dirs.append(Path.home() / ".config" / "morning-ai" / "entities")
    project_root = Path(__file__).resolve().parent.parent
    dirs.append(project_root / "entities" / "custom")
    return dirs


def load_custom_entities() -> Dict[str, Dict[str, Any]]:
    """Load all custom entity files from search directories.

    Returns:
        Merged dictionary of all custom entities across all files and directories.
    """
    merged = {key: {} for key in PLATFORM_KEYS.values()}

    for search_dir in _get_search_dirs():
        if not search_dir.is_dir():
            continue
        for md_file in sorted(search_dir.glob("*.md")):
            if md_file.name == "custom-example.md":
                continue
            try:
                parsed = parse_custom_file(md_file)
                for dict_key in PLATFORM_KEYS.values():
                    merged[dict_key].update(parsed[dict_key])
            except Exception as e:
                print(f"[morning-ai] Warning: failed to parse {md_file}: {e}", file=sys.stderr)

    return merged


def merge_into_registries(
    x_handles: dict,
    github_sources: dict,
    huggingface_authors: dict,
    arxiv_queries: dict,
    web_queries: dict,
    reddit_keywords: dict,
    reddit_subreddits: dict,
    hn_keywords: dict,
) -> None:
    """Load custom entities and merge into the provided registry dicts."""
    custom = load_custom_entities()

    x_handles.update(custom["x_handles"])
    github_sources.update(custom["github_sources"])
    huggingface_authors.update(custom["huggingface_authors"])
    arxiv_queries.update(custom["arxiv_queries"])
    web_queries.update(custom["web_queries"])
    reddit_keywords.update(custom["reddit_keywords"])
    reddit_subreddits.update(custom["reddit_subreddits"])
    hn_keywords.update(custom["hn_keywords"])


path: lib/dedupe.py
content: """Near-duplicate detection for morning-ai.

Adapted from last30days dedupe.py.
Supports within-source dedup and cross-source linking.
"""

import re
from typing import List, Set, Tuple

from .schema import TrackerItem

# Stopwords for token-based matching
STOPWORDS = frozenset({
    "the", "a", "an", "to", "for", "how", "is", "in", "of", "on",
    "and", "with", "from", "by", "at", "this", "that", "it", "my",
    "your", "i", "me", "we", "you", "what", "are", "do", "can",
    "its", "be", "or", "not", "no", "so", "if", "but", "about",
    "all", "just", "get", "has", "have", "was", "will", "new",
    "ai", "model", "release", "update", "now", "available",
})


def normalize_text(text: str) -> str:
    """Normalize text for comparison."""
    text = text.lower()
    text = re.sub(r"[^\w\s]", " ", text)
    text = re.sub(r"\s+", " ", text)
    return text.strip()


def get_ngrams(text: str, n: int = 3) -> Set[str]:
    """Get character n-grams from text."""
    text = normalize_text(text)
    if len(text) < n:
        return {text}
    return {text[i:i + n] for i in range(len(text) - n + 1)}


def jaccard_similarity(set1: Set[str], set2: Set[str]) -> float:
    """Compute Jaccard similarity between two sets."""
    if not set1 or not set2:
        return 0.0
    intersection = len(set1 & set2)
    union = len(set1 | set2)
    return intersection / union if union > 0 else 0.0


def get_item_text(item: TrackerItem) -> str:
    """Get comparable text from a TrackerItem."""
    if item.title:
        return item.title
    return item.raw_text[:150]


def _tokenize(text: str) -> Set[str]:
    """Tokenize text for token-level Jaccard."""
    words = re.sub(r"[^\w\s]", " ", text.lower()).split()
    return {w for w in words if w not in STOPWORDS and len(w) > 1}


def _token_jaccard(text_a: str, text_b: str) -> float:
    """Token-level Jaccard similarity."""
    tokens_a = _tokenize(text_a)
    tokens_b = _tokenize(text_b)
    if not tokens_a or not tokens_b:
        return 0.0
    intersection = len(tokens_a & tokens_b)
    union = len(tokens_a | tokens_b)
    return intersection / union if union else 0.0


def hybrid_similarity(text_a: str, text_b: str) -> float:
    """Hybrid similarity: max of char-trigram Jaccard and token Jaccard."""
    trigram_sim = jaccard_similarity(get_ngrams(text_a), get_ngrams(text_b))
    token_sim = _token_jaccard(text_a, text_b)
    return max(trigram_sim, token_sim)


def find_duplicates(
    items: List[TrackerItem],
    threshold: float = 0.7,
) -> List[Tuple[int, int]]:
    """Find near-duplicate pairs by Jaccard similarity.

    Args:
        items: List of items to check
        threshold: Similarity threshold (0-1)

    Returns:
        List of (i, j) index pairs where items are similar
    """
    duplicates = []
    ngrams = [get_ngrams(get_item_text(item)) for item in items]

    for i in range(len(items)):
        for j in range(i + 1, len(items)):
            similarity = jaccard_similarity(ngrams[i], ngrams[j])
            if similarity >= threshold:
                duplicates.append((i, j))

    return duplicates


def dedupe_items(
    items: List[TrackerItem],
    threshold: float = 0.7,
) -> List[TrackerItem]:
    """Remove near-duplicates, keeping highest-scored item.

    Args:
        items: List of items (should be pre-sorted by importance desc)
        threshold: Similarity threshold

    Returns:
        Deduplicated items
    """
    if len(items) <= 1:
        return items

    dup_pairs = find_duplicates(items, threshold)

    to_remove = set()
    for i, j in dup_pairs:
        if items[i].importance >= items[j].importance:
            to_remove.add(j)
        else:
            to_remove.add(i)

    return [item for idx, item in enumerate(items) if idx not in to_remove]


def dedupe_by_source(items: List[TrackerItem], threshold: float = 0.7) -> List[TrackerItem]:
    """Deduplicate items within each source separately.

    More efficient than global dedup for large item sets.

    Args:
        items: All collected items
        threshold: Similarity threshold

    Returns:
        Deduplicated items
    """
    by_source = {}
    for item in items:
        by_source.setdefault(item.source, []).append(item)

    result = []
    for source, source_items in by_source.items():
        source_items.sort(key=lambda x: x.importance, reverse=True)
        deduped = dedupe_items(source_items, threshold)
        result.extend(deduped)

    return result


def cross_source_link(
    items: List[TrackerItem],
    threshold: float = 0.40,
) -> List[TrackerItem]:
    """Link similar items across different sources.

    Uses hybrid similarity (char-trigram + token Jaccard) for cross-source
    matching. Adds bidirectional cross_refs when similarity exceeds threshold.

    Also considers entity matching: items about the same entity get a
    similarity boost for more accurate cross-referencing.

    Args:
        items: All items from all sources
        threshold: Similarity threshold for cross-linking

    Returns:
        Items with cross_refs populated
    """
    if len(items) <= 1:
        return items

    # Pre-compute texts for comparison (truncate long texts for fairness)
    texts = []
    for item in items:
        text = get_item_text(item)
        if item.source == "x":
            text = text[:100]
        texts.append(text)

    for i in range(len(items)):
        for j in range(i + 1, len(items)):
            # Skip same-source comparisons
            if items[i].source == items[j].source:
                continue

            similarity = hybrid_similarity(texts[i], texts[j])

            # Entity match bonus: same entity increases likelihood of cross-ref
            if items[i].entity and items[i].entity == items[j].entity:
                similarity += 0.1

            if similarity >= threshold:
                if items[j].id not in items[i].cross_refs:
                    items[i].cross_refs.append(items[j].id)
                if items[i].id not in items[j].cross_refs:
                    items[j].cross_refs.append(items[i].id)

    return items


path: lib/entities.py
content: """Entity registry for morning-ai.

Loads 80+ tracked entities from ``entities/*.md`` at import time.
Custom entities are merged from:
    1. CUSTOM_ENTITIES_DIR env var
    2. ~/.config/morning-ai/entities/
    3. {project_root}/entities/custom/
"""

from lib.custom_entities import load_builtin_entities, merge_into_registries

# Load built-in entities from entities/*.md
_builtin = load_builtin_entities()

X_HANDLES = _builtin["x_handles"]
GITHUB_SOURCES = _builtin["github_sources"]
HUGGINGFACE_AUTHORS = _builtin["huggingface_authors"]
ARXIV_QUERIES = _builtin["arxiv_queries"]
WEB_QUERIES = _builtin["web_queries"]
REDDIT_KEYWORDS = _builtin["reddit_keywords"]
REDDIT_SUBREDDITS = _builtin["reddit_subreddits"]
HN_KEYWORDS = _builtin["hn_keywords"]

# Merge custom entities on top
merge_into_registries(
    X_HANDLES,
    GITHUB_SOURCES,
    HUGGINGFACE_AUTHORS,
    ARXIV_QUERIES,
    WEB_QUERIES,
    REDDIT_KEYWORDS,
    REDDIT_SUBREDDITS,
    HN_KEYWORDS,
)


path: lib/env.py
content: """Environment and configuration management for morning-ai."""

import os
from pathlib import Path
from typing import Any, Dict, List, Optional


# Skill root search paths (priority order)
SKILL_SEARCH_PATHS: List[Optional[str]] = [
    ".",                                           # Current checkout
    os.environ.get("CLAUDE_PLUGIN_ROOT"),           # Claude Code plugin
    str(Path.home() / ".claude" / "plugins" / "marketplaces" / "MorningAI"),
    str(Path.home() / ".claude" / "skills" / "morning-ai"),
    str(Path.home() / ".agents" / "skills" / "morning-ai"),
    str(Path.home() / ".codex" / "skills" / "morning-ai"),
]


def find_skill_root() -> str:
    """Find the skill root directory by searching known install paths."""
    for p in SKILL_SEARCH_PATHS:
        if p and Path(p).is_dir() and (Path(p) / "SKILL.md").exists():
            return str(Path(p).resolve())
    return "."


def load_env_file(path: str) -> Dict[str, str]:
    """Parse a .env file into a dict."""
    env = {}
    p = Path(path)
    if not p.exists():
        return env
    for line in p.read_text().splitlines():
        line = line.strip()
        if not line or line.startswith("#"):
            continue
        if "=" not in line:
            continue
        key, _, value = line.partition("=")
        key = key.strip()
        value = value.strip().strip("'\"")
        env[key] = value
    return env


def get_config() -> Dict[str, Any]:
    """Load config from environment + .env files.

    Priority: env vars > project .env > project .claude/morning-ai.env > global config
    """
    global_env = load_env_file(str(Path.home() / ".config" / "morning-ai" / ".env"))
    project_claude_env = load_env_file(".claude/morning-ai.env")
    project_env = load_env_file(".env")
    local_env = load_env_file(".env.local")

    config = {}
    config.update(global_env)
    config.update(project_claude_env)
    config.update(project_env)
    config.update(local_env)
    config.update({k: v for k, v in os.environ.items() if k.startswith(("MORNING_AI_", "GITHUB_", "IMAGE_GEN_", "IMAGE_STYLE", "GEMINI_", "MINIMAX_", "SOCIAL_", "MESSAGE_"))})

    return config


def get_key(config: Dict[str, Any], key: str) -> Optional[str]:
    """Get a config value, returning None if empty."""
    val = config.get(key, "")
    return val if val else None


def get_available_sources(config: Dict[str, Any]) -> Dict[str, bool]:
    """Check which data sources are available based on configured API keys."""
    return {
        "reddit": True,  # public JSON, no key needed
        "hackernews": True,  # Algolia API, free
        "github": bool(get_key(config, "GITHUB_TOKEN")),
        "huggingface": True,  # public API
        "arxiv": True,  # public API
    }


path: lib/github.py
content: """GitHub collector for morning-ai.

Monitors releases, trending repos, and commit activity via GitHub API.
New collector — no last30days equivalent.
"""

import math
import sys
from typing import Any, Dict, List, Optional
from urllib.parse import urlencode, quote

from . import http
from .schema import TrackerItem, Engagement, CollectionResult, SOURCE_GITHUB

GITHUB_API = "https://api.github.com"

DEPTH_CONFIG = {"quick": 5, "default": 10, "deep": 20}


def _log(msg: str):
    if sys.stderr.isatty():
        sys.stderr.write(f"[GitHub] {msg}\n")
        sys.stderr.flush()


def _gh_headers([REDACTED_SECRET] = None) -> Dict[str, str]:
    headers = {
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
    }
    if [REDACTED_SECRET]"Authorization"] = f"Bearer {token}"
    return headers


def _parse_date(date_str: Optional[str]) -> Optional[str]:
    if not date_str:
        return None
    if len(date_str) >= 10:
        return date_str[:10]
    return None


def get_org_releases(
    org: str,
    from_date: str,
    to_date: str,
    [REDACTED_SECRET] = None,
    depth: str = "default",
) -> List[Dict[str, Any]]:
    """Get recent releases from an org's repos."""
    per_page = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"])
    headers = _gh_headers(token)

    # Search for recent releases via GitHub search API
    query = f"org:{org}"
    params = urlencode({
        "q": query,
        "sort": "updated",
        "order": "desc",
        "per_page": str(per_page),
    })
    url = f"{GITHUB_API}/search/repositories?{params}"

    try:
        response = http.get(url, headers=headers, timeout=30)
    except Exception as e:
        _log(f"Failed to search org {org}: {e}")
        return []

    repos = response.get("items", [])
    releases = []

    for repo in repos[:per_page]:
        repo_name = repo.get("full_name", "")
        releases_url = f"{GITHUB_API}/repos/{repo_name}/releases?per_page=5"

        try:
            repo_releases = http.get(releases_url, headers=headers, timeout=15)
        except Exception:
            continue

        if not isinstance(repo_releases, list):
            continue

        for rel in repo_releases:
            pub_date = _parse_date(rel.get("published_at"))
            if pub_date and from_date <= pub_date <= to_date:
                releases.append({
                    "repo": repo_name,
                    "tag": rel.get("tag_name", ""),
                    "name": rel.get("name", ""),
                    "body": rel.get("body", ""),
                    "url": rel.get("html_url", ""),
                    "date": pub_date,
                    "prerelease": rel.get("prerelease", False),
                    "stars": repo.get("stargazers_count", 0),
                    "forks": repo.get("forks_count", 0),
                })

    return releases


def get_repo_releases(
    repo: str,
    from_date: str,
    to_date: str,
    [REDACTED_SECRET] = None,
) -> List[Dict[str, Any]]:
    """Get recent releases from a specific repo."""
    headers = _gh_headers(token)
    url = f"{GITHUB_API}/repos/{repo}/releases?per_page=10"

    try:
        response = http.get(url, headers=headers, timeout=15)
    except Exception as e:
        _log(f"Failed to get releases for {repo}: {e}")
        return []

    if not isinstance(response, list):
        return []

    releases = []
    for rel in response:
        pub_date = _parse_date(rel.get("published_at"))
        if pub_date and from_date <= pub_date <= to_date:
            releases.append({
                "repo": repo,
                "tag": rel.get("tag_name", ""),
                "name": rel.get("name", ""),
                "body": (rel.get("body") or "")[:500],
                "url": rel.get("html_url", ""),
                "date": pub_date,
                "prerelease": rel.get("prerelease", False),
            })

    return releases


def search_trending(
    query: str,
    from_date: str,
    to_date: str,
    [REDACTED_SECRET] = None,
    depth: str = "default",
) -> List[Dict[str, Any]]:
    """Search for trending repos related to a query."""
    per_page = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"])
    headers = _gh_headers(token)

    params = urlencode({
        "q": f"{query} created:>{from_date}",
        "sort": "stars",
        "order": "desc",
        "per_page": str(per_page),
    })
    url = f"{GITHUB_API}/search/repositories?{params}"

    try:
        response = http.get(url, headers=headers, timeout=30)
    except Exception as e:
        _log(f"Trending search '{query}' failed: {e}")
        return []

    return response.get("items", [])


def collect(
    entities: Dict[str, Dict[str, Any]],
    from_date: str,
    to_date: str,
    [REDACTED_SECRET] = None,
    depth: str = "default",
) -> CollectionResult:
    """Collect GitHub data for tracked entities.

    Args:
        entities: Dict mapping entity name -> {
            "orgs": ["openai", "anthropics"],  # GitHub org names
            "repos": ["owner/repo"],           # Specific repos
        }
        from_date: Start date YYYY-MM-DD
        to_date: End date YYYY-MM-DD
        [REDACTED_SECRET] token (optional, increases rate limit)
        depth: Search depth

    Returns:
        CollectionResult
    """
    result = CollectionResult(source=SOURCE_GITHUB)
    all_items = []

    for entity_name, sources in entities.items():
        result.entities_checked += 1
        entity_found = False

        # Check specific repos
        for repo in sources.get("repos", []):
            releases = get_repo_releases(repo, from_date, to_date, token)
            for rel in releases:
                tag = rel.get("tag", "")
                name = rel.get("name") or tag
                body = rel.get("body", "")

                all_items.append(TrackerItem(
                    id=f"GH-{repo}-{tag}",
                    title=f"{repo} {name}",
                    summary=body[:300] if body else f"New release: {name}",
                    entity=entity_name,
                    source=SOURCE_GITHUB,
                    source_url=rel.get("url", f"https://github.com/{repo}/releases"),
                    source_label=f"GitHub {repo}",
                    date=rel.get("date"),
                    date_confidence="high",
                    raw_text=body,
                    engagement=Engagement(
                        stars=rel.get("stars", 0),
                        forks=rel.get("forks", 0),
                    ),
                    relevance=0.8 if not rel.get("prerelease") else 0.5,
                ))
                entity_found = True

        # Check org releases
        for org in sources.get("orgs", []):
            releases = get_org_releases(org, from_date, to_date, token, depth)
            for rel in releases:
                tag = rel.get("tag", "")
                name = rel.get("name") or tag
                repo = rel.get("repo", org)
                body = rel.get("body", "")

                all_items.append(TrackerItem(
                    id=f"GH-{repo}-{tag}",
                    title=f"{repo} {name}",
                    summary=body[:300] if body else f"New release: {name}",
                    entity=entity_name,
                    source=SOURCE_GITHUB,
                    source_url=rel.get("url", f"https://github.com/{repo}"),
                    source_label=f"GitHub {repo}",
                    date=rel.get("date"),
                    date_confidence="high",
                    raw_text=body,
                    engagement=Engagement(
                        stars=rel.get("stars", 0),
                        forks=rel.get("forks", 0),
                    ),
                    relevance=0.8 if not rel.get("prerelease") else 0.5,
                ))
                entity_found = True

        if entity_found:
            result.entities_with_updates += 1

    result.items = all_items
    _log(f"Collected {len(all_items)} GitHub releases from {result.entities_checked} entities")
    return result


path: lib/hackernews.py
content: """Hacker News collector for morning-ai (Algolia API, free).

Adapted from last30days hackernews.py.
"""

import html
import math
import re
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from typing import Any, Dict, List, Optional
from urllib.parse import urlencode

from . import http
from .schema import TrackerItem, Engagement, CollectionResult, SOURCE_HACKERNEWS

ALGOLIA_SEARCH_URL = "https://hn.algolia.com/api/v1/search"
ALGOLIA_ITEM_URL = "https://hn.algolia.com/api/v1/items"

DEPTH_CONFIG = {"quick": 15, "default": 30, "deep": 60}
ENRICH_LIMITS = {"quick": 3, "default": 5, "deep": 10}


def _log(msg: str):
    if sys.stderr.isatty():
        sys.stderr.write(f"[HN] {msg}\n")
        sys.stderr.flush()


def _date_to_unix(date_str: str) -> int:
    import datetime
    parts = date_str.split("-")
    dt = datetime.datetime(int(parts[0]), int(parts[1]), int(parts[2]), tzinfo=datetime.timezone.utc)
    return int(dt.timestamp())


def _unix_to_date(ts: int) -> str:
    import datetime
    dt = datetime.datetime.fromtimestamp(ts, tz=datetime.timezone.utc)
    return dt.strftime("%Y-%m-%d")


def _strip_html(text: str) -> str:
    text = html.unescape(text)
    text = re.sub(r"<p>", "\n", text)
    text = re.sub(r"<[^>]+>", "", text)
    return text.strip()


def search(
    query: str,
    from_date: str,
    to_date: str,
    depth: str = "default",
) -> List[Dict[str, Any]]:
    """Search HN via Algolia API."""
    count = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"])
    from_ts = _date_to_unix(from_date)
    to_ts = _date_to_unix(to_date) + 86400

    params = {
        "query": query,
        "tags": "story",
        "numericFilters": f"created_at_i>{from_ts},created_at_i<{to_ts},points>2",
        "hitsPerPage": str(count),
    }
    url = f"{ALGOLIA_SEARCH_URL}?{urlencode(params)}"

    try:
        response = http.get(url, timeout=30)
    except Exception as e:
        _log(f"Search failed: {e}")
        return []

    return response.get("hits", [])


def _fetch_comments(object_id: str, max_comments: int = 5) -> List[str]:
    """Fetch top comments for a story."""
    url = f"{ALGOLIA_ITEM_URL}/{object_id}"
    try:
        data = http.get(url, timeout=15)
    except Exception:
        return []

    children = data.get("children", [])
    real_comments = [c for c in children if c.get("text") and c.get("author")]
    real_comments.sort(key=lambda c: c.get("points") or 0, reverse=True)

    insights = []
    for c in real_comments[:max_comments]:
        text = _strip_html(c.get("text", ""))
        first_sentence = text.split(". ")[0].split("\n")[0][:200]
        if first_sentence:
            insights.append(first_sentence)
    return insights


def collect(
    entities: Dict[str, List[str]],
    from_date: str,
    to_date: str,
    depth: str = "default",
) -> CollectionResult:
    """Collect HN stories for tracked entities.

    Args:
        entities: Dict mapping entity name -> list of search keywords
        from_date: Start date YYYY-MM-DD
        to_date: End date YYYY-MM-DD
        depth: Search depth

    Returns:
        CollectionResult
    """
    result = CollectionResult(source=SOURCE_HACKERNEWS)
    all_items = []
    enrich_limit = ENRICH_LIMITS.get(depth, ENRICH_LIMITS["default"])

    for entity_name, keywords in entities.items():
        result.entities_checked += 1
        entity_found = False

        for keyword in keywords:
            hits = search(keyword, from_date, to_date, depth)
            if not hits:
                continue

            for i, hit in enumerate(hits):
                object_id = hit.get("objectID", "")
                points = hit.get("points") or 0
                num_comments_val = hit.get("num_comments") or 0
                created_at_i = hit.get("created_at_i")
                date_str = _unix_to_date(created_at_i) if created_at_i else None
                article_url = hit.get("url") or ""
                hn_url = f"https://news.ycombinator.com/item?id={object_id}"

                rank_score = max(0.3, 1.0 - (i * 0.02))
                engagement_boost = min(0.2, math.log1p(points) / 40)
                relevance = min(1.0, rank_score * 0.7 + engagement_boost + 0.1)

                all_items.append(TrackerItem(
                    id=f"HN-{object_id}",
                    title=hit.get("title", ""),
                    summary=f"HN discussion ({points} pts, {num_comments_val} comments)",
                    entity=entity_name,
                    source=SOURCE_HACKERNEWS,
                    source_url=article_url or hn_url,
                    source_label=f"Hacker News",
                    date=date_str,
                    date_confidence="high" if date_str else "low",
                    raw_text=hit.get("title", ""),
                    engagement=Engagement(points=points, num_comments=num_comments_val),
                    relevance=round(relevance, 2),
                ))
                entity_found = True

        if entity_found:
            result.entities_with_updates += 1

    # Enrich top stories with comments
    top_items = sorted(all_items, key=lambda x: x.engagement.points, reverse=True)[:enrich_limit]
    with ThreadPoolExecutor(max_workers=5) as executor:
        futures = {}
        for item in top_items:
            oid = item.id.replace("HN-", "")
            futures[executor.submit(_fetch_comments, oid)] = item
        for future in as_completed(futures):
            item = futures[future]
            try:
                insights = future.result(timeout=15)
                if insights:
                    item.summary += " | Top insights: " + "; ".join(insights[:3])
            except Exception:
                pass

    result.items = all_items
    _log(f"Collected {len(all_items)} HN stories from {result.entities_checked} entities")
    return result


path: lib/http.py
content: """HTTP utilities for morning-ai (stdlib only, adapted from last30days)."""

import json
import os
import sys
import time
import urllib.error
import urllib.request
from typing import Any, Dict, Optional
from urllib.parse import urlencode

DEFAULT_TIMEOUT = 30
MAX_RETRIES = 5
RETRY_DELAY = 2.0
USER_AGENT = "morning-ai/1.0 (AI Industry Tracker)"
DEBUG = os.environ.get("MORNING_AI_DEBUG", "").lower() in ("1", "true", "yes")


def log(msg: str):
    if DEBUG:
        sys.stderr.write(f"[HTTP] {msg}\n")
        sys.stderr.flush()


class HTTPError(Exception):
    def __init__(self, message: str, status_code: Optional[int] = None, body: Optional[str] = None):
        super().__init__(message)
        self.status_code = status_code
        self.body = body


def request(
    method: str,
    url: str,
    headers: Optional[Dict[str, str]] = None,
    json_data: Optional[Dict[str, Any]] = None,
    timeout: int = DEFAULT_TIMEOUT,
    retries: int = MAX_RETRIES,
    raw: bool = False,
) -> Any:
    headers = headers or {}
    headers.setdefault("User-Agent", USER_AGENT)

    data = None
    if json_data is not None:
        data = json.dumps(json_data).encode("utf-8")
        headers.setdefault("Content-Type", "application/json")

    req = urllib.request.Request(url, data=data, headers=headers, method=method)
    log(f"{method} {url}")

    last_error = None
    for attempt in range(retries):
        try:
            with urllib.request.urlopen(req, timeout=timeout) as response:
                body = response.read().decode("utf-8")
                log(f"Response: {response.status} ({len(body)} bytes)")
                if raw:
                    return body
                return json.loads(body) if body else {}
        except urllib.error.HTTPError as e:
            body = None
            try:
                body = e.read().decode("utf-8")
            except Exception:
                pass
            log(f"HTTP Error {e.code}: {e.reason}")
            last_error = HTTPError(f"HTTP {e.code}: {e.reason}", e.code, body)

            if 400 <= e.code < 500 and e.code != 429:
                raise last_error

            if attempt < retries - 1:
                if e.code == 429:
                    retry_after = e.headers.get("Retry-After") if hasattr(e, "headers") else None
                    if retry_after:
                        try:
                            delay = float(retry_after)
                        except ValueError:
                            delay = RETRY_DELAY * (2 ** attempt) + 1
                    else:
                        delay = RETRY_DELAY * (2 ** attempt) + 1
                    log(f"Rate limited (429). Waiting {delay:.1f}s before retry {attempt + 2}/{retries}")
                else:
                    delay = RETRY_DELAY * (2 ** attempt)
                time.sleep(delay)
        except urllib.error.URLError as e:
            log(f"URL Error: {e.reason}")
            last_error = HTTPError(f"URL Error: {e.reason}")
            if attempt < retries - 1:
                time.sleep(RETRY_DELAY * (attempt + 1))
        except json.JSONDecodeError as e:
            last_error = HTTPError(f"Invalid JSON response: {e}")
            raise last_error
        except (OSError, TimeoutError, ConnectionResetError) as e:
            log(f"Connection error: {type(e).__name__}: {e}")
            last_error = HTTPError(f"Connection error: {type(e).__name__}: {e}")
            if attempt < retries - 1:
                time.sleep(RETRY_DELAY * (attempt + 1))

    if last_error:
        raise last_error
    raise HTTPError("Request failed with no error details")


def get(url: str, headers: Optional[Dict[str, str]] = None, **kwargs) -> Any:
    return request("GET", url, headers=headers, **kwargs)


def get_text(url: str, headers: Optional[Dict[str, str]] = None, **kwargs) -> str:
    """GET request returning raw text (not parsed as JSON)."""
    return request("GET", url, headers=headers, raw=True, **kwargs)


def post(url: str, json_data: Dict[str, Any], headers: Optional[Dict[str, str]] = None, **kwargs) -> Any:
    return request("POST", url, headers=headers, json_data=json_data, **kwargs)


path: lib/huggingface.py
content: """HuggingFace collector for morning-ai.

Monitors model/space/dataset updates via HuggingFace API.
New collector — no last30days equivalent.
"""

import sys
from typing import Any, Dict, List, Optional
from urllib.parse import urlencode, quote

from . import http
from .schema import TrackerItem, Engagement, CollectionResult, SOURCE_HUGGINGFACE

HF_API = "https://huggingface.co/api"

DEPTH_CONFIG = {"quick": 5, "default": 10, "deep": 20}


def _log(msg: str):
    if sys.stderr.isatty():
        sys.stderr.write(f"[HF] {msg}\n")
        sys.stderr.flush()


def _parse_date(date_str: Optional[str]) -> Optional[str]:
    if not date_str:
        return None
    if len(date_str) >= 10:
        return date_str[:10]
    return None


def get_author_models(
    author: str,
    from_date: str,
    to_date: str,
    limit: int = 10,
) -> List[Dict[str, Any]]:
    """Get recent models from an author/org on HuggingFace.

    Args:
        author: HuggingFace username or org (e.g. "meta-llama")
        from_date: Start date YYYY-MM-DD
        to_date: End date YYYY-MM-DD
        limit: Max results

    Returns:
        List of model info dicts
    """
    params = urlencode({
        "author": author,
        "sort": "lastModified",
        "direction": "-1",
        "limit": str(limit),
    })
    url = f"{HF_API}/models?{params}"

    try:
        response = http.get(url, timeout=30)
    except Exception as e:
        _log(f"Failed to fetch models for {author}: {e}")
        return []

    if not isinstance(response, list):
        return []

    models = []
    for model in response:
        # Prefer lastModified (most models are created once, updated often)
        created = _parse_date(model.get("createdAt"))
        modified = _parse_date(model.get("lastModified"))
        date = modified or created

        if not date:
            continue

        # Check if either created or modified falls in range
        in_range = (from_date <= date <= to_date)
        if not in_range and created and (from_date <= created <= to_date):
            date = created
            in_range = True
        if not in_range:
            continue

        downloads = model.get("downloads", 0) or 0
        likes = model.get("likes", 0) or 0

        models.append({
            "model_id": model.get("modelId", model.get("id", "")),
            "pipeline_tag": model.get("pipeline_tag", ""),
            "tags": model.get("tags", []),
            "downloads": downloads,
            "likes": likes,
            "date": date,
            "created": created,
            "modified": modified,
            "url": f"https://huggingface.co/{model.get('modelId', model.get('id', ''))}",
            "private": model.get("private", False),
            "library_name": model.get("library_name", ""),
        })

    _log(f"{author}: {len(models)} models in date range")
    return models


def get_trending_models(
    limit: int = 20,
) -> List[Dict[str, Any]]:
    """Get trending models from HuggingFace."""
    params = urlencode({
        "sort": "trending",
        "direction": "-1",
        "limit": str(limit),
    })
    url = f"{HF_API}/models?{params}"

    try:
        response = http.get(url, timeout=30)
    except Exception as e:
        _log(f"Failed to fetch trending models: {e}")
        return []

    if not isinstance(response, list):
        return []
    return response


def search_models(
    query: str,
    from_date: str,
    to_date: str,
    limit: int = 10,
) -> List[Dict[str, Any]]:
    """Search for models by keyword."""
    params = urlencode({
        "search": query,
        "sort": "lastModified",
        "direction": "-1",
        "limit": str(limit),
    })
    url = f"{HF_API}/models?{params}"

    try:
        response = http.get(url, timeout=30)
    except Exception as e:
        _log(f"Model search '{query}' failed: {e}")
        return []

    if not isinstance(response, list):
        return []

    models = []
    for model in response:
        created = _parse_date(model.get("createdAt"))
        modified = _parse_date(model.get("lastModified"))
        date = modified or created
        if date and (date < from_date or date > to_date):
            continue
        models.append(model)

    return models


def collect(
    entities: Dict[str, List[str]],
    from_date: str,
    to_date: str,
    depth: str = "default",
) -> CollectionResult:
    """Collect HuggingFace model updates for tracked entities.

    Args:
        entities: Dict mapping entity name -> list of HF author/org names
            e.g. {"Meta AI": ["meta-llama"], "Google": ["google"]}
        from_date: Start date YYYY-MM-DD
        to_date: End date YYYY-MM-DD
        depth: Search depth

    Returns:
        CollectionResult
    """
    result = CollectionResult(source=SOURCE_HUGGINGFACE)
    all_items = []
    limit = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"])

    for entity_name, hf_authors in entities.items():
        result.entities_checked += 1
        entity_found = False

        for author in hf_authors:
            models = get_author_models(author, from_date, to_date, limit)

            for model in models:
                model_id = model.get("model_id", "")
                pipeline = model.get("pipeline_tag", "")
                downloads = model.get("downloads", 0)
                likes = model.get("likes", 0)
                tags = model.get("tags", [])

                tag_str = ", ".join(tags[:5]) if tags else ""
                summary_parts = []
                if pipeline:
                    summary_parts.append(f"Pipeline: {pipeline}")
                if downloads:
                    summary_parts.append(f"{downloads:,} downloads")
                if likes:
                    summary_parts.append(f"{likes} likes")
                if tag_str:
                    summary_parts.append(f"Tags: {tag_str}")

                all_items.append(TrackerItem(
                    id=f"HF-{model_id}",
                    title=model_id,
                    summary=" | ".join(summary_parts) if summary_parts else model_id,
                    entity=entity_name,
                    source=SOURCE_HUGGINGFACE,
                    source_url=model.get("url", f"https://huggingface.co/{model_id}"),
                    source_label=f"HuggingFace {author}",
                    date=model.get("date"),
                    date_confidence="high" if model.get("date") else "low",
                    raw_text=model_id,
                    engagement=Engagement(
                        likes=likes,
                        views=downloads,
                    ),
                    relevance=min(1.0, 0.5 + min(0.5, (downloads / 10000 + likes / 100))),
                ))
                entity_found = True

        if entity_found:
            result.entities_with_updates += 1

    result.items = all_items
    _log(f"Collected {len(all_items)} HF models from {result.entities_checked} entities")
    return result


path: lib/image_gen.py
content: """Pluggable image generation for morning-ai.

Dispatches to external APIs (Gemini, MiniMax) or skips (none).
Provider is selected via IMAGE_GEN_PROVIDER env var.
"""

import base64
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional

from . import http
from .env import get_key

IMAGE_GEN_TIMEOUT = 120


def _log(msg: str):
    sys.stderr.write(f"[ImageGen] {msg}\n")
    sys.stderr.flush()


class ImageGenError(Exception):
    pass


# ---------------------------------------------------------------------------
# Providers
# ---------------------------------------------------------------------------

def _generate_none(prompt: str, output_path: str, config: Dict[str, Any], *, aspect_ratio: Optional[str] = None) -> str:
    """No-op provider — skip image generation."""
    _log("Provider is 'none', skipping image generation")
    return ""


def _generate_gemini(prompt: str, output_path: str, config: Dict[str, Any], *, aspect_ratio: Optional[str] = None) -> str:
    """Generate image via Google Gemini (Imagen) API."""
    [REDACTED_SECRET], "GEMINI_API_KEY")
    if not api_key:
        raise ImageGenError("GEMINI_API_KEY not configured")

    model = get_key(config, "GEMINI_IMAGE_MODEL") or "imagen-3.0-generate-002"
    url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:predict"

    headers = {
        "x-goog-api-key": api_key,
        "Content-Type": "application/json",
    }
    payload = {
        "instances": [{"prompt": prompt}],
        "parameters": {
            "aspectRatio": aspect_ratio or "9:16",
            "sampleCount": 1,
        },
    }

    _log(f"Calling Gemini Imagen ({model})...")
    try:
        response = http.post(url, json_data=payload, headers=headers, timeout=IMAGE_GEN_TIMEOUT)
    except http.HTTPError as e:
        raise ImageGenError(f"Gemini API error: {e}") from e

    predictions = response.get("predictions", [])
    if not predictions:
        raise ImageGenError("Gemini returned no predictions")

    b64_data = predictions[0].get("bytesBase64Encoded", "")
    if not b64_data:
        raise ImageGenError("Gemini response missing image data")

    _write_image(b64_data, output_path)
    _log(f"Gemini image saved to {output_path}")
    return output_path


def _generate_minimax(prompt: str, output_path: str, config: Dict[str, Any], *, aspect_ratio: Optional[str] = None) -> str:
    """Generate image via MiniMax API."""
    [REDACTED_SECRET], "MINIMAX_API_KEY")
    if not api_key:
        raise ImageGenError("MINIMAX_API_KEY not configured")

    model = get_key(config, "MINIMAX_IMAGE_MODEL") or "image-01"
    region = (get_key(config, "MINIMAX_API_REGION") or "intl").strip().lower()
    if region == "cn":
        url = "https://api.minimaxi.com/v1/image_generation"
    else:
        url = "https://api.minimax.io/v1/image_generation"

    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    }
    payload = {
        "model": model,
        "prompt": prompt,
        "aspect_ratio": aspect_ratio or "9:16",
        "response_format": "b64_json",
        "n": 1,
    }

    _log(f"Calling MiniMax ({model})...")
    try:
        response = http.post(url, json_data=payload, headers=headers, timeout=IMAGE_GEN_TIMEOUT)
    except http.HTTPError as e:
        raise ImageGenError(f"MiniMax API error: {e}") from e

    data = response.get("data", {})
    image_list = data.get("image_base64", [])
    if not image_list:
        raise ImageGenError("MiniMax returned no image data")

    b64_data = image_list[0] if isinstance(image_list, list) else image_list
    if not b64_data:
        raise ImageGenError("MiniMax response missing image data")

    _write_image(b64_data, output_path)
    _log(f"MiniMax image saved to {output_path}")
    return output_path


# ---------------------------------------------------------------------------
# Provider registry
# ---------------------------------------------------------------------------

PROVIDERS = {
    "none": _generate_none,
    "gemini": _generate_gemini,
    "minimax": _generate_minimax,
}


# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------

def generate(prompt: str, output_path: str, config: Dict[str, Any], *, aspect_ratio: Optional[str] = None) -> str:
    """Generate an image using the configured provider.

    Args:
        prompt: Image generation prompt text
        output_path: Path to save the output PNG
        config: Configuration dict (from env.get_config())
        aspect_ratio: Optional aspect ratio (e.g. "9:16", "16:9", "1:1").
                      Defaults to "9:16" when not specified.

    Returns:
        Output file path on success, empty string if skipped (none provider)

    Raises:
        ImageGenError: On API or configuration errors
    """
    provider_name = get_key(config, "IMAGE_GEN_PROVIDER") or "none"

    if provider_name not in PROVIDERS:
        available = ", ".join(sorted(PROVIDERS.keys()))
        raise ImageGenError(f"Unknown provider '{provider_name}'. Available: {available}")

    provider_fn = PROVIDERS[provider_name]
    return provider_fn(prompt, output_path, config, aspect_ratio=aspect_ratio)


def list_providers() -> list:
    """Return list of available provider names."""
    return sorted(PROVIDERS.keys())


def generate_batch(
    items: List[Dict[str, str]],
    config: Dict[str, Any],
) -> List[str]:
    """Generate multiple images sequentially.

    Args:
        items: List of {"prompt": str, "output": str} dicts.
               Optional "aspect_ratio" key per item (e.g. "9:16").
               Entries with empty prompt are skipped.
        config: Configuration dict (from env.get_config())

    Returns:
        List of successfully generated file paths (empty strings for skipped)
    """
    results = []
    total = len(items)

    for i, item in enumerate(items):
        prompt = item.get("prompt", "").strip()
        output_path = item.get("output", "")
        aspect_ratio = item.get("aspect_ratio")

        if not prompt:
            _log(f"[{i + 1}/{total}] Skipping (empty prompt): {output_path}")
            results.append("")
            continue

        if not output_path:
            _log(f"[{i + 1}/{total}] Skipping (no output path)")
            results.append("")
            continue

        _log(f"[{i + 1}/{total}] Generating: {output_path}")
        try:
            result = generate(prompt, output_path, config, aspect_ratio=aspect_ratio)
            results.append(result)
        except ImageGenError as e:
            _log(f"[{i + 1}/{total}] Failed: {e}")
            results.append("")

    succeeded = sum(1 for r in results if r)
    _log(f"Batch complete: {succeeded}/{total} images generated")
    return results


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------

def _write_image(b64_data: str, output_path: str):
    """Decode base64 image data and write to file."""
    try:
        image_bytes = base64.b64decode(b64_data)
    except Exception as e:
        raise ImageGenError(f"Failed to decode base64 image data: {e}") from e

    path = Path(output_path)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(image_bytes)


path: lib/image_stitch.py
content: """Stitch multiple images into a single vertical long image.

Requires Pillow: pip install Pillow
"""

import sys
from pathlib import Path
from typing import List, Tuple

try:
    from PIL import Image
except ImportError:
    Image = None


def _log(msg: str):
    sys.stderr.write(f"[ImageStitch] {msg}\n")
    sys.stderr.flush()


def stitch_images(
    image_paths: List[str],
    output_path: str,
    *,
    gap: int = 20,
    padding: int = 30,
    bg_color: Tuple[int, int, int] = (245, 245, 240),
) -> str:
    """Stitch multiple images vertically into a single long image.

    Args:
        image_paths: Ordered list of image file paths to combine.
                     Non-existent paths are silently skipped.
        output_path: Path for the combined output PNG.
        gap: Pixel gap between images (filled with bg_color).
        padding: Top and bottom padding in pixels.
        bg_color: RGB background/gap color. Default matches
                  infographic style (#F5F5F0).

    Returns:
        Output file path on success.

    Raises:
        ImportError: If Pillow is not installed.
        ValueError: If no valid images found.
    """
    if Image is None:
        raise ImportError(
            "Pillow is required for image stitching. Install with: pip install Pillow"
        )

    # Filter to existing files
    valid_paths = [p for p in image_paths if Path(p).exists()]
    if not valid_paths:
        raise ValueError("No valid image files found to stitch")

    _log(f"Stitching {len(valid_paths)} images...")

    images = [Image.open(p) for p in valid_paths]

    # Uniform width (use max)
    max_width = max(img.width for img in images)

    resized = []
    for img in images:
        if img.width != max_width:
            ratio = max_width / img.width
            new_height = int(img.height * ratio)
            img = img.resize((max_width, new_height), Image.LANCZOS)
        resized.append(img)

    # Canvas dimensions
    total_height = sum(img.height for img in resized) + gap * (len(resized) - 1) + padding * 2

    canvas = Image.new("RGB", (max_width, total_height), bg_color)
    y_offset = padding
    for img in resized:
        canvas.paste(img, (0, y_offset))
        y_offset += img.height + gap

    Path(output_path).parent.mkdir(parents=True, exist_ok=True)
    canvas.save(output_path, "PNG")

    # Close originals first, then any resized copies
    originals = set(id(img) for img in images)
    for img in images:
        img.close()
    for img in resized:
        if id(img) not in originals:
            img.close()
    canvas.close()

    _log(f"Combined image saved to {output_path} ({max_width}x{total_height})")
    return output_path


path: lib/reddit.py
content: """Reddit collector for morning-ai (public JSON, no API key needed).

Adapted from last30days reddit_public.py.
"""

import json
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from typing import Any, Dict, List, Optional

from .schema import TrackerItem, Engagement, CollectionResult, SOURCE_REDDIT

USER_AGENT = "morning-ai/1.0 (AI Industry Tracker)"

DEPTH_LIMITS = {
    "quick": 10,
    "default": 25,
    "deep": 50,
}

# AI-focused subreddits
DEFAULT_SUBREDDITS = ["MachineLearning", "LocalLLaMA", "artificial", "singularity"]

MAX_RETRIES = 3
BASE_BACKOFF = 2.0


def _log(msg: str):
    if sys.stderr.isatty():
        sys.stderr.write(f"[Reddit] {msg}\n")
        sys.stderr.flush()


def _fetch_json(url: str, timeout: int = 15) -> Optional[Dict[str, Any]]:
    headers = {"User-Agent": USER_AGENT, "Accept": "application/json"}
    req = urllib.request.Request(url, headers=headers)

    for attempt in range(MAX_RETRIES):
        try:
            with urllib.request.urlopen(req, timeout=timeout) as resp:
                content_type = resp.headers.get("Content-Type", "")
                if "json" not in content_type and "text/html" in content_type:
                    return None
                body = resp.read().decode("utf-8")
                return json.loads(body)
        except urllib.error.HTTPError as e:
            if e.code == 429 and attempt < MAX_RETRIES - 1:
                delay = BASE_BACKOFF * (2 ** attempt)
                _log(f"429 rate limited, retry after {delay:.1f}s")
                time.sleep(delay)
                continue
            return None
        except (urllib.error.URLError, OSError, json.JSONDecodeError):
            return None
    return None


def _parse_date(created_utc: Any) -> Optional[str]:
    if not created_utc:
        return None
    try:
        from datetime import datetime, timezone
        dt = datetime.fromtimestamp(float(created_utc), tz=timezone.utc)
        return dt.strftime("%Y-%m-%d")
    except (ValueError, TypeError, OSError):
        return None


def search_subreddit(
    query: str,
    subreddit: str,
    from_date: str,
    to_date: str,
    depth: str = "default",
) -> List[TrackerItem]:
    """Search a subreddit via public JSON endpoint."""
    limit = DEPTH_LIMITS.get(depth, DEPTH_LIMITS["default"])
    encoded = urllib.parse.quote_plus(query)
    url = (
        f"https://www.reddit.com/r/{subreddit}/search.json"
        f"?q={encoded}&restrict_sr=on&sort=relevance&t=month&limit={limit}&raw_json=1"
    )

    data = _fetch_json(url)
    if not data:
        return []

    children = data.get("data", {}).get("children", [])
    items = []

    for i, child in enumerate(children):
        if child.get("kind") != "t3":
            continue
        post = child.get("data", {})
        permalink = str(post.get("permalink", "")).strip()
        if not permalink or "/comments/" not in permalink:
            continue

        date = _parse_date(post.get("created_utc"))
        if date and (date < from_date or date > to_date):
            continue

        score_val = int(post.get("score", 0) or 0)
        num_comments = int(post.get("num_comments", 0) or 0)

        items.append(TrackerItem(
            id=f"R-{subreddit}-{i}",
            title=str(post.get("title", "")).strip(),
            summary=str(post.get("selftext", ""))[:300],
            entity=query,
            source=SOURCE_REDDIT,
            source_url=f"https://www.reddit.com{permalink}",
            source_label=f"r/{subreddit}",
            date=date,
            date_confidence="high" if date else "low",
            raw_text=str(post.get("selftext", "")),
            engagement=Engagement(
                score=score_val,
                num_comments=num_comments,
                upvote_ratio=float(post.get("upvote_ratio", 0) or 0),
            ),
            relevance=_compute_relevance(score_val, num_comments),
        ))

    _log(f"r/{subreddit} '{query}': {len(items)} posts")
    return items


def _compute_relevance(score: int, num_comments: int) -> float:
    score_c = min(1.0, max(0.0, score / 500.0))
    comments_c = min(1.0, max(0.0, num_comments / 200.0))
    return round(score_c * 0.6 + comments_c * 0.4, 3)


def fetch_subreddit(
    subreddit: str,
    from_date: str,
    to_date: str,
    depth: str = "default",
) -> List[TrackerItem]:
    """Fetch hot posts from a dedicated subreddit (no keyword filter).

    Used for entity-specific subreddits where all posts are relevant.
    """
    limit = DEPTH_LIMITS.get(depth, DEPTH_LIMITS["default"])
    url = (
        f"https://www.reddit.com/r/{subreddit}/hot.json"
        f"?limit={limit}&raw_json=1"
    )

    data = _fetch_json(url)
    if not data:
        return []

    children = data.get("data", {}).get("children", [])
    items = []

    for i, child in enumerate(children):
        if child.get("kind") != "t3":
            continue
        post = child.get("data", {})
        permalink = str(post.get("permalink", "")).strip()
        if not permalink or "/comments/" not in permalink:
            continue

        # Skip pinned/stickied posts
        if post.get("stickied"):
            continue

        date = _parse_date(post.get("created_utc"))
        if date and (date < from_date or date > to_date):
            continue

        score_val = int(post.get("score", 0) or 0)
        num_comments = int(post.get("num_comments", 0) or 0)

        items.append(TrackerItem(
            id=f"R-{subreddit}-h{i}",
            title=str(post.get("title", "")).strip(),
            summary=str(post.get("selftext", ""))[:300],
            entity="",  # filled by caller
            source=SOURCE_REDDIT,
            source_url=f"https://www.reddit.com{permalink}",
            source_label=f"r/{subreddit}",
            date=date,
            date_confidence="high" if date else "low",
            raw_text=str(post.get("selftext", "")),
            engagement=Engagement(
                score=score_val,
                num_comments=num_comments,
                upvote_ratio=float(post.get("upvote_ratio", 0) or 0),
            ),
            relevance=_compute_relevance(score_val, num_comments),
        ))

    _log(f"r/{subreddit} (hot): {len(items)} posts")
    return items


def collect(
    entities: Dict[str, List[str]],
    from_date: str,
    to_date: str,
    entity_subreddits: Optional[Dict[str, List[str]]] = None,
    subreddits: Optional[List[str]] = None,
    depth: str = "default",
) -> CollectionResult:
    """Collect Reddit posts for tracked entities.

    Args:
        entities: Dict mapping entity name -> list of search keywords
        from_date: Start date YYYY-MM-DD
        to_date: End date YYYY-MM-DD
        entity_subreddits: Dict mapping entity name -> list of dedicated subreddit names
        subreddits: General subreddits to search (default: AI-focused)
        depth: Search depth

    Returns:
        CollectionResult
    """
    subs = subreddits or DEFAULT_SUBREDDITS
    result = CollectionResult(source=SOURCE_REDDIT)
    all_items = []
    seen_urls = set()

    # Phase 1: Fetch hot posts from entity-specific subreddits
    if entity_subreddits:
        for entity_name, entity_subs in entity_subreddits.items():
            result.entities_checked += 1
            entity_found = False

            for sub in entity_subs:
                items = fetch_subreddit(sub, from_date, to_date, depth)
                if items:
                    for item in items:
                        item.entity = entity_name
                        seen_urls.add(item.source_url)
                    all_items.extend(items)
                    entity_found = True
                time.sleep(0.5)

            if entity_found:
                result.entities_with_updates += 1

    # Phase 2: Keyword search in general subreddits
    for entity_name, keywords in entities.items():
        if entity_name not in (entity_subreddits or {}):
            result.entities_checked += 1
        entity_found = False

        for keyword in keywords:
            for sub in subs:
                items = search_subreddit(keyword, sub, from_date, to_date, depth)
                if items:
                    for item in items:
                        item.entity = entity_name
                    # Dedupe against entity-specific results
                    items = [i for i in items if i.source_url not in seen_urls]
                    for item in items:
                        seen_urls.add(item.source_url)
                    all_items.extend(items)
                    entity_found = True
                # Rate limit: small delay between requests
                time.sleep(0.5)

        if entity_found and entity_name not in (entity_subreddits or {}):
            result.entities_with_updates += 1

    result.items = all_items
    _log(f"Collected {len(all_items)} Reddit posts from {result.entities_checked} entities")
    return result


path: lib/schema.py
content: """Data schema for morning-ai items."""

from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional


# AI Tracker content types (matching config.yaml)
TYPE_PRODUCT = "product"
TYPE_MODEL = "model"
TYPE_BENCHMARK = "benchmark"
TYPE_FINANCING = "financing"

VALID_TYPES = {TYPE_PRODUCT, TYPE_MODEL, TYPE_BENCHMARK, TYPE_FINANCING}

# Data source identifiers
SOURCE_X = "x"
SOURCE_REDDIT = "reddit"
SOURCE_HACKERNEWS = "hackernews"
SOURCE_GITHUB = "github"
SOURCE_HUGGINGFACE = "huggingface"
SOURCE_ARXIV = "arxiv"
SOURCE_WEB = "web"


@dataclass
class Engagement:
    """Platform-specific engagement metrics."""
    # X/Twitter
    likes: int = 0
    reposts: int = 0
    replies: int = 0
    quotes: int = 0
    # Reddit
    score: int = 0
    num_comments: int = 0
    upvote_ratio: float = 0.0
    # YouTube / HN
    views: int = 0
    points: int = 0
    # GitHub
    stars: int = 0
    forks: int = 0
    # General
    total: int = 0

    def to_dict(self) -> Dict[str, Any]:
        d = {}
        for k, v in self.__dict__.items():
            if v:
                d[k] = v
        return d

    @classmethod
    def from_dict(cls, d: Dict[str, Any]) -> "Engagement":
        return cls(**{k: v for k, v in d.items() if k in cls.__dataclass_fields__})


@dataclass
class TrackerItem:
    """A single tracked update from any source."""
    id: str = ""
    title: str = ""
    summary: str = ""
    entity: str = ""  # tracked entity name, e.g. "OpenAI", "Anthropic"
    content_type: str = ""  # product | model | benchmark | financing
    source: str = ""  # x | reddit | hackernews | github | huggingface | arxiv | web
    source_url: str = ""
    source_label: str = ""  # human-readable source, e.g. "@OpenAI on X"
    date: Optional[str] = None  # YYYY-MM-DD
    date_confidence: str = "low"  # high | med | low
    raw_text: str = ""
    engagement: Engagement = field(default_factory=Engagement)
    relevance: float = 0.0  # 0-1 from collector
    importance: float = 0.0  # 1-10 morning-ai scale (set by scoring)
    cross_refs: List[str] = field(default_factory=list)
    verified: bool = False
    verify_sources: List[str] = field(default_factory=list)

    def to_dict(self) -> Dict[str, Any]:
        return {
            "id": self.id,
            "title": self.title,
            "summary": self.summary,
            "entity": self.entity,
            "content_type": self.content_type,
            "source": self.source,
            "source_url": self.source_url,
            "source_label": self.source_label,
            "date": self.date,
            "date_confidence": self.date_confidence,
            "raw_text": self.raw_text,
            "engagement": self.engagement.to_dict(),
            "relevance": self.relevance,
            "importance": self.importance,
            "cross_refs": self.cross_refs,
            "verified": self.verified,
            "verify_sources": self.verify_sources,
        }

    @classmethod
    def from_dict(cls, d: Dict[str, Any]) -> "TrackerItem":
        engagement = Engagement.from_dict(d.get("engagement", {}))
        return cls(
            id=d.get("id", ""),
            title=d.get("title", ""),
            summary=d.get("summary", ""),
            entity=d.get("entity", ""),
            content_type=d.get("content_type", ""),
            source=d.get("source", ""),
            source_url=d.get("source_url", ""),
            source_label=d.get("source_label", ""),
            date=d.get("date"),
            date_confidence=d.get("date_confidence", "low"),
            raw_text=d.get("raw_text", ""),
            engagement=engagement,
            relevance=d.get("relevance", 0.0),
            importance=d.get("importance", 0.0),
            cross_refs=d.get("cross_refs", []),
            verified=d.get("verified", False),
            verify_sources=d.get("verify_sources", []),
        )


@dataclass
class CollectionResult:
    """Result from a single collector run."""
    source: str
    items: List[TrackerItem] = field(default_factory=list)
    errors: List[str] = field(default_factory=list)
    entities_checked: int = 0
    entities_with_updates: int = 0

    def to_dict(self) -> Dict[str, Any]:
        return {
            "source": self.source,
            "items": [item.to_dict() for item in self.items],
            "errors": self.errors,
            "entities_checked": self.entities_checked,
            "entities_with_updates": self.entities_with_updates,
        }


@dataclass
class DailyReport:
    """Aggregated daily report from all sources."""
    date: str = ""
    time_window: str = ""  # e.g. "2026-04-02 08:00 ~ 2026-04-03 08:00 UTC+8"
    generated_at: str = ""
    items: List[TrackerItem] = field(default_factory=list)
    collection_results: List[CollectionResult] = field(default_factory=list)
    stats: Dict[str, Any] = field(default_factory=dict)

    def to_dict(self) -> Dict[str, Any]:
        return {
            "date": self.date,
            "time_window": self.time_window,
            "generated_at": self.generated_at,
            "items": [item.to_dict() for item in self.items],
            "collection_results": [r.to_dict() for r in self.collection_results],
            "stats": self.stats,
        }


path: lib/score.py
content: """Scoring module for morning-ai.

Maps collector relevance scores to morning-ai's 1-10 importance scale.
Combines engagement, source reliability, and content relevance.
Adapted from last30days score.py composite scoring approach.
"""

import math
from typing import List

from .schema import TrackerItem, SOURCE_X, SOURCE_GITHUB, SOURCE_HUGGINGFACE, SOURCE_ARXIV, SOURCE_WEB, SOURCE_REDDIT, SOURCE_HACKERNEWS

# Source reliability weights (official sources > community)
SOURCE_RELIABILITY = {
    SOURCE_GITHUB: 0.9,
    SOURCE_HUGGINGFACE: 0.85,
    SOURCE_ARXIV: 0.8,
    SOURCE_WEB: 0.7,
    SOURCE_X: 0.65,
    SOURCE_REDDIT: 0.5,
    SOURCE_HACKERNEWS: 0.55,
}

# Scoring component weights — Stage 1 automated scoring dimensions.
# These are computable from collector metadata (not the same as the
# 5-dimension agent evaluation criteria in tracking-list/SKILL.md).
WEIGHT_RELEVANCE = 0.35
WEIGHT_ENGAGEMENT = 0.30
WEIGHT_SOURCE_RELIABILITY = 0.20
WEIGHT_RECENCY = 0.15


def _compute_engagement_score(item: TrackerItem) -> float:
    """Compute normalized engagement score (0-1) based on source type."""
    eng = item.engagement

    if item.source == SOURCE_X:
        raw = (
            0.55 * math.log1p(eng.likes)
            + 0.25 * math.log1p(eng.reposts)
            + 0.15 * math.log1p(eng.replies)
            + 0.05 * math.log1p(eng.quotes)
        )
        return min(1.0, raw / 10.0)

    elif item.source == SOURCE_REDDIT:
        raw = (
            0.55 * math.log1p(eng.score)
            + 0.45 * math.log1p(eng.num_comments)
        )
        return min(1.0, raw / 8.0)

    elif item.source == SOURCE_HACKERNEWS:
        raw = (
            0.55 * math.log1p(eng.points)
            + 0.45 * math.log1p(eng.num_comments)
        )
        return min(1.0, raw / 8.0)

    elif item.source == SOURCE_GITHUB:
        raw = (
            0.60 * math.log1p(eng.stars)
            + 0.40 * math.log1p(eng.forks)
        )
        return min(1.0, raw / 10.0)

    elif item.source == SOURCE_HUGGINGFACE:
        raw = (
            0.50 * math.log1p(eng.views)  # downloads
            + 0.50 * math.log1p(eng.likes)
        )
        return min(1.0, raw / 12.0)

    elif item.source == SOURCE_ARXIV:
        return 0.5  # arXiv doesn't have engagement metrics

    elif item.source == SOURCE_WEB:
        return 0.4  # web search results have limited engagement info

    return 0.3


def _compute_recency_score(item: TrackerItem) -> float:
    """Score based on date confidence."""
    if item.date_confidence == "high":
        return 1.0
    elif item.date_confidence == "med":
        return 0.7
    return 0.4


def score_item(item: TrackerItem) -> float:
    """Compute importance score for a single item on 1-10 scale.

    Combines:
    - Relevance (from collector): 35%
    - Engagement (platform-specific): 30%
    - Source reliability: 20%
    - Recency confidence: 15%

    Returns:
        Score from 1.0 to 10.0
    """
    relevance = item.relevance
    engagement = _compute_engagement_score(item)
    reliability = SOURCE_RELIABILITY.get(item.source, 0.5)
    recency = _compute_recency_score(item)

    composite = (
        WEIGHT_RELEVANCE * relevance
        + WEIGHT_ENGAGEMENT * engagement
        + WEIGHT_SOURCE_RELIABILITY * reliability
        + WEIGHT_RECENCY * recency
    )

    # Map 0-1 composite to 1-10 scale
    score = max(1.0, min(10.0, composite * 10.0))
    return round(score, 1)


def score_items(items: List[TrackerItem]) -> List[TrackerItem]:
    """Score all items and sort by importance descending.

    Args:
        items: List of TrackerItem to score

    Returns:
        Same items with importance field set, sorted by importance desc
    """
    for item in items:
        item.importance = score_item(item)

    items.sort(key=lambda x: x.importance, reverse=True)
    return items


def apply_verification_bonus(items: List[TrackerItem]) -> List[TrackerItem]:
    """Apply cross-verification bonus per morning-ai rules.

    Items with 2+ cross-references get a verification bonus.
    Items scoring 7+ with <2 independent sources get a penalty.

    Args:
        items: Pre-scored items with cross_refs populated

    Returns:
        Items with adjusted importance scores
    """
    for item in items:
        num_refs = len(item.cross_refs)

        if num_refs >= 2:
            item.verified = True
            item.importance = min(10.0, item.importance + 0.5)
        elif item.importance >= 7.0 and num_refs < 2:
            # High-score items need verification per morning-ai rules
            item.importance = max(6.9, item.importance - 0.5)
            item.verified = False

    items.sort(key=lambda x: x.importance, reverse=True)
    return items


path: lib/snapshots.py
content: """Lightweight SQLite snapshot store for leaderboard tracking.

Stores daily snapshots of leaderboard state (model name, rank, score)
and diffs against the previous snapshot to detect new models, removals,
and ranking/score changes.

DB location: data/leaderboard.db (auto-created on first use).
"""

import json
import os
import sqlite3
from pathlib import Path

DB_PATH = Path(__file__).resolve().parent.parent / "data" / "leaderboard.db"

_CREATE_TABLE = """
CREATE TABLE IF NOT EXISTS leaderboard_entries (
    leaderboard   TEXT NOT NULL,
    model         TEXT NOT NULL,
    rank          INTEGER,
    score         REAL,
    metadata      TEXT,
    snapshot_date TEXT NOT NULL,
    PRIMARY KEY (leaderboard, model, snapshot_date)
)
"""

_CREATE_INDEX = """
CREATE INDEX IF NOT EXISTS idx_lb_date
ON leaderboard_entries (leaderboard, snapshot_date)
"""


def _connect():
    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
    conn = sqlite3.connect(str(DB_PATH))
    conn.row_factory = sqlite3.Row
    return conn


def init_db():
    with _connect() as conn:
        conn.execute(_CREATE_TABLE)
        conn.execute(_CREATE_INDEX)


def save_snapshot(leaderboard: str, date: str, entries: list[dict]):
    """Save a list of model entries for a leaderboard on a given date.

    Each entry: {"model": str, "rank": int|None, "score": float|None, ...}
    Extra keys are stored in the metadata JSON column.
    """
    init_db()
    with _connect() as conn:
        for entry in entries:
            model = entry["model"]
            rank = entry.get("rank")
            score = entry.get("score")
            extra = {k: v for k, v in entry.items() if k not in ("model", "rank", "score")}
            metadata = json.dumps(extra, ensure_ascii=False) if extra else None
            conn.execute(
                """INSERT OR REPLACE INTO leaderboard_entries
                   (leaderboard, model, rank, score, metadata, snapshot_date)
                   VALUES (?, ?, ?, ?, ?, ?)""",
                (leaderboard, model, rank, score, metadata, date),
            )


def _get_snapshot(conn, leaderboard: str, date: str) -> dict[str, dict]:
    """Return {model_name: {rank, score, metadata}} for a specific date."""
    rows = conn.execute(
        """SELECT model, rank, score, metadata FROM leaderboard_entries
           WHERE leaderboard = ? AND snapshot_date = ?""",
        (leaderboard, date),
    ).fetchall()
    result = {}
    for r in rows:
        result[r["model"]] = {
            "rank": r["rank"],
            "score": r["score"],
            "metadata": json.loads(r["metadata"]) if r["metadata"] else {},
        }
    return result


def _get_prev_date(conn, leaderboard: str, date: str) -> str | None:
    """Find the most recent snapshot date before `date`."""
    row = conn.execute(
        """SELECT DISTINCT snapshot_date FROM leaderboard_entries
           WHERE leaderboard = ? AND snapshot_date < ?
           ORDER BY snapshot_date DESC LIMIT 1""",
        (leaderboard, date),
    ).fetchone()
    return row["snapshot_date"] if row else None


def diff_snapshot(leaderboard: str, date: str) -> dict:
    """Compare the snapshot at `date` with the previous one.

    Returns:
        {
            "leaderboard": str,
            "date": str,
            "prev_date": str | None,
            "is_initial": bool,
            "new_models": [{"model", "rank", "score"}, ...],
            "removed_models": [str, ...],
            "rank_changes": [{"model", "old_rank", "new_rank"}, ...],
            "score_changes": [{"model", "old_score", "new_score"}, ...],
        }
    """
    init_db()
    with _connect() as conn:
        current = _get_snapshot(conn, leaderboard, date)
        prev_date = _get_prev_date(conn, leaderboard, date)

        if prev_date is None:
            return {
                "leaderboard": leaderboard,
                "date": date,
                "prev_date": None,
                "is_initial": True,
                "new_models": [
                    {"model": m, "rank": d["rank"], "score": d["score"]}
                    for m, d in current.items()
                ],
                "removed_models": [],
                "rank_changes": [],
                "score_changes": [],
            }

        previous = _get_snapshot(conn, leaderboard, prev_date)
        curr_names = set(current)
        prev_names = set(previous)

        new_models = [
            {"model": m, "rank": current[m]["rank"], "score": current[m]["score"]}
            for m in sorted(curr_names - prev_names)
        ]
        removed_models = sorted(prev_names - curr_names)

        rank_changes = []
        score_changes = []
        for m in sorted(curr_names & prev_names):
            c, p = current[m], previous[m]
            if c["rank"] is not None and p["rank"] is not None and c["rank"] != p["rank"]:
                rank_changes.append({"model": m, "old_rank": p["rank"], "new_rank": c["rank"]})
            if c["score"] is not None and p["score"] is not None and c["score"] != p["score"]:
                score_changes.append({"model": m, "old_score": p["score"], "new_score": c["score"]})

        return {
            "leaderboard": leaderboard,
            "date": date,
            "prev_date": prev_date,
            "is_initial": False,
            "new_models": new_models,
            "removed_models": removed_models,
            "rank_changes": rank_changes,
            "score_changes": score_changes,
        }


def get_latest_snapshot(leaderboard: str) -> dict:
    """Return the most recent snapshot for a leaderboard.

    Returns: {"leaderboard", "date", "entries": [{"model", "rank", "score"}, ...]}
    """
    init_db()
    with _connect() as conn:
        row = conn.execute(
            """SELECT DISTINCT snapshot_date FROM leaderboard_entries
               WHERE leaderboard = ?
               ORDER BY snapshot_date DESC LIMIT 1""",
            (leaderboard,),
        ).fetchone()
        if not row:
            return {"leaderboard": leaderboard, "date": None, "entries": []}

        date = row["snapshot_date"]
        snapshot = _get_snapshot(conn, leaderboard, date)
        return {
            "leaderboard": leaderboard,
            "date": date,
            "entries": [
                {"model": m, "rank": d["rank"], "score": d["score"]}
                for m, d in snapshot.items()
            ],
        }


path: scripts/sync.sh
content: #!/usr/bin/env bash
# sync.sh — Deploy morning-ai skill to multiple AI tool directories
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SOURCE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"

TARGETS=(
    "$HOME/.claude/skills/morning-ai"
    "$HOME/.agents/skills/morning-ai"
    "$HOME/.codex/skills/morning-ai"
    "$HOME/.config/opencode/skills/morning-ai"
)

SYNC_DIRS=(lib skills entities templates hooks scripts .codex-plugin .agents .claude-plugin)
SYNC_FILES=(SKILL.md AGENTS.md .clawhubignore)

for target in "${TARGETS[@]}"; do
    echo "Syncing to $target ..."
    mkdir -p "$target"

    # Sync directories
    for dir in "${SYNC_DIRS[@]}"; do
        if [[ -d "$SOURCE_DIR/$dir" ]]; then
            rsync -a --delete "$SOURCE_DIR/$dir/" "$target/$dir/"
        fi
    done

    # Sync individual files
    for f in "${SYNC_FILES[@]}"; do
        if [[ -f "$SOURCE_DIR/$f" ]]; then
            cp "$SOURCE_DIR/$f" "$target/$f"
        fi
    done

    echo "  Done."
done

echo "Sync complete. Deployed to ${#TARGETS[@]} targets."


path: skills/gen-infographic/agents/openai.yaml
content: interface:
  display_name: "News Infographic Generator"
  short_description: "Generate cover and per-type infographics for AI news reports"
  default_prompt: "Generate infographic images for the daily AI news report."


path: skills/gen-infographic/scripts/gen_infographic.py
content: #!/usr/bin/env python3
"""Generate cover and per-type infographics via external image generation API.

Supports single-image mode (--prompt) and batch mode (--batch manifest.json).

Usage:
    # Single image
    python3 skills/gen-infographic/scripts/gen_infographic.py --prompt "9:16 portrait infographic, AI News Daily ..." -o cover.png

    # Batch mode (multiple images from manifest)
    python3 skills/gen-infographic/scripts/gen_infographic.py --batch manifest.json

    # Pipe prompt via stdin
    echo "prompt text" | python3 skills/gen-infographic/scripts/gen_infographic.py --provider minimax

Manifest JSON format:
    [
      {"prompt": "9:16 portrait infographic, AI News Daily ...", "output": "news_infographic_2026-04-08.png"},
      {"prompt": "9:16 portrait infographic, Model Updates ...", "output": "news_infographic_2026-04-08_model.png"}
    ]
"""

import argparse
import json
import sys
from datetime import datetime
from pathlib import Path

# Add project root to path for imports
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
sys.path.insert(0, str(Path(__file__).resolve().parent))

from lib import env
from lib.image_gen import generate, generate_batch, list_providers, ImageGenError
from lib.image_stitch import stitch_images
from styles import (
    STYLE_PRESETS, DEFAULT_STYLE, CONTENT_DENSITY,
    SECTION_SUFFIX, SECTION_STYLE_OVERRIDES, STYLE_BG_COLORS,
)


def _log(msg: str):
    sys.stderr.write(f"[gen-infographic] {msg}\n")
    sys.stderr.flush()


def _inject_style(prompt: str, config: dict) -> str:
    """Inject style preset into prompt based on IMAGE_STYLE config."""
    style_name = (config.get("IMAGE_STYLE") or DEFAULT_STYLE).strip().lower()
    if style_name not in STYLE_PRESETS:
        _log(f"Warning: Unknown style '{style_name}', falling back to '{DEFAULT_STYLE}'")
        style_name = DEFAULT_STYLE

    style_block = STYLE_PRESETS[style_name] + CONTENT_DENSITY

    # Detect section prompts (used in --stitch mode) and append continuity rules
    is_section = "section" in prompt[:120].lower()
    if is_section:
        style_block += SECTION_SUFFIX
        extra = SECTION_STYLE_OVERRIDES.get(style_name, "")
        if extra:
            style_block += extra

    if "{STYLE_BLOCK}" in prompt:
        prompt = prompt.replace("{STYLE_BLOCK}", style_block)
    else:
        prompt = prompt.rstrip() + "\n\n" + style_block

    _log(f"Style: {style_name}{' (section mode)' if is_section else ''}")
    return prompt


def _run_single(args, config):
    """Single-image mode: one prompt → one image."""
    # Read prompt
    if args.prompt:
        prompt = args.prompt
    elif args.prompt_file:
        prompt = Path(args.prompt_file).read_text(encoding="utf-8").strip()
    elif not sys.stdin.isatty():
        prompt = sys.stdin.read().strip()
    else:
        _log("Error: No prompt provided. Use --prompt, --prompt-file, or pipe to stdin.")
        sys.exit(1)

    if not prompt:
        _log("Error: Empty prompt")
        sys.exit(1)

    prompt = _inject_style(prompt, config)

    output_path = args.output or f"news_infographic_{args.date}.png"
    provider_name = config.get("IMAGE_GEN_PROVIDER", "none")
    _log(f"Provider: {provider_name}, Output: {output_path}")

    try:
        result = generate(prompt, output_path, config)
    except ImageGenError as e:
        _log(f"Error: {e}")
        sys.exit(1)

    if result:
        _log(f"Success: {result}")
        print(result)
    else:
        _log("Skipped (provider=none)")


def _run_batch(args, config):
    """Batch mode: manifest JSON → multiple images, optional stitch."""
    manifest_path = Path(args.batch)
    if not manifest_path.exists():
        _log(f"Error: Manifest file not found: {args.batch}")
        sys.exit(1)

    try:
        items = json.loads(manifest_path.read_text(encoding="utf-8"))
    except (json.JSONDecodeError, OSError) as e:
        _log(f"Error: Failed to read manifest: {e}")
        sys.exit(1)

    if not isinstance(items, list):
        _log("Error: Manifest must be a JSON array of {\"prompt\": ..., \"output\": ...}")
        sys.exit(1)

    provider_name = config.get("IMAGE_GEN_PROVIDER", "none")
    _log(f"Batch mode: {len(items)} images, provider: {provider_name}")

    for item in items:
        if item.get("prompt", "").strip():
            item["prompt"] = _inject_style(item["prompt"], config)

    results = generate_batch(items, config)
    succeeded = [r for r in results if r]

    for path in succeeded:
        print(path)

    _log(f"Done: {len(succeeded)}/{len(items)} images generated")
    if not succeeded and provider_name != "none":
        sys.exit(1)

    # Stitch into a single long image
    if args.stitch and len(succeeded) >= 2:
        stitch_output = args.stitch_output or f"news_infographic_{args.date}_combined.png"
        style_name = (config.get("IMAGE_STYLE") or DEFAULT_STYLE).strip().lower()
        bg_color = STYLE_BG_COLORS.get(style_name, (245, 245, 240))
        try:
            combined = stitch_images(succeeded, stitch_output, bg_color=bg_color)
            print(combined)
            _log(f"Stitched image: {combined}")
        except (ImportError, ValueError) as e:
            _log(f"Stitch skipped: {e}")
    elif args.stitch:
        _log("Stitch skipped: need at least 2 images")


def main():
    parser = argparse.ArgumentParser(description="Generate infographics via external API")
    parser.add_argument("--prompt", "-p", default=None,
                        help="Image generation prompt text (single-image mode)")
    parser.add_argument("--prompt-file", default=None,
                        help="Path to file containing the prompt (single-image mode)")
    parser.add_argument("--output", "-o", default=None,
                        help="Output PNG file path (default: news_infographic_{date}.png)")
    parser.add_argument("--batch", default=None,
                        help="Path to manifest JSON for batch generation")
    parser.add_argument("--provider", default=None,
                        choices=list_providers(),
                        help="Image generation provider (overrides IMAGE_GEN_PROVIDER env var)")
    parser.add_argument("--date", default=datetime.now().strftime("%Y-%m-%d"),
                        help="Date for default output filename (default: today)")
    parser.add_argument("--stitch", action="store_true",
                        help="Stitch batch images into a single vertical long image (requires Pillow)")
    parser.add_argument("--stitch-output", default=None,
                        help="Output path for stitched image (default: news_infographic_{date}_combined.png)")
    args = parser.parse_args()

    # Load config and apply provider override
    config = env.get_config()
    if args.provider:
        config["IMAGE_GEN_PROVIDER"] = args.provider

    if args.batch:
        _run_batch(args, config)
    else:
        _run_single(args, config)


if __name__ == "__main__":
    main()


path: skills/gen-infographic/scripts/styles.py
content: """Infographic style presets for image generation.

Each style block is a text description inserted into the image generation prompt
to control visual appearance. Selected via IMAGE_STYLE config (default: classic).
"""

STYLE_PRESETS = {
    "classic": (
        'Style: Clean editorial magazine layout. Off-white (#F5F5F0) background with subtle warm gray grid lines. '
        'Bold sans-serif header "AI News Daily" in black with a vivid accent color underline. '
        'Each card is a white rectangle with soft drop shadow (4px blur, 10% black), compact spacing between cards. '
        'Use a refined accent palette: deep navy (#1A2744) for card titles, muted teal (#2A9D8F) for bullet icons, '
        'slate gray (#4A5568) for body text. NO gradients, NO textures, NO background patterns \u2014 pure flat white space.\n'
        'Layout: Information-dense. Cards arranged in a balanced grid with tight gutters (16px). '
        'Maximize space for card content (titles and bullet points). '
        'Do NOT display score numbers or score badges \u2014 let card size and content density convey importance.\n'
        'Card design: Card title in 15pt bold navy sans-serif, subtitle in 10pt gray italic. '
        'Bullet points with small teal dot markers, 12pt regular weight, tight line spacing (1.2x). '
        'Thin 1px light gray top-border on each card for subtle separation. '
        'NO icons, NO illustrations, NO decorative elements inside cards \u2014 text only with strong typographic hierarchy. '
        'Pack content tightly \u2014 minimize whitespace within and between cards.'
    ),
    "dark": (
        'Style: Dark mode editorial layout. Deep charcoal (#1A1A2E) background. '
        'Bold sans-serif header "AI News Daily" in white (#FAFAFA) with electric blue (#00D4FF) accent underline. '
        'Each card is a dark slate (#16213E) rectangle with subtle 1px border in muted blue (#0F3460), '
        'compact spacing between cards. Accent palette: white (#FAFAFA) for card titles, '
        'soft violet (#7B68EE) for bullet icons, light gray (#B0BEC5) for body text. '
        'NO gradients, NO glow effects \u2014 clean flat dark surfaces.\n'
        'Layout: Information-dense. Cards arranged in a balanced grid with tight gutters (16px). '
        'Maximize space for card content (titles and bullet points). '
        'Do NOT display score numbers or score badges \u2014 let card size and content density convey importance.\n'
        'Card design: Card title in 15pt bold white sans-serif, subtitle in 10pt light gray italic. '
        'Bullet points with small violet dot markers, 12pt regular weight, tight line spacing (1.2x). '
        'Thin 1px muted blue top-border on each card. '
        'NO icons, NO illustrations \u2014 text only with strong typographic hierarchy on dark background. '
        'Pack content tightly \u2014 minimize whitespace within and between cards.'
    ),
    "glassmorphism": (
        'Style: Glassmorphism editorial layout. Soft gradient background blending from lavender (#E8EAF6) top-left '
        'to pale rose (#FCE4EC) bottom-right. Bold sans-serif header "AI News Daily" in dark charcoal (#212121) '
        'with warm coral (#FF6B6B) accent underline. Each card is a semi-transparent frosted white panel '
        '(rgba(255,255,255,0.65)) with backdrop blur effect, rounded corners (16px), and subtle white border '
        '(1px, 30% opacity). Accent palette: charcoal (#212121) for card titles, soft indigo (#5C6BC0) for bullet icons, '
        'medium gray (#546E7A) for body text. Soft diffused shadows (8px blur, 5% black) behind each card.\n'
        'Layout: Information-dense. Cards arranged in a balanced grid with tight gutters (16px). '
        'Maximize space for card content (titles and bullet points). '
        'Do NOT display score numbers or score badges \u2014 let card size and content density convey importance.\n'
        'Card design: Card title in 15pt bold charcoal sans-serif, subtitle in 10pt gray italic. '
        'Bullet points with small indigo dot markers, 12pt regular weight, tight line spacing (1.2x). '
        'NO hard borders \u2014 rely on frosted glass contrast for separation. '
        'Pack content tightly \u2014 minimize whitespace within and between cards.'
    ),
    "newspaper": (
        'Style: Classic newspaper editorial layout. Warm cream (#FFF8E7) background with very faint paper texture grain. '
        'Bold serif header "AI News Daily" in deep black (#1A1A1A) with crimson red (#B71C1C) thin rule line below. '
        'Each card is separated by thin black hairline rules (1px) \u2014 NO card backgrounds, NO shadows, NO boxes. '
        'Content flows in a column-based newspaper grid. Accent palette: deep black (#1A1A1A) for card titles in bold serif, '
        'dark gray (#333333) for bullet text in serif, medium gray (#666666) for subtitles in italic serif.\n'
        'Layout: Information-dense. Multi-column newspaper grid (2-3 columns). Large stories span full width at top, '
        'smaller stories in side-by-side columns below. Separated by horizontal and vertical hairline rules. '
        'NO cards, NO boxes \u2014 pure typographic layout. Tight column gaps (12px). '
        'Do NOT display score numbers or score indicators \u2014 let column placement and headline size convey importance.\n'
        'Card design: Card title in 15pt bold black serif, subtitle in 10pt gray italic serif. '
        'Bullet points with small em-dash markers, 12pt regular serif weight, tight line spacing (1.2x). '
        'Feels like the front page of a prestigious broadsheet. '
        'Pack content tightly \u2014 minimize whitespace between stories.'
    ),
    "tech": (
        'Style: Tech terminal aesthetic layout. Near-black background (#0D1117) with very faint dot grid pattern '
        '(8px spacing, 5% white). Bold monospace header "AI News Daily" in bright cyan (#00FFCC) with a blinking cursor '
        'underscore effect. Each card is a dark panel (#161B22) with 1px border in dim cyan (#1A3A3A), rounded corners (4px). '
        'Accent palette: bright green (#39FF14) for card titles in monospace bold, amber (#FFB000) for bullet markers '
        'as `>` symbols, light gray (#C9D1D9) for body text in monospace. '
        'Each card has a subtle top-left label like `// MODEL` or `// PRODUCT` in dim green (#1A4A1A).\n'
        'Layout: Information-dense. Cards arranged in a balanced grid with tight gutters (16px). '
        'Compact spacing. Maximize space for card content (titles and bullet points). '
        'Do NOT display score numbers or score badges \u2014 let card size and content density convey importance.\n'
        'Card design: Card title in 14pt bold green monospace, subtitle in 10pt gray monospace. '
        'Bullet points with amber `>` markers, 12pt regular monospace, tight line spacing (1.2x). '
        'Thin 1px dim cyan border. '
        'Feels like a developer dashboard or terminal readout. '
        'Pack content tightly \u2014 minimize whitespace within and between cards.'
    ),
}

DEFAULT_STYLE = "classic"

# Appended to every style block to enforce information density.
CONTENT_DENSITY = (
    "\n\nCONTENT RENDERING RULES:\n"
    "- Display EVERY bullet point in FULL — do NOT truncate, summarize, abbreviate, or drop any text.\n"
    "- Each bullet should include specific details (numbers, metrics, dates, names) exactly as provided.\n"
    "- Use compact font sizes to fit all content — shrink text further if needed to avoid omitting any bullet.\n"
    "- Minimize whitespace: tight margins, tight line spacing, tight card gaps.\n"
    "- Fill the entire image area with content — avoid large empty or decorative spaces.\n"
    "- Card titles must display the COMPLETE text including entity name, event subject, and description."
)

# Section continuity instructions appended to style blocks for section (stitch) prompts.
# Overrides header/branding from the main style block so stitched images flow seamlessly.
SECTION_SUFFIX = (
    "\n\nSECTION CONTINUITY RULES (override any conflicting style instructions above):\n"
    "- Do NOT render any top header, banner, branding, or title bar — "
    'no "AI News Daily" header or similar. Start directly with the section title.\n'
    "- Top edge: no decorative border, rule, or extra padding — "
    "content starts near the top so the previous section flows into this one.\n"
    "- Bottom edge: no footer, closing border, or extra padding — "
    "the next section will continue directly below.\n"
    "- Keep consistent left/right margins so adjacent sections align when stacked vertically."
)

# Extra section overrides per style (appended after SECTION_SUFFIX).
SECTION_STYLE_OVERRIDES = {
    "newspaper": (
        "\n- Use a SINGLE-COLUMN vertical layout for this section (NOT multi-column). "
        "Stack all items top-to-bottom separated by horizontal hairline rules. "
        "This ensures visual continuity when multiple sections are stitched into one long image.\n"
        "- Maintain the warm cream (#FFF8E7) background with consistent margins."
    ),
}

# Background colors (RGB) per style, used for stitch gap/padding fill.
STYLE_BG_COLORS = {
    "classic":        (245, 245, 240),   # #F5F5F0
    "dark":           (26, 26, 46),      # #1A1A2E
    "glassmorphism":  (232, 234, 246),   # #E8EAF6
    "newspaper":      (255, 248, 231),   # #FFF8E7
    "tech":           (13, 17, 23),      # #0D1117
}


path: skills/gen-message/templates/digest.md
content: # Message Digest Template

This template defines the exact output format for `message_{DATE}.md`.

---

## Language-Specific Text

### Headers

| `--lang` | Header Line 1 | Header Line 2 |
|----------|---------------|----------------|
| `zh` | `AI 每日速报 {YYYY-MM-DD}` | `共 {N} 条重要更新` |
| `en` | `AI Daily Digest {YYYY-MM-DD}` | `{N} notable updates today` |
| `ja` | `AI デイリーダイジェスト {YYYY-MM-DD}` | `本日の注目 {N} 件` |

### Footer

| `--lang` | Footer |
|----------|--------|
| `zh` | `Powered by MorningAI \| 完整报告: report_{DATE}.md` |
| `en` | `Powered by MorningAI \| Full report: report_{DATE}.md` |
| `ja` | `Powered by MorningAI \| 完全レポート: report_{DATE}.md` |

---

## Template Structure

```
{Header Line 1}

{Header Line 2}

{Item 1}

{Item 2}

...

{Item N}

---
{Footer}
```

---

## Item Template

### Standard Item

```
{emoji} **{Entity} {Event Description}**
{1-2 sentence summary with key metrics and specifics.}
🔗 {source_url}
```

**Emoji selection:**
- Score 9-10 → `🔥`
- Score 7-8 → `⭐`
- Score 5-6 → `🔷`

**Title rules:**
- Bold the entire title
- Include entity name + concise event description
- Keep titles under 30 characters (Chinese) or 60 characters (English) when possible
- Do NOT include score numbers

**Summary rules:**
- Condense to 1-2 sentences maximum
- Include the most important specific details: numbers, versions, benchmarks, pricing
- No filler phrases like "major update" without specifics
- End with a period

**Link rules:**
- Each item MUST end with a `🔗 {source_url}` line
- The URL comes from the item's `source_url` field
- Always use the full URL, do NOT shorten or omit

### GitHub Trending Item (Special)

When an item is from GitHub Trending or has notable star/fork metrics:

```
🔥 **GitHub Trending: {repo-name} (Stars Surging!)**
⭐ {star_count}(+{delta}) | {one-line description}
🔗 {source_url}
```

---

## Link Placement

### Inline Format (`MESSAGE_LINKS=inline`, default)

Each item ends with a source link — this is the standard format:

```
{emoji} **{Entity} {Event Description}**
{Summary sentence.}
🔗 {source_url}
```

### Bottom Format (`MESSAGE_LINKS=bottom`, alternative)

When set to `bottom`, omit `🔗` lines from items and group links as a reference list at the end:

```
---
[1] {Entity Event Short Label} - {source_url}
[2] {Entity Event Short Label} - {source_url}
...
```

- One link per item, in the same order as items appear
- Short label: `{Entity} {Event}` (e.g., "Anthropic Claude 4.5 Sonnet")
- Full URL from `source_url` field

---

## Complete Example

```
AI Daily Digest 2026-04-08

8 notable updates today

🔥 **Anthropic Releases Claude 4.5 Sonnet**
New mid-tier model with +18% SWE-Bench, 200K context, 40% faster output. Available via API and claude.ai.
🔗 https://x.com/AnthropicAI/status/example

⭐ **Google Gemini 2.5 Flash Enters Public Preview**
Flash-tier model with native multimodal reasoning, 1M context. Free tier on AI Studio.
🔗 https://x.com/GoogleDeepMind/status/example

⭐ **Cursor Background Agents Now Generally Available**
Autonomous background agents for multi-file refactoring, test generation, and PR creation. Max 10 concurrent on Pro.
🔗 https://cursor.com/changelog/background-agents-ga

⭐ **DeepSeek Open-Sources V3-0407 Model**
Updated V3 with 671B MoE architecture, MIT license. Weights available on HuggingFace.
🔗 https://github.com/deepseek-ai/DeepSeek-V3

⭐ **OpenAI Open-Sources Codex CLI**
Terminal-based coding agent with suggest, auto-edit, and full-auto modes. MIT license.
🔗 https://github.com/openai/codex

⭐ **LMSYS Chatbot Arena April Rankings Update**
Claude 4.5 Sonnet enters #2 overall (ELO 1287). Gemini 2.5 Pro holds #1 in coding.
🔗 https://lmarena.ai

⭐ **GitHub Copilot Coding Agent Public Preview**
Autonomous agent handles issues and creates PRs in secure cloud sandbox. Free for Pro+ and Enterprise.
🔗 https://github.blog/changelog/copilot-coding-agent

⭐ **Windsurf Raises $200M Series C at $3B Valuation**
Largest round in AI coding tools space. Plans to hire 200 engineers and expand enterprise features.
🔗 https://techcrunch.com/2026/04/07/windsurf-raises-200m

---
Powered by MorningAI | Full report: report_2026-04-08.md
```


path: skills/gen-social/agents/openai.yaml
content: interface:
  display_name: "Social Content Generator"
  short_description: "Generate platform-specific social media copy and images for X and Xiaohongshu"
  default_prompt: "Generate social media content for today's AI news report."


path: skills/gen-social/templates/x/insider.md
content: ---
platform: x
style: insider
default_lang: en
default_items: 3
min_score: 7
persona: tech-insider
image_style: tech
image_lang: en
---

<!-- EXAMPLE TEMPLATE — This is a reference persona for X/Twitter.
     To create your own persona:
       cp skills/gen-social/templates/x/insider.md skills/gen-social/templates/x/my-style.md
     Then edit the copy. Custom templates in this directory are gitignored.
     Set your channel config style to "my-style" to use it. -->

## Persona: Tech Insider

You're an AI industry insider who's been following this space since GPT-2. You know the players, you see patterns others miss, and you translate what just happened into what it actually means. Think @swyx or @simonw — someone who builds, ships, and pays attention.

**Voice traits:**
- Conversational but informed — you talk like someone at a good dinner party, not a press release
- Dry humor when the moment calls for it ("another day, another billion-parameter model")
- You connect dots — "this is interesting because..." / "the real story here isn't X, it's Y"
- You use contractions naturally (it's, doesn't, won't, they're)
- You occasionally address the reader directly ("worth keeping an eye on", "if you're building with...")
- You drop context that only insiders would know ("remember when X said Y last month?")

**What you never do:**
- Sound like a press release or corporate announcement
- Use "BREAKING:" or "JUST IN:" or any cable news energy
- Write generic filler ("In a move that could reshape the industry...")
- Over-explain things your audience already understands
- Use hashtags as decoration — 1-2 max, only if genuinely useful for discovery

## Constraints

- **One tweet per item**, max 280 characters each
- Up to 3 standalone tweets (not a thread)
- Reserve ~23 chars for a t.co link
- **Effective copy limit: ~255 characters**
- Hashtags are optional — skip them if the tweet is stronger without

## Content Structure

Each tweet is a self-contained take. No fixed formula — vary the structure across tweets. Some approaches:

- Lead with the implication, follow with the fact
- Open with a short punchy sentence, then the detail
- Ask a rhetorical question, then answer it
- State the fact, then add the "why you should care" angle

## Character Limit Rules

| Component | Budget |
|-----------|--------|
| Main copy | ~255 chars |
| Source link | ~23 chars |
| Hashtag (optional) | ~15 chars |
| **Total** | **≤ 280 chars** |

If over 280: cut qualifiers and secondary details first. The insight matters more than the completeness.

## Output Format

```
---tweet---
{copy}

{link}

---media---
{image_filename or "none"}
```

## Examples

```
---tweet---
Claude 4.5 Sonnet just dropped. +18% SWE-Bench, 200K context, 40% faster. The interesting part: Anthropic is closing the Opus-Sonnet gap fast enough that "just use Opus" stops being good advice.

https://anthropic.com/news/claude-4-5-sonnet

---media---
social_2026-04-08_x_insider_en_cover.png

---tweet---
Cursor shipped background agents to GA. They run in the cloud, no IDE needed, up to 10 at once. We're about 6 months from "I reviewed the PR my agent's agent wrote" being a normal sentence.

https://cursor.com/changelog

---media---
none

---tweet---
DeepSeek just open-sourced a 671B MoE model under MIT. Benchmarks put it near GPT-4o at a fraction of the cost. The "open-source can't compete at the frontier" argument had a rough week.

https://huggingface.co/deepseek-ai

---media---
none
```


path: skills/gen-social/templates/xiaohongshu/educational.md
content: ---
platform: xiaohongshu
style: educational
default_lang: zh
default_items: 5
min_score: 6
image_style: classic
image_lang: zh
---

<!-- 示例模板 — 这是小红书科普体的参考模板。
     创建你自己的风格：
       cp skills/gen-social/templates/xiaohongshu/educational.md skills/gen-social/templates/xiaohongshu/my-style.md
     然后编辑内容。此目录下的自定义模板会被 gitignore。
     在 channel 配置中将 style 设为 "my-style" 即可使用。 -->

## Constraints

- **Title**: ≤ 20 characters, structured and informative
- **Body**: ≤ 1000 characters
- **Tags**: 5-10 hashtags at end of body, format: `#标签`
- **Images**: 3:4 portrait or 1:1 square, carousel supported (3-6 images)
- **Emoji**: Moderate use — structured and clean, not excessive

## Tone & Voice

- Educational, structured, "one article to understand it all"
- Authoritative but approachable
- Use framing like "一文读懂" "看完这篇就够了" "3分钟了解"
- Focus on explaining **why it matters**, not just what happened
- Help readers build knowledge, not just consume headlines

## Content Structure

### Title
- Format: `{emoji} {structured phrase}` (≤ 20 chars)
- Emphasis on comprehensiveness or clarity
- Examples: "📚一文读懂今日AI大事", "🔍AI日报｜5个重要更新", "💡今日AI圈发生了什么"

### Body
1. **Opening** (1 sentence) — context and date
2. **Numbered items** — each with clear structure:
   - `{number}️⃣ {Entity} — {Event}`
   - **What**: 1 sentence on what happened
   - **Why it matters**: 1 sentence on significance
   - **Key metric** (if available): number or comparison
3. **Summary takeaway** — "今日关键词：..." or trend observation
4. **Tags** — 5-10 relevant hashtags

### Body Format Rules
- Numbered list (1️⃣ 2️⃣ 3️⃣ etc.)
- Clear separation between "what" and "why"
- Bold key terms with 【】brackets
- Use ▸ or → for sub-points
- Line breaks between each numbered item

## Character Limit Rules

| Component | Budget |
|-----------|--------|
| Title | ≤ 20 chars |
| Opening | ~30-50 chars |
| Per item block | ~100-180 chars |
| Summary takeaway | ~50-80 chars |
| Tags | ~100-150 chars |
| **Total body** | **≤ 1000 chars** |

## Output Format

```
---xiaohongshu---

---title---
{emoji} {title text}

---body---
{structured body with numbered items}

#tag1 #tag2 #tag3 #tag4 #tag5

---images---
1: {image_filename}
2: {image_filename}
3: {image_filename}
4: {image_filename}
```

## Example Output

```
---xiaohongshu---

---title---
📚一文读懂今日AI大事

---body---
4月8日AI圈5个重要更新，看完这篇就够了👇

1️⃣ Anthropic — Claude 4.5 Sonnet 发布
▸ 新一代中端模型，编程能力提升18%
▸ 上下文扩展到200K，速度提升40%
→ 意味着：日常AI编程助手体验将大幅升级

2️⃣ Google — Gemini 2.5 Flash 公测
▸ 100万上下文窗口，支持多模态推理
▸ AI Studio免费用，Vertex AI付费版
→ 意味着：长文档处理能力进入新阶段

3️⃣ DeepSeek — V3-0407 开源
▸ 671B参数MoE架构，MIT协议
▸ 性能接近GPT-4o，成本极低
→ 意味着：开源模型正式追平闭源前沿

4️⃣ Cursor — 后台Agent正式上线
▸ 无需盯屏幕，后台自动重构和提PR
▸ Pro版最多10个Agent并发
→ 意味着：编程工作流正在走向全自动

5️⃣ Windsurf — 2亿美元C轮融资
▸ 估值30亿美元，Insight Partners领投
→ 意味着：AI编程工具赛道资本持续加注

📌 今日关键词：模型升级、开源突破、编程自动化

#AI日报 #人工智能 #Claude #Gemini #DeepSeek #Cursor #AI科普 #科技圈 #大模型

---images---
1: social_2026-04-08_xhs_kepu_zh_1.png
2: social_2026-04-08_xhs_kepu_zh_2.png
3: social_2026-04-08_xhs_kepu_zh_3.png
4: social_2026-04-08_xhs_kepu_zh_4.png
```


path: skills/leaderboard/scripts/leaderboard_snapshot.py
content: #!/usr/bin/env python3
"""CLI for leaderboard snapshot management.

Usage:
    # Save snapshot and print diff against previous
    python3 skills/leaderboard/scripts/leaderboard_snapshot.py save \\
        --leaderboard lmsys --date 2026-04-08 \\
        --data '[{"model":"GPT-4o","rank":1,"score":1287}]'

    # Show the latest saved snapshot
    python3 skills/leaderboard/scripts/leaderboard_snapshot.py latest --leaderboard lmsys
"""

import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
from lib.snapshots import diff_snapshot, get_latest_snapshot, save_snapshot


def cmd_save(args):
    entries = json.loads(args.data)
    save_snapshot(args.leaderboard, args.date, entries)
    result = diff_snapshot(args.leaderboard, args.date)
    print(json.dumps(result, indent=2, ensure_ascii=False))


def cmd_latest(args):
    result = get_latest_snapshot(args.leaderboard)
    print(json.dumps(result, indent=2, ensure_ascii=False))


def main():
    parser = argparse.ArgumentParser(description="Leaderboard snapshot tool")
    sub = parser.add_subparsers(dest="command", required=True)

    p_save = sub.add_parser("save", help="Save snapshot and show diff")
    p_save.add_argument("--leaderboard", required=True)
    p_save.add_argument("--date", required=True, help="YYYY-MM-DD")
    p_save.add_argument("--data", required=True, help="JSON array of model entries")
    p_save.set_defaults(func=cmd_save)

    p_latest = sub.add_parser("latest", help="Show latest snapshot")
    p_latest.add_argument("--leaderboard", required=True)
    p_latest.set_defaults(func=cmd_latest)

    args = parser.parse_args()
    args.func(args)


if __name__ == "__main__":
    main()


path: skills/tracking-list/agents/openai.yaml
content: interface:
  display_name: "AI News Tracking Spec"
  short_description: "Scoring, formatting, and validation rules for AI news items"
  default_prompt: "Apply the AI news tracking specification for scoring and formatting news items."


path: skills/tracking-list/scripts/collect.py
content: #!/usr/bin/env python3
"""morning-ai data collection orchestrator.

Runs all collectors concurrently, normalizes, scores, deduplicates,
and outputs a structured JSON report.

Usage:
    python3 skills/tracking-list/scripts/collect.py [--date YYYY-MM-DD] [--depth quick|default|deep] [--output PATH]
"""

import argparse
import json
import sys
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional

# Add project root to path for imports
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))

from lib import env, cache
from lib.schema import TrackerItem, CollectionResult, DailyReport
from lib.score import score_items, apply_verification_bonus
from lib.dedupe import dedupe_by_source, cross_source_link
from lib import entities


def _log(msg: str):
    sys.stderr.write(f"[collect] {msg}\n")
    sys.stderr.flush()


def get_time_window(date_str: str) -> tuple:
    """Get 24-hour collection window (08:00 UTC+8 yesterday to 08:00 UTC+8 today).

    Returns:
        (from_date, to_date) as YYYY-MM-DD strings
    """
    dt = datetime.strptime(date_str, "%Y-%m-%d")
    from_date = (dt - timedelta(days=1)).strftime("%Y-%m-%d")
    to_date = date_str
    return from_date, to_date


def collect_reddit(config: Dict[str, Any], from_date: str, to_date: str, depth: str) -> CollectionResult:
    """Collect from Reddit."""
    from lib import reddit
    return reddit.collect(
        entities.REDDIT_KEYWORDS, from_date, to_date,
        entity_subreddits=entities.REDDIT_SUBREDDITS, depth=depth,
    )


def collect_hackernews(config: Dict[str, Any], from_date: str, to_date: str, depth: str) -> CollectionResult:
    """Collect from Hacker News."""
    from lib import hackernews
    return hackernews.collect(entities.HN_KEYWORDS, from_date, to_date, depth)


def collect_github(config: Dict[str, Any], from_date: str, to_date: str, depth: str) -> CollectionResult:
    """Collect from GitHub."""
    from lib import github

    [REDACTED_SECRET], "GITHUB_TOKEN")
    return github.collect(entities.GITHUB_SOURCES, from_date, to_date, token, depth)


def collect_huggingface(config: Dict[str, Any], from_date: str, to_date: str, depth: str) -> CollectionResult:
    """Collect from HuggingFace."""
    from lib import huggingface
    return huggingface.collect(entities.HUGGINGFACE_AUTHORS, from_date, to_date, depth)


def collect_arxiv(config: Dict[str, Any], from_date: str, to_date: str, depth: str) -> CollectionResult:
    """Collect from arXiv."""
    from lib import arxiv
    return arxiv.collect(entities.ARXIV_QUERIES, from_date, to_date, depth)


# All collectors with their names
COLLECTORS = {
    "reddit": collect_reddit,
    "hackernews": collect_hackernews,
    "github": collect_github,
    "huggingface": collect_huggingface,
    "arxiv": collect_arxiv,
}


def run_collection(
    date_str: str,
    depth: str = "default",
    sources: Optional[List[str]] = None,
) -> DailyReport:
    """Run full data collection pipeline.

    Args:
        date_str: Target date YYYY-MM-DD
        depth: Collection depth (quick|default|deep)
        sources: Optional list of sources to collect (default: all)

    Returns:
        DailyReport with all collected, scored, and deduplicated items
    """
    config = env.get_config()
    from_date, to_date = get_time_window(date_str)
    available = env.get_available_sources(config)

    _log(f"Date: {date_str}, Window: {from_date} ~ {to_date}, Depth: {depth}")
    _log(f"Available sources: {[k for k, v in available.items() if v]}")

    # Determine which collectors to run
    active_collectors = {}
    for name, func in COLLECTORS.items():
        if sources and name not in sources:
            continue
        active_collectors[name] = func

    # Run collectors concurrently
    start_time = time.time()
    collection_results = []
    all_items: List[TrackerItem] = []

    with ThreadPoolExecutor(max_workers=len(active_collectors)) as executor:
        futures = {
            executor.submit(func, config, from_date, to_date, depth): name
            for name, func in active_collectors.items()
        }

        for future in as_completed(futures):
            name = futures[future]
            try:
                result = future.result(timeout=120)
                collection_results.append(result)
                all_items.extend(result.items)
                _log(f"[{name}] Done: {len(result.items)} items, {len(result.errors)} errors")
            except Exception as e:
                _log(f"[{name}] Failed: {e}")
                collection_results.append(
                    CollectionResult(source=name, errors=[str(e)])
                )

    collect_time = time.time() - start_time
    _log(f"Collection complete: {len(all_items)} raw items in {collect_time:.1f}s")

    # Pipeline: Score → Dedupe by source → Cross-source link → Verification bonus → Final dedupe
    _log("Scoring items...")
    all_items = score_items(all_items)

    _log("Deduplicating within sources...")
    all_items = dedupe_by_source(all_items)
    _log(f"After source dedup: {len(all_items)} items")

    _log("Cross-source linking...")
    all_items = cross_source_link(all_items)

    _log("Applying verification bonus...")
    all_items = apply_verification_bonus(all_items)

    # Re-assign sequential IDs
    for i, item in enumerate(all_items):
        item.id = f"T{i + 1:03d}"

    # Build stats
    stats = {
        "total_raw": sum(len(r.items) for r in collection_results),
        "total_after_dedup": len(all_items),
        "by_source": {},
        "by_importance": {
            "9-10 (industry-changing)": len([i for i in all_items if i.importance >= 9]),
            "7-8 (important)": len([i for i in all_items if 7 <= i.importance < 9]),
            "5-6 (routine)": len([i for i in all_items if 5 <= i.importance < 7]),
            "3-4 (minor)": len([i for i in all_items if 3 <= i.importance < 5]),
            "1-2 (trivial)": len([i for i in all_items if i.importance < 3]),
        },
        "verified_count": len([i for i in all_items if i.verified]),
        "collection_time_seconds": round(collect_time, 1),
        "errors": [],
    }

    for r in collection_results:
        stats["by_source"][r.source] = {
            "items": len(r.items),
            "entities_checked": r.entities_checked,
            "entities_with_updates": r.entities_with_updates,
            "errors": r.errors,
        }
        stats["errors"].extend(r.errors)

    # Build report
    report = DailyReport(
        date=date_str,
        time_window=f"{from_date} 08:00 ~ {to_date} 08:00 UTC+8",
        generated_at=datetime.now(timezone.utc).isoformat(),
        items=all_items,
        collection_results=collection_results,
        stats=stats,
    )

    _log(f"Report ready: {len(all_items)} items, {stats['verified_count']} verified")
    return report


def main():
    parser = argparse.ArgumentParser(description="morning-ai data collection")
    parser.add_argument("--date", default=datetime.now().strftime("%Y-%m-%d"),
                        help="Target date (default: today)")
    parser.add_argument("--depth", default="default", choices=["quick", "default", "deep"],
                        help="Collection depth")
    parser.add_argument("--sources", nargs="*",
                        help="Specific sources to collect (default: all)")
    parser.add_argument("--output", "-o", default=None,
                        help="Output JSON file path (default: stdout)")
    parser.add_argument("--cache-ttl", type=int, default=1,
                        help="Cache TTL in hours (default: 1, 0 to disable)")
    parser.add_argument("--no-cache", action="store_true",
                        help="Skip cache entirely (equivalent to --cache-ttl 0)")
    parser.add_argument("--clear-cache", action="store_true",
                        help="Clear all cached data and exit")
    args = parser.parse_args()

    # Clear cache and exit
    if args.clear_cache:
        cache.clear_cache()
        _log("Cache cleared")
        return

    # Check cache
    ttl = 0 if args.no_cache else args.cache_ttl
    cache_key = cache.get_cache_key("daily", args.date, args.depth)
    if ttl > 0:
        cached = cache.load_cache(cache_key, ttl)
        if cached:
            _log("Using cached report")
            if args.output:
                Path(args.output).write_text(json.dumps(cached, ensure_ascii=False, indent=2))
            else:
                json.dump(cached, sys.stdout, ensure_ascii=False, indent=2)
            return

    # Run collection
    report = run_collection(args.date, args.depth, args.sources)
    report_dict = report.to_dict()

    # Save cache
    if ttl > 0:
        cache.save_cache(cache_key, report_dict)

    # Output
    if args.output:
        output_path = Path(args.output)
        output_path.parent.mkdir(parents=True, exist_ok=True)
        output_path.write_text(json.dumps(report_dict, ensure_ascii=False, indent=2))
        _log(f"Report saved to {args.output}")
    else:
        json.dump(report_dict, sys.stdout, ensure_ascii=False, indent=2)


if __name__ == "__main__":
    main()


path: templates/report.md
content: # AI News Daily {DATE}

**Coverage Period**: [{YESTERDAY} 08:00, {TODAY} 08:00) UTC+8
**Generated At**: {TIMESTAMP}
**Tracking Types**: {ACTIVE_TYPES}

---

<!-- Insert cover image here if generated: ![Cover Image](news_infographic_{DATE}.png) -->

---

## TLDR

> Today's important updates with 7+ scores (across all types)

<!-- List by score from high to low for all 7+ updates, format:
- **[Type] Entity name - Event description** (X.X points): One-line summary with specific details (numbers, versions, capabilities).
  _Impact: One sentence explaining why this matters — competitive context, industry significance, or user implications._ [[Source](URL)]
Every TLDR item MUST include at least one source link so readers can click through to the original.
-->

---

## Model Updates

<!-- Insert type image if generated: ![Model Updates](news_infographic_{DATE}_model.png) -->

### Major Releases (9-10 Scores)

<!-- List in tracking-list format. Each item MUST include:
     - Summary with 5-8 specific bullet points covering: what, why, technical specs, competitive context, availability, pricing, ecosystem impact
     - "Why It Matters" analysis paragraph (2-4 sentences: industry impact, competitive context, user implications)
     - "Key Data" table with quantitative metrics (benchmarks, parameters, pricing, performance deltas)
     - Multi-source verification (2+ sources)
-->

### Important Updates (7-8 Scores)

<!-- List in tracking-list format. Each item MUST include:
     - Summary with 4-6 specific bullet points covering: what happened, key specs/numbers, why it matters, who's affected
     - "Why It Matters" analysis paragraph (1-2 sentences)
     - "Key Data" table when applicable
     - Multi-source verification (2+ sources)
-->

### Regular Updates (5-6 Scores)

<!-- Multi-line format per item with 3-4 concrete points:
- **Entity** (X.X): Event description with specifics (version, capability, metric).
  - Detail 1: what changed, key numbers, comparison
  - Detail 2: additional context, availability
  - Detail 3: implications or notable aspects
  Source: [Name](URL)
-->

### Minor Updates (3-4 Scores)

<!-- Table format — Source column MUST contain clickable [Name](URL) links -->

| Entity | Update | Score | Source |
|------|---------|------|------|

---

## Product Updates

<!-- Insert type image if generated: ![Product Updates](news_infographic_{DATE}_product.png) -->

### Major Updates (9-10 Scores)

<!-- List in tracking-list format with full detail: Summary (5-8 bullets with technical specs, availability, pricing, competitive context), Why It Matters (2-4 sentences), Key Data table, multi-source verification -->

### Important Updates (7-8 Scores)

<!-- List in tracking-list format with full detail: Summary (4-6 bullets), Why It Matters (1-2 sentences), Key Data table (if applicable), multi-source verification -->

### Regular Updates (5-6 Scores)

<!-- Multi-line format per item with 3-4 concrete points:
- **Product** (X.X): Event description with specifics (version, capability, metric).
  - Detail 1: what changed, key numbers, comparison
  - Detail 2: additional context, availability
  - Detail 3: implications or notable aspects
  Source: [Name](URL)
-->

### Minor Updates (3-4 Scores)

<!-- Source column MUST contain clickable [Name](URL) links -->

| Product | Update | Score | Source |
|------|---------|------|------|

---

## Benchmarks & Research

<!-- Insert type image if generated: ![Benchmark Updates](news_infographic_{DATE}_benchmark.png) -->

### Major Benchmarks (7+ Scores)

<!-- List in tracking-list format with full detail: Summary (4-8 bullets depending on score), Why It Matters (analytical, not descriptive), Key Data table (benchmark scores with deltas), multi-source verification -->

### Academic Papers

<!-- Use academic paper record format. Each paper MUST include:
     - Core Innovation (one sentence)
     - Research Significance (who is impacted, what does this enable, concrete implications for practitioners)
     - Key Data table (metrics, benchmark results with deltas, comparisons to baselines and SOTA)
     - 3-5 bullet points covering: method, key results, limitations, code/weights availability
-->

### Regular Benchmarks (5-6 Scores)

<!-- Multi-line format per item with 3-4 points:
- **Entity** (X.X): Event description with specifics.
  - Detail 1: what changed in the ranking/results, key numbers
  - Detail 2: comparison with previous rankings or competing models
  - Detail 3: methodology or significance
  Source: [Name](URL)
-->

---

## Funding

<!-- Insert type image if generated: ![Funding Updates](news_infographic_{DATE}_funding.png) -->

### Major Events (7+ Scores)

<!-- List in tracking-list format with full detail: Summary (4-6 bullets covering amount, valuation, investors, strategic rationale, competitive context, planned use of funds), Why It Matters (competitive/strategic context, 1-4 sentences based on score), Key Data table (amount, valuation, investors, previous round comparison), multi-source verification -->

### Regular Events

| Entity | Event | Amount/Scale | Score | Source |
|------|------|----------|------|------|

<!-- Source column MUST contain clickable [Name](URL) links -->

---

## Entities with No Updates

<!-- List entities with no updates today -->

---

## Statistics Summary

### By Type

| Type | Total | 9-10 | 7-8 | 5-6 | 3-4 | 1-2 |
|------|------|------|-----|-----|-----|-----|
| Model | | | | | | |
| Product | | | | | | |
| Benchmark | | | | | | |
| Funding | | | | | | |
| **Total** | | | | | | |

### By Agent

| Agent | Discoveries | 7+ Scores | Completion Rate |
|-------|--------|------|--------|
| ai-labs | | | |
| model-infra | | | |
| coding-agent | | | |
| ai-apps | | | |
| vision-media | | | |
| benchmarks-academic | | | |
| kol | | | |
| trending-discovery | | | |

---

## References

| # | Source | URL |
|---|------|-----|

---

## Scoring Criteria

| Score | Level | Criteria |
|------|------|------|
| 9-10 | Major Event | Industry landscape breakthrough |
| 7-8 | Important Update | Noteworthy significant progress |
| 5-6 | Regular Update | Routine update, good to know |
| 3-4 | Minor Update | Secondary information |
| 1-2 | Trivial Update | Negligible details |


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)