Skill: 2813223285/xiaohongshu-note-fetcher (version 1.0.1)

--- name: xiaohongshu-note-fetcher description: 抓取并整理小红书笔记公开页面信息（标题、正文摘要、作者、发布时间、互动数据、标签、封面图等）为结构化 JSON 或 Markdown。用于“根据笔记链接提取内容”“批量收集笔记基础信息”“生成笔记摘要素材”等场景；当用户提供小红书笔记 URL、URL 列表或需要导出机器可读结果时触发。 --- # 小红书笔记获取 ## 极简规则 输入 `TikHub API Key` + `需求（关键词、页码）` 即可抓取小红书搜索数据。 最小示例： ```bash make tikhub-fetch \ KEYWORD="女性力量" \ TIKHUB_PAGE=2 \ TIKHUB_TOKEN="<YOUR_TIKHUB_KEY>" \ TIKHUB_ENDPOINT=web \ TIKHUB_AUTH_MODE=bearer ``` 输出文件： - `workspace/xiaohongshu-note-fetcher-skill-data/tikhub_search_page2.json` ## 快速开始 1. 收集输入来源： - 单条链接：直接给 URL。 - 批量链接：准备一个文本文件，每行一个 URL。 - 若目标页面需登录态：提供 Cookie 字符串或 Cookie 文件。 2. 运行脚本抓取（页面解析模式）： ```bash python3 scripts/fetch_xiaohongshu_notes.py \ --url "https://www.xiaohongshu.com/explore/<note_id>" \ --format both \ --output result.json ``` 3. 批量抓取示例： ```bash python3 scripts/fetch_xiaohongshu_notes.py \ --url-file ./urls.txt \ --format json \ --output notes.json ``` 4. 运行 TikHub API 搜索（推荐用于关键词搜笔记）： ```bash python3 scripts/search_notes_tikhub.py \ --token "<YOUR_TIKHUB_TOKEN>" \ --keyword "女性主义" \ --page 1 \ --output search_page1.json ``` 也可以使用 Makefile 一键调用（推荐长期使用）： ```bash make tikhub-fetch \ KEYWORD="美食" \ TIKHUB_TOKEN="<YOUR_TIKHUB_TOKEN>" \ TIKHUB_ENDPOINT=web \ TIKHUB_AUTH_MODE=bearer ``` 若你不想每次手输 token，可把 token 放到文件（如 `./.tikhub_token`）： ```bash make tikhub-fetch \ KEYWORD="美食" \ TIKHUB_TOKEN_FILE=./.tikhub_token \ TIKHUB_ENDPOINT=web \ TIKHUB_AUTH_MODE=bearer ``` 5. 使用其他服务商 API（通用适配）： ```bash python3 scripts/search_notes_generic.py \ --base-url "https://your-api.example.com/search_notes" \ --auth-mode bearer \ --auth-header Authorization \ --token "<YOUR_API_TOKEN>" \ --keyword "美食推荐" \ --page 1 \ --param sort_type=general \ --output generic_search.json ``` 6. 从 TikHub 响应生成文章列表（按点赞过滤）： ```bash python3 scripts/build_article_list_from_tikhub.py \ --input ./tikhub_search.json \ --min-likes 1000 \ --rank-by hot \ --md-output ./xhs_article_list.md \ --csv-output ./xhs_article_list.csv \ --json-output ./xhs_article_list.json \ --template-output ./xhs_publish_templates.md ``` 7. 自动翻页抓取并合并（直接调 TikHub）： ```bash python3 scripts/build_article_list_from_tikhub.py \ --token "<YOUR_TIKHUB_TOKEN>" \ --keyword "美食" \ --pages 5 \ --sort general \ --note-type _0 \ --min-likes 1000 \ --rank-by hot \ --top 50 ``` 8. 交互式筛选与查看（会先问你筛选方式，含默认方案）： ```bash python3 scripts/interactive_filter_view.py \ --input ./tikhub_search.json ``` 直接回车会采用默认方案： - 点赞阈值：`1000` - 排序：`hot`（综合热度） - 条数：`20` - 查看：`summary` - 同时导出 `md/csv` 若你不想交互，直接用默认方案： ```bash python3 scripts/interactive_filter_view.py \ --input ./tikhub_search.json \ --non-interactive ``` 9. 导图输出（保留）： ```bash python3 scripts/generate_wow_pack.py \ --input ./xhs_article_list.json \ --keyword 美食 \ --url-output ./xhs_topic_mindmap_url.txt ``` 6. 本机浏览器抓取（不走第三方 API，推荐）： ```bash node scripts/fetch_xiaohongshu_note_playwright.js \ --url "https://www.xiaohongshu.com/explore/<note_id>" \ --cookie-file ./cookie.txt \ --output note_browser.json \ --screenshot note_browser.png \ --html-out note_browser.html ``` 若首次运行提示 `playwright_not_installed`，先安装： ```bash cd scripts npm i playwright npx playwright install chromium ``` ## 参数说明 - `--url`: 单条小红书笔记 URL。 - `--url-file`: 批量 URL 文件（每行一个 URL，支持 `#` 注释行）。 - `--cookie`: 原始 Cookie 请求头字符串。 - `--cookie-file`: Cookie 文件（纯文本 Cookie 字符串）。 - `--format`: `json`、`md`、`both`，默认 `json`。 - `--output`: 输出路径；`json`/`both` 写该文件，`md` 会在同目录生成 `.md`。 - `--timeout`: 请求超时秒数，默认 `20`。 - `search_notes_tikhub.py` 关键参数： - `--keyword`、`--page` 必填。 - `--sort-type` 可选：`general`、`time_descending`、`popularity_descending` 等。 - `--note-type` 支持中英文：`不限`/`all`、`视频笔记`/`video`、`普通笔记`/`image`、`直播笔记`/`live`。 - `--time-filter` 支持中英文：`不限`/`all`、`一天内`/`day`、`一周内`/`week`、`半年内`/`half_year`。 - `--ai-mode` 使用整数 `0` 或 `1`。 - `search_notes_generic.py` 关键参数： - `--base-url`：新 API 的搜索端点。 - `--auth-mode`：`none`、`bearer`、`apikey`。 - `--auth-header`：鉴权头名称，默认 `Authorization`。 - `--keyword-param` / `--page-param`：当对方字段不是 `keyword/page` 时改这里。 - `--param key=value`：补充任意查询参数，可重复。 - `--header key=value`：补充任意请求头，可重复。 - `fetch_xiaohongshu_note_playwright.js` 关键参数： - `--url`：笔记 URL。 - `--cookie-file`：浏览器 Cookie 文本（建议提供，提高字段完整度）。 - `--headed`：显示浏览器窗口调试。 - `--screenshot` / `--html-out`：输出调试文件，方便排查风控页和登录页。 `--url` 与 `--url-file` 至少给一个。 ## 工作流程 1. 读取 URL 列表。 2. 发送 HTTP 请求拉取页面 HTML（带浏览器 UA，可选 Cookie）。 3. 从页面中提取： - OpenGraph 元信息（`og:title`、`og:description`、`og:image`） - JSON-LD（发布时间、作者、关键词、互动统计） - 页面脚本中的 `noteId`（若存在） 4. 产出统一字段结构，见 `references/output-schema.md`。 ## 常见问题处理 1. 仅返回基础字段或计数缺失： - 小红书前端结构会变化，且部分互动字段需要更完整会话态；优先补充 `--cookie` 后重试。 2. 返回 403 或风控页： - 更换网络环境或降低抓取频率，避免高并发。 - 仅用于你有权访问的数据，不要绕过平台安全机制。 3. 需要深度字段（如评论明细）： - 本 skill 默认仅做页面级基础信息抽取；如需更深层数据，先确认合规性，再扩展独立脚本。 4. TikHub `400 Request failed`： - 首次请求尽量只保留 `keyword`、`page`，确认成功后再加筛选项。 - 避免把数值参数写成字符串语义值：`page` 用整数、`ai_mode` 用 `0/1`。 - 分页时优先使用上一页返回的 `search_id`、`search_session_id`。 - 先用官方默认演示参数跑通，再逐项加入自定义参数。 ## 合规边界 - 只抓取你有权限访问的公开或授权内容。 - 遵守小红书服务条款、robots、隐私和数据保护要求。 - 不执行账号盗用、验证码绕过、反爬绕过等违规行为。 ## 资源 - 脚本：[scripts/fetch_xiaohongshu_notes.py](scripts/fetch_xiaohongshu_notes.py) - API 搜索脚本：[scripts/search_notes_tikhub.py](scripts/search_notes_tikhub.py) - 通用 API 适配脚本：[scripts/search_notes_generic.py](scripts/search_notes_generic.py) - 响应转列表脚本：[scripts/build_article_list_from_tikhub.py](scripts/build_article_list_from_tikhub.py) - 交互筛选脚本：[scripts/interactive_filter_view.py](scripts/interactive_filter_view.py) - 惊艳组合脚本：[scripts/generate_wow_pack.py](scripts/generate_wow_pack.py) - 浏览器抓取脚本：[scripts/fetch_xiaohongshu_note_playwright.js](scripts/fetch_xiaohongshu_note_playwright.js) - 字段说明：[references/output-schema.md](references/output-schema.md)

path: PUBLISHING_NOTES.txt
content: Slug: xiaohongshu-note-fetcher
Display Name: 小红书笔记抓取与选题助手
Version: 1.0.0
Tags: latest,xiaohongshu,content,automation,mindmap
License: MIT-0

Changelog:
- 首次发布：支持 TikHub API 抓取小红书搜索数据
- 支持关键词+页码抓取与结构化输出
- 支持按点赞阈值筛选并导出 md/csv/json
- 支持思维导读链接生成


path: scripts/fetch_xiaohongshu_note_playwright.js
content: #!/usr/bin/env node
"use strict";

const fs = require("node:fs");
const path = require("node:path");

function parseArgs(argv) {
  const args = {
    url: "",
    output: "xhs_note_browser.json",
    cookieFile: "",
    timeout: 45000,
    waitMs: 2500,
    headed: false,
    screenshot: "",
    htmlOut: "",
  };
  for (let i = 2; i < argv.length; i += 1) {
    const k = argv[i];
    const v = argv[i + 1];
    if (k === "--url") args.url = v || "";
    else if (k === "--output") args.output = v || args.output;
    else if (k === "--cookie-file") args.cookieFile = v || "";
    else if (k === "--timeout") args.timeout = Number(v || args.timeout);
    else if (k === "--wait-ms") args.waitMs = Number(v || args.waitMs);
    else if (k === "--screenshot") args.screenshot = v || "";
    else if (k === "--html-out") args.htmlOut = v || "";
    else if (k === "--headed") args.headed = true;
    else if (k === "--help" || k === "-h") {
      printHelp();
      process.exit(0);
    }
  }
  return args;
}

function printHelp() {
  console.log(`Usage:
  node fetch_xiaohongshu_note_playwright.js --url <note_url> [options]

Options:
  --url <url>            Xiaohongshu note URL
  --output <file>        Output JSON file (default: xhs_note_browser.json)
  --cookie-file <file>   Raw Cookie header value saved in a text file
  --timeout <ms>         Navigation timeout in ms (default: 45000)
  --wait-ms <ms>         Extra wait after load (default: 2500)
  --headed               Run non-headless for debugging
  --screenshot <file>    Optional screenshot output path
  --html-out <file>      Optional rendered HTML output path
  --help                 Show help`);
}

function parseCookieHeader(raw) {
  const out = [];
  const text = (raw || "").trim();
  if (!text) return out;
  for (const part of text.split(";")) {
    const p = part.trim();
    if (!p) continue;
    const idx = p.indexOf("=");
    if (idx <= 0) continue;
    const name = p.slice(0, idx).trim();
    const value = p.slice(idx + 1).trim();
    if (!name) continue;
    out.push({ name, value, domain: ".xiaohongshu.com", path: "/" });
  }
  return out;
}

function pick(v, fallback = null) {
  return v === undefined || v === null || v === "" ? fallback : v;
}

function toIntMaybe(v) {
  if (v === undefined || v === null) return null;
  const n = Number(v);
  return Number.isFinite(n) ? n : null;
}

function extractNoteId(url) {
  const m = String(url || "").match(/\/explore\/([0-9a-zA-Z]+)/);
  return m ? m[1] : null;
}

function deepFind(obj, keys) {
  if (!obj || typeof obj !== "object") return [];
  const res = [];
  const stack = [obj];
  while (stack.length) {
    const cur = stack.pop();
    if (!cur || typeof cur !== "object") continue;
    for (const [k, v] of Object.entries(cur)) {
      if (keys.includes(k)) res.push(v);
      if (v && typeof v === "object") stack.push(v);
    }
  }
  return res;
}

async function run() {
  const args = parseArgs(process.argv);
  if (!args.url) {
    printHelp();
    process.exit(2);
  }

  let playwright;
  try {
    playwright = require("playwright");
  } catch (e) {
    console.error("playwright_not_installed");
    console.error("Install: npm i playwright && npx playwright install chromium");
    process.exit(3);
  }

  const payload = {
    ok: false,
    url: args.url,
    final_url: args.url,
    note_id: extractNoteId(args.url),
    title: null,
    description: null,
    author: null,
    published_at: null,
    tags: [],
    images: [],
    interaction: {
      like_count: null,
      comment_count: null,
      share_count: null,
      collect_count: null,
    },
    login_popup_detected: false,
    login_popup_closed: false,
    fetched_at_utc: new Date().toISOString(),
    source_status: "error",
    error: "",
  };

  const browser = await playwright.chromium.launch({ headless: !args.headed });
  const context = await browser.newContext({
    userAgent:
      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 " +
      "(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
    locale: "zh-CN",
    viewport: { width: 1440, height: 2000 },
  });

  try {
    if (args.cookieFile) {
      const cookieText = fs.readFileSync(args.cookieFile, "utf-8");
      const cookies = parseCookieHeader(cookieText);
      if (cookies.length > 0) await context.addCookies(cookies);
    }

    const page = await context.newPage();
    await page.goto(args.url, { waitUntil: "domcontentloaded", timeout: args.timeout });
    await page.waitForTimeout(args.waitMs);
    payload.final_url = page.url();

    const isLoginRedirect = payload.final_url.includes("/login");
    if (isLoginRedirect) {
      payload.ok = false;
      payload.source_status = "login_required";
      payload.error = "redirected_to_login; provide cookie-file from logged-in browser";
      const htmlEarly = await page.content();
      if (args.htmlOut) fs.writeFileSync(args.htmlOut, htmlEarly, "utf-8");
      if (args.screenshot) await page.screenshot({ path: args.screenshot, fullPage: true });
    }
    const popupResult = await page.evaluate(() => {
      const hasLoginText = /登录查看更多内容|登录后查看更多内容/.test(document.body.innerText || "");
      const closeSelectors = [
        'button[aria-label*="关闭"]',
        'button[class*="close"]',
        'div[class*="close"]',
        'i[class*="close"]',
        '.login-container button',
        '.login-guide button',
      ];
      let clicked = false;
      for (const sel of closeSelectors) {
        const nodes = Array.from(document.querySelectorAll(sel));
        for (const n of nodes) {
          const txt = (n.textContent || "").trim();
          const cls = n.className ? String(n.className) : "";
          if (/关闭|close|icon-close|cancel/i.test(txt + " " + cls)) {
            try {
              n.click();
              clicked = true;
            } catch (_) {}
          }
        }
      }
      // Remove common overlay layers if present.
      const overlaySelectors = [
        '.mask',
        '.modal-mask',
        '.overlay',
        '.login-container',
        '.login-guide',
        '[class*="login-panel"]',
      ];
      for (const sel of overlaySelectors) {
        const nodes = document.querySelectorAll(sel);
        nodes.forEach((el) => {
          const c = el.className ? String(el.className) : "";
          if (/login|modal|mask|overlay/i.test(c)) {
            try {
              el.remove();
            } catch (_) {}
          }
        });
      }
      return { hasLoginText, clicked };
    });
    payload.login_popup_detected = !!(popupResult && popupResult.hasLoginText);
    payload.login_popup_closed = !!(popupResult && popupResult.clicked);
    if (payload.login_popup_detected) {
      await page.waitForTimeout(1200);
    }
    if (!isLoginRedirect) {
      const html = await page.content();
      if (args.htmlOut) {
        fs.writeFileSync(args.htmlOut, html, "utf-8");
      }
      if (args.screenshot) {
        await page.screenshot({ path: args.screenshot, fullPage: true });
      }

      const extracted = await page.evaluate(() => {
        const readMeta = (name) => {
          const p = document.querySelector(`meta[property="${name}"]`);
          if (p && p.content) return p.content;
          const n = document.querySelector(`meta[name="${name}"]`);
          if (n && n.content) return n.content;
          return null;
        };

        const getJsonLd = () => {
          const blocks = Array.from(document.querySelectorAll('script[type="application/ld+json"]'));
          const out = [];
          for (const b of blocks) {
            try {
              const v = JSON.parse(b.textContent || "");
              out.push(v);
            } catch (_) {}
          }
          return out;
        };

        return {
          title: readMeta("og:title") || document.title || null,
          description: readMeta("og:description") || null,
          ogImage: readMeta("og:image") || null,
          jsonLd: getJsonLd(),
        };
      });

      payload.title = pick(extracted.title);
      payload.description = pick(extracted.description);
      if (extracted.ogImage) payload.images.push(extracted.ogImage);

      const ldList = Array.isArray(extracted.jsonLd) ? extracted.jsonLd : [];
      const merged = {};
      for (const item of ldList) {
        if (item && typeof item === "object") Object.assign(merged, item);
      }

      if (merged.author && typeof merged.author === "object") {
        payload.author = pick(merged.author.name);
      }
      payload.published_at = pick(merged.datePublished);

      const keywords = pick(merged.keywords, "");
      if (typeof keywords === "string" && keywords.trim()) {
        payload.tags = keywords
          .split(/[，,]/)
          .map((s) => s.trim())
          .filter(Boolean);
      }

      const ldImage = merged.image;
      if (typeof ldImage === "string") payload.images.push(ldImage);
      if (Array.isArray(ldImage)) {
        for (const i of ldImage) {
          if (typeof i === "string") payload.images.push(i);
        }
      }
      payload.images = Array.from(new Set(payload.images.filter(Boolean)));

      // Regex fallback on rendered HTML for note description/author.
      const descPatterns = [/"desc":"([^"]{1,5000})"/, /"description":"([^"]{1,5000})"/];
      for (const p of descPatterns) {
        const m = html.match(p);
        if (m && m[1]) {
          const cand = m[1].replace(/\\n/g, "\n").replace(/\\"/g, '"').trim();
          if (cand && !/3 亿人的生活经验，都在小红书/.test(cand)) {
            payload.description = cand;
            break;
          }
        }
      }
      if (!payload.author) {
        const m = html.match(/"nickname":"([^"]{1,200})"/);
        if (m && m[1]) payload.author = m[1].replace(/\\"/g, '"');
      }

      const stats = Array.isArray(merged.interactionStatistic) ? merged.interactionStatistic : [];
      for (const s of stats) {
        if (!s || typeof s !== "object") continue;
        const t = String(s.interactionType || "").toLowerCase();
        const c = toIntMaybe(s.userInteractionCount);
        if (t.includes("like")) payload.interaction.like_count = c;
        else if (t.includes("comment")) payload.interaction.comment_count = c;
        else if (t.includes("share")) payload.interaction.share_count = c;
      }

      // Fallback: extract lightweight fields from visible DOM only.
      const domFallback = await page.evaluate(() => {
        const pickText = (selectors) => {
          for (const s of selectors) {
            const el = document.querySelector(s);
            const text = el && el.textContent ? el.textContent.trim() : "";
            if (text) return text;
          }
          return null;
        };
        return {
          title: pickText(["h1", ".note-content .title", ".content .title"]),
          author: pickText([".author .name", ".author-name", ".username", ".user-name", ".author"]),
          description: pickText([".note-content .desc", ".content .desc", ".note-desc", ".desc"]),
        };
      });
      if (!payload.title && domFallback && domFallback.title) payload.title = domFallback.title;
      if (!payload.author && domFallback && domFallback.author) payload.author = domFallback.author;
      if (!payload.description && domFallback && domFallback.description) payload.description = domFallback.description;
      if (payload.description && /3 亿人的生活经验，都在小红书/.test(payload.description)) {
        payload.description = null;
      }

      payload.ok = true;
      payload.source_status = "ok";
      if (!/\/explore\/[0-9a-zA-Z]+/.test(payload.final_url)) {
        payload.source_status = "redirected";
        payload.error = "redirected_to_non_note_page; try valid note URL and cookie-file";
        payload.ok = false;
      }
    }
  } catch (e) {
    payload.error = String(e && e.message ? e.message : e);
    payload.source_status = "error";
  } finally {
    await context.close();
    await browser.close();
  }

  fs.writeFileSync(args.output, `${JSON.stringify(payload, null, 2)}\n`, "utf-8");
  console.log(`Saved: ${path.resolve(args.output)}`);
  if (!payload.ok) process.exit(1);
}

run().catch((e) => {
  console.error(String(e));
  process.exit(1);
});


path: scripts/generate_wow_pack.py
content: #!/usr/bin/env python3
"""Generate an online mindmap URL from article list."""

from __future__ import annotations

import argparse
import base64
import json
import re
from collections import Counter
from pathlib import Path
from typing import Any, Dict, List


def load_rows(path: Path) -> List[Dict[str, Any]]:
    data = json.loads(path.read_text(encoding="utf-8"))
    if not isinstance(data, list):
        raise ValueError("Input must be a JSON list from xhs_article_list.json")
    out = []
    for x in data:
        if isinstance(x, dict):
            out.append(x)
    return out


def pick_keywords(text: str) -> List[str]:
    tags = re.findall(r"#([^\s#]{2,20})", text or "")
    return [t.strip() for t in tags if t.strip()]


def build_mindmap_data(rows: List[Dict[str, Any]], keyword: str) -> Dict[str, Any]:
    top_tags: Counter[str] = Counter()
    for r in rows:
        top_tags.update(pick_keywords((r.get("title") or "") + " " + (r.get("desc") or "")))
    tag_children = [f"{k} ({v})" for k, v in top_tags.most_common(6)] or ["无明显标签聚类"]

    top_notes = rows[:5]
    top_note_children = [f"{(r.get('title') or 'Untitled')[:18]} | ❤️{r.get('liked_count', 0)}" for r in top_notes]

    strategy_children = [
        "爆款结构复盘（标题+开头钩子）",
        "系列化发布（同标签连续5天）",
        "封面/标题 AB 测试",
        "评论区关键词二次扩展",
    ]
    risk_children = [
        "避免内容搬运与侵权",
        "数据仅作选题参考",
        "商业合作需标注与合规",
    ]

    return {
        "central": f"小红书高赞选题池（{keyword or '关键词'}）",
        "branches": [
            {"label": "🏆 高赞Top", "children": top_note_children},
            {"label": "🏷️ 话题聚类", "children": tag_children},
            {"label": "🚀 下周动作", "children": strategy_children},
            {"label": "⚠️ 风险注意", "children": risk_children},
        ],
    }


def _sanitize_text(text: str) -> str:
    s = (text or "").replace("\n", " ").replace('"', "'").strip()
    return s[:120]


def build_mermaid_mindmap_text(mindmap_data: Dict[str, Any]) -> str:
    central = _sanitize_text(str(mindmap_data.get("central") or "选题导图"))
    lines = ["mindmap", f'  root["{central}"]']
    branch_index = 0
    for branch in mindmap_data.get("branches", []):
        if not isinstance(branch, dict):
            continue
        branch_index += 1
        label = _sanitize_text(str(branch.get("label") or "分支"))
        branch_id = f"b{branch_index}"
        lines.append(f'    {branch_id}["{label}"]')
        child_index = 0
        for child in branch.get("children", []):
            child_index += 1
            child_text = _sanitize_text(str(child))
            child_id = f"{branch_id}_{child_index}"
            lines.append(f'      {child_id}["{child_text}"]')
    return "\n".join(lines)


def build_mindmap_url(mindmap_data: Dict[str, Any]) -> str:
    mermaid_text = build_mermaid_mindmap_text(mindmap_data)
    payload = base64.urlsafe_b64encode(mermaid_text.encode("utf-8")).decode("ascii").rstrip("=")
    # Mermaid Ink accepts base64-encoded mermaid text in path.
    return f"https://mermaid.ink/svg/{payload}"


def main() -> int:
    parser = argparse.ArgumentParser(description="Generate an online mindmap URL from article list.")
    parser.add_argument("--input", required=True, help="Input article list JSON (xhs_article_list.json).")
    parser.add_argument("--keyword", default="", help="Keyword label override.")
    parser.add_argument(
        "--url-output",
        default="",
        help="Optional output txt path for the generated mindmap URL.",
    )
    args = parser.parse_args()

    rows = load_rows(Path(args.input))
    if not rows:
        raise SystemExit("Input rows empty.")
    keyword = args.keyword or "高赞内容"

    rows_sorted = sorted(rows, key=lambda x: int(x.get("liked_count", 0)), reverse=True)

    mindmap_data = build_mindmap_data(rows_sorted, keyword)
    mindmap_url = build_mindmap_url(mindmap_data)

    if args.url_output.strip():
        out_path = Path(args.url_output).resolve()
        out_path.parent.mkdir(parents=True, exist_ok=True)
        out_path.write_text(mindmap_url + "\n", encoding="utf-8")
        print(f"Saved URL: {out_path}")

    print(f"Mindmap URL: {mindmap_url}")
    print("\nOutput guide:")
    print("- 直接把上面的 Mindmap URL 发给用户即可，无需再发导图文件。")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())


path: scripts/build_article_list_from_tikhub.py
content: #!/usr/bin/env python3
"""Build hot-note article list from TikHub search_notes JSON response.

Features:
1) Merge multi-page responses (from local files or direct API pagination)
2) Rank by custom hot score (likes/collects/comments/shares)
3) Generate publish-ready markdown templates
"""

from __future__ import annotations

import argparse
import csv
import json
from pathlib import Path
from typing import Any, Dict, List
from urllib.parse import urlencode
from urllib.request import Request, urlopen


SEARCH_NOTES_WEB_URL = "https://api.tikhub.io/api/v1/xiaohongshu/web/search_notes"


def extract_items(payload: Dict[str, Any]) -> List[Dict[str, Any]]:
    items = (
        payload.get("data", {})
        .get("data", {})
        .get("items", [])
    )
    if not isinstance(items, list):
        return []
    return [x for x in items if isinstance(x, dict)]


def fetch_pages_from_api(
    *,
    token: str,
    keyword: str,
    pages: int,
    sort: str,
    note_type: str,
    timeout: int,
) -> List[Dict[str, Any]]:
    out: List[Dict[str, Any]] = []
    for page in range(1, max(1, pages) + 1):
        params = {
            "keyword": keyword,
            "page": str(page),
            "sort": sort,
            "noteType": note_type,
        }
        url = f"{SEARCH_NOTES_WEB_URL}?{urlencode(params)}"
        req = Request(url, headers={"Authorization": f"Bearer {token}"}, method="GET")
        with urlopen(req, timeout=timeout) as resp:
            payload = json.loads(resp.read().decode("utf-8", errors="replace"))
        out.extend(extract_items(payload))
    return out


def build_rows(items: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
    rows: List[Dict[str, Any]] = []
    for item in items:
        note = item.get("note", {})
        if not isinstance(note, dict):
            continue
        user = note.get("user", {})
        if not isinstance(user, dict):
            user = {}
        note_id = str(note.get("id") or "").strip()
        xsec_token = str(note.get("xsec_token") or "").strip()
        url = ""
        if note_id:
            url = f"https://www.xiaohongshu.com/explore/{note_id}"
            if xsec_token:
                url += f"?xsec_token={xsec_token}&xsec_source=pc_search"
        rows.append(
            {
                "id": note_id,
                "title": str(note.get("title") or "").strip(),
                "author": str(user.get("nickname") or "").strip(),
                "liked_count": int(note.get("liked_count") or 0),
                "collected_count": int(note.get("collected_count") or 0),
                "comments_count": int(note.get("comments_count") or 0),
                "shared_count": int(note.get("shared_count") or 0),
                "desc": str(note.get("desc") or "").strip(),
                "url": url,
            }
        )
    return rows


def hot_score(row: Dict[str, Any], w_like: float, w_collect: float, w_comment: float, w_share: float) -> float:
    return (
        row["liked_count"] * w_like
        + row["collected_count"] * w_collect
        + row["comments_count"] * w_comment
        + row["shared_count"] * w_share
    )


def write_markdown(rows: List[Dict[str, Any]], out: Path, keyword: str, min_likes: int) -> None:
    lines = [
        f"# 小红书文章列表（点赞 > {min_likes}）",
        "",
        f"- 关键词: {keyword}",
        f"- 命中数量: {len(rows)}",
        "",
    ]
    if not rows:
        lines.append("暂无符合条件的文章。")
    else:
        for idx, r in enumerate(rows, start=1):
            lines.extend(
                [
                    f"{idx}. {r['title'] or 'Untitled'}",
                    f"- 作者: {r['author'] or 'N/A'}",
                    f"- 点赞: {r['liked_count']}",
                    f"- 收藏: {r['collected_count']}",
                    f"- 评论: {r['comments_count']}",
                    f"- 分享: {r['shared_count']}",
                    f"- 链接: {r['url'] or 'N/A'}",
                    "",
                ]
            )
    out.write_text("\n".join(lines).strip() + "\n", encoding="utf-8")


def write_publish_templates(rows: List[Dict[str, Any]], out: Path, keyword: str) -> None:
    lines = [
        f"# 可发布文章模板（关键词：{keyword}）",
        "",
    ]
    if not rows:
        lines.append("暂无可生成模板的数据。")
        out.write_text("\n".join(lines).strip() + "\n", encoding="utf-8")
        return
    for idx, r in enumerate(rows, start=1):
        title = r["title"] or "未命名笔记"
        desc = (r.get("desc") or "").replace("\n", " ").strip()
        if len(desc) > 120:
            desc = desc[:120].rstrip() + "..."
        if not desc:
            desc = "这篇内容聚焦实用经验与真实体验，适合做同主题选题参考。"
        lines.extend(
            [
                f"## 模板 {idx}",
                f"建议标题：{title}",
                f"导语：围绕“{keyword}”筛选到一篇高互动笔记，适合做选题延展。",
                f"内容摘要：{desc}",
                f"原笔记作者：{r.get('author') or '未知'}",
                f"数据参考：点赞 {r['liked_count']} / 收藏 {r['collected_count']} / 评论 {r['comments_count']} / 分享 {r['shared_count']}",
                f"原链接：{r.get('url') or 'N/A'}",
                "",
            ]
        )
    out.write_text("\n".join(lines).strip() + "\n", encoding="utf-8")


def write_csv(rows: List[Dict[str, Any]], out: Path) -> None:
    with out.open("w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(
            f,
            fieldnames=[
                "id",
                "title",
                "author",
                "liked_count",
                "collected_count",
                "comments_count",
                "shared_count",
                "hot_score",
                "url",
                "desc",
            ],
        )
        writer.writeheader()
        writer.writerows(rows)


def main() -> int:
    parser = argparse.ArgumentParser(description="Build filtered article list from TikHub JSON response.")
    parser.add_argument("--input", action="append", default=[], help="Path to TikHub response JSON file. Repeatable.")
    parser.add_argument("--input-dir", default="", help="Directory containing page JSON files.")
    parser.add_argument("--input-glob", default="*.json", help="Glob pattern when --input-dir is set.")
    parser.add_argument("--token", default="", help="TikHub token for direct API pagination mode.")
    parser.add_argument("--keyword", default="", help="Keyword for direct API pagination mode.")
    parser.add_argument("--pages", type=int, default=0, help="Fetch N pages directly from API when token+keyword provided.")
    parser.add_argument("--sort", default="general", help="Sort for API mode, default general.")
    parser.add_argument("--note-type", default="_0", help="noteType for API mode, default _0.")
    parser.add_argument("--timeout", type=int, default=30, help="Request timeout seconds for API mode.")
    parser.add_argument("--min-likes", type=int, default=1000, help="Like threshold. Default 1000.")
    parser.add_argument("--top", type=int, default=0, help="Keep top N after sorting desc. 0 means all.")
    parser.add_argument("--rank-by", choices=["likes", "hot"], default="likes", help="Ranking strategy.")
    parser.add_argument("--w-like", type=float, default=1.0, help="Hot-score weight for likes.")
    parser.add_argument("--w-collect", type=float, default=1.2, help="Hot-score weight for collects.")
    parser.add_argument("--w-comment", type=float, default=2.0, help="Hot-score weight for comments.")
    parser.add_argument("--w-share", type=float, default=1.5, help="Hot-score weight for shares.")
    parser.add_argument("--md-output", default="xhs_article_list.md", help="Markdown output path.")
    parser.add_argument("--csv-output", default="xhs_article_list.csv", help="CSV output path.")
    parser.add_argument("--json-output", default="xhs_article_list.json", help="Filtered JSON output path.")
    parser.add_argument(
        "--template-output",
        default="xhs_publish_templates.md",
        help="Publish-ready template markdown output path.",
    )
    args = parser.parse_args()

    items: List[Dict[str, Any]] = []
    keyword = args.keyword or ""

    if args.token and args.keyword and args.pages > 0:
        items.extend(
            fetch_pages_from_api(
                [REDACTED_SECRET],
                keyword=args.keyword,
                pages=args.pages,
                sort=args.sort,
                note_type=args.note_type,
                timeout=args.timeout,
            )
        )
    else:
        input_paths: List[Path] = []
        input_paths.extend([Path(p) for p in args.input if p.strip()])
        if args.input_dir:
            input_paths.extend(sorted(Path(args.input_dir).glob(args.input_glob)))
        if not input_paths:
            raise SystemExit("Provide --input/--input-dir or token+keyword+pages.")
        for p in input_paths:
            payload = json.loads(p.read_text(encoding="utf-8"))
            if not keyword:
                keyword = str(payload.get("params", {}).get("keyword") or "")
            items.extend(extract_items(payload))

    rows = build_rows(items)
    # de-dup by id
    dedup: Dict[str, Dict[str, Any]] = {}
    for r in rows:
        key = r["id"] or r["url"] or str(len(dedup))
        old = dedup.get(key)
        if old is None or r["liked_count"] > old["liked_count"]:
            dedup[key] = r
    rows = list(dedup.values())

    for r in rows:
        r["hot_score"] = round(
            hot_score(r, args.w_like, args.w_collect, args.w_comment, args.w_share),
            2,
        )
    rows = [r for r in rows if r["liked_count"] > args.min_likes]
    if args.rank_by == "likes":
        rows.sort(key=lambda x: x["liked_count"], reverse=True)
    else:
        rows.sort(key=lambda x: x["hot_score"], reverse=True)
    if args.top and args.top > 0:
        rows = rows[: args.top]

    md_path = Path(args.md_output).resolve()
    csv_path = Path(args.csv_output).resolve()
    json_path = Path(args.json_output).resolve()
    tpl_path = Path(args.template_output).resolve()

    write_markdown(rows, md_path, keyword=keyword, min_likes=args.min_likes)
    write_csv(rows, csv_path)
    json_path.write_text(json.dumps(rows, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
    write_publish_templates(rows, tpl_path, keyword=keyword)

    print(f"Saved MD:   {md_path}")
    print(f"Saved CSV:  {csv_path}")
    print(f"Saved JSON: {json_path}")
    print(f"Saved TPL:  {tpl_path}")
    print(f"Rows: {len(rows)}")
    print("\nOutput guide:")
    print(f"- {md_path.name}: 给用户快速浏览的文章清单（标题/作者/互动数据/链接）。")
    print(f"- {csv_path.name}: 适合 Excel 或 BI 二次筛选、排序、透视分析。")
    print(f"- {json_path.name}: 标准结构化结果，可用于后续脚本或系统接口。")
    print(f"- {tpl_path.name}: 基于结果自动生成的发布文案模板，可直接改写使用。")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())


path: scripts/interactive_filter_view.py
content: #!/usr/bin/env python3
"""Interactive filter & view tool for Xiaohongshu note JSON."""

from __future__ import annotations

import argparse
import csv
import json
from pathlib import Path
from typing import Any, Dict, List


def extract_items(payload: Dict[str, Any]) -> List[Dict[str, Any]]:
    items = payload.get("data", {}).get("data", {}).get("items", [])
    if not isinstance(items, list):
        return []
    return [x for x in items if isinstance(x, dict)]


def build_rows(items: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
    rows: List[Dict[str, Any]] = []
    for item in items:
        note = item.get("note", {})
        if not isinstance(note, dict):
            continue
        user = note.get("user", {}) if isinstance(note.get("user"), dict) else {}
        rows.append(
            {
                "id": str(note.get("id") or ""),
                "title": str(note.get("title") or "").strip(),
                "author": str(user.get("nickname") or "").strip(),
                "liked_count": int(note.get("liked_count") or 0),
                "collected_count": int(note.get("collected_count") or 0),
                "comments_count": int(note.get("comments_count") or 0),
                "shared_count": int(note.get("shared_count") or 0),
                "desc": str(note.get("desc") or "").strip(),
            }
        )
    return rows


def hot_score(r: Dict[str, Any]) -> float:
    return (
        r["liked_count"] * 1.0
        + r["collected_count"] * 1.2
        + r["comments_count"] * 2.0
        + r["shared_count"] * 1.5
    )


def ask(prompt: str, default: str) -> str:
    value = input(f"{prompt} [默认: {default}]：").strip()
    return value or default


def write_md(rows: List[Dict[str, Any]], out: Path, keyword: str, min_likes: int, rank_by: str) -> None:
    lines = [
        "# 小红书文章列表",
        "",
        f"- 关键词: {keyword}",
        f"- 筛选: 点赞 > {min_likes}",
        f"- 排序: {rank_by}",
        f"- 数量: {len(rows)}",
        "",
    ]
    for i, r in enumerate(rows, 1):
        lines.extend(
            [
                f"{i}. {r['title'] or 'Untitled'}",
                f"- 作者: {r['author'] or 'N/A'}",
                f"- 点赞: {r['liked_count']}",
                f"- 收藏: {r['collected_count']}",
                f"- 评论: {r['comments_count']}",
                f"- 分享: {r['shared_count']}",
                f"- 热度分: {r['hot_score']}",
                "",
            ]
        )
    out.write_text("\n".join(lines).strip() + "\n", encoding="utf-8")


def write_csv(rows: List[Dict[str, Any]], out: Path) -> None:
    with out.open("w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(
            f,
            fieldnames=[
                "id",
                "title",
                "author",
                "liked_count",
                "collected_count",
                "comments_count",
                "shared_count",
                "hot_score",
                "desc",
            ],
        )
        writer.writeheader()
        writer.writerows(rows)


def main() -> int:
    parser = argparse.ArgumentParser(description="Interactive filter and view for TikHub JSON.")
    parser.add_argument("--input", required=True, help="Path to JSON response file.")
    parser.add_argument("--non-interactive", action="store_true", help="Use default scheme directly.")
    parser.add_argument("--min-likes", type=int, default=1000)
    parser.add_argument("--rank-by", choices=["likes", "hot"], default="hot")
    parser.add_argument("--top", type=int, default=20)
    parser.add_argument("--view", choices=["summary", "full", "export"], default="summary")
    parser.add_argument("--md-output", default="xhs_article_list.md")
    parser.add_argument("--csv-output", default="xhs_article_list.csv")
    args = parser.parse_args()

    payload = json.loads(Path(args.input).read_text(encoding="utf-8"))
    keyword = str(payload.get("params", {}).get("keyword") or "")
    rows = build_rows(extract_items(payload))
    for r in rows:
        r["hot_score"] = round(hot_score(r), 2)

    min_likes = args.min_likes
    rank_by = args.rank_by
    top = args.top
    view_mode = args.view

    if not args.non_interactive:
        min_likes = int(ask("点赞阈值（仅保留大于此值）", str(min_likes)))
        rank_by = ask("排序方式（likes/hot）", rank_by)
        top = int(ask("最多显示条数", str(top)))
        view_mode = ask("查看方式（summary/full/export）", view_mode)

    rows = [r for r in rows if r["liked_count"] > min_likes]
    if rank_by == "likes":
        rows.sort(key=lambda x: x["liked_count"], reverse=True)
    else:
        rows.sort(key=lambda x: x["hot_score"], reverse=True)
    if top > 0:
        rows = rows[:top]

    # default scheme: summary + export files
    if view_mode in ("summary", "full", "export"):
        print(f"\n关键词: {keyword} | 条数: {len(rows)} | 筛选: 点赞>{min_likes} | 排序:{rank_by}\n")
        for i, r in enumerate(rows, 1):
            print(f"{i}. {r['title']} | 点赞:{r['liked_count']} | 作者:{r['author']}")
            if view_mode == "full":
                print(f"   收藏:{r['collected_count']} 评论:{r['comments_count']} 分享:{r['shared_count']} 热度:{r['hot_score']}")
                if r["desc"]:
                    print(f"   摘要:{r['desc'][:120]}")
    write_md(rows, Path(args.md_output), keyword=keyword, min_likes=min_likes, rank_by=rank_by)
    write_csv(rows, Path(args.csv_output))
    print(f"\n已导出: {Path(args.md_output).resolve()}")
    print(f"已导出: {Path(args.csv_output).resolve()}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())


path: scripts/package-lock.json
content: {
  "name": "scripts",
  "version": "1.0.0",
  "lockfileVersion": 3,
  "requires": true,
  "packages": {
    "": {
      "name": "scripts",
      "version": "1.0.0",
      "license": "ISC",
      "dependencies": {
        "playwright": "^1.59.1"
      }
    },
    "node_modules/fsevents": {
      "version": "2.3.2",
      "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
      "integrity": [REDACTED_SECRET],
      "hasInstallScript": true,
      "license": "MIT",
      "optional": true,
      "os": [
        "darwin"
      ],
      "engines": {
        "node": "^8.16.0 || ^10.6.0 || >=11.0.0"
      }
    },
    "node_modules/playwright": {
      "version": "1.59.1",
      "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.59.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "Apache-2.0",
      "dependencies": {
        "playwright-core": "1.59.1"
      },
      "bin": {
        "playwright": "cli.js"
      },
      "engines": {
        "node": ">=18"
      },
      "optionalDependencies": {
        "fsevents": "2.3.2"
      }
    },
    "node_modules/playwright-core": {
      "version": "1.59.1",
      "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.59.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "Apache-2.0",
      "bin": {
        "playwright-core": "cli.js"
      },
      "engines": {
        "node": ">=18"
      }
    }
  }
}


path: scripts/package.json
content: {
  "name": "scripts",
  "version": "1.0.0",
  "description": "",
  "main": "fetch_xiaohongshu_note_playwright.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "type": "commonjs",
  "dependencies": {
    "playwright": "^1.59.1"
  }
}


path: scripts/search_notes_generic.py
content: #!/usr/bin/env python3
"""Generic Xiaohongshu note-search API caller."""

from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path
from typing import Dict, List, Tuple
from urllib.parse import urlencode
from urllib.request import Request, urlopen


def parse_kv(items: List[str]) -> List[Tuple[str, str]]:
    out: List[Tuple[str, str]] = []
    for item in items:
        if "=" not in item:
            raise ValueError(f"Invalid --param format: {item}. Use key=value")
        k, v = item.split("=", 1)
        k = k.strip()
        if not k:
            raise ValueError(f"Invalid key in --param: {item}")
        out.append((k, v))
    return out


def build_headers(args: argparse.Namespace) -> Dict[str, str]:
    headers: Dict[str, str] = {"Accept": "application/json"}
    if args.auth_mode == "bearer":
        headers[args.auth_header] = f"Bearer {args.token}"
    elif args.auth_mode == "apikey":
        headers[args.auth_header] = args.token
    for k, v in parse_kv(args.header):
        headers[k] = v
    return headers


def main() -> int:
    parser = argparse.ArgumentParser(description="Call a custom Xiaohongshu notes API endpoint.")
    parser.add_argument("--base-url", required=True, help="API endpoint URL, e.g. https://example.com/search")
    parser.add_argument("--token", default="", help="API token/key")
    parser.add_argument("--token-file", default="", help="Read token/key from file")
    parser.add_argument("--auth-mode", choices=["none", "bearer", "apikey"], default="bearer")
    parser.add_argument("--auth-header", default="Authorization", help="Header name for auth token")
    parser.add_argument("--keyword", required=True, help="Search keyword")
    parser.add_argument("--page", type=int, default=1, help="Page number, start from 1")
    parser.add_argument("--keyword-param", default="keyword", help="Keyword query parameter name")
    parser.add_argument("--page-param", default="page", help="Page query parameter name")
    parser.add_argument("--param", action="append", default=[], help="Extra query params in key=value format")
    parser.add_argument("--header", action="append", default=[], help="Extra headers in key=value format")
    parser.add_argument("--timeout", type=int, default=30, help="Request timeout in seconds")
    parser.add_argument("--output", default="", help="Optional path to save response JSON/text")
    args = parser.parse_args()

    if args.page < 1:
        parser.error("--page must be >= 1")

    [REDACTED_SECRET]
    if args.token_file:
        [REDACTED_SECRET]).read_text(encoding="utf-8").strip()
    if args.auth_mode != "none" and not [REDACTED_SECRET]"Provide --token or --token-file when auth-mode is bearer/apikey")
    args.token = token

    query_items: List[Tuple[str, str]] = [
        (args.keyword_param, args.keyword),
        (args.page_param, str(args.page)),
    ]
    query_items.extend(parse_kv(args.param))
    query_string = urlencode(query_items)

    sep = "&" if "?" in args.base_url else "?"
    url = f"{args.base_url}{sep}{query_string}"
    req = Request(url=url, headers=build_headers(args), method="GET")

    with urlopen(req, timeout=args.timeout) as resp:
        raw = resp.read().decode("utf-8", errors="replace")

    if args.output:
        Path(args.output).write_text(raw, encoding="utf-8")

    try:
        payload = json.loads(raw)
        print(json.dumps(payload, ensure_ascii=False, indent=2))
    except json.JSONDecodeError:
        print(raw)
    return 0


if __name__ == "__main__":
    sys.exit(main())


path: scripts/search_notes_tikhub.py
content: #!/usr/bin/env python3
"""Call TikHub Xiaohongshu search_notes endpoint safely."""

from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path
from typing import Dict, Optional
from urllib.parse import urlencode
from urllib.request import Request, urlopen


APP_V2_API_URL = "https://api.tikhub.io/api/v1/xiaohongshu/app_v2/search_notes"
WEB_API_URL = "https://api.tikhub.io/api/v1/xiaohongshu/web/search_notes"

SORT_TYPES = {
    "general",
    "time_descending",
    "popularity_descending",
    "comment_descending",
    "collect_descending",
    "english_preferred",
}

NOTE_TYPE_MAP = {
    "all": "不限",
    "video": "视频笔记",
    "image": "普通笔记",
    "live": "直播笔记",
    "不限": "不限",
    "视频笔记": "视频笔记",
    "普通笔记": "普通笔记",
    "直播笔记": "直播笔记",
}

TIME_FILTER_MAP = {
    "all": "不限",
    "day": "一天内",
    "week": "一周内",
    "half_year": "半年内",
    "不限": "不限",
    "一天内": "一天内",
    "一周内": "一周内",
    "半年内": "半年内",
}


def normalize_note_type(value: Optional[str]) -> Optional[str]:
    if value is None:
        return None
    key = value.strip()
    if not key:
        return None
    return NOTE_TYPE_MAP.get(key, key)


def normalize_time_filter(value: Optional[str]) -> Optional[str]:
    if value is None:
        return None
    key = value.strip()
    if not key:
        return None
    return TIME_FILTER_MAP.get(key, key)


def build_params(args: argparse.Namespace) -> Dict[str, str]:
    params: Dict[str, str] = {"keyword": args.keyword, "page": str(args.page)}

    # For first page requests, keep params minimal by default.
    first_page = args.page == 1 and not args.search_id and not args.search_session_id
    if first_page and not args.force_all_params:
        if args.sort_type and args.sort_type != "general":
            params["sort_type"] = args.sort_type
        if args.note_type:
            note_type = normalize_note_type(args.note_type)
            if note_type and note_type != "不限":
                params["note_type"] = note_type
        if args.time_filter:
            time_filter = normalize_time_filter(args.time_filter)
            if time_filter and time_filter != "不限":
                params["time_filter"] = time_filter
        if args.source and args.source != "explore_feed":
            params["source"] = args.source
        if args.ai_mode is not None and args.ai_mode != 0:
            params["ai_mode"] = str(args.ai_mode)
        return params

    # Pagination or force-all mode.
    if args.sort_type:
        params["sort_type"] = args.sort_type
    if args.note_type:
        params["note_type"] = normalize_note_type(args.note_type) or "不限"
    if args.time_filter:
        params["time_filter"] = normalize_time_filter(args.time_filter) or "不限"
    if args.search_id:
        params["search_id"] = args.search_id
    if args.search_session_id:
        params["search_session_id"] = args.search_session_id
    if args.source:
        params["source"] = args.source
    if args.ai_mode is not None:
        params["ai_mode"] = str(args.ai_mode)
    return params


def main() -> int:
    parser = argparse.ArgumentParser(description="Search Xiaohongshu notes via TikHub API.")
    parser.add_argument("--token", help="TikHub bearer token. Can be omitted when --token-file is provided.")
    parser.add_argument("--token-file", help="File containing TikHub bearer token.")
    parser.add_argument("--keyword", required=True, help="Search keyword.")
    parser.add_argument("--page", type=int, default=1, help="Page number, start from 1.")
    parser.add_argument("--sort-type", default="general", choices=sorted(SORT_TYPES))
    parser.add_argument("--note-type", default="不限", help="all/video/image/live or Chinese enum.")
    parser.add_argument("--time-filter", default="不限", help="all/day/week/half_year or Chinese enum.")
    parser.add_argument("--search-id", default="", help="search_id from first response for pagination.")
    parser.add_argument("--search-session-id", default="", help="search_session_id from first response.")
    parser.add_argument("--source", default="explore_feed")
    parser.add_argument("--ai-mode", type=int, choices=[0, 1], default=0)
    parser.add_argument("--endpoint", choices=["app_v2", "web"], default="app_v2", help="TikHub endpoint family.")
    parser.add_argument(
        "--auth-mode",
        choices=["bearer", "apikey"],
        default="bearer",
        help="Use Authorization: Bearer <token> or custom API key header.",
    )
    parser.add_argument(
        "--auth-header",
        default="X-API-KEY",
        help="Header name used when --auth-mode apikey, default X-API-KEY.",
    )
    parser.add_argument("--timeout", type=int, default=30, help="HTTP timeout in seconds.")
    parser.add_argument("--force-all-params", action="store_true", help="Always send all query params.")
    parser.add_argument("--output", default="", help="Optional path to save full JSON response.")
    args = parser.parse_args()

    if args.page < 1:
        parser.error("--page must be >= 1")

    [REDACTED_SECRET]
    if args.token_file:
        [REDACTED_SECRET]).read_text(encoding="utf-8").strip()
    if not [REDACTED_SECRET]"Provide --token or --token-file")

    params = build_params(args)
    base_url = APP_V2_API_URL if args.endpoint == "app_v2" else WEB_API_URL
    # web endpoint expects sort/noteType; app_v2 expects sort_type/note_type.
    if args.endpoint == "web":
        if "sort_type" in params:
            params["sort"] = params.pop("sort_type")
        if "note_type" in params:
            params["noteType"] = params.pop("note_type")

    url = f"{base_url}?{urlencode(params)}"
    if args.auth_mode == "bearer":
        headers = {"Authorization": f"Bearer {token}"}
    else:
        headers = {args.auth_header: token}
    req = Request(url, headers=headers, method="GET")

    with urlopen(req, timeout=args.timeout) as resp:
        payload = resp.read().decode("utf-8", errors="replace")

    try:
        data = json.loads(payload)
    except json.JSONDecodeError:
        print(payload)
        return 0

    if args.output:
        Path(args.output).write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")

    print(json.dumps(data, ensure_ascii=False, indent=2))
    return 0


if __name__ == "__main__":
    sys.exit(main())


path: scripts/fetch_xiaohongshu_notes.py
content: #!/usr/bin/env python3
"""Fetch Xiaohongshu note pages and extract basic structured fields."""

from __future__ import annotations

import argparse
import json
import re
import sys
from dataclasses import dataclass, asdict
from datetime import datetime, timezone
from html import unescape
from pathlib import Path
from typing import Dict, List, Optional
from urllib import error, request


UA = (
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
    "AppleWebKit/537.36 (KHTML, like Gecko) "
    "Chrome/124.0.0.0 Safari/537.36"
)


@dataclass
class NoteRecord:
    url: str
    note_id: Optional[str]
    title: Optional[str]
    description: Optional[str]
    author: Optional[str]
    published_at: Optional[str]
    images: List[str]
    tags: List[str]
    interaction: Dict[str, Optional[int]]
    fetched_at_utc: str
    source_status: str


def read_urls(one_url: Optional[str], url_file: Optional[str]) -> List[str]:
    urls: List[str] = []
    if one_url:
        urls.append(one_url.strip())
    if url_file:
        for line in Path(url_file).read_text(encoding="utf-8").splitlines():
            line = line.strip()
            if not line or line.startswith("#"):
                continue
            urls.append(line)
    deduped = []
    seen = set()
    for u in urls:
        if u and u not in seen:
            deduped.append(u)
            seen.add(u)
    return deduped


def fetch_html(url: str, timeout: int, cookie: Optional[str]) -> str:
    headers = {"User-Agent": UA, "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8"}
    if cookie:
        headers["Cookie"] = cookie
    req = request.Request(url=url, headers=headers)
    with request.urlopen(req, timeout=timeout) as resp:
        return resp.read().decode("utf-8", errors="ignore")


def _extract_meta(html: str, prop: str) -> Optional[str]:
    pattern = (
        r'<meta[^>]+(?:property|name)=["\']'
        + re.escape(prop)
        + r'["\'][^>]+content=["\']([^"\']+)["\']'
    )
    m = re.search(pattern, html, flags=re.IGNORECASE)
    return unescape(m.group(1).strip()) if m else None


def _extract_note_id(url: str, html: str) -> Optional[str]:
    m = re.search(r"/explore/([0-9a-zA-Z]+)", url)
    if m:
        return m.group(1)
    m = re.search(r'"noteId"\s*:\s*"([^"]+)"', html)
    if m:
        return m.group(1)
    return None


def _extract_json_ld(html: str) -> Dict[str, object]:
    out: Dict[str, object] = {}
    blocks = re.findall(
        r'<script[^>]+type=["\']application/ld\+json["\'][^>]*>(.*?)</script>',
        html,
        flags=re.IGNORECASE | re.DOTALL,
    )
    for raw in blocks:
        raw = raw.strip()
        if not raw:
            continue
        try:
            payload = json.loads(raw)
        except json.JSONDecodeError:
            continue
        if isinstance(payload, list):
            for item in payload:
                if isinstance(item, dict):
                    out.update(item)
        elif isinstance(payload, dict):
            out.update(payload)
    return out


def _parse_iso(value: Optional[str]) -> Optional[str]:
    if not value:
        return None
    try:
        dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
    except ValueError:
        return value
    return dt.astimezone(timezone.utc).isoformat()


def _extract_interaction_from_ld(ld: Dict[str, object]) -> Dict[str, Optional[int]]:
    metrics = {"like_count": None, "comment_count": None, "share_count": None, "collect_count": None}
    stats = ld.get("interactionStatistic")
    if not isinstance(stats, list):
        return metrics
    for item in stats:
        if not isinstance(item, dict):
            continue
        raw_type = str(item.get("interactionType", "")).lower()
        count = item.get("userInteractionCount")
        if not isinstance(count, int):
            try:
                count = int(str(count))
            except (TypeError, ValueError):
                continue
        if "like" in raw_type:
            metrics["like_count"] = count
        elif "comment" in raw_type:
            metrics["comment_count"] = count
        elif "share" in raw_type:
            metrics["share_count"] = count
    return metrics


def parse_note(url: str, html: str) -> NoteRecord:
    ld = _extract_json_ld(html)
    title = _extract_meta(html, "og:title") or ld.get("headline")
    if isinstance(title, str):
        title = title.strip()
    else:
        title = None

    desc = _extract_meta(html, "og:description") or ld.get("description")
    if isinstance(desc, str):
        desc = desc.strip()
    else:
        desc = None

    author = None
    author_obj = ld.get("author")
    if isinstance(author_obj, dict):
        author_name = author_obj.get("name")
        if isinstance(author_name, str):
            author = author_name.strip()

    images: List[str] = []
    first_image = _extract_meta(html, "og:image")
    if first_image:
        images.append(first_image)
    ld_image = ld.get("image")
    if isinstance(ld_image, str):
        images.append(ld_image)
    elif isinstance(ld_image, list):
        images.extend([x for x in ld_image if isinstance(x, str)])
    images = list(dict.fromkeys([x.strip() for x in images if x and x.strip()]))

    tags: List[str] = []
    keywords = ld.get("keywords")
    if isinstance(keywords, str):
        tags = [x.strip() for x in re.split(r"[,，]", keywords) if x.strip()]

    note = NoteRecord(
        url=url,
        note_id=_extract_note_id(url, html),
        title=title,
        description=desc,
        author=author,
        published_at=_parse_iso(ld.get("datePublished") if isinstance(ld, dict) else None),
        images=images,
        tags=tags,
        interaction=_extract_interaction_from_ld(ld),
        fetched_at_utc=datetime.now(timezone.utc).isoformat(),
        source_status="ok",
    )
    return note


def to_markdown(note: NoteRecord) -> str:
    lines = [
        f"# {note.title or 'Untitled'}",
        "",
        f"- URL: {note.url}",
        f"- Note ID: {note.note_id or 'N/A'}",
        f"- Author: {note.author or 'N/A'}",
        f"- Published: {note.published_at or 'N/A'}",
        f"- Like: {note.interaction.get('like_count')}",
        f"- Comment: {note.interaction.get('comment_count')}",
        f"- Share: {note.interaction.get('share_count')}",
        f"- Collect: {note.interaction.get('collect_count')}",
        "",
        "## Description",
        "",
        note.description or "",
        "",
        "## Tags",
        "",
        ", ".join(note.tags) if note.tags else "N/A",
        "",
        "## Images",
        "",
    ]
    if note.images:
        lines.extend([f"- {img}" for img in note.images])
    else:
        lines.append("- N/A")
    return "\n".join(lines).strip() + "\n"


def main() -> int:
    parser = argparse.ArgumentParser(description="Fetch and parse Xiaohongshu notes.")
    parser.add_argument("--url", help="Single Xiaohongshu note URL.")
    parser.add_argument("--url-file", help="File containing note URLs, one per line.")
    parser.add_argument("--cookie", help="Raw cookie header value.")
    parser.add_argument("--cookie-file", help="Read cookie header value from file.")
    parser.add_argument("--format", choices=["json", "md", "both"], default="json")
    parser.add_argument("--output", default="xhs_notes.json", help="Output path for JSON.")
    parser.add_argument("--timeout", type=int, default=20, help="HTTP timeout in seconds.")
    args = parser.parse_args()

    if not args.url and not args.url_file:
        parser.error("Provide at least one of --url or --url-file.")

    cookie = args.cookie
    if args.cookie_file:
        cookie = Path(args.cookie_file).read_text(encoding="utf-8").strip()

    urls = read_urls(args.url, args.url_file)
    if not urls:
        parser.error("No valid URLs found.")

    records: List[NoteRecord] = []
    for url in urls:
        try:
            html = fetch_html(url, timeout=args.timeout, cookie=cookie)
            note = parse_note(url, html)
        except error.HTTPError as e:
            note = NoteRecord(
                url=url,
                note_id=None,
                title=None,
                description=f"HTTPError: {e.code}",
                author=None,
                published_at=None,
                images=[],
                tags=[],
                interaction={"like_count": None, "comment_count": None, "share_count": None, "collect_count": None},
                fetched_at_utc=datetime.now(timezone.utc).isoformat(),
                source_status=f"http_error_{e.code}",
            )
        except Exception as e:  # noqa: BLE001
            note = NoteRecord(
                url=url,
                note_id=None,
                title=None,
                description=f"Error: {e}",
                author=None,
                published_at=None,
                images=[],
                tags=[],
                interaction={"like_count": None, "comment_count": None, "share_count": None, "collect_count": None},
                fetched_at_utc=datetime.now(timezone.utc).isoformat(),
                source_status="error",
            )
        records.append(note)

    output_path = Path(args.output)
    data = [asdict(r) for r in records]

    if args.format in ("json", "both"):
        output_path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")

    if args.format in ("md", "both"):
        if len(records) == 1:
            md_path = output_path.with_suffix(".md")
            md_path.write_text(to_markdown(records[0]), encoding="utf-8")
        else:
            folder = output_path.parent / (output_path.stem + "_md")
            folder.mkdir(parents=True, exist_ok=True)
            for idx, note in enumerate(records, start=1):
                suffix = note.note_id or f"note_{idx}"
                safe = re.sub(r"[^0-9A-Za-z_-]+", "_", suffix)
                (folder / f"{safe}.md").write_text(to_markdown(note), encoding="utf-8")

    print(f"Processed {len(records)} URL(s).")
    print(f"Output base: {output_path}")
    return 0


if __name__ == "__main__":
    sys.exit(main())


path: scripts/list_hot_notes_playwright.js
content: #!/usr/bin/env node
"use strict";

const fs = require("node:fs");
const path = require("node:path");
const { chromium } = require("playwright");

function parseArgs(argv) {
  const args = {
    keyword: "",
    cookieFile: "",
    maxItems: 50,
    minLikes: 1000,
    output: "xhs_hot_notes.json",
    mdOutput: "xhs_hot_notes.md",
    timeout: 45000,
    scrollRounds: 10,
    waitMs: 1800,
    headed: false,
  };
  for (let i = 2; i < argv.length; i += 1) {
    const k = argv[i];
    const v = argv[i + 1];
    if (k === "--keyword") args.keyword = v || "";
    else if (k === "--cookie-file") args.cookieFile = v || "";
    else if (k === "--max-items") args.maxItems = Number(v || args.maxItems);
    else if (k === "--min-likes") args.minLikes = Number(v || args.minLikes);
    else if (k === "--output") args.output = v || args.output;
    else if (k === "--md-output") args.mdOutput = v || args.mdOutput;
    else if (k === "--timeout") args.timeout = Number(v || args.timeout);
    else if (k === "--scroll-rounds") args.scrollRounds = Number(v || args.scrollRounds);
    else if (k === "--wait-ms") args.waitMs = Number(v || args.waitMs);
    else if (k === "--headed") args.headed = true;
    else if (k === "--help" || k === "-h") {
      console.log(`Usage:
  node list_hot_notes_playwright.js --keyword <关键词> --cookie-file <cookie.txt> [options]

Options:
  --max-items <n>       Max notes to collect (default: 50)
  --min-likes <n>       Filter threshold (default: 1000)
  --output <file>       JSON output path
  --md-output <file>    Markdown output path
  --scroll-rounds <n>   Scroll rounds (default: 10)
  --wait-ms <ms>        Wait between rounds (default: 1800)
  --headed              Run non-headless for debug
`);
      process.exit(0);
    }
  }
  return args;
}

function parseCookieHeader(raw) {
  const out = [];
  const text = (raw || "").trim();
  if (!text) return out;
  for (const part of text.split(";")) {
    const p = part.trim();
    if (!p) continue;
    const idx = p.indexOf("=");
    if (idx <= 0) continue;
    const name = p.slice(0, idx).trim();
    const value = p.slice(idx + 1).trim();
    if (!name) continue;
    out.push({ name, value, domain: ".xiaohongshu.com", path: "/" });
  }
  return out;
}

function parseLikes(raw) {
  const s = String(raw || "").trim().toLowerCase();
  if (!s) return null;
  const clean = s.replace(/,/g, "");
  const wan = clean.match(/^(\d+(?:\.\d+)?)\s*万$/);
  if (wan) return Math.round(Number(wan[1]) * 10000);
  const qian = clean.match(/^(\d+(?:\.\d+)?)\s*k$/);
  if (qian) return Math.round(Number(qian[1]) * 1000);
  const num = clean.match(/^\d+(?:\.\d+)?$/);
  if (num) return Math.round(Number(clean));
  return null;
}

function toMarkdown(data) {
  const lines = [];
  lines.push(`# 小红书高赞文章列表（点赞 > ${data.min_likes}）`);
  lines.push("");
  lines.push(`- 关键词: ${data.keyword}`);
  lines.push(`- 抓取时间: ${data.fetched_at_utc}`);
  lines.push(`- 总抓取: ${data.total_collected}`);
  lines.push(`- 高赞数量: ${data.hot_notes.length}`);
  lines.push("");
  if (data.hot_notes.length === 0) {
    lines.push("暂无符合条件的笔记。");
    return `${lines.join("\n")}\n`;
  }
  for (let i = 0; i < data.hot_notes.length; i += 1) {
    const n = data.hot_notes[i];
    lines.push(`${i + 1}. ${n.title || "Untitled"}`);
    lines.push(`   - 点赞: ${n.likes ?? "N/A"}`);
    lines.push(`   - 作者: ${n.author || "N/A"}`);
    lines.push(`   - 链接: ${n.url}`);
  }
  return `${lines.join("\n")}\n`;
}

async function main() {
  const args = parseArgs(process.argv);
  if (!args.keyword) throw new Error("missing --keyword");
  if (!args.cookieFile) throw new Error("missing --cookie-file");

  const browser = await chromium.launch({ headless: !args.headed });
  const context = await browser.newContext({
    userAgent:
      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 " +
      "(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
    locale: "zh-CN",
    viewport: { width: 1400, height: 2200 },
  });

  try {
    const cookieText = fs.readFileSync(args.cookieFile, "utf-8");
    const cookies = parseCookieHeader(cookieText);
    if (cookies.length > 0) await context.addCookies(cookies);

    const page = await context.newPage();
    const url = `https://www.xiaohongshu.com/search_result?keyword=${encodeURIComponent(args.keyword)}&source=web_explore_feed`;
    await page.goto(url, { waitUntil: "domcontentloaded", timeout: args.timeout });
    await page.waitForTimeout(args.waitMs);

    const loginState = await page.evaluate(() => {
      try {
        const st = window.__INITIAL_STATE__;
        if (st && st.user && typeof st.user.loggedIn === "boolean") {
          return { known: true, loggedIn: st.user.loggedIn };
        }
      } catch (_) {}
      return { known: false, loggedIn: null };
    });
    if (loginState.known && !loginState.loggedIn) {
      throw new Error("cookie_not_logged_in: refresh cookie from logged-in browser request");
    }

    // Try closing login overlays.
    await page.evaluate(() => {
      const sels = ['button[aria-label*="关闭"]', 'button[class*="close"]', '.close'];
      for (const s of sels) {
        for (const el of document.querySelectorAll(s)) {
          try {
            el.click();
          } catch (_) {}
        }
      }
    });

    for (let i = 0; i < args.scrollRounds; i += 1) {
      await page.mouse.wheel(0, 2600);
      await page.waitForTimeout(args.waitMs);
    }

    const cards = await page.evaluate(() => {
      const toAbs = (href) => {
        if (!href) return null;
        if (href.startsWith("http")) return href;
        if (href.startsWith("/")) return `https://www.xiaohongshu.com${href}`;
        return null;
      };
      const out = [];
      const anchors = Array.from(document.querySelectorAll('a[href*="/explore/"], a[href*="/discovery/item/"]'));
      for (const a of anchors) {
        const href = toAbs(a.getAttribute("href"));
        if (!href) continue;
        const card = a.closest("section, article, div") || a.parentElement;
        const text = (card && card.innerText) || a.innerText || "";
        const lines = text.split("\n").map((x) => x.trim()).filter(Boolean);
        const title = lines[0] || "";
        let author = "";
        let likesText = "";
        for (const line of lines) {
          if (!likesText && /^(\d+(\.\d+)?万?|\d+(\.\d+)?k)$/i.test(line)) likesText = line;
          if (!author && /@|作者|发布/.test(line) === false && line.length <= 24 && line !== title) author = line;
        }
        out.push({ url: href, title, author, likes_text: likesText });
      }
      return out;
    });

    const dedup = [];
    const seen = new Set();
    for (const c of cards) {
      if (!c.url || seen.has(c.url)) continue;
      seen.add(c.url);
      dedup.push(c);
      if (dedup.length >= args.maxItems) break;
    }

    const notes = dedup.map((n) => ({
      ...n,
      likes: parseLikes(n.likes_text),
    }));
    const hot = notes.filter((n) => (n.likes || 0) > args.minLikes);

    const result = {
      ok: true,
      keyword: args.keyword,
      min_likes: args.minLikes,
      total_collected: notes.length,
      fetched_at_utc: new Date().toISOString(),
      notes,
      hot_notes: hot,
    };

    fs.writeFileSync(args.output, `${JSON.stringify(result, null, 2)}\n`, "utf-8");
    fs.writeFileSync(args.mdOutput, toMarkdown(result), "utf-8");
    console.log(`Saved JSON: ${path.resolve(args.output)}`);
    console.log(`Saved MD:   ${path.resolve(args.mdOutput)}`);
    console.log(`Collected: ${notes.length}, Hot(>${args.minLikes}): ${hot.length}`);
  } finally {
    await context.close();
    await browser.close();
  }
}

main().catch((e) => {
  console.error(String(e && e.message ? e.message : e));
  process.exit(1);
});


path: agents/openai.yaml
content: interface:
  display_name: "获取小红书笔记"
  short_description: "根据小红书笔记链接提取结构化内容"
  default_prompt: "请根据我提供的小红书笔记链接，调用脚本提取并输出结构化信息（JSON或Markdown）。"


path: references/output-schema.md
content: # Output Schema

The script outputs a JSON array. Each item follows this schema:

```json
{
  "url": "https://www.xiaohongshu.com/explore/<note_id>",
  "note_id": "string | null",
  "title": "string | null",
  "description": "string | null",
  "author": "string | null",
  "published_at": "ISO-8601 UTC string | null",
  "images": ["string"],
  "tags": ["string"],
  "interaction": {
    "like_count": "number | null",
    "comment_count": "number | null",
    "share_count": "number | null",
    "collect_count": "number | null"
  },
  "fetched_at_utc": "ISO-8601 UTC string",
  "source_status": "ok | http_error_<code> | error"
}
```

Notes:
- Field availability depends on page structure and whether valid cookies are provided.
- `collect_count` is currently a placeholder and may stay `null` unless upstream data includes it.


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)