Skill: justao/url-reader (version 0.1.1)

--- name: url-reader description: 智能读取任意URL内容，支持微信公众号、小红书、今日头条、抖音、淘宝、天猫、京东、百度等中国主流平台，自动识别平台类型并提取核心内容。自动保存内容为Markdown，下载图片到本地。 --- # URL Reader - 智能网页内容读取器 一键读取任意URL的内容，自动识别平台类型，智能选择最佳读取策略，**自动保存内容和图片到本地**。 ## 默认保存目录 ``` /Users/ys/laoyang知识库/nickys/素材/ ``` 保存格式： ``` 素材/ └── 2026-01-30_文章标题/ ├── content.md # Markdown内容 ├── img_01.webp # 图片1 ├── img_02.webp # 图片2 └── ... ``` ## 核心技术方案 ### 三层读取策略（自动降级） ``` ┌─────────────────────────────────────────────────────────────────┐ │ URL Reader 技术架构 │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ 用户输入 URL │ │ ↓ │ │ ┌─────────────┐ │ │ │ 平台识别器 │ → 识别URL所属平台（微信/小红书/淘宝等） │ │ └─────────────┘ │ │ ↓ │ │ ┌─────────────────────────────────────────────────────────────┐│ │ │ 策略选择器 ││ │ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ ││ │ │ │ 策略1 │ │ 策略2 │ │ 策略3 │ ││ │ │ │ Firecrawl │→│ Jina │→│ Playwright │ ││ │ │ │ (首选) │ │ (备选) │ │ (兜底) │ ││ │ │ └───────────┘ └───────────┘ └───────────┘ ││ │ └─────────────────────────────────────────────────────────────┘│ │ ↓ │ │ ┌─────────────┐ │ │ │ 内容提取器 │ → 提取标题、正文、作者、时间等 │ │ └─────────────┘ │ │ ↓ │ │ ┌─────────────┐ │ │ │ 格式化输出 │ → Markdown 格式 │ │ └─────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ### 策略1：Firecrawl API（首选） **特点**： - AI 驱动的网页抓取 - 自动处理 JavaScript 渲染 - 自动绕过反爬机制 - 直接返回干净的 Markdown - 支持 96% 的网站 **API 调用**： ```python from firecrawl import Firecrawl app = Firecrawl([REDACTED_SECRET]") result = app.scrape(url, formats=["markdown"]) ``` **定价**： - 免费：500 页/月 - 付费：按量计费 ### 策略2：Jina Reader API（备选） **特点**： - 完全免费 - 无需 API Key - 支持动态渲染 - 返回 Markdown 格式 **使用方式**： ``` https://r.jina.ai/{原始URL} ``` ### 策略3：Playwright 浏览器自动化（兜底） **特点**： - 支持登录态保持 - 可处理任何网站 - 需要首次手动登录 **适用场景**： - 微信公众号（需要登录） - 需要登录的平台 - 前两种策略都失败时 ## 使用方式 ### 方式1：直接对话 ``` 用户：帮我读取这个链接 https://mp.weixin.qq.com/s/xxxxx 用户：看看这个小红书 https://www.xiaohongshu.com/explore/xxxxx 用户：读一下这个网页 https://example.com/article ``` ### 方式2：命令行调用 ```bash /url-reader https://example.com/article ``` ## 支持的平台 | 平台 | 域名 | 推荐策略 | 备注 | |------|------|----------|------| | 微信公众号 | mp.weixin.qq.com | Firecrawl → Playwright | 可能需要登录 | | 小红书 | xiaohongshu.com | Firecrawl → Jina | 短链接需解析 | | 今日头条 | toutiao.com | Firecrawl → Jina | - | | 抖音 | douyin.com | Firecrawl | 提取视频描述 | | 淘宝 | taobao.com | Firecrawl → Playwright | 可能需要登录 | | 天猫 | tmall.com | Firecrawl → Playwright | 可能需要登录 | | 京东 | jd.com | Firecrawl → Jina | - | | 百度 | baidu.com | Firecrawl → Jina | - | | 知乎 | zhihu.com | Firecrawl → Jina | - | | 微博 | weibo.com | Firecrawl → Playwright | 可能需要登录 | | B站 | bilibili.com | Firecrawl → Jina | - | | 通用网站 | * | Firecrawl → Jina | - | ## 工作流程 ``` 1. 接收 URL 2. 识别平台类型 3. 选择读取策略： ├─ 尝试 Firecrawl API │ ├─ 成功 → 返回内容 │ └─ 失败 → 继续 ├─ 尝试 Jina Reader │ ├─ 成功 → 返回内容 │ └─ 失败 → 继续 └─ 尝试 Playwright（需要登录态） ├─ 有登录态 → 读取内容 └─ 无登录态 → 提示用户设置 4. 提取核心内容 5. 格式化输出 ``` ## 输出格式 ```markdown # [文章标题] **来源**：[平台名称] **作者**：[作者名称] **发布时间**：[时间] **原文链接**：[URL] --- [正文内容] --- **互动数据**（如有）： - 阅读/播放：xxx - 点赞：xxx - 评论：xxx ``` ## 配置说明 ### Firecrawl API Key 配置 1. 访问 https://www.firecrawl.dev/ 注册账号 2. 获取 API Key 3. 配置环境变量： ```bash export FIRECRAWL_API_KEY="fc-YOUR_API_KEY" ``` ### Playwright 登录态设置（可选） 用于需要登录的平台（如微信公众号）： ```bash cd ~/.claude/skills/url-reader source .venv/bin/activate python scripts/wechat_reader.py setup ``` ## 目录结构 ``` url-reader/ ├── skill.md # 本文档 ├── metadata.json # 元数据 ├── scripts/ │ ├── url_reader.py # 主读取器（整合三种策略） │ ├── firecrawl_reader.py # Firecrawl 策略 │ ├── jina_reader.py # Jina 策略 │ ├── wechat_reader.py # Playwright 策略（微信） │ └── url_identifier.py # URL 平台识别器 └── data/ └── wechat_auth.json # 微信登录态（自动生成） ``` ## 依赖安装 ```bash cd ~/.claude/skills/url-reader python3 -m venv .venv source .venv/bin/activate # 核心依赖 pip install firecrawl-py requests # Playwright（可选，用于需要登录的平台） pip install playwright playwright install chromium ``` ## 常见问题 ### Q: 为什么有些网站读取失败？ A: 可能原因： 1. 网站有强反爬机制 → 尝试 Playwright 2. 需要登录 → 设置登录态 3. 内容已删除 → 无法读取 ### Q: Firecrawl 免费额度用完了怎么办？ A: 1. 自动降级到 Jina Reader（免费） 2. 或升级 Firecrawl 付费计划 ### Q: 微信公众号总是读取失败？ A: 微信反爬最严格，建议： 1. 使用 Playwright + 登录态 2. 或手动复制内容 ## 版本历史 - **v2.0**：整合 Firecrawl + Jina + Playwright 三层策略 - **v1.1**：添加 Playwright 浏览器自动化 - **v1.0**：基础功能

path: README.md
content: # URL Reader - 智能网页内容读取器

一键读取任意URL的内容，自动识别平台类型，智能选择最佳读取策略，自动保存内容和图片到本地。

## 功能特点

- 🔍 **智能平台识别**：自动识别微信公众号、小红书、今日头条、抖音、淘宝、天猫、京东、百度、知乎、微博、B站等平台
- 🔄 **三层读取策略**：Firecrawl → Jina → Playwright 自动降级
- 📝 **Markdown输出**：干净的Markdown格式输出
- 💾 **自动保存**：自动保存内容和图片到本地

## 技术架构

```
用户输入 URL
     ↓
┌─────────────┐
│ 平台识别器   │ → 识别URL所属平台
└─────────────┘
     ↓
┌─────────────────────────────────────┐
│           策略选择器                 │
│  Firecrawl → Jina → Playwright      │
│  (首选)     (备选)   (兜底)          │
└─────────────────────────────────────┘
     ↓
┌─────────────┐
│ 内容提取器   │ → 提取标题、正文、作者等
└─────────────┘
     ↓
┌─────────────┐
│ 格式化输出   │ → Markdown 格式
└─────────────┘
```

## 安装

```bash
cd ~/.claude/skills/url-reader
python3 -m venv .venv
source .venv/bin/activate

# 核心依赖
pip install firecrawl-py requests

# Playwright（可选，用于需要登录的平台）
pip install playwright
playwright install chromium
```

## 配置

### Firecrawl API Key

1. 访问 https://www.firecrawl.dev/ 注册账号
2. 获取 API Key
3. 配置环境变量：
   ```bash
   export FIRECRAWL_API_KEY="fc-YOUR_API_KEY"
   ```

## 使用方式

### Claude Code 中使用

```
用户：帮我读取这个链接 https://mp.weixin.qq.com/s/xxxxx
用户：看看这个小红书 https://www.xiaohongshu.com/explore/xxxxx
```

### 命令行调用

```bash
/url-reader https://example.com/article
```

## 支持的平台

| 平台 | 域名 | 推荐策略 |
|------|------|----------|
| 微信公众号 | mp.weixin.qq.com | Firecrawl → Playwright |
| 小红书 | xiaohongshu.com | Firecrawl → Jina |
| 今日头条 | toutiao.com | Firecrawl → Jina |
| 抖音 | douyin.com | Firecrawl |
| 淘宝 | taobao.com | Firecrawl → Playwright |
| 天猫 | tmall.com | Firecrawl → Playwright |
| 京东 | jd.com | Firecrawl → Jina |
| 百度 | baidu.com | Firecrawl → Jina |
| 知乎 | zhihu.com | Firecrawl → Jina |
| 微博 | weibo.com | Firecrawl → Playwright |
| B站 | bilibili.com | Firecrawl → Jina |
| 通用网站 | * | Firecrawl → Jina |

## 目录结构

```
url-reader/
├── skill.md              # Skill文档
├── metadata.json         # 元数据
├── scripts/
│   ├── url_reader.py     # 主读取器
│   ├── url_identifier.py # URL平台识别器
│   ├── save_content.py   # 内容保存
│   └── wechat_reader.py  # 微信读取器
└── data/                 # 数据目录（不上传）
```

## License

MIT


path: metadata.json
content: {
  "name": "url-reader",
  "version": "1.0.0",
  "description": "智能读取任意URL内容，支持微信公众号、小红书、今日头条、抖音、淘宝、天猫、京东、百度等中国主流平台",
  "author": "ys",
  "created": "2026-01-29",
  "tags": ["url", "reader", "scraper", "wechat", "xiaohongshu", "toutiao", "douyin", "taobao", "jd"],
  "platforms": [
    "微信公众号",
    "小红书",
    "今日头条",
    "抖音",
    "淘宝",
    "天猫",
    "京东",
    "百度",
    "知乎",
    "微博",
    "B站"
  ]
}


path: scripts/save_content.py
content: #!/usr/bin/env python3
"""
保存URL内容到本地
- 内容保存为 Markdown
- 图片下载到同级目录
"""

import os
import sys
import re
import requests
import hashlib
from pathlib import Path
from datetime import datetime
from urllib.parse import urlparse, unquote

# 默认保存目录
DEFAULT_OUTPUT_DIR = "/Users/ys/laoyang知识库/nickys/素材"


def sanitize_filename(name: str, max_length: int = 50) -> str:
    """清理文件名，移除非法字符"""
    # 移除非法字符
    name = re.sub(r'[<>:"/\\|?*\n\r\t]', '', name)
    # 移除多余空格
    name = re.sub(r'\s+', ' ', name).strip()
    # 截断长度
    if len(name) > max_length:
        name = name[:max_length]
    return name or "untitled"


def extract_title_from_content(content: str) -> str:
    """从内容中提取标题"""
    # 尝试从 Markdown 标题提取
    match = re.search(r'^#\s+(.+)$', content, re.MULTILINE)
    if match:
        return match.group(1).strip()

    # 尝试从第一行提取
    lines = content.strip().split('\n')
    if lines:
        first_line = lines[0].strip()
        if first_line and len(first_line) < 100:
            return first_line

    return "untitled"


def extract_images_from_content(content: str) -> list:
    """从内容中提取图片URL"""
    # Markdown 图片格式: ![alt](url)
    md_images = re.findall(r'!\[.*?\]\((https?://[^\s\)]+)\)', content)

    # 直接的图片URL
    direct_images = re.findall(r'(https?://[^\s\)]+\.(?:jpg|jpeg|png|gif|webp|bmp)[^\s\)]*)', content, re.IGNORECASE)

    # 小红书图片URL
    xhs_images = re.findall(r'(https?://sns-webpic[^\s\)]+)', content)

    # 合并去重
    all_images = list(dict.fromkeys(md_images + direct_images + xhs_images))

    return all_images


def download_image(url: str, save_dir: Path, index: int) -> str:
    """下载图片并返回本地文件名"""
    try:
        headers = {
            'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36',
            'Referer': 'https://www.xiaohongshu.com/'
        }

        response = requests.get(url, headers=headers, timeout=30)
        response.raise_for_status()

        # 确定文件扩展名
        content_type = response.headers.get('content-type', '')
        if 'webp' in content_type or 'webp' in url:
            ext = '.webp'
        elif 'png' in content_type or 'png' in url:
            ext = '.png'
        elif 'gif' in content_type or 'gif' in url:
            ext = '.gif'
        else:
            ext = '.jpg'

        # 生成文件名
        filename = f"img_{index:02d}{ext}"
        filepath = save_dir / filename

        # 保存图片
        with open(filepath, 'wb') as f:
            f.write(response.content)

        print(f"  ✓ 下载图片: {filename}")
        return filename

    except Exception as e:
        print(f"  ✗ 下载失败: {url[:50]}... ({e})")
        return None


def save_content(content: str, url: str, output_dir: str = None, title: str = None) -> dict:
    """
    保存内容到本地

    Args:
        content: Markdown 内容
        url: 原始 URL
        output_dir: 输出目录
        title: 自定义标题（可选）

    Returns:
        保存结果信息
    """
    output_dir = Path(output_dir or DEFAULT_OUTPUT_DIR)
    output_dir.mkdir(parents=True, exist_ok=True)

    # 提取或使用标题
    if not title:
        title = extract_title_from_content(content)

    # 生成文件夹名（日期_标题）
    date_str = datetime.now().strftime("%Y-%m-%d")
    folder_name = f"{date_str}_{sanitize_filename(title)}"

    # 创建内容目录
    content_dir = output_dir / folder_name
    content_dir.mkdir(parents=True, exist_ok=True)

    # 提取并下载图片
    images = extract_images_from_content(content)
    image_mapping = {}  # 原始URL -> 本地文件名

    if images:
        print(f"发现 {len(images)} 张图片，正在下载...")
        for i, img_url in enumerate(images, 1):
            local_name = download_image(img_url, content_dir, i)
            if local_name:
                image_mapping[img_url] = local_name

    # 替换内容中的图片URL为本地路径
    updated_content = content
    for orig_url, local_name in image_mapping.items():
        updated_content = updated_content.replace(orig_url, local_name)

    # 添加元信息
    meta = f"""---
title: {title}
url: {url}
saved_at: {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}
images: {len(image_mapping)}
---

"""

    # 保存 Markdown 文件
    md_path = content_dir / "content.md"
    with open(md_path, 'w', encoding='utf-8') as f:
        f.write(meta + updated_content)

    print(f"\n✓ 内容已保存到: {content_dir}")
    print(f"  - Markdown: content.md")
    print(f"  - 图片: {len(image_mapping)} 张")

    return {
        'success': True,
        'dir': str(content_dir),
        'md_file': str(md_path),
        'images': len(image_mapping),
        'title': title
    }


def main():
    if len(sys.argv) < 2:
        print("用法: python save_content.py <markdown_file_or_content>")
        print(f"\n默认保存目录: {DEFAULT_OUTPUT_DIR}")
        return

    # 如果是文件路径
    if os.path.isfile(sys.argv[1]):
        with open(sys.argv[1], 'r', encoding='utf-8') as f:
            content = f.read()
        url = sys.argv[2] if len(sys.argv) > 2 else "unknown"
    else:
        content = sys.argv[1]
        url = sys.argv[2] if len(sys.argv) > 2 else "unknown"

    save_content(content, url)


if __name__ == "__main__":
    main()


path: scripts/url_identifier.py
content: #!/usr/bin/env python3
"""
通用URL内容读取器
支持多种平台和读取策略
"""

import re
import json
import sys
from urllib.parse import urlparse, unquote


def identify_platform(url: str) -> dict:
    """识别URL所属平台"""
    parsed = urlparse(url)
    domain = parsed.netloc.lower()

    platforms = {
        'wechat': {
            'name': '微信公众号',
            'domains': ['mp.weixin.qq.com'],
            'strategy': 'browser',
            'script': 'wechat_reader.py'
        },
        'xiaohongshu': {
            'name': '小红书',
            'domains': ['xiaohongshu.com', 'xhslink.com', 'www.xiaohongshu.com'],
            'strategy': 'browser',
            'script': 'xiaohongshu_reader.py'
        },
        'toutiao': {
            'name': '今日头条',
            'domains': ['toutiao.com', 'www.toutiao.com'],
            'strategy': 'jina',
        },
        'douyin': {
            'name': '抖音',
            'domains': ['douyin.com', 'www.douyin.com', 'v.douyin.com'],
            'strategy': 'browser',
        },
        'taobao': {
            'name': '淘宝',
            'domains': ['taobao.com', 'item.taobao.com', 'world.taobao.com'],
            'strategy': 'browser',
        },
        'tmall': {
            'name': '天猫',
            'domains': ['tmall.com', 'detail.tmall.com'],
            'strategy': 'browser',
        },
        'jd': {
            'name': '京东',
            'domains': ['jd.com', 'item.jd.com'],
            'strategy': 'jina',
        },
        'baidu': {
            'name': '百度',
            'domains': ['baidu.com', 'www.baidu.com', 'baijiahao.baidu.com'],
            'strategy': 'jina',
        },
        'zhihu': {
            'name': '知乎',
            'domains': ['zhihu.com', 'www.zhihu.com', 'zhuanlan.zhihu.com'],
            'strategy': 'jina',
        },
        'weibo': {
            'name': '微博',
            'domains': ['weibo.com', 'www.weibo.com', 'm.weibo.cn'],
            'strategy': 'browser',
        },
        'bilibili': {
            'name': 'B站',
            'domains': ['bilibili.com', 'www.bilibili.com', 'b23.tv'],
            'strategy': 'jina',
        },
    }

    for platform_id, info in platforms.items():
        for d in info['domains']:
            if d in domain:
                return {
                    'id': platform_id,
                    **info
                }

    # 通用网站
    return {
        'id': 'generic',
        'name': '通用网站',
        'domains': [],
        'strategy': 'jina'
    }


def get_jina_url(url: str) -> str:
    """获取 Jina Reader URL"""
    return f"https://r.jina.ai/{url}"


def main():
    if len(sys.argv) < 2:
        print("用法: python url_identifier.py <url>")
        return

    url = sys.argv[1]
    platform = identify_platform(url)

    print(json.dumps(platform, ensure_ascii=False, indent=2))

    print(f"\n平台: {platform['name']}")
    print(f"推荐策略: {platform['strategy']}")

    if platform['strategy'] == 'jina':
        print(f"Jina URL: {get_jina_url(url)}")
    elif platform['strategy'] == 'browser':
        script = platform.get('script', '需要浏览器自动化')
        print(f"脚本: {script}")


if __name__ == "__main__":
    main()


path: scripts/url_reader.py
content: #!/usr/bin/env python3
"""
URL Reader - 智能网页内容读取器
策略：Firecrawl（首选）→ Jina（备选）→ Playwright（兜底）
自动保存内容和图片到指定目录
"""

import os
import sys
import json
import asyncio
import requests
import re
import hashlib
from urllib.parse import urlparse
from pathlib import Path
from datetime import datetime

# 配置
FIRECRAWL_API_KEY = os.environ.get("FIRECRAWL_API_KEY", "")
JINA_BASE_URL = "https://r.jina.ai/"
SCRIPT_DIR = Path(__file__).parent
DATA_DIR = SCRIPT_DIR.parent / "data"
WECHAT_AUTH_FILE = DATA_DIR / "wechat_auth.json"

# 默认保存目录
DEFAULT_OUTPUT_DIR = "/Users/ys/laoyang知识库/nickys/素材"


def identify_platform(url: str) -> dict:
    """识别URL所属平台"""
    parsed = urlparse(url)
    domain = parsed.netloc.lower()

    platforms = {
        'wechat': {
            'name': '微信公众号',
            'domains': ['mp.weixin.qq.com'],
            'need_login': True
        },
        'xiaohongshu': {
            'name': '小红书',
            'domains': ['xiaohongshu.com', 'xhslink.com'],
            'need_login': False
        },
        'toutiao': {
            'name': '今日头条',
            'domains': ['toutiao.com'],
            'need_login': False
        },
        'douyin': {
            'name': '抖音',
            'domains': ['douyin.com', 'v.douyin.com'],
            'need_login': False
        },
        'taobao': {
            'name': '淘宝',
            'domains': ['taobao.com', 'item.taobao.com'],
            'need_login': True
        },
        'tmall': {
            'name': '天猫',
            'domains': ['tmall.com', 'detail.tmall.com'],
            'need_login': True
        },
        'jd': {
            'name': '京东',
            'domains': ['jd.com', 'item.jd.com'],
            'need_login': False
        },
        'zhihu': {
            'name': '知乎',
            'domains': ['zhihu.com', 'zhuanlan.zhihu.com'],
            'need_login': False
        },
        'weibo': {
            'name': '微博',
            'domains': ['weibo.com', 'm.weibo.cn'],
            'need_login': True
        },
        'bilibili': {
            'name': 'B站',
            'domains': ['bilibili.com', 'b23.tv'],
            'need_login': False
        },
        'baidu': {
            'name': '百度',
            'domains': ['baidu.com', 'baijiahao.baidu.com'],
            'need_login': False
        },
    }

    for platform_id, info in platforms.items():
        for d in info['domains']:
            if d in domain:
                return {'id': platform_id, **info}

    return {
        'id': 'generic',
        'name': '通用网站',
        'domains': [],
        'need_login': False
    }


def read_with_firecrawl(url: str) -> dict:
    """策略A：使用 Firecrawl API 读取"""
    if not FIRECRAWL_API_KEY:
        return {'success': False, 'error': 'FIRECRAWL_API_KEY 未设置'}

    try:
        from firecrawl import Firecrawl
        app = Firecrawl([REDACTED_SECRET])
        result = app.scrape(url)

        if result:
            # Firecrawl v2 返回 Document 对象
            markdown = getattr(result, 'markdown', '') or ''
            metadata = getattr(result, 'metadata', None)
            if metadata:
                metadata = metadata.model_dump() if hasattr(metadata, 'model_dump') else {}
            else:
                metadata = {}

            if markdown and len(markdown) > 100:
                # 检查是否是验证页面
                if '环境异常' in markdown or '验证' in markdown[:200]:
                    return {'success': False, 'error': '页面需要验证'}

                return {
                    'success': True,
                    'strategy': 'Firecrawl',
                    'content': markdown,
                    'metadata': metadata
                }

        return {'success': False, 'error': 'Firecrawl 返回内容为空'}

    except Exception as e:
        return {'success': False, 'error': f'Firecrawl 错误: {str(e)}'}


def read_with_jina(url: str) -> dict:
    """策略B-1：使用 Jina Reader API 读取（免费）"""
    try:
        jina_url = f"{JINA_BASE_URL}{url}"
        headers = {
            'Accept': 'text/markdown',
            'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36'
        }

        response = requests.get(jina_url, headers=headers, timeout=30)

        if response.status_code == 200:
            content = response.text

            # 检查是否是有效内容
            if '环境异常' in content or '完成验证' in content:
                return {'success': False, 'error': '页面需要验证'}

            if len(content) < 100:
                return {'success': False, 'error': '内容太短，可能读取失败'}

            return {
                'success': True,
                'strategy': 'Jina Reader',
                'content': content,
                'metadata': {}
            }

        return {'success': False, 'error': f'HTTP {response.status_code}'}

    except Exception as e:
        return {'success': False, 'error': f'Jina 错误: {str(e)}'}


async def read_with_playwright_async(url: str, platform_id: str) -> dict:
    """策略B-2：使用 Playwright 浏览器读取（需要登录态）"""
    try:
        from playwright.async_api import async_playwright
        import time

        # 检查是否有登录态
        auth_file = None
        if platform_id == 'wechat' and WECHAT_AUTH_FILE.exists():
            auth_file = str(WECHAT_AUTH_FILE)

        async with async_playwright() as p:
            # 使用移动端 User-Agent 可能更容易通过验证
            browser = await p.chromium.launch(headless=True)

            # 使用微信内置浏览器的 User-Agent
            mobile_ua = 'Mozilla/5.0 (iPhone; CPU iPhone OS 16_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 MicroMessenger/8.0.38(0x18002629) NetType/WIFI Language/zh_CN'

            if auth_file:
                context = await browser.new_context(
                    storage_state=auth_file,
                    user_agent=mobile_ua
                )
            else:
                context = await browser.new_context(
                    user_agent=mobile_ua
                )

            page = await context.new_page()

            await page.goto(url, wait_until="networkidle", timeout=30000)

            # 等待页面完全加载
            await page.wait_for_timeout(2000)

            # 检查是否需要验证
            content = await page.content()
            if '环境异常' in content or '完成验证' in content:
                # 尝试点击验证按钮
                verify_btn = await page.query_selector("text=去验证")
                if verify_btn:
                    await verify_btn.click()
                    await page.wait_for_timeout(3000)
                    await page.wait_for_load_state("networkidle", timeout=15000)
                    content = await page.content()

                if '环境异常' in content or '完成验证' in content:
                    await browser.close()
                    return {'success': False, 'error': '需要手动验证，请运行 setup 命令登录'}

            # 等待文章内容加载
            if platform_id == 'wechat':
                try:
                    await page.wait_for_selector('#js_content', timeout=10000)
                except:
                    pass

            # 提取内容
            if platform_id == 'wechat':
                result = await page.evaluate("""
                    () => {
                        const title = document.querySelector('#activity-name')?.innerText?.trim() || '';
                        const author = document.querySelector('#js_name')?.innerText?.trim() || '';
                        const content = document.querySelector('#js_content')?.innerText?.trim() || '';
                        const publishTime = document.querySelector('#publish_time')?.innerText?.trim() || '';
                        return { title, author, content, publishTime };
                    }
                """)
            else:
                # 通用提取
                result = await page.evaluate("""
                    () => {
                        const title = document.querySelector('h1')?.innerText?.trim() ||
                                     document.querySelector('title')?.innerText?.trim() || '';
                        const content = document.body.innerText || '';
                        return { title, author: '', content, publishTime: '' };
                    }
                """)

            await browser.close()

            if result.get('content') and len(result['content']) > 100:
                # 格式化为 Markdown
                markdown = f"# {result.get('title', '无标题')}\n\n"
                if result.get('author'):
                    markdown += f"**作者**: {result['author']}\n"
                if result.get('publishTime'):
                    markdown += f"**发布时间**: {result['publishTime']}\n"
                markdown += f"\n---\n\n{result.get('content', '')}"

                return {
                    'success': True,
                    'strategy': 'Playwright',
                    'content': markdown,
                    'metadata': {
                        'title': result.get('title', ''),
                        'author': result.get('author', ''),
                        'publishTime': result.get('publishTime', '')
                    }
                }

            return {'success': False, 'error': '页面内容提取失败'}

    except Exception as e:
        return {'success': False, 'error': f'Playwright 错误: {str(e)}'}


def read_with_playwright(url: str, platform_id: str) -> dict:
    """Playwright 同步包装"""
    return asyncio.run(read_with_playwright_async(url, platform_id))


def read_url(url: str, verbose: bool = True) -> dict:
    """
    智能读取URL内容
    策略顺序：Firecrawl → Jina → Playwright
    """
    # 识别平台
    platform = identify_platform(url)
    if verbose:
        print(f"📍 平台识别: {platform['name']}")

    errors = []

    # 策略1: Firecrawl（如果有 API Key）
    if FIRECRAWL_API_KEY:
        if verbose:
            print("🔄 尝试策略 A: Firecrawl...")
        result = read_with_firecrawl(url)
        if result.get('success'):
            if verbose:
                print("✅ Firecrawl 读取成功")
            result['platform'] = platform
            return result
        errors.append(f"Firecrawl: {result.get('error')}")
        if verbose:
            print(f"❌ {result.get('error')}")

    # 策略2: Jina Reader（免费，不需要登录的平台优先尝试）
    if not platform.get('need_login'):
        if verbose:
            print("🔄 尝试策略 B-1: Jina Reader...")
        result = read_with_jina(url)
        if result.get('success'):
            if verbose:
                print("✅ Jina Reader 读取成功")
            result['platform'] = platform
            return result
        errors.append(f"Jina: {result.get('error')}")
        if verbose:
            print(f"❌ {result.get('error')}")

    # 策略3: Playwright（需要登录的平台或前面都失败）
    if verbose:
        print("🔄 尝试策略 B-2: Playwright 浏览器...")
    result = read_with_playwright(url, platform['id'])
    if result.get('success'):
        if verbose:
            print("✅ Playwright 读取成功")
        result['platform'] = platform
        return result
    errors.append(f"Playwright: {result.get('error')}")
    if verbose:
        print(f"❌ {result.get('error')}")

    # 如果是需要登录的平台，Jina 作为最后尝试
    if platform.get('need_login'):
        if verbose:
            print("🔄 最后尝试: Jina Reader...")
        result = read_with_jina(url)
        if result.get('success'):
            if verbose:
                print("✅ Jina Reader 读取成功")
            result['platform'] = platform
            return result
        errors.append(f"Jina (fallback): {result.get('error')}")

    return {
        'success': False,
        'platform': platform,
        'errors': errors
    }


def format_output(result: dict, url: str) -> str:
    """格式化输出为 Markdown"""
    if not result.get('success'):
        output = ["# ❌ 读取失败\n"]
        output.append(f"**URL**: {url}")
        output.append(f"**平台**: {result.get('platform', {}).get('name', '未知')}")
        output.append("\n**尝试的策略及错误**:")
        for err in result.get('errors', []):
            output.append(f"- {err}")
        output.append("\n**建议**:")
        output.append("1. 如果是微信公众号，请运行 `python wechat_reader.py setup` 设置登录态")
        output.append("2. 设置 FIRECRAWL_API_KEY 环境变量以使用 Firecrawl")
        output.append("3. 或手动复制文章内容")
        return "\n".join(output)

    platform = result.get('platform', {})
    metadata = result.get('metadata', {})
    content = result.get('content', '')

    output = []
    output.append(f"**来源**: {platform.get('name', '未知')}")
    output.append(f"**读取策略**: {result.get('strategy', '未知')}")
    output.append(f"**原文链接**: {url}")
    output.append("\n---\n")
    output.append(content)

    return "\n".join(output)


# ============== 保存功能 ==============

def sanitize_filename(name: str, max_length: int = 50) -> str:
    """清理文件名，移除非法字符"""
    name = re.sub(r'[<>:"/\\|?*\n\r\t]', '', name)
    name = re.sub(r'\s+', ' ', name).strip()
    if len(name) > max_length:
        name = name[:max_length]
    return name or "untitled"


def extract_title_from_content(content: str) -> str:
    """从内容中提取标题"""
    # 尝试从 Markdown 一级标题提取
    match = re.search(r'^#\s+(.+)$', content, re.MULTILINE)
    if match:
        title = match.group(1).strip()
        # 排除一些无意义的标题
        if title and not title.startswith('来源') and not title.startswith('**') and len(title) > 2:
            return title

    # 尝试从 **标题**: 格式提取
    match = re.search(r'\*\*标题\*\*[：:]\s*(.+)', content)
    if match:
        return match.group(1).strip()

    # 尝试从第一个有意义的行提取
    lines = content.strip().split('\n')
    for line in lines:
        line = line.strip()
        # 跳过空行、元数据行
        if not line or line.startswith('**') or line.startswith('---') or line.startswith('#'):
            continue
        if len(line) > 5 and len(line) < 100:
            return line[:50]

    return "untitled"


def extract_images_from_content(content: str) -> list:
    """从内容中提取图片URL"""
    md_images = re.findall(r'!\[.*?\]\((https?://[^\s\)]+)\)', content)
    direct_images = re.findall(r'(https?://[^\s\)]+\.(?:jpg|jpeg|png|gif|webp|bmp)[^\s\)]*)', content, re.IGNORECASE)
    xhs_images = re.findall(r'(https?://sns-webpic[^\s\)]+)', content)
    feishu_images = re.findall(r'(https?://[^\s\)]*feishu[^\s\)]+\.(?:jpg|jpeg|png|gif|webp)[^\s\)]*)', content, re.IGNORECASE)
    qq_images = re.findall(r'(https?://docimg[^\s\)]+\.(?:jpg|jpeg|png|gif|webp)[^\s\)]*)', content, re.IGNORECASE)
    all_images = list(dict.fromkeys(md_images + direct_images + xhs_images + feishu_images + qq_images))
    return all_images


def download_image(url: str, save_dir: Path, index: int) -> str:
    """下载图片并返回本地文件名"""
    try:
        headers = {
            'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36',
            'Referer': 'https://www.xiaohongshu.com/'
        }
        response = requests.get(url, headers=headers, timeout=30)
        response.raise_for_status()

        content_type = response.headers.get('content-type', '')
        if 'webp' in content_type or 'webp' in url:
            ext = '.webp'
        elif 'png' in content_type or 'png' in url:
            ext = '.png'
        elif 'gif' in content_type or 'gif' in url:
            ext = '.gif'
        else:
            ext = '.jpg'

        filename = f"img_{index:02d}{ext}"
        filepath = save_dir / filename

        with open(filepath, 'wb') as f:
            f.write(response.content)

        return filename
    except Exception as e:
        return None


def save_content(content: str, url: str, platform_name: str = "", output_dir: str = None, title: str = None, verbose: bool = True) -> dict:
    """
    保存内容到本地
    """
    output_dir = Path(output_dir or DEFAULT_OUTPUT_DIR)
    output_dir.mkdir(parents=True, exist_ok=True)

    if not title:
        title = extract_title_from_content(content)

    date_str = datetime.now().strftime("%Y-%m-%d")
    folder_name = f"{date_str}_{sanitize_filename(title)}"

    content_dir = output_dir / folder_name
    content_dir.mkdir(parents=True, exist_ok=True)

    images = extract_images_from_content(content)
    image_mapping = {}

    if images and verbose:
        print(f"📷 发现 {len(images)} 张图片，正在下载...")

    for i, img_url in enumerate(images, 1):
        local_name = download_image(img_url, content_dir, i)
        if local_name:
            image_mapping[img_url] = local_name
            if verbose:
                print(f"  ✓ {local_name}")

    updated_content = content
    for orig_url, local_name in image_mapping.items():
        updated_content = updated_content.replace(orig_url, local_name)

    meta = f"""---
title: {title}
platform: {platform_name}
url: {url}
saved_at: {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}
images: {len(image_mapping)}
---

"""

    md_path = content_dir / "content.md"
    with open(md_path, 'w', encoding='utf-8') as f:
        f.write(meta + updated_content)

    if verbose:
        print(f"\n💾 已保存到: {content_dir}")
        print(f"   - content.md")
        print(f"   - 图片: {len(image_mapping)} 张")

    return {
        'success': True,
        'dir': str(content_dir),
        'md_file': str(md_path),
        'images': len(image_mapping),
        'title': title
    }


def read_and_save(url: str, output_dir: str = None, verbose: bool = True) -> dict:
    """
    读取URL内容并保存到本地
    """
    result = read_url(url, verbose=verbose)

    if not result.get('success'):
        return result

    platform = result.get('platform', {})
    content = format_output(result, url)

    save_result = save_content(
        content=content,
        url=url,
        platform_name=platform.get('name', '未知'),
        output_dir=output_dir,
        verbose=verbose
    )

    result['save'] = save_result
    return result


def main():
    if len(sys.argv) < 2:
        print("=" * 60)
        print("URL Reader - 智能网页内容读取器")
        print("=" * 60)
        print("\n用法:")
        print("  python url_reader.py <url>              # 读取并显示")
        print("  python url_reader.py <url> --save       # 读取并保存")
        print("\n示例:")
        print("  python url_reader.py https://mp.weixin.qq.com/s/xxxxx --save")
        print("\n保存目录:")
        print(f"  {DEFAULT_OUTPUT_DIR}")
        print("\n策略优先级:")
        print("  1. Firecrawl (需要 API Key)")
        print("  2. Jina Reader (免费)")
        print("  3. Playwright (需要登录态)")
        print("\n环境变量:")
        print(f"  FIRECRAWL_API_KEY: {'已设置' if FIRECRAWL_API_KEY else '未设置'}")
        print(f"  微信登录态: {'已设置' if WECHAT_AUTH_FILE.exists() else '未设置'}")
        return

    url = sys.argv[1]
    save_mode = '--save' in sys.argv

    print(f"\n{'=' * 60}")
    print(f"正在读取: {url}")
    print(f"{'=' * 60}\n")

    if save_mode:
        result = read_and_save(url)
        if result.get('success') and result.get('save'):
            print(f"\n{'=' * 60}")
            print("✅ 读取并保存成功")
            print(f"{'=' * 60}")
    else:
        result = read_url(url)
        output = format_output(result, url)
        print(f"\n{'=' * 60}")
        print("读取结果")
        print(f"{'=' * 60}\n")
        print(output)


if __name__ == "__main__":
    main()


path: scripts/wechat_reader.py
content: #!/usr/bin/env python3
"""
微信公众号文章读取器
使用 Playwright 浏览器自动化，支持登录态保持
"""

import asyncio
import json
import sys
import os
from pathlib import Path
from datetime import datetime

# 数据目录
DATA_DIR = Path(__file__).parent.parent / "data"
AUTH_FILE = DATA_DIR / "wechat_auth.json"

async def setup_auth(wait_seconds: int = 120):
    """首次登录，保存认证状态"""
    from playwright.async_api import async_playwright

    print("正在启动浏览器，请扫码登录微信...")

    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=False)
        context = await browser.new_context()
        page = await context.new_page()

        # 打开微信文章页面（任意一个）
        await page.goto("https://mp.weixin.qq.com/")

        print("\n请在浏览器中完成以下操作：")
        print("1. 点击右上角「登录」")
        print("2. 使用微信扫码登录")
        print(f"3. 登录成功后会自动保存（等待 {wait_seconds} 秒）")
        print("\n等待登录中...")

        # 等待登录成功（检测页面变化）
        try:
            # 等待登录成功后的元素出现
            await page.wait_for_selector(".weui-desktop-account__nickname", timeout=wait_seconds * 1000)
            print("✓ 检测到登录成功！")
        except:
            # 超时后也保存，可能用户已经登录
            print("等待超时，尝试保存当前状态...")

        # 保存认证状态
        storage = await context.storage_state()
        DATA_DIR.mkdir(parents=True, exist_ok=True)

        with open(AUTH_FILE, 'w', encoding='utf-8') as f:
            json.dump(storage, f, ensure_ascii=False, indent=2)

        await browser.close()

    print(f"✓ 认证状态已保存到: {AUTH_FILE}")
    return True


async def read_article(url: str, headless: bool = True):
    """读取微信公众号文章"""
    from playwright.async_api import async_playwright

    if not AUTH_FILE.exists():
        print("错误：未找到认证状态，请先运行 setup 命令")
        print("运行: python wechat_reader.py setup")
        return None

    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=headless)

        # 加载已保存的认证状态
        context = await browser.new_context(storage_state=str(AUTH_FILE))
        page = await context.new_page()

        try:
            print(f"正在读取: {url}")
            await page.goto(url, wait_until="networkidle", timeout=30000)

            # 检查是否需要验证
            if await page.query_selector("text=环境异常"):
                print("检测到验证页面，尝试自动处理...")
                # 点击验证按钮
                verify_btn = await page.query_selector("text=去验证")
                if verify_btn:
                    await verify_btn.click()
                    await page.wait_for_load_state("networkidle", timeout=30000)

            # 等待文章内容加载
            await page.wait_for_selector("#js_content", timeout=10000)

            # 提取文章信息
            result = await page.evaluate("""
                () => {
                    const title = document.querySelector('#activity-name')?.innerText?.trim() || '';
                    const author = document.querySelector('#js_name')?.innerText?.trim() || '';
                    const content = document.querySelector('#js_content')?.innerText?.trim() || '';
                    const publishTime = document.querySelector('#publish_time')?.innerText?.trim() || '';
                    const originalUrl = window.location.href;

                    // 提取图片
                    const images = [];
                    document.querySelectorAll('#js_content img').forEach(img => {
                        if (img.dataset.src) {
                            images.push(img.dataset.src);
                        }
                    });

                    return {
                        title,
                        author,
                        content,
                        publishTime,
                        originalUrl,
                        images
                    };
                }
            """)

            await browser.close()
            return result

        except Exception as e:
            print(f"读取失败: {e}")

            # 如果是验证问题，提示重新登录
            if "环境异常" in str(e) or "验证" in str(e):
                print("\n认证可能已过期，请重新登录：")
                print("运行: python wechat_reader.py setup")

            await browser.close()
            return None


async def check_auth():
    """检查认证状态"""
    if not AUTH_FILE.exists():
        print("✗ 未找到认证状态")
        print("运行: python wechat_reader.py setup")
        return False

    # 检查文件修改时间
    mtime = datetime.fromtimestamp(AUTH_FILE.stat().st_mtime)
    age = datetime.now() - mtime

    print(f"✓ 认证文件存在")
    print(f"  路径: {AUTH_FILE}")
    print(f"  创建时间: {mtime.strftime('%Y-%m-%d %H:%M:%S')}")
    print(f"  已过去: {age.days} 天 {age.seconds // 3600} 小时")

    if age.days > 7:
        print("\n⚠ 认证可能已过期，建议重新登录")

    return True


def format_article(article: dict) -> str:
    """格式化文章输出"""
    if not article:
        return "无法读取文章内容"

    output = []
    output.append(f"# {article.get('title', '无标题')}")
    output.append("")
    output.append(f"**来源**：微信公众号")
    output.append(f"**作者**：{article.get('author', '未知')}")
    output.append(f"**发布时间**：{article.get('publishTime', '未知')}")
    output.append(f"**原文链接**：{article.get('originalUrl', '')}")
    output.append("")
    output.append("---")
    output.append("")
    output.append(article.get('content', ''))

    if article.get('images'):
        output.append("")
        output.append("---")
        output.append("")
        output.append(f"**文章图片**：共 {len(article['images'])} 张")

    return "\n".join(output)


async def main():
    if len(sys.argv) < 2:
        print("用法:")
        print("  python wechat_reader.py setup          # 首次登录")
        print("  python wechat_reader.py status         # 检查认证状态")
        print("  python wechat_reader.py read <url>     # 读取文章")
        print("  python wechat_reader.py read <url> --show  # 显示浏览器")
        return

    command = sys.argv[1]

    if command == "setup":
        await setup_auth()

    elif command == "status":
        await check_auth()

    elif command == "read":
        if len(sys.argv) < 3:
            print("请提供文章URL")
            return

        url = sys.argv[2]
        headless = "--show" not in sys.argv

        article = await read_article(url, headless=headless)
        if article:
            print("\n" + "=" * 50)
            print(format_article(article))

    else:
        print(f"未知命令: {command}")


if __name__ == "__main__":
    asyncio.run(main())


path: scripts/wechat_reader_v2.py
content: #!/usr/bin/env python3
"""
微信公众号文章读取器 - 增强版
支持自动处理验证页面
"""

import asyncio
import json
import sys
import os
from pathlib import Path
from datetime import datetime

DATA_DIR = Path(__file__).parent.parent / "data"
AUTH_FILE = DATA_DIR / "wechat_auth.json"


async def read_wechat_article(url: str, headless: bool = True, wait_for_manual: bool = False) -> dict:
    """
    读取微信公众号文章

    Args:
        url: 文章链接
        headless: 是否无头模式
        wait_for_manual: 是否等待手动验证
    """
    from playwright.async_api import async_playwright

    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=headless)

        # 加载已保存的认证状态
        if AUTH_FILE.exists():
            context = await browser.new_context(storage_state=str(AUTH_FILE))
        else:
            context = await browser.new_context()

        page = await context.new_page()

        # 设置微信客户端 User-Agent
        await page.set_extra_http_headers({
            'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 16_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 MicroMessenger/8.0.38(0x18002629) NetType/WIFI Language/zh_CN'
        })

        try:
            print(f"正在访问: {url[:60]}...")
            await page.goto(url, wait_until="networkidle", timeout=30000)
            await page.wait_for_timeout(2000)

            # 检查是否需要验证
            content = await page.content()

            if '环境异常' in content or '完成验证' in content:
                print("检测到需要验证...")

                if wait_for_manual and not headless:
                    print("\n" + "=" * 50)
                    print("请在浏览器中完成验证")
                    print("验证完成后按 Enter 继续...")
                    print("=" * 50)

                    # 等待用户手动验证（最多等待 120 秒）
                    for i in range(120):
                        await page.wait_for_timeout(1000)
                        content = await page.content()
                        if '环境异常' not in content and 'js_content' in content:
                            print("验证成功！")
                            break

                    # 保存新的认证状态
                    storage = await context.storage_state()
                    DATA_DIR.mkdir(parents=True, exist_ok=True)
                    with open(AUTH_FILE, 'w', encoding='utf-8') as f:
                        json.dump(storage, f, ensure_ascii=False, indent=2)
                    print(f"认证状态已保存")
                else:
                    await browser.close()
                    return {
                        'success': False,
                        'error': '需要验证',
                        'suggestion': '请运行: python wechat_reader_v2.py --verify <url>'
                    }

            # 等待文章内容加载
            try:
                await page.wait_for_selector('#js_content', timeout=10000)
            except:
                pass

            # 提取文章内容
            result = await page.evaluate("""
                () => {
                    const title = document.querySelector('#activity-name')?.innerText?.trim() || '';
                    const author = document.querySelector('#js_name')?.innerText?.trim() || '';
                    const content = document.querySelector('#js_content')?.innerText?.trim() || '';
                    const publishTime = document.querySelector('#publish_time')?.innerText?.trim() || '';
                    const originalUrl = window.location.href;

                    return {
                        title,
                        author,
                        content,
                        publishTime,
                        originalUrl
                    };
                }
            """)

            await browser.close()

            if result.get('content') and len(result['content']) > 50:
                return {
                    'success': True,
                    **result
                }
            else:
                return {
                    'success': False,
                    'error': '无法提取文章内容',
                    'suggestion': '文章可能已被删除或需要特殊权限'
                }

        except Exception as e:
            await browser.close()
            return {
                'success': False,
                'error': str(e)
            }


def format_output(result: dict) -> str:
    """格式化输出"""
    if not result.get('success'):
        output = ["# 读取失败\n"]
        output.append(f"**错误**: {result.get('error', '未知错误')}")
        if result.get('suggestion'):
            output.append(f"**建议**: {result['suggestion']}")
        return "\n".join(output)

    output = []
    output.append(f"# {result.get('title', '无标题')}\n")
    output.append(f"**作者**: {result.get('author', '未知')}")
    output.append(f"**发布时间**: {result.get('publishTime', '未知')}")
    output.append(f"**原文链接**: {result.get('originalUrl', '')}")
    output.append("\n---\n")
    output.append(result.get('content', ''))

    return "\n".join(output)


async def main():
    if len(sys.argv) < 2:
        print("=" * 60)
        print("微信公众号文章读取器 - 增强版")
        print("=" * 60)
        print("\n用法:")
        print("  python wechat_reader_v2.py <url>              # 普通读取")
        print("  python wechat_reader_v2.py --verify <url>     # 需要验证时使用")
        print("  python wechat_reader_v2.py --show <url>       # 显示浏览器")
        print("\n示例:")
        print("  python wechat_reader_v2.py https://mp.weixin.qq.com/s/xxxxx")
        print("  python wechat_reader_v2.py --verify 'https://mp.weixin.qq.com/s?__biz=xxx&mid=xxx&sn=xxx'")
        return

    # 解析参数
    verify_mode = '--verify' in sys.argv
    show_mode = '--show' in sys.argv

    # 获取 URL（最后一个非选项参数）
    url = None
    for arg in sys.argv[1:]:
        if not arg.startswith('--'):
            url = arg

    if not url:
        print("错误: 请提供 URL")
        return

    headless = not (verify_mode or show_mode)
    wait_for_manual = verify_mode

    print(f"\n正在读取文章...")
    if verify_mode:
        print("验证模式: 浏览器将打开，请手动完成验证")

    result = await read_wechat_article(url, headless=headless, wait_for_manual=wait_for_manual)

    print("\n" + "=" * 60)
    print(format_output(result))


if __name__ == "__main__":
    asyncio.run(main())


path: scripts/wechat_url_converter.py
content: #!/usr/bin/env python3
"""
微信公众号长链接转短链接工具
将带参数的长链接转换为短链接格式
"""

import asyncio
import sys
import re
from urllib.parse import urlparse, parse_qs


async def convert_long_to_short(long_url: str) -> str:
    """
    将微信公众号长链接转换为短链接

    长链接格式: https://mp.weixin.qq.com/s?__biz=xxx&mid=xxx&idx=xxx&sn=xxx...
    短链接格式: https://mp.weixin.qq.com/s/xxxxxx
    """
    from playwright.async_api import async_playwright

    print(f"正在转换: {long_url[:80]}...")

    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=True)
        context = await browser.new_context()
        page = await context.new_page()

        # 设置 User-Agent
        await page.set_extra_http_headers({
            'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 16_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 MicroMessenger/8.0.38'
        })

        try:
            # 访问长链接，获取重定向后的短链接
            response = await page.goto(long_url, wait_until="domcontentloaded", timeout=15000)

            # 获取最终URL
            final_url = page.url

            # 检查是否已经是短链接
            if '/s/' in final_url and '?' not in final_url.split('/s/')[1].split('#')[0]:
                await browser.close()
                return final_url

            # 尝试从页面中提取短链接
            # 方法1: 从 meta 标签获取
            short_url = await page.evaluate("""
                () => {
                    // 尝试从 og:url 获取
                    const ogUrl = document.querySelector('meta[property="og:url"]');
                    if (ogUrl) return ogUrl.content;

                    // 尝试从 canonical 获取
                    const canonical = document.querySelector('link[rel="canonical"]');
                    if (canonical) return canonical.href;

                    return null;
                }
            """)

            if short_url and '/s/' in short_url:
                await browser.close()
                return short_url

            # 方法2: 从页面源码中查找
            content = await page.content()

            # 查找 msg_link 或其他短链接模式
            patterns = [
                r'var\s+msg_link\s*=\s*["\']([^"\']+)["\']',
                r'href="(https://mp\.weixin\.qq\.com/s/[a-zA-Z0-9_-]+)"',
                r'"url":"(https://mp\.weixin\.qq\.com/s/[a-zA-Z0-9_-]+)"',
            ]

            for pattern in patterns:
                match = re.search(pattern, content)
                if match:
                    found_url = match.group(1).replace('\\/', '/')
                    if '/s/' in found_url and len(found_url.split('/s/')[1]) > 10:
                        await browser.close()
                        return found_url

            await browser.close()

            # 如果无法获取短链接，返回原始URL
            return final_url

        except Exception as e:
            await browser.close()
            print(f"转换失败: {e}")
            return long_url


def is_long_url(url: str) -> bool:
    """判断是否是长链接格式"""
    parsed = urlparse(url)
    if 'mp.weixin.qq.com' not in parsed.netloc:
        return False

    # 长链接特征: 包含 __biz, mid, sn 等参数
    query = parse_qs(parsed.query)
    return '__biz' in query or 'mid' in query


def extract_short_id(url: str) -> str:
    """从短链接中提取ID"""
    match = re.search(r'/s/([a-zA-Z0-9_-]+)', url)
    return match.group(1) if match else None


async def main():
    if len(sys.argv) < 2:
        print("=" * 60)
        print("微信公众号长链接转短链接工具")
        print("=" * 60)
        print("\n用法: python wechat_url_converter.py <长链接>")
        print("\n示例:")
        print('  python wechat_url_converter.py "https://mp.weixin.qq.com/s?__biz=xxx&mid=xxx&idx=1&sn=xxx"')
        print("\n说明:")
        print("  长链接格式: https://mp.weixin.qq.com/s?__biz=xxx&mid=xxx&idx=xxx&sn=xxx")
        print("  短链接格式: https://mp.weixin.qq.com/s/xxxxxx")
        return

    long_url = sys.argv[1]

    if not is_long_url(long_url):
        # 检查是否已经是短链接
        short_id = extract_short_id(long_url)
        if short_id:
            print(f"已经是短链接格式: https://mp.weixin.qq.com/s/{short_id}")
            return
        print("错误: 不是有效的微信公众号链接")
        return

    short_url = await convert_long_to_short(long_url)

    print("\n" + "=" * 60)
    print("转换结果")
    print("=" * 60)
    print(f"\n原始链接: {long_url[:80]}...")
    print(f"\n短链接: {short_url}")

    short_id = extract_short_id(short_url)
    if short_id:
        print(f"\n文章ID: {short_id}")


if __name__ == "__main__":
    asyncio.run(main())


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)