Skill: robertstarry-gif/liuzln-openclaw-skills-wechat-article-fetcher (version 1.0.0)

--- name: wechat-article-fetcher description: Fetch and save WeChat Official Account articles with full content and images. Supports any WeChat article URL, automatic image download, JSON export, and full-page screenshots. Use when you need to archive, save, or analyze WeChat Official Account articles. --- # WeChat Article Fetcher Skill 微信公众号文章爬取 Skill，支持任意微信公众号文章 URL，自动下载图片，导出 JSON，保存完整截图。 ## 功能特性 - ✅ 支持任意微信公众号文章 URL - ✅ 自动提取标题、作者、发布时间 - ✅ 提取完整文章内容 - ✅ 自动下载并保存文章图片 - ✅ 保存完整页面截图 - ✅ 导出 JSON 格式结果 - ✅ 支持虚拟环境（如 playwright-env） - ✅ 无头模式（无 UI 服务器部署） - ✅ 命令行工具支持 ## 快速开始 ### 1. 使用命令行工具 ```bash # 爬取单篇文章 python3 skills/wechat-article-fetcher/scripts/fetch.py \ https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw # 指定输出目录 python3 skills/wechat-article-fetcher/scripts/fetch.py \ https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \ --output ./my_articles # 不保存图片 python3 skills/wechat-article-fetcher/scripts/fetch.py \ https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \ --no-images ``` ### 2. 使用虚拟环境运行 ```bash # 方式 1: 激活虚拟环境后运行 source playwright-env/bin/activate python3 skills/wechat-article-fetcher/scripts/fetch.py <url> # 方式 2: 使用提供的包装脚本 python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \ <url> --venv /path/to/playwright-env ``` ### 3. 在代码中使用 ```python from wechat_article_fetcher import fetch_article # 爬取文章 result = fetch_article("https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw") print(f"标题: {result['title']}") print(f"作者: {result['author']}") print(f"内容长度: {result['length']}") print(f"图片数量: {result['images_count']}") ``` ## 命令行工具 ### fetch.py - 主要爬取工具 ```bash python3 skills/wechat-article-fetcher/scripts/fetch.py [OPTIONS] URL 参数: URL 微信公众号文章 URL（必需） 选项: -o, --output PATH 输出目录（默认: ./wechat_articles） --no-images 不保存图片 --no-screenshot 不保存截图 --headless BOOLEAN 无头模式（默认: true） --timeout INTEGER 超时时间（毫秒，默认: 60000） -h, --help 显示帮助信息 ``` ### batch_fetch.py - 批量爬取工具 ```bash # 从文件中读取 URL 列表 python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \ --urls-file urls.txt # 或者直接在命令行指定多个 URL python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \ --urls "url1" "url2" "url3" ``` ### run_in_venv.py - 虚拟环境运行工具 ```bash python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \ <url> --venv /path/to/playwright-env ``` ## 输出说明 每次爬取会创建一个以时间戳命名的目录： ``` wechat_articles/ └── 20260302_125500/ ├── article.json # JSON 格式结果 ├── article.png # 完整页面截图 └── images/ # 文章图片目录 ├── 001_image1.jpg ├── 002_image2.png └── ... ``` ### article.json 结构 ```json { "title": "文章标题", "author": "作者名称", "publish_date": "2026-03-02", "url": "https://mp.weixin.qq.com/s/...", "content": "完整文章内容...", "images": [ { "index": 1, "url": "https://mmbiz.qpic.cn/...", "alt": "图片描述", "filename": "001_image.jpg", "success": true } ], "images_count": 5, "images_dir": [REDACTED_SECRET], "fetch_time": "2026-03-02T12:55:00.000000", "length": 15000 } ``` ## 配置文件 可以创建 `config.json` 来自定义默认配置： ```json { "headless": true, "timeout": 60000, "output_dir": "./wechat_articles", "save_images": true, "save_screenshot": true, "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36..." } ``` ## 使用示例 ### 示例 1: 爬取单篇文章 ```bash python3 skills/wechat-article-fetcher/scripts/fetch.py \ https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw ``` ### 示例 2: 批量爬取 创建 `urls.txt`： ``` https://mp.weixin.qq.com/s/xxx https://mp.weixin.qq.com/s/yyy https://mp.weixin.qq.com/s/zzz ``` 然后运行： ```bash python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \ --urls-file urls.txt ``` ### 示例 3: 在虚拟环境中使用 ```bash python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \ https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \ --venv /home/user/playwright-env ``` ## 最佳实践 1. **使用虚拟环境**: 隔离依赖，避免冲突 2. **合理设置超时**: 根据网络情况调整 3. **批量爬取时添加延迟**: 避免给服务器造成压力 4. **定期检查输出**: 确保图片和内容完整 5. **遵守 robots.txt**: 尊重网站的爬取规则 ## 故障排除 ### 问题: 找不到模块 **解决方案**: 确保在正确的虚拟环境中运行，或安装依赖： ```bash pip install playwright playwright install chromium ``` ### 问题: 浏览器无法启动 **解决方案**: 安装系统依赖： ```bash sudo apt install -y libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 \ libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \ libxfixes3 libxrandr2 libgbm1 libasound2 ``` ### 问题: 图片下载失败 **解决方案**: 检查网络连接，或使用 `--no-images` 跳过图片下载。 ### 问题: 页面加载超时 **解决方案**: 增加超时时间： ```bash python3 skills/wechat-article-fetcher/scripts/fetch.py \ <url> --timeout 90000 ``` ## 相关资源 - [Playwright 官方文档](https://playwright.dev/python/) - [微信公众号开放平台](https://open.weixin.qq.com/) ## 更新日志 ### v1.0.0 - 初始版本 - 支持单篇文章爬取 - 支持图片下载 - 支持 JSON 导出 - 支持完整页面截图 - 支持虚拟环境 - 提供命令行工具 - 提供批量爬取工具

path: README_QUICKSTART.md
content: # WeChat Article Fetcher - 快速使用指南

## 📦 Skill 结构

```
skills/wechat-article-fetcher/
├── SKILL.md                    # 完整文档
├── README_QUICKSTART.md         # 本文件
├── scripts/                     # 工具脚本
│   ├── fetch.py                # 单篇文章爬取（主工具）
│   ├── batch_fetch.py          # 批量爬取工具
│   └── run_in_venv.py         # 虚拟环境运行工具
└── examples/                    # 示例脚本
    └── fetch_wechat_article.py # 示例脚本
```

## 🚀 快速开始

### 1. 爬取单篇文章（最简单）

```bash
# 基本用法
python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw

# 指定输出目录
python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \
  -o ./my_articles
```

### 2. 在虚拟环境中运行（推荐）

```bash
# 使用虚拟环境运行
python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \
  --venv /path/to/playwright-env
```

### 3. 批量爬取

创建 `urls.txt`：
```
https://mp.weixin.qq.com/s/xxx
https://mp.weixin.qq.com/s/yyy
https://mp.weixin.qq.com/s/zzz
```

然后运行：
```bash
# 从文件批量爬取
python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \
  --urls-file urls.txt

# 或者在虚拟环境中批量爬取
python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \
  --urls-file urls.txt --venv /path/to/playwright-env --batch
```

## 📝 命令详解

### fetch.py - 单篇文章爬取

```bash
python3 skills/wechat-article-fetcher/scripts/fetch.py [OPTIONS] URL

必需参数:
  URL                   微信公众号文章 URL

选项:
  -o, --output PATH     输出目录（默认: ./wechat_articles）
  --no-images           不保存图片
  --no-screenshot       不保存截图
  --headless BOOLEAN    无头模式（默认: true）
  --timeout INTEGER     超时时间（毫秒，默认: 60000）
  -h, --help            显示帮助信息
```

### batch_fetch.py - 批量爬取

```bash
python3 skills/wechat-article-fetcher/scripts/batch_fetch.py [OPTIONS]

必需参数（二选一）:
  --urls URL1 URL2...   直接指定多个 URL
  --urls-file FILE      从文件读取 URL 列表

选项:
  -o, --output PATH     输出目录（默认: ./wechat_articles）
  --no-images           不保存图片
  --no-screenshot       不保存截图
  --headless BOOLEAN    无头模式（默认: true）
  --timeout INTEGER     超时时间（毫秒，默认: 60000）
  --delay FLOAT         每次爬取之间的延迟（秒，默认: 2.0）
  -h, --help            显示帮助信息
```

### run_in_venv.py - 虚拟环境运行

```bash
python3 skills/wechat-article-fetcher/scripts/run_in_venv.py [OPTIONS] [URL_OR_ARGS...]

必需参数:
  --venv, -v PATH       虚拟环境路径

选项:
  --batch               使用批量爬取模式
  --script, -s PATH     指定要运行的脚本路径
  --script-args ...     传递给脚本的其他参数
  -h, --help            显示帮助信息
```

## 📁 输出目录结构

每次爬取会创建一个以时间戳命名的目录：

```
wechat_articles/
└── 20260302_125500/
    ├── article.json          # JSON 格式结果
    ├── article.png           # 完整页面截图
    └── images/               # 文章图片目录
        ├── 001_image1.jpg
        ├── 002_image2.png
        └── ...
```

## 💡 使用示例

### 示例 1: 快速爬取一篇文章

```bash
python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw
```

### 示例 2: 只保存文字，不保存图片

```bash
python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \
  --no-images
```

### 示例 3: 使用虚拟环境运行

```bash
python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \
  --venv /home/liuzln/playwright-env
```

### 示例 4: 批量爬取 10 篇文章

创建 `my_urls.txt`：
```
https://mp.weixin.qq.com/s/xxx1
https://mp.weixin.qq.com/s/xxx2
...
https://mp.weixin.qq.com/s/xxx10
```

然后运行：
```bash
python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \
  --urls-file my_urls.txt \
  --delay 3
```

## 🔧 配置说明

所有脚本都是通用的，可以处理任意微信公众号文章 URL，不需要修改代码！

## 📚 更多文档

详细文档请查看：`skills/wechat-article-fetcher/SKILL.md`


path: __init__.py
content: #!/usr/bin/env python3
"""
微信公众号文章爬取 - 简单调用接口

可以被直接调用的简单接口，用于快速爬取微信公众号文章。
"""

import sys
import json
import asyncio
from pathlib import Path

# 添加模块路径
skill_dir = Path(__file__).parent
sys.path.insert(0, str(skill_dir))


def fetch(url: str, venv_path: str = None, output_dir: str = "./wechat_articles"):
    """
    爬取微信公众号文章（简单接口）
    
    Args:
        url: 文章 URL
        venv_path: 虚拟环境路径（可选）
        output_dir: 输出目录
    
    Returns:
        爬取结果字典
    """
    if venv_path:
        # 使用虚拟环境方式
        from wechat_article_fetcher import fetch_article_with_venv
        return fetch_article_with_venv(
            url=url,
            venv_path=venv_path,
            output_dir=output_dir
        )
    else:
        # 直接调用方式
        from wechat_article_fetcher import fetch_article_sync
        return fetch_article_sync(
            url=url,
            output_dir=output_dir
        )


def print_summary(result: dict):
    """打印文章摘要"""
    if not result:
        print("❌ 爬取失败")
        return
    
    print("="*80)
    print("📰 微信公众号文章")
    print("="*80)
    print(f"标题: {result['title']}")
    if result.get('author'):
        print(f"作者: {result['author']}")
    if result.get('publish_date'):
        print(f"发布时间: {result['publish_date']}")
    print(f"链接: {result['url']}")
    print(f"内容长度: {result['length']} 字符")
    if result.get('images_count'):
        print(f"图片: {result['images_count']} 张")
    print("="*80)
    print("\n📖 内容摘要:")
    content = result['content']
    print(content[:1000] + "..." if len(content) > 1000 else content)
    print("\n" + "="*80)
    if result.get('base_output_dir'):
        print(f"\n📁 保存位置: {result['base_output_dir']}/{result['article_dir']}/")


if __name__ == "__main__":
    import argparse
    
    parser = argparse.ArgumentParser(description="微信公众号文章快速爬取")
    parser.add_argument("url", help="文章 URL")
    parser.add_argument("--venv", help="虚拟环境路径（可选）")
    parser.add_argument("-o", "--output", default="./wechat_articles", help="输出目录")
    parser.add_argument("--summary", action="store_true", help="只打印摘要")
    
    args = parser.parse_args()
    
    result = fetch(args.url, args.venv, args.output)
    
    if args.summary:
        print_summary(result)
    elif result:
        print(json.dumps(result, ensure_ascii=False, indent=2))


path: examples/fetch_wechat_article.py
content: #!/usr/bin/env python3
"""
微信公众号文章爬取脚本（增强版）

专门用于爬取微信公众号文章内容，支持保存图片。
"""

import asyncio
import json
import logging
import os
import re
from datetime import datetime
from pathlib import Path
from typing import List, Dict, Any
from urllib.parse import urlparse, unquote

try:
    from playwright.async_api import async_playwright, Page
except ImportError:
    print("请先安装 Playwright: pip install playwright")
    print("然后安装浏览器: playwright install chromium")
    exit(1)


# 配置日志
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)


# 配置
CONFIG = {
    "headless": True,
    "timeout": 60000,
    "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
    "url": "https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw",
    "save_images": True,  # 是否保存图片
    "images_dir": "images",  # 图片保存目录
    "image_quality": 90  # 图片质量 (0-100)
}


def sanitize_filename(filename: str) -> str:
    """清理文件名，移除非法字符"""
    filename = re.sub(r'[<>:"/\\|?*]', '_', filename)
    filename = filename.strip()
    if len(filename) > 100:
        filename = filename[:100]
    return filename


async def download_image(page: Page, img_url: str, save_path: Path) -> bool:
    """下载图片"""
    try:
        # 使用 page.request 下载图片
        response = await page.request.get(img_url)
        if response.ok:
            image_data = await response.body()
            with open(save_path, 'wb') as f:
                f.write(image_data)
            logger.info(f"✅ 已保存图片: {save_path}")
            return True
        else:
            logger.warning(f"❌ 下载图片失败 (HTTP {response.status}): {img_url}")
            return False
    except Exception as e:
        logger.warning(f"❌ 下载图片异常: {img_url} - {e}")
        return False


async def extract_and_save_images(page: Page, images_dir: Path) -> List[Dict[str, Any]]:
    """提取并保存文章中的图片"""
    images_info = []
    
    try:
        # 获取所有图片元素
        img_elements = await page.query_selector_all("#js_content img, .rich_media_content img")
        
        if not img_elements:
            logger.info("未找到图片元素")
            return images_info
        
        logger.info(f"找到 {len(img_elements)} 张图片")
        
        for idx, img_element in enumerate(img_elements, 1):
            try:
                # 获取图片 URL
                src = await img_element.get_attribute("src")
                data_src = await img_element.get_attribute("data-src")
                
                # 优先使用 data-src（微信公众号常用）
                img_url = data_src or src
                
                if not img_url:
                    continue
                
                # 确保 URL 完整
                if img_url.startswith("//"):
                    img_url = "https:" + img_url
                elif img_url.startswith("/"):
                    img_url = "https://mp.weixin.qq.com" + img_url
                
                # 获取图片 alt 作为文件名
                alt = await img_element.get_attribute("alt") or f"image_{idx}"
                alt = sanitize_filename(alt)
                
                # 确定文件扩展名
                parsed = urlparse(img_url)
                path = unquote(parsed.path)
                ext = Path(path).suffix.lower()
                if not ext or ext not in ['.jpg', '.jpeg', '.png', '.gif', '.webp']:
                    ext = '.jpg'
                
                # 保存图片
                img_filename = f"{idx:03d}_{alt}{ext}"
                img_path = images_dir / img_filename
                
                success = await download_image(page, img_url, img_path)
                
                images_info.append({
                    "index": idx,
                    "url": img_url,
                    "alt": alt,
                    "filename": img_filename,
                    "success": success
                })
                
            except Exception as e:
                logger.warning(f"处理图片 {idx} 时出错: {e}")
                continue
        
    except Exception as e:
        logger.error(f"提取图片时出错: {e}", exc_info=True)
    
    return images_info


async def fetch_wechat_article(url: str = None):
    """爬取微信公众号文章"""
    if url is None:
        url = CONFIG["url"]
    
    async with async_playwright() as p:
        # 启动浏览器
        browser = await p.chromium.launch(
            headless=CONFIG["headless"],
            args=["--no-sandbox", "--disable-dev-shm-usage"]
        )
        page = await browser.new_page(
            user_agent=CONFIG["user_agent"],
            viewport={"width": 1920, "height": 1080}
        )
        page.set_default_timeout(CONFIG["timeout"])
        
        try:
            logger.info(f"正在访问微信公众号文章: {url}")
            
            # 访问页面
            try:
                await page.goto(url, wait_until="domcontentloaded")
            except Exception as e:
                logger.warning(f"domcontentloaded 超时，尝试 load: {e}")
                await page.goto(url, wait_until="load")
            
            # 等待页面渲染
            await asyncio.sleep(5)
            
            timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
            
            # 创建目录
            screenshots_dir = Path("screenshots")
            results_dir = Path("results")
            images_dir = Path(CONFIG["images_dir"]) / timestamp
            
            screenshots_dir.mkdir(exist_ok=True)
            results_dir.mkdir(exist_ok=True)
            if CONFIG["save_images"]:
                images_dir.mkdir(parents=True, exist_ok=True)
            
            # 截图
            try:
                screenshot_file = screenshots_dir / f"wechat_article_{timestamp}.png"
                await page.screenshot(path=str(screenshot_file), full_page=True)
                logger.info(f"已保存完整截图: {screenshot_file}")
            except Exception as e:
                logger.warning(f"截图失败: {e}")
            
            # 提取文章标题
            title = ""
            title_selectors = [
                "#activity-name",
                ".rich_media_title",
                "h1",
                "title"
            ]
            
            for selector in title_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text and len(text.strip()) > 5:
                            title = text.strip()
                            logger.info(f"找到标题 (选择器: {selector}): {title}")
                            break
                except Exception as e:
                    logger.debug(f"选择器 {selector} 失败: {e}")
                    continue
            
            # 提取作者
            author = ""
            author_selectors = [
                "#js_name",
                ".rich_media_meta_nickname",
                ".profile_nickname"
            ]
            
            for selector in author_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text:
                            author = text.strip()
                            logger.info(f"找到作者 (选择器: {selector}): {author}")
                            break
                except Exception as e:
                    logger.debug(f"选择器 {selector} 失败: {e}")
                    continue
            
            # 提取发布时间
            publish_date = ""
            date_selectors = [
                "#publish_time",
                ".rich_media_meta_text",
                "em#post-date"
            ]
            
            for selector in date_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text:
                            publish_date = text.strip()
                            logger.info(f"找到时间 (选择器: {selector}): {publish_date}")
                            break
                except Exception as e:
                    logger.debug(f"选择器 {selector} 失败: {e}")
                    continue
            
            # 提取文章内容
            content = ""
            content_selectors = [
                "#js_content",
                ".rich_media_content",
                "#img-content"
            ]
            
            for selector in content_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text and len(text) > 100:
                            content = text
                            logger.info(f"找到内容 (选择器: {selector})，长度: {len(content)} 字符")
                            break
                except Exception as e:
                    logger.debug(f"选择器 {selector} 失败: {e}")
                    continue
            
            if not content:
                try:
                    content = await page.inner_text("body")
                    logger.info("使用 body 内容")
                except Exception as e:
                    logger.warning(f"获取 body 失败: {e}")
                    content = "无法提取内容"
            
            # 提取并保存图片
            images_info = []
            if CONFIG["save_images"]:
                logger.info("开始提取图片...")
                images_info = await extract_and_save_images(page, images_dir)
            
            # 构造结果
            result = {
                "title": title,
                "author": author,
                "publish_date": publish_date,
                "url": url,
                "content": content,
                "images": images_info,
                "images_count": len([i for i in images_info if i["success"]]),
                "images_dir": str(images_dir) if CONFIG["save_images"] else None,
                "fetch_time": datetime.now().isoformat(),
                "length": len(content)
            }
            
            # 保存结果
            result_file = results_dir / f"wechat_article_{timestamp}.json"
            with open(result_file, 'w', encoding='utf-8') as f:
                json.dump(result, f, ensure_ascii=False, indent=2)
            logger.info(f"已保存结果: {result_file}")
            
            return result
            
        finally:
            await browser.close()


async def main():
    """主函数"""
    print("="*80)
    print("微信公众号文章爬取脚本（增强版）")
    print("="*80)
    
    url = CONFIG["url"]
    print(f"📄 目标文章: {url}")
    print(f"🖼️  保存图片: {'是' if CONFIG['save_images'] else '否'}")
    if CONFIG['save_images']:
        print(f"📁 图片目录: {CONFIG['images_dir']}/")
    print("="*80)
    
    result = await fetch_wechat_article(url)
    
    if result:
        print("\n" + "="*80)
        print(f"📰 标题: {result['title']}")
        if result['author']:
            print(f"✍️  作者: {result['author']}")
        if result['publish_date']:
            print(f"📅 发布时间: {result['publish_date']}")
        print(f"🔗 链接: {result['url']}")
        print(f"📝 内容长度: {result['length']} 字符")
        if CONFIG['save_images']:
            print(f"🖼️  图片保存: {result['images_count']}/{len(result['images'])} 张")
            print(f"📁 图片目录: {result['images_dir']}")
        print("="*80)
        print("\n📖 内容摘要:")
        print(result['content'][:1000] + "..." if len(result['content']) > 1000 else result['content'])
        print("\n" + "="*80)
        print("\n✅ 爬取完成！")
        print(f"📄 JSON 结果: results/wechat_article_*.json")
        print(f"🖼️  完整截图: screenshots/wechat_article_*.png")
        if CONFIG['save_images']:
            print(f"📷 文章图片: {result['images_dir']}/")
    else:
        print("\n❌ 未能获取文章内容")


if __name__ == "__main__":
    asyncio.run(main())


path: fetch_direct.py
content: #!/usr/bin/env python3
"""
微信公众号文章爬取 - 直接调用版本

配置好虚拟环境路径后，可以直接调用此脚本爬取文章。
"""

import sys
import json
import subprocess
from pathlib import Path


# ================= 配置区域 =================
# 在这里配置你的虚拟环境路径
VENV_PATH = "/opt/playwright-env"  # 已配置好虚拟环境路径
# ===========================================


def fetch_article(url: str, output_dir: str = "./wechat_articles"):
    """
    爬取微信公众号文章
    
    Args:
        url: 文章 URL
        output_dir: 输出目录
    
    Returns:
        爬取结果字典
    """
    # 检查虚拟环境
    venv_python = Path(VENV_PATH) / "bin" / "python"
    if not venv_python.exists():
        venv_python = Path(VENV_PATH) / "Scripts" / "python.exe"
    
    if not venv_python.exists():
        print(f"❌ 找不到虚拟环境的 Python: {VENV_PATH}")
        print("请修改脚本中的 VENV_PATH 配置")
        return None
    
    # 找到脚本路径
    skill_dir = Path(__file__).parent
    fetch_script = skill_dir / "scripts" / "fetch.py"
    
    if not fetch_script.exists():
        print(f"❌ 找不到脚本: {fetch_script}")
        return None
    
    # 构建命令
    cmd = [str(venv_python), str(fetch_script), url, "-o", output_dir]
    
    print("="*80)
    print("📰 微信公众号文章爬取")
    print("="*80)
    print(f"📄 URL: {url}")
    print(f"🐍 虚拟环境: {VENV_PATH}")
    print("="*80)
    print()
    
    # 运行命令
    try:
        result = subprocess.run(
            cmd,
            cwd=str(skill_dir.parent),
            capture_output=False,
            text=True
        )
        
        if result.returncode == 0:
            # 尝试找到最新的结果
            output_path = Path(output_dir)
            if output_path.exists():
                dirs = sorted([d for d in output_path.iterdir() if d.is_dir()], 
                            key=lambda x: x.stat().st_mtime, reverse=True)
                if dirs:
                    latest_dir = dirs[0]
                    json_path = latest_dir / "article.json"
                    if json_path.exists():
                        with open(json_path, 'r', encoding='utf-8') as f:
                            data = json.load(f)
                        
                        print("\n" + "="*80)
                        print("✅ 爬取成功！")
                        print("="*80)
                        print(f"📰 标题: {data['title']}")
                        if data.get('author'):
                            print(f"✍️  作者: {data['author']}")
                        if data.get('publish_date'):
                            print(f"📅 发布时间: {data['publish_date']}")
                        print(f"📝 内容长度: {data['length']} 字符")
                        if data.get('images_count'):
                            print(f"🖼️  图片: {data['images_count']} 张")
                        print("="*80)
                        print("\n📖 内容摘要:")
                        content = data['content']
                        print(content[:1000] + "..." if len(content) > 1000 else content)
                        print("\n" + "="*80)
                        print(f"\n📁 保存位置: {output_dir}/{data['article_dir']}/")
                        
                        return data
        
        return None
        
    except Exception as e:
        print(f"❌ 运行失败: {e}")
        return None


if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("使用方法:")
        print(f"  python3 {sys.argv[0]} <微信公众号文章URL>")
        print()
        print("示例:")
        print(f"  python3 {sys.argv[0]} https://mp.weixin.qq.com/s/xxx")
        sys.exit(1)
    
    url = sys.argv[1]
    fetch_article(url)


path: scripts/batch_fetch.py
content: #!/usr/bin/env python3
"""
微信公众号文章批量爬取工具

支持从文件或命令行读取多个 URL 进行批量爬取。
"""

import argparse
import asyncio
import logging
import sys
from pathlib import Path
from typing import List

# 导入 fetch.py 中的函数
sys.path.insert(0, str(Path(__file__).parent))
from fetch import fetch_article


# 配置日志
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)


async def batch_fetch(
    urls: List[str],
    output_dir: str = "./wechat_articles",
    save_images: bool = True,
    save_screenshot: bool = True,
    headless: bool = True,
    timeout: int = 60000,
    delay: float = 2.0
):
    """
    批量爬取文章
    
    Args:
        urls: URL 列表
        output_dir: 输出目录
        save_images: 是否保存图片
        save_screenshot: 是否保存截图
        headless: 是否无头模式
        timeout: 超时时间
        delay: 每次爬取之间的延迟（秒）
    """
    results = []
    total = len(urls)
    
    print("="*80)
    print(f"批量爬取工具 - 共 {total} 篇文章")
    print("="*80)
    
    for i, url in enumerate(urls, 1):
        print(f"\n[{i}/{total}] 正在处理: {url[:60]}...")
        
        try:
            result = await fetch_article(
                url=url,
                output_dir=output_dir,
                save_images=save_images,
                save_screenshot=save_screenshot,
                headless=headless,
                timeout=timeout
            )
            
            if result:
                results.append({
                    "index": i,
                    "url": url,
                    "title": result['title'],
                    "success": True,
                    "article_dir": result['article_dir']
                })
                print(f"✅ [{i}/{total}] 成功: {result['title'][:50]}...")
            else:
                results.append({
                    "index": i,
                    "url": url,
                    "success": False,
                    "error": "爬取失败"
                })
                print(f"❌ [{i}/{total}] 失败")
        
        except Exception as e:
            logger.error(f"爬取异常: {e}", exc_info=True)
            results.append({
                "index": i,
                "url": url,
                "success": False,
                "error": str(e)
            })
            print(f"❌ [{i}/{total}] 异常: {e}")
        
        # 延迟（除了最后一个）
        if i < total and delay > 0:
            print(f"⏳ 等待 {delay} 秒...")
            await asyncio.sleep(delay)
    
    # 总结
    print("\n" + "="*80)
    print("批量爬取完成")
    print("="*80)
    
    success_count = sum(1 for r in results if r['success'])
    print(f"\n📊 统计: {success_count}/{total} 成功")
    
    if success_count < total:
        print("\n❌ 失败的文章:")
        for r in results:
            if not r['success']:
                print(f"  - [{r['index']}] {r['url'][:60]}...")
                if 'error' in r:
                    print(f"    错误: {r['error']}")
    
    return results


def read_urls_from_file(file_path: str) -> List[str]:
    """从文件中读取 URL"""
    urls = []
    path = Path(file_path)
    
    if not path.exists():
        logger.error(f"文件不存在: {file_path}")
        return urls
    
    with open(path, 'r', encoding='utf-8') as f:
        for line in f:
            line = line.strip()
            if line and not line.startswith('#'):
                urls.append(line)
    
    return urls


def main():
    """主函数"""
    parser = argparse.ArgumentParser(
        description="微信公众号文章批量爬取工具",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
示例:
  # 从命令行指定多个 URL
  python batch_fetch.py --urls "url1" "url2" "url3"
  
  # 从文件中读取 URL 列表
  python batch_fetch.py --urls-file urls.txt
  
  # 不保存图片，减少延迟
  python batch_fetch.py --urls-file urls.txt --no-images --delay 1
        """
    )
    
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument("--urls", nargs="+", help="URL 列表（多个）")
    group.add_argument("--urls-file", help="包含 URL 列表的文件路径")
    
    parser.add_argument("-o", "--output", default="./wechat_articles", help="输出目录 (默认: ./wechat_articles)")
    parser.add_argument("--no-images", action="store_true", help="不保存图片")
    parser.add_argument("--no-screenshot", action="store_true", help="不保存截图")
    parser.add_argument("--headless", type=bool, default=True, help="无头模式 (默认: true)")
    parser.add_argument("--timeout", type=int, default=60000, help="超时时间，毫秒 (默认: 60000)")
    parser.add_argument("--delay", type=float, default=2.0, help="每次爬取之间的延迟，秒 (默认: 2.0)")
    
    args = parser.parse_args()
    
    # 获取 URL 列表
    if args.urls_file:
        urls = read_urls_from_file(args.urls_file)
    else:
        urls = args.urls
    
    if not urls:
        print("❌ 没有找到有效的 URL")
        sys.exit(1)
    
    # 运行
    asyncio.run(batch_fetch(
        urls=urls,
        output_dir=args.output,
        save_images=not args.no_images,
        save_screenshot=not args.no_screenshot,
        headless=args.headless,
        timeout=args.timeout,
        delay=args.delay
    ))


if __name__ == "__main__":
    main()


path: scripts/fetch.py
content: #!/usr/bin/env python3
"""
微信公众号文章爬取工具 - 命令行版本

支持任意微信公众号文章 URL，自动提取内容并保存图片。
"""

import argparse
import asyncio
import json
import logging
import os
import re
import sys
from datetime import datetime
from pathlib import Path
from typing import List, Dict, Any, Optional
from urllib.parse import urlparse, unquote

try:
    from playwright.async_api import async_playwright, Page
except ImportError:
    print("请先安装 Playwright: pip install playwright")
    print("然后安装浏览器: playwright install chromium")
    sys.exit(1)


# 配置日志
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)


def sanitize_filename(filename: str) -> str:
    """清理文件名，移除非法字符"""
    filename = re.sub(r'[<>:"/\\|?*]', '_', filename)
    filename = filename.strip()
    if len(filename) > 100:
        filename = filename[:100]
    return filename


async def download_image(page: Page, img_url: str, save_path: Path) -> bool:
    """下载图片"""
    try:
        response = await page.request.get(img_url)
        if response.ok:
            image_data = await response.body()
            with open(save_path, 'wb') as f:
                f.write(image_data)
            logger.info(f"✅ 已保存图片: {save_path.name}")
            return True
        else:
            logger.warning(f"❌ 下载图片失败 (HTTP {response.status}): {img_url}")
            return False
    except Exception as e:
        logger.warning(f"❌ 下载图片异常: {e}")
        return False


async def extract_and_save_images(page: Page, images_dir: Path) -> List[Dict[str, Any]]:
    """提取并保存文章中的图片"""
    images_info = []
    
    try:
        img_elements = await page.query_selector_all("#js_content img, .rich_media_content img")
        
        if not img_elements:
            logger.info("未找到图片元素")
            return images_info
        
        logger.info(f"找到 {len(img_elements)} 张图片")
        
        for idx, img_element in enumerate(img_elements, 1):
            try:
                src = await img_element.get_attribute("src")
                data_src = await img_element.get_attribute("data-src")
                img_url = data_src or src
                
                if not img_url:
                    continue
                
                if img_url.startswith("//"):
                    img_url = "https:" + img_url
                elif img_url.startswith("/"):
                    img_url = "https://mp.weixin.qq.com" + img_url
                
                alt = await img_element.get_attribute("alt") or f"image_{idx}"
                alt = sanitize_filename(alt)
                
                parsed = urlparse(img_url)
                path = unquote(parsed.path)
                ext = Path(path).suffix.lower()
                if not ext or ext not in ['.jpg', '.jpeg', '.png', '.gif', '.webp']:
                    ext = '.jpg'
                
                img_filename = f"{idx:03d}_{alt}{ext}"
                img_path = images_dir / img_filename
                
                success = await download_image(page, img_url, img_path)
                
                images_info.append({
                    "index": idx,
                    "url": img_url,
                    "alt": alt,
                    "filename": img_filename,
                    "success": success
                })
                
            except Exception as e:
                logger.warning(f"处理图片 {idx} 时出错: {e}")
                continue
        
    except Exception as e:
        logger.error(f"提取图片时出错: {e}", exc_info=True)
    
    return images_info


async def fetch_article(
    url: str,
    output_dir: str = "./wechat_articles",
    save_images: bool = True,
    save_screenshot: bool = True,
    headless: bool = True,
    timeout: int = 60000
) -> Optional[Dict[str, Any]]:
    """
    爬取微信公众号文章
    
    Args:
        url: 文章 URL
        output_dir: 输出目录
        save_images: 是否保存图片
        save_screenshot: 是否保存截图
        headless: 是否无头模式
        timeout: 超时时间（毫秒）
    
    Returns:
        爬取结果字典，失败返回 None
    """
    # 验证 URL
    if not url or "mp.weixin.qq.com" not in url:
        logger.error("请提供有效的微信公众号文章 URL")
        return None
    
    async with async_playwright() as p:
        # 启动浏览器
        browser = await p.chromium.launch(
            headless=headless,
            args=["--no-sandbox", "--disable-dev-shm-usage"]
        )
        page = await browser.new_page(
            user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
            viewport={"width": 1920, "height": 1080}
        )
        page.set_default_timeout(timeout)
        
        try:
            logger.info(f"正在访问: {url}")
            
            # 访问页面
            try:
                await page.goto(url, wait_until="domcontentloaded")
            except Exception as e:
                logger.warning(f"domcontentloaded 超时，尝试 load: {e}")
                await page.goto(url, wait_until="load")
            
            await asyncio.sleep(5)
            
            # 创建输出目录
            timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
            base_output_dir = Path(output_dir)
            article_dir = base_output_dir / timestamp
            
            base_output_dir.mkdir(parents=True, exist_ok=True)
            article_dir.mkdir(exist_ok=True)
            
            images_dir = article_dir / "images"
            if save_images:
                images_dir.mkdir(exist_ok=True)
            
            # 保存截图
            screenshot_path = None
            if save_screenshot:
                try:
                    screenshot_path = article_dir / "article.png"
                    await page.screenshot(path=str(screenshot_path), full_page=True)
                    logger.info(f"已保存截图: {screenshot_path.name}")
                except Exception as e:
                    logger.warning(f"截图失败: {e}")
            
            # 提取标题
            title = ""
            title_selectors = ["#activity-name", ".rich_media_title", "h1", "title"]
            for selector in title_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text and len(text.strip()) > 5:
                            title = text.strip()
                            break
                except:
                    continue
            
            # 提取作者
            author = ""
            author_selectors = ["#js_name", ".rich_media_meta_nickname", ".profile_nickname"]
            for selector in author_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text:
                            author = text.strip()
                            break
                except:
                    continue
            
            # 提取发布时间
            publish_date = ""
            date_selectors = ["#publish_time", ".rich_media_meta_text", "em#post-date"]
            for selector in date_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text:
                            publish_date = text.strip()
                            break
                except:
                    continue
            
            # 提取内容
            content = ""
            content_selectors = ["#js_content", ".rich_media_content", "#img-content"]
            for selector in content_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text and len(text) > 100:
                            content = text
                            break
                except:
                    continue
            
            if not content:
                try:
                    content = await page.inner_text("body")
                except Exception as e:
                    logger.warning(f"获取 body 失败: {e}")
                    content = "无法提取内容"
            
            # 提取并保存图片
            images_info = []
            if save_images:
                logger.info("开始提取图片...")
                images_info = await extract_and_save_images(page, images_dir)
            
            # 构造结果
            result = {
                "title": title,
                "author": author,
                "publish_date": publish_date,
                "url": url,
                "content": content,
                "images": images_info,
                "images_count": len([i for i in images_info if i["success"]]),
                "images_dir": str(images_dir.relative_to(base_output_dir)) if save_images else None,
                "screenshot": str(screenshot_path.relative_to(base_output_dir)) if screenshot_path else None,
                "article_dir": str(article_dir.relative_to(base_output_dir)),
                "fetch_time": datetime.now().isoformat(),
                "length": len(content)
            }
            
            # 保存 JSON
            json_path = article_dir / "article.json"
            with open(json_path, 'w', encoding='utf-8') as f:
                json.dump(result, f, ensure_ascii=False, indent=2)
            logger.info(f"已保存结果: {json_path.name}")
            
            return result
            
        finally:
            await browser.close()


def main():
    """主函数"""
    parser = argparse.ArgumentParser(
        description="微信公众号文章爬取工具",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
示例:
  # 基本用法
  python fetch.py https://mp.weixin.qq.com/s/xxx
  
  # 指定输出目录
  python fetch.py https://mp.weixin.qq.com/s/xxx -o ./my_articles
  
  # 不保存图片
  python fetch.py https://mp.weixin.qq.com/s/xxx --no-images
  
  # 不保存截图
  python fetch.py https://mp.weixin.qq.com/s/xxx --no-screenshot
        """
    )
    
    parser.add_argument("url", help="微信公众号文章 URL")
    parser.add_argument("-o", "--output", default="./wechat_articles", help="输出目录 (默认: ./wechat_articles)")
    parser.add_argument("--no-images", action="store_true", help="不保存图片")
    parser.add_argument("--no-screenshot", action="store_true", help="不保存截图")
    parser.add_argument("--headless", type=bool, default=True, help="无头模式 (默认: true)")
    parser.add_argument("--timeout", type=int, default=60000, help="超时时间，毫秒 (默认: 60000)")
    
    args = parser.parse_args()
    
    print("="*80)
    print("微信公众号文章爬取工具")
    print("="*80)
    print(f"📄 URL: {args.url}")
    print(f"📁 输出目录: {args.output}")
    print(f"🖼️  保存图片: {'否' if args.no_images else '是'}")
    print(f"📸 保存截图: {'否' if args.no_screenshot else '是'}")
    print("="*80)
    
    # 运行
    result = asyncio.run(fetch_article(
        url=args.url,
        output_dir=args.output,
        save_images=not args.no_images,
        save_screenshot=not args.no_screenshot,
        headless=args.headless,
        timeout=args.timeout
    ))
    
    if result:
        print("\n" + "="*80)
        print(f"📰 标题: {result['title']}")
        if result['author']:
            print(f"✍️  作者: {result['author']}")
        if result['publish_date']:
            print(f"📅 发布时间: {result['publish_date']}")
        print(f"📝 内容长度: {result['length']} 字符")
        if not args.no_images:
            print(f"🖼️  图片保存: {result['images_count']}/{len(result['images'])} 张")
        print("="*80)
        print("\n📖 内容摘要:")
        print(result['content'][:800] + "..." if len(result['content']) > 800 else result['content'])
        print("\n" + "="*80)
        print(f"\n✅ 爬取完成！")
        print(f"📁 文章目录: {args.output}/{result['article_dir']}/")
        print(f"📄 JSON 结果: {args.output}/{result['article_dir']}/article.json")
        if not args.no_screenshot and result['screenshot']:
            print(f"📸 完整截图: {args.output}/{result['screenshot']}")
        if not args.no_images and result['images_dir']:
            print(f"📷 文章图片: {args.output}/{result['images_dir']}/")
    else:
        print("\n❌ 爬取失败")
        sys.exit(1)


if __name__ == "__main__":
    main()


path: scripts/run_in_venv.py
content: #!/usr/bin/env python3
"""
在虚拟环境中运行微信公众号文章爬取工具
"""

import argparse
import os
import sys
import subprocess
from pathlib import Path


def main():
    parser = argparse.ArgumentParser(
        description="在虚拟环境中运行微信公众号文章爬取工具",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
示例:
  # 爬取单篇文章
  python run_in_venv.py https://mp.weixin.qq.com/s/xxx --venv /path/to/playwright-env
  
  # 批量爬取
  python run_in_venv.py --urls-file urls.txt --venv /path/to/playwright-env --batch
        """
    )
    
    parser.add_argument("url_or_args", nargs="*", help="文章 URL 或批量参数")
    parser.add_argument("--venv", "-v", required=True, help="虚拟环境路径")
    parser.add_argument("--batch", action="store_true", help="使用批量爬取模式")
    parser.add_argument("--script", "-s", help="要运行的脚本路径（可选）")
    parser.add_argument("--script-args", nargs=argparse.REMAINDER, help="传递给脚本的参数")
    
    args = parser.parse_args()
    
    # 检查虚拟环境
    venv_path = Path(args.venv)
    if not venv_path.exists():
        print(f"❌ 虚拟环境不存在: {venv_path}")
        sys.exit(1)
    
    # 找到虚拟环境的 Python
    python_path = venv_path / "bin" / "python"
    if not python_path.exists():
        python_path = venv_path / "Scripts" / "python.exe"
    
    if not python_path.exists():
        print(f"❌ 找不到虚拟环境的 Python: {python_path}")
        sys.exit(1)
    
    # 确定要运行的脚本
    script_dir = Path(__file__).parent
    
    if args.script:
        script_path = Path(args.script)
    elif args.batch:
        script_path = script_dir / "batch_fetch.py"
    else:
        script_path = script_dir / "fetch.py"
    
    if not script_path.exists():
        print(f"❌ 脚本不存在: {script_path}")
        sys.exit(1)
    
    # 构建参数
    cmd = [str(python_path), str(script_path)]
    
    if args.url_or_args:
        cmd.extend(args.url_or_args)
    
    if args.script_args:
        cmd.extend(args.script_args)
    
    print("="*80)
    print("微信公众号文章爬取工具（虚拟环境模式）")
    print("="*80)
    print(f"🐍 虚拟环境: {venv_path}")
    print(f"📜 运行脚本: {script_path.name}")
    print(f"🚀 执行命令: {' '.join(cmd)}")
    print("="*80)
    print()
    
    # 运行脚本
    try:
        result = subprocess.run(cmd, cwd=os.getcwd())
        sys.exit(result.returncode)
    except KeyboardInterrupt:
        print("\n⚠️  被用户中断")
        sys.exit(130)


if __name__ == "__main__":
    main()


path: wechat-article-fetcher-SKILL.md
content: ---
name: wechat-article-fetcher
description: 抓取微信公众号文章全文内容与图片，支持JSON导出和整页截图
metadata:
  version: 1.0.1
  author: Liuzln
  category: cli-utilities
---

# 微信公众号文章抓取器

抓取任意微信公众号文章 URL，自动提取标题、作者、发布时间与完整正文，下载并本地保存文章内图片，生成整页截图并导出结构化 JSON。

## 使用方式

命令行工具：
- `fetch.py` — 单篇抓取
- `batch_fetch.py` — 批量抓取
- `run_in_venv.py` — Playwright 虚拟环境兼容运行

## 可选参数
- 无头模式（服务器部署）
- 超时控制
- 不保存图片/截图

## 适用场景
内容归档、离线备份、舆情监测、文本/图像数据采集、NLP 数据准备。

## 注意事项
- Cookie 过期需手动更新
- 建议先用少量文章测试确认正常后再批量抓取


path: wechat_article_fetcher.py
content: #!/usr/bin/env python3
"""
微信公众号文章爬取模块

可以直接导入使用的模块，方便被其他程序调用。
"""

import asyncio
import json
import logging
import os
import re
import sys
from datetime import datetime
from pathlib import Path
from typing import List, Dict, Any, Optional
from urllib.parse import urlparse, unquote


# 配置日志
logger = logging.getLogger(__name__)


def _sanitize_filename(filename: str) -> str:
    """清理文件名，移除非法字符"""
    filename = re.sub(r'[<>:"/\\|?*]', '_', filename)
    filename = filename.strip()
    if len(filename) > 100:
        filename = filename[:100]
    return filename


async def _download_image(page, img_url: str, save_path: Path) -> bool:
    """下载图片"""
    try:
        response = await page.request.get(img_url)
        if response.ok:
            image_data = await response.body()
            with open(save_path, 'wb') as f:
                f.write(image_data)
            logger.info(f"✅ 已保存图片: {save_path.name}")
            return True
        else:
            logger.warning(f"❌ 下载图片失败 (HTTP {response.status}): {img_url}")
            return False
    except Exception as e:
        logger.warning(f"❌ 下载图片异常: {e}")
        return False


async def _extract_and_save_images(page, images_dir: Path) -> List[Dict[str, Any]]:
    """提取并保存文章中的图片"""
    images_info = []
    
    try:
        img_elements = await page.query_selector_all("#js_content img, .rich_media_content img")
        
        if not img_elements:
            logger.info("未找到图片元素")
            return images_info
        
        logger.info(f"找到 {len(img_elements)} 张图片")
        
        for idx, img_element in enumerate(img_elements, 1):
            try:
                src = await img_element.get_attribute("src")
                data_src = await img_element.get_attribute("data-src")
                img_url = data_src or src
                
                if not img_url:
                    continue
                
                if img_url.startswith("//"):
                    img_url = "https:" + img_url
                elif img_url.startswith("/"):
                    img_url = "https://mp.weixin.qq.com" + img_url
                
                alt = await img_element.get_attribute("alt") or f"image_{idx}"
                alt = _sanitize_filename(alt)
                
                parsed = urlparse(img_url)
                path = unquote(parsed.path)
                ext = Path(path).suffix.lower()
                if not ext or ext not in ['.jpg', '.jpeg', '.png', '.gif', '.webp']:
                    ext = '.jpg'
                
                img_filename = f"{idx:03d}_{alt}{ext}"
                img_path = images_dir / img_filename
                
                success = await _download_image(page, img_url, img_path)
                
                images_info.append({
                    "index": idx,
                    "url": img_url,
                    "alt": alt,
                    "filename": img_filename,
                    "success": success
                })
                
            except Exception as e:
                logger.warning(f"处理图片 {idx} 时出错: {e}")
                continue
        
    except Exception as e:
        logger.error(f"提取图片时出错: {e}", exc_info=True)
    
    return images_info


async def fetch_article(
    url: str,
    output_dir: str = "./wechat_articles",
    save_images: bool = True,
    save_screenshot: bool = True,
    headless: bool = True,
    timeout: int = 60000,
    playwright_path: Optional[str] = None
) -> Optional[Dict[str, Any]]:
    """
    爬取微信公众号文章（异步版本）
    
    Args:
        url: 文章 URL
        output_dir: 输出目录
        save_images: 是否保存图片
        save_screenshot: 是否保存截图
        headless: 是否无头模式
        timeout: 超时时间（毫秒）
        playwright_path: Playwright 虚拟环境路径（可选）
    
    Returns:
        爬取结果字典，失败返回 None
    """
    # 验证 URL
    if not url or "mp.weixin.qq.com" not in url:
        logger.error("请提供有效的微信公众号文章 URL")
        return None
    
    # 如果指定了虚拟环境路径，使用该环境的 Playwright
    if playwright_path:
        venv_python = Path(playwright_path) / "bin" / "python"
        if venv_python.exists():
            logger.info(f"使用虚拟环境: {playwright_path}")
            # 注意：这里需要用 subprocess 调用，因为不能在运行时切换解释器
            # 所以这个参数主要是给 run_in_venv 用的
            pass
    
    try:
        from playwright.async_api import async_playwright
    except ImportError:
        logger.error("请先安装 Playwright: pip install playwright")
        logger.error("然后安装浏览器: playwright install chromium")
        return None
    
    async with async_playwright() as p:
        # 启动浏览器
        browser = await p.chromium.launch(
            headless=headless,
            args=["--no-sandbox", "--disable-dev-shm-usage"]
        )
        page = await browser.new_page(
            user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
            viewport={"width": 1920, "height": 1080}
        )
        page.set_default_timeout(timeout)
        
        try:
            logger.info(f"正在访问: {url}")
            
            # 访问页面
            try:
                await page.goto(url, wait_until="domcontentloaded")
            except Exception as e:
                logger.warning(f"domcontentloaded 超时，尝试 load: {e}")
                await page.goto(url, wait_until="load")
            
            await asyncio.sleep(5)
            
            # 创建输出目录
            timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
            base_output_dir = Path(output_dir)
            article_dir = base_output_dir / timestamp
            
            base_output_dir.mkdir(parents=True, exist_ok=True)
            article_dir.mkdir(exist_ok=True)
            
            images_dir = article_dir / "images"
            if save_images:
                images_dir.mkdir(exist_ok=True)
            
            # 保存截图
            screenshot_path = None
            if save_screenshot:
                try:
                    screenshot_path = article_dir / "article.png"
                    await page.screenshot(path=str(screenshot_path), full_page=True)
                    logger.info(f"已保存截图: {screenshot_path.name}")
                except Exception as e:
                    logger.warning(f"截图失败: {e}")
            
            # 提取标题
            title = ""
            title_selectors = ["#activity-name", ".rich_media_title", "h1", "title"]
            for selector in title_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text and len(text.strip()) > 5:
                            title = text.strip()
                            break
                except:
                    continue
            
            # 提取作者
            author = ""
            author_selectors = ["#js_name", ".rich_media_meta_nickname", ".profile_nickname"]
            for selector in author_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text:
                            author = text.strip()
                            break
                except:
                    continue
            
            # 提取发布时间
            publish_date = ""
            date_selectors = ["#publish_time", ".rich_media_meta_text", "em#post-date"]
            for selector in date_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text:
                            publish_date = text.strip()
                            break
                except:
                    continue
            
            # 提取内容
            content = ""
            content_selectors = ["#js_content", ".rich_media_content", "#img-content"]
            for selector in content_selectors:
                try:
                    element = await page.query_selector(selector)
                    if element:
                        text = await element.inner_text()
                        if text and len(text) > 100:
                            content = text
                            break
                except:
                    continue
            
            if not content:
                try:
                    content = await page.inner_text("body")
                except Exception as e:
                    logger.warning(f"获取 body 失败: {e}")
                    content = "无法提取内容"
            
            # 提取并保存图片
            images_info = []
            if save_images:
                logger.info("开始提取图片...")
                images_info = await _extract_and_save_images(page, images_dir)
            
            # 构造结果
            result = {
                "title": title,
                "author": author,
                "publish_date": publish_date,
                "url": url,
                "content": content,
                "images": images_info,
                "images_count": len([i for i in images_info if i["success"]]),
                "images_dir": str(images_dir.relative_to(base_output_dir)) if save_images else None,
                "screenshot": str(screenshot_path.relative_to(base_output_dir)) if screenshot_path else None,
                "article_dir": str(article_dir.relative_to(base_output_dir)),
                "base_output_dir": str(base_output_dir.absolute()),
                "fetch_time": datetime.now().isoformat(),
                "length": len(content)
            }
            
            # 保存 JSON
            json_path = article_dir / "article.json"
            with open(json_path, 'w', encoding='utf-8') as f:
                json.dump(result, f, ensure_ascii=False, indent=2)
            logger.info(f"已保存结果: {json_path.name}")
            
            return result
            
        finally:
            await browser.close()


def fetch_article_sync(
    url: str,
    output_dir: str = "./wechat_articles",
    save_images: bool = True,
    save_screenshot: bool = True,
    headless: bool = True,
    timeout: int = 60000,
    playwright_path: Optional[str] = None
) -> Optional[Dict[str, Any]]:
    """
    爬取微信公众号文章（同步版本）
    
    这是一个包装函数，可以在非异步代码中直接调用。
    
    Args:
        url: 文章 URL
        output_dir: 输出目录
        save_images: 是否保存图片
        save_screenshot: 是否保存截图
        headless: 是否无头模式
        timeout: 超时时间（毫秒）
        playwright_path: Playwright 虚拟环境路径（可选）
    
    Returns:
        爬取结果字典，失败返回 None
    """
    try:
        # 检查是否已经有事件循环在运行
        loop = asyncio.get_event_loop()
        if loop.is_running():
            # 如果已经在运行，创建一个新任务
            async def _run():
                return await fetch_article(
                    url=url,
                    output_dir=output_dir,
                    save_images=save_images,
                    save_screenshot=save_screenshot,
                    headless=headless,
                    timeout=timeout,
                    playwright_path=playwright_path
                )
            # 注意：在已运行的循环中需要用不同的方式处理
            # 这里简化处理，直接用 run_until_complete
            return asyncio.run(_run())
        else:
            # 如果没有运行的循环，直接运行
            return asyncio.run(fetch_article(
                url=url,
                output_dir=output_dir,
                save_images=save_images,
                save_screenshot=save_screenshot,
                headless=headless,
                timeout=timeout,
                playwright_path=playwright_path
            ))
    except RuntimeError as e:
        # 如果事件循环有问题，尝试新的策略
        if "event loop is closed" in str(e):
            import asyncio
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
            return loop.run_until_complete(fetch_article(
                url=url,
                output_dir=output_dir,
                save_images=save_images,
                save_screenshot=save_screenshot,
                headless=headless,
                timeout=timeout,
                playwright_path=playwright_path
            ))
        else:
            raise


def fetch_article_with_venv(
    url: str,
    venv_path: str,
    output_dir: str = "./wechat_articles",
    save_images: bool = True,
    save_screenshot: bool = True,
    headless: bool = True,
    timeout: int = 60000
) -> Optional[Dict[str, Any]]:
    """
    在指定虚拟环境中爬取微信公众号文章
    
    这个函数会使用 subprocess 调用虚拟环境中的 Python 来运行脚本。
    
    Args:
        url: 文章 URL
        venv_path: 虚拟环境路径
        output_dir: 输出目录
        save_images: 是否保存图片
        save_screenshot: 是否保存截图
        headless: 是否无头模式
        timeout: 超时时间（毫秒）
    
    Returns:
        爬取结果字典，失败返回 None
    """
    import subprocess
    import tempfile
    
    # 检查虚拟环境
    venv_python = Path(venv_path) / "bin" / "python"
    if not venv_python.exists():
        venv_python = Path(venv_path) / "Scripts" / "python.exe"
    
    if not venv_python.exists():
        logger.error(f"找不到虚拟环境的 Python: {venv_path}")
        return None
    
    # 创建一个临时脚本
    script_content = f'''
import asyncio
import sys
import json
from pathlib import Path

# 添加脚本目录到路径
skill_dir = Path(r"{Path(__file__).parent}")
sys.path.insert(0, str(skill_dir / "scripts"))

from fetch import fetch_article

async def main():
    result = await fetch_article(
        url=r"{url}",
        output_dir=r"{output_dir}",
        save_images={save_images},
        save_screenshot={save_screenshot},
        headless={headless},
        timeout={timeout}
    )
    if result:
        print(json.dumps(result, ensure_ascii=False))

if __name__ == "__main__":
    asyncio.run(main())
'''
    
    # 这里简化处理，直接调用现有的 fetch.py 脚本
    script_dir = Path(__file__).parent
    fetch_script = script_dir / "scripts" / "fetch.py"
    
    if not fetch_script.exists():
        logger.error(f"找不到脚本: {fetch_script}")
        return None
    
    # 构建命令
    cmd = [str(venv_python), str(fetch_script), url, "-o", output_dir]
    if not save_images:
        cmd.append("--no-images")
    if not save_screenshot:
        cmd.append("--no-screenshot")
    
    logger.info(f"在虚拟环境中运行: {venv_path}")
    logger.info(f"执行命令: {' '.join(cmd)}")
    
    try:
        # 运行命令
        result = subprocess.run(
            cmd,
            cwd=str(script_dir.parent.parent),
            capture_output=False,
            text=True
        )
        
        # 注意：这里只是运行脚本，不解析输出
        # 实际结果保存在 output_dir 中
        
        if result.returncode == 0:
            logger.info("✅ 爬取完成！")
            # 尝试找到最新的结果
            output_path = Path(output_dir)
            if output_path.exists():
                # 找到最新的目录
                dirs = sorted([d for d in output_path.iterdir() if d.is_dir()], 
                            key=lambda x: x.stat().st_mtime, reverse=True)
                if dirs:
                    latest_dir = dirs[0]
                    json_path = latest_dir / "article.json"
                    if json_path.exists():
                        with open(json_path, 'r', encoding='utf-8') as f:
                            return json.load(f)
        
        return None
        
    except Exception as e:
        logger.error(f"运行失败: {e}", exc_info=True)
        return None


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)