Skill: hwl1413520/competitor-monitor-ai (version 1.0.0)

--- name: competitor-monitor description: 竞品数据定时监控Skill，定时访问指定电商或社交平台页面，抓取点赞、评论、销量等数据，检测数据异常波动（如出现爆款），自动截图并发送提醒通知。适用于竞品分析、市场监控、爆款发现等场景。 license: MIT compatibility: 需要网络访问，需要Python 3.8+环境，需要Playwright或Selenium进行网页抓取 metadata: author: AI Assistant version: "1.0" category: data-monitoring language: zh-CN --- # 竞品数据定时监控 Skill ## 用途 本 Skill 用于定时监控竞品在电商平台或社交平台的数据表现，帮助用户： - 自动抓取竞品的点赞、评论、销量等关键数据 - 追踪数据变化趋势，生成可视化报表 - 检测数据异常波动（如出现爆款、数据暴增） - 自动截图保存关键页面 - 及时发送提醒通知 ## 适用平台 ### 电商平台 - **淘宝/天猫**：商品销量、评价数、价格 - **京东**：商品销量、评价数、价格 - **拼多多**：销量、评价、价格 - **抖音电商**：销量、点赞、评论 - **小红书电商**：销量、点赞、收藏 ### 社交平台 - **抖音**：视频点赞、评论、转发、播放量 - **小红书**：笔记点赞、收藏、评论 - **微博**：转发、评论、点赞 - **B站**：播放量、点赞、投币、收藏 - **快手**：点赞、评论、播放量 ## 核心功能 ### 1. 定时数据采集 - 支持自定义监控频率（每小时/每天/每周） - 支持多平台同时监控 - 支持批量监控多个竞品链接 ### 2. 数据抓取 - 自动抓取点赞数、评论数、销量等关键指标 - 支持JavaScript渲染页面 - 自动处理反爬机制 ### 3. 异常检测 - 检测数据异常波动（如点赞数暴增） - 识别潜在爆款 - 支持自定义异常阈值 ### 4. 自动截图 - 异常时自动截图保存 - 支持全页面截图 - 截图带时间戳存档 ### 5. 提醒通知 - 支持多种通知方式（邮件、微信、钉钉、飞书） - 支持自定义提醒内容 - 支持异常摘要报告 ## 工作流程 ### 第一步：配置监控任务 创建监控配置文件 `config/monitor_tasks.json`： ```json { "tasks": [ { "id": "task_001", "name": "竞品A-抖音监控", "platform": "douyin", "url": "https://www.douyin.com/video/xxxxx", "schedule": "0 */2 * * *", "metrics": ["likes", "comments", "shares", "views"], "alert_threshold": { "likes": {"increase_percent": 50, "min_value": 1000}, "comments": {"increase_percent": 30, "min_value": 100} } }, { "id": "task_002", "name": "竞品B-小红书监控", "platform": "xiaohongshu", "url": "https://www.xiaohongshu.com/explore/xxxxx", "schedule": "0 */4 * * *", "metrics": ["likes", "collections", "comments"], "alert_threshold": { "likes": {"increase_percent": 100, "min_value": 500} } } ] } ``` ### 第二步：启动监控服务 ```bash # 启动监控服务 python scripts/monitor_service.py --config assets/config/monitor_tasks.json # 后台运行 nohup python scripts/monitor_service.py --config assets/config/monitor_tasks.json > monitor.log 2>&1 & ``` ### 第三步：异常检测与提醒 当检测到数据异常时： 1. 自动截图保存到 `assets/screenshots/` 2. 发送提醒通知 3. 记录异常日志 ## 监控配置说明 ### 配置文件结构 ```json { "tasks": [ { "id": "任务ID", "name": "任务名称", "platform": "平台类型", "url": "监控页面URL", "schedule": "定时规则(cron格式)", "metrics": ["监控指标列表"], "selectors": { "likes": "点赞数CSS选择器", "comments": "评论数CSS选择器" }, "alert_threshold": { "指标名": { "increase_percent": "增长率阈值(%)", "min_value": "最小触发值" } }, "screenshot_on_alert": true, "notification": { "email": ["[REDACTED_SECRET]"], "webhook": "https://oapi.dingtalk.com/robot/send?access_token=xxx" } } ], "global": { "data_retention_days": 30, "screenshot_retention_days": 7, "default_schedule": "0 */6 * * *" } } ``` ### 平台预设配置 Skill内置了常见平台的CSS选择器预设： | 平台 | 点赞 | 评论 | 分享/转发 | 播放量/浏览 | |------|------|------|-----------|-------------| | 抖音 | .like-count | .comment-count | .share-count | .play-count | | 小红书 | .like-count | .comment-count | .collect-count | - | | 微博 | .like-count | .comment-count | .repost-count | .read-count | | B站 | .like-count | .comment-count | .coin-count | .view-count | | 淘宝 | - | .rate-count | - | .sell-count | ### 定时规则（Cron格式） ``` * * * * * │ │ │ │ │ │ │ │ │ └─── 星期 (0-7, 0和7都是周日) │ │ │ └───── 月份 (1-12) │ │ └─────── 日期 (1-31) │ └───────── 小时 (0-23) └─────────── 分钟 (0-59) 示例： 0 */2 * * * # 每2小时 0 9,18 * * * # 每天9点和18点 0 */6 * * 1-5 # 工作日每6小时 ``` ## 异常检测规则 ### 检测类型 1. **增长率检测** - 检测指标增长率是否超过阈值 - 示例：点赞数2小时内增长超过50% 2. **绝对值检测** - 检测指标是否达到某个绝对值 - 示例：评论数超过1000 3. **趋势检测** - 检测数据是否呈现异常趋势 - 示例：连续3次采集数据持续上升 4. **爆款检测** - 综合多指标判断是否为爆款 - 示例：点赞+评论+分享同时暴增 ### 阈值配置示例 ```json { "alert_threshold": { "likes": { "increase_percent": 50, "min_value": 1000, "window": "2h" }, "comments": { "increase_percent": 30, "min_value": 100 }, "views": { "increase_percent": 100, "min_value": 10000 } }, "burst_detection": { "enabled": true, "min_metrics": 2, "threshold": { "likes": 5000, "comments": 500, "shares": 1000 } } } ``` ## 通知方式 ### 邮件通知 ```json { "notification": { "email": { "smtp_server": "smtp.qq.com", "smtp_port": 587, "username": "[REDACTED_SECRET]", "password": "your_auth_code", "to": ["[REDACTED_SECRET]"] } } } ``` ### 钉钉机器人 ```json { "notification": { "dingtalk": { "webhook": "https://oapi.dingtalk.com/robot/send?access_token=xxx", "secret": "your_secret" } } } ``` ### 飞书机器人 ```json { "notification": { "feishu": { "webhook": "https://open.feishu.cn/open-apis/bot/v2/hook/xxx" } } } ``` ### 企业微信 ```json { "notification": { "wecom": { "webhook": "https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxx" } } } ``` ## 数据存储 ### 数据文件结构 ``` assets/data/ ├── raw/ # 原始抓取数据 │ ├── task_001/ │ │ ├── 2024-01-15.json │ │ └── 2024-01-16.json │ └── task_002/ ├── processed/ # 处理后数据 │ └── trends/ └── alerts/ # 异常记录 └── 2024-01-16.json ``` ### 数据格式 ```json { "task_id": "task_001", "timestamp": "2024-01-16T10:00:00", "url": "https://...", "data": { "likes": 15234, "comments": 892, "shares": 3456, "views": 125000 }, "changes": { "likes": {"value": 1234, "percent": 8.8}, "comments": {"value": 89, "percent": 11.1} }, "alert_triggered": false } ``` ## 使用方法 ### 快速开始 1. **复制配置文件** ```bash cp assets/config/monitor_tasks.example.json assets/config/monitor_tasks.json ``` 2. **编辑配置** ```bash vim assets/config/monitor_tasks.json ``` 3. **启动监控** ```bash python scripts/monitor_service.py start ``` ### 命令行工具 ```bash # 添加监控任务 python scripts/monitor_cli.py add --name "竞品A" --url "https://..." --platform douyin # 列出所有任务 python scripts/monitor_cli.py list # 手动执行一次监控 python scripts/monitor_cli.py run --task task_001 # 查看监控数据 python scripts/monitor_cli.py data --task task_001 --days 7 # 生成趋势报告 python scripts/monitor_cli.py report --task task_001 --output report.html # 停止监控服务 python scripts/monitor_service.py stop ``` ### 单次监控任务 ```bash # 单次抓取数据 python scripts/scraper.py --url "https://..." --platform douyin --output data.json # 带截图 python scripts/scraper.py --url "https://..." --platform douyin --screenshot ``` ## 技术实现 ### 依赖安装 ```bash pip install playwright schedule pandas requests playwright install chromium ``` ### 核心模块 | 模块 | 功能 | |------|------| | `scraper.py` | 网页数据抓取 | | `detector.py` | 异常检测算法 | | `notifier.py` | 通知发送 | | `scheduler.py` | 定时任务调度 | | `storage.py` | 数据存储管理 | | `monitor_service.py` | 监控服务主程序 | ## 注意事项 1. **合规性**：请遵守各平台的robots协议和使用条款 2. **频率控制**：避免过于频繁的抓取，建议间隔不小于1小时 3. **反爬处理**：部分平台可能有反爬机制，需要适当调整 4. **数据隐私**：注意保护抓取的数据，不要泄露敏感信息 5. **稳定性**：建议将监控服务部署在稳定的服务器上 ## 文件结构 ``` competitor-monitor/ ├── SKILL.md # Skill主文件 ├── README.md # 使用说明 ├── INSTALL_GUIDE.md # 安装指南 ├── install.sh # 安装脚本 ├── scripts/ │ ├── monitor_service.py # 监控服务主程序 │ ├── monitor_cli.py # 命令行工具 │ ├── scraper.py # 数据抓取模块 │ ├── detector.py # 异常检测模块 │ ├── notifier.py # 通知模块 │ ├── scheduler.py # 定时任务模块 │ ├── storage.py # 数据存储模块 │ └── dashboard.py # 数据可视化 ├── assets/ │ ├── config/ │ │ ├── monitor_tasks.example.json # 配置示例 │ │ └── platforms.json # 平台预设配置 │ ├── screenshots/ # 截图存档 │ └── data/ # 数据存档 └── references/ └── PLATFORM_SELECTORS.md # 平台选择器参考 ``` ## 示例场景 ### 场景1：监控竞品抖音视频 ```json { "id": "douyin_001", "name": "竞品抖音视频监控", "platform": "douyin", "url": "https://www.douyin.com/video/1234567890", "schedule": "0 */2 * * *", "metrics": ["likes", "comments", "shares"], "alert_threshold": { "likes": {"increase_percent": 50, "min_value": 5000} }, "screenshot_on_alert": true } ``` ### 场景2：监控小红书爆款笔记 ```json { "id": "xhs_001", "name": "小红书爆款监控", "platform": "xiaohongshu", "url": "https://www.xiaohongshu.com/explore/abcdef", "schedule": "0 */4 * * *", "metrics": ["likes", "collections", "comments"], "alert_threshold": { "likes": {"increase_percent": 100, "min_value": 1000}, "collections": {"increase_percent": 80, "min_value": 500} }, "burst_detection": { "enabled": true, "min_metrics": 2 } } ``` ### 场景3：监控淘宝商品销量 ```json { "id": "taobao_001", "name": "竞品销量监控", "platform": "taobao", "url": "https://item.taobao.com/item.htm?id=123456", "schedule": "0 9,18 * * *", "metrics": ["sell_count", "rate_count", "price"], "alert_threshold": { "sell_count": {"increase_percent": 30, "min_value": 100} } } ```

path: assets/config/monitor_tasks.example.json
content: {
  "tasks": [
    {
      "id": "douyin_001",
      "name": "竞品抖音视频-爆款监控",
      "platform": "douyin",
      "url": "https://www.douyin.com/video/1234567890",
      "schedule": "0 */2 * * *",
      "metrics": ["likes", "comments", "shares", "views"],
      "selectors": {
        "likes": "[data-e2e=\"like-count\"]",
        "comments": "[data-e2e=\"comment-count\"]",
        "shares": "[data-e2e=\"share-count\"]",
        "views": "[data-e2e=\"play-count\"]"
      },
      "alert_threshold": {
        "likes": {
          "increase_percent": 50,
          "min_value": 5000,
          "window": 3
        },
        "comments": {
          "increase_percent": 30,
          "min_value": 500,
          "window": 3
        },
        "shares": {
          "increase_percent": 40,
          "min_value": 1000,
          "window": 3
        }
      },
      "burst_detection": {
        "enabled": true,
        "min_metrics": 2,
        "threshold": {
          "likes": 10000,
          "comments": 1000,
          "shares": 5000
        }
      },
      "screenshot_on_alert": true,
      "notification": {
        "dingtalk": {
          "webhook": "https://oapi.dingtalk.com/robot/send?access_token=your_token_here",
          "secret": "your_secret_here"
        },
        "email": {
          "smtp_server": "smtp.qq.com",
          "smtp_port": 587,
          "username": "[REDACTED_SECRET]",
          "password": "your_auth_code",
          "to": ["[REDACTED_SECRET]"]
        }
      }
    },
    {
      "id": "xhs_001",
      "name": "小红书笔记-热度监控",
      "platform": "xiaohongshu",
      "url": "https://www.xiaohongshu.com/explore/abcdefghij",
      "schedule": "0 */4 * * *",
      "metrics": ["likes", "collections", "comments"],
      "selectors": {
        "likes": ".like-wrapper .count",
        "collections": ".collect-wrapper .count",
        "comments": ".comment-wrapper .count"
      },
      "alert_threshold": {
        "likes": {
          "increase_percent": 100,
          "min_value": 1000,
          "window": 3
        },
        "collections": {
          "increase_percent": 80,
          "min_value": 500,
          "window": 3
        }
      },
      "burst_detection": {
        "enabled": true,
        "min_metrics": 2,
        "threshold": {
          "likes": 5000,
          "collections": 2000
        }
      },
      "screenshot_on_alert": true,
      "notification": {
        "feishu": {
          "webhook": "https://open.feishu.cn/open-apis/bot/v2/hook/your_webhook_here"
        }
      }
    },
    {
      "id": "weibo_001",
      "name": "微博热搜-话题监控",
      "platform": "weibo",
      "url": "https://weibo.com/1234567890/AbCdEfGhIj",
      "schedule": "0 */1 * * *",
      "metrics": ["likes", "comments", "reposts", "views"],
      "alert_threshold": {
        "likes": {
          "increase_percent": 30,
          "min_value": 10000,
          "window": 2
        },
        "reposts": {
          "increase_percent": 50,
          "min_value": 5000,
          "window": 2
        }
      },
      "screenshot_on_alert": false,
      "notification": {
        "wecom": {
          "webhook": "https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=your_key_here"
        }
      }
    },
    {
      "id": "bilibili_001",
      "name": "B站视频-播放量监控",
      "platform": "bilibili",
      "url": "https://www.bilibili.com/video/BV1xx411c7mD",
      "schedule": "0 */6 * * *",
      "metrics": ["likes", "coins", "collections", "views", "comments"],
      "alert_threshold": {
        "views": {
          "increase_percent": 100,
          "min_value": 50000,
          "window": 3
        },
        "likes": {
          "increase_percent": 50,
          "min_value": 5000,
          "window": 3
        }
      },
      "screenshot_on_alert": true,
      "notification": {
        "email": {
          "smtp_server": "smtp.qq.com",
          "smtp_port": 587,
          "username": "[REDACTED_SECRET]",
          "password": "your_auth_code",
          "to": ["[REDACTED_SECRET]"]
        }
      }
    },
    {
      "id": "taobao_001",
      "name": "淘宝商品-销量监控",
      "platform": "taobao",
      "url": "https://item.taobao.com/item.htm?id=1234567890",
      "schedule": "0 9,18 * * *",
      "metrics": ["sell_count", "rate_count", "price"],
      "selectors": {
        "sell_count": ".sell-count",
        "rate_count": ".rate-count",
        "price": ".tb-rmb-num"
      },
      "alert_threshold": {
        "sell_count": {
          "increase_percent": 30,
          "min_value": 100,
          "window": 2
        }
      },
      "screenshot_on_alert": true,
      "notification": {
        "dingtalk": {
          "webhook": "https://oapi.dingtalk.com/robot/send?access_token=your_token_here"
        }
      }
    }
  ],
  "global": {
    "data_dir": "../assets/data",
    "screenshot_dir": "../assets/screenshots",
    "data_retention_days": 30,
    "screenshot_retention_days": 7,
    "default_schedule": "0 */6 * * *",
    "log_level": "INFO"
  }
}


path: install.sh
content: #!/bin/bash
# 竞品数据定时监控 Skill 安装脚本

set -e

# 颜色输出
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m'

SKILL_NAME="competitor-monitor"
SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"

echo "========================================"
echo "  📊 竞品数据监控 Skill 安装器"
echo "========================================"
echo ""

# 检测Python
if ! command -v python3 &> /dev/null; then
    echo -e "${RED}✗ 未检测到 Python3${NC}"
    exit 1
fi

echo -e "${GREEN}✓${NC} Python3: $(python3 --version)"
echo ""

# 安装Python依赖
echo "📦 安装Python依赖..."
pip3 install playwright schedule requests pandas 2>/dev/null || pip install playwright schedule requests pandas

# 安装浏览器
echo "🌐 安装Chromium浏览器..."
python3 -m playwright install chromium

# 查找Skills目录
detect_skills_dir() {
    local possible_dirs=(
        "$HOME/.claude/skills"
        "$HOME/.config/claude/skills"
    )
    
    for dir in "${possible_dirs[@]}"; do
        if [ -d "$dir" ]; then
            echo "$dir"
            return 0
        fi
    done
    
    echo "$HOME/.claude/skills"
}

TARGET_DIR=${1:-$(detect_skills_dir)}
echo -e "${BLUE}ℹ${NC} 安装目录: $TARGET_DIR"
echo ""

# 确认安装
if [ -d "$TARGET_DIR/$SKILL_NAME" ]; then
    echo -e "${YELLOW}!${NC} Skill 已存在"
    read -p "是否覆盖安装? (y/N): " confirm
    if [[ ! "$confirm" =~ ^[Yy]$ ]]; then
        echo "安装已取消"
        exit 0
    fi
    rm -rf "$TARGET_DIR/$SKILL_NAME"
fi

# 创建目录并复制
echo "📁 创建目录..."
mkdir -p "$TARGET_DIR"

echo "📦 复制 Skill 文件..."
cp -r "$SKILL_DIR" "$TARGET_DIR/"

# 创建数据目录
mkdir -p "$TARGET_DIR/$SKILL_NAME/assets/data"
mkdir -p "$TARGET_DIR/$SKILL_NAME/assets/screenshots"

# 设置权限
echo "🔧 设置权限..."
chmod 755 "$TARGET_DIR/$SKILL_NAME"
chmod -R 755 "$TARGET_DIR/$SKILL_NAME/scripts"

echo ""
echo -e "${GREEN}========================================${NC}"
echo -e "${GREEN}  ✅ 安装成功!${NC}"
echo -e "${GREEN}========================================${NC}"
echo ""
echo "📍 安装位置: $TARGET_DIR/$SKILL_NAME"
echo ""
echo "🚀 快速开始:"
echo ""
echo "  1. 复制配置文件:"
echo "     cp $TARGET_DIR/$SKILL_NAME/assets/config/monitor_tasks.example.json \\"
echo "        $TARGET_DIR/$SKILL_NAME/assets/config/monitor_tasks.json"
echo ""
echo "  2. 编辑配置:"
echo "     vim $TARGET_DIR/$SKILL_NAME/assets/config/monitor_tasks.json"
echo ""
echo "  3. 启动监控:"
echo "     python3 $TARGET_DIR/$SKILL_NAME/scripts/monitor_service.py start"
echo ""
echo "📖 更多信息请查看 README.md"
echo ""


path: README.md
content: # 竞品数据定时监控 Skill

定时访问指定电商或社交平台页面，抓取点赞、评论、销量等数据，检测数据异常波动（如出现爆款），自动截图并发送提醒通知。

## 功能特点

- ⏰ **定时监控**：支持自定义监控频率（每小时/每天/每周）
- 📊 **数据抓取**：自动抓取点赞、评论、销量等关键指标
- 🔍 **异常检测**：检测数据暴增、骤降、持续趋势等异常
- 🔥 **爆款识别**：综合多指标判断是否为爆款
- 📸 **自动截图**：异常时自动截图保存
- 📢 **多平台通知**：支持邮件、钉钉、飞书、企业微信

## 支持平台

| 平台 | 支持指标 |
|------|----------|
| **抖音** | 点赞、评论、分享、播放量 |
| **小红书** | 点赞、收藏、评论 |
| **微博** | 转发、评论、点赞、阅读量 |
| **B站** | 播放量、点赞、投币、收藏、评论 |
| **快手** | 点赞、评论、播放量 |
| **淘宝/天猫** | 销量、评价数、价格 |
| **京东** | 评价数、价格 |
| **拼多多** | 销量、价格 |

## 快速开始

### 1. 安装依赖

```bash
pip install playwright schedule requests pandas
playwright install chromium
```

### 2. 配置监控任务

```bash
# 复制配置文件示例
cp assets/config/monitor_tasks.example.json assets/config/monitor_tasks.json

# 编辑配置
vim assets/config/monitor_tasks.json
```

### 3. 启动监控服务

```bash
# 启动定时监控服务
python scripts/monitor_service.py start --config assets/config/monitor_tasks.json

# 后台运行
nohup python scripts/monitor_service.py start > monitor.log 2>&1 &
```

### 4. 单次测试

```bash
# 执行一次监控任务
python scripts/monitor_service.py run --task douyin_001

# 抓取单个页面
python scripts/scraper.py --url "https://www.douyin.com/video/xxxxx" --platform douyin --screenshot
```

## 配置说明

### 任务配置

```json
{
  "id": "任务唯一ID",
  "name": "任务名称",
  "platform": "平台类型",
  "url": "监控页面URL",
  "schedule": "定时规则(cron格式)",
  "metrics": ["监控指标列表"],
  "alert_threshold": {
    "指标名": {
      "increase_percent": "增长率阈值(%)",
      "min_value": "最小触发值",
      "window": "历史数据窗口"
    }
  },
  "screenshot_on_alert": true,
  "notification": {
    "dingtalk": {...},
    "email": {...}
  }
}
```

### 定时规则（Cron格式）

| 规则 | 说明 |
|------|------|
| `0 */2 * * *` | 每2小时 |
| `0 */4 * * *` | 每4小时 |
| `0 9,18 * * *` | 每天9点和18点 |
| `0 */1 * * *` | 每小时 |
| `0 */6 * * 1-5` | 工作日每6小时 |

### 异常检测阈值

```json
{
  "alert_threshold": {
    "likes": {
      "increase_percent": 50,  // 增长率超过50%
      "min_value": 5000,       // 且当前值超过5000
      "window": 3              // 与最近3次数据比较
    }
  },
  "burst_detection": {
    "enabled": true,
    "min_metrics": 2,  // 至少2个指标同时触发
    "threshold": {
      "likes": 10000,
      "comments": 1000
    }
  }
}
```

## 通知配置

### 钉钉机器人

```json
{
  "notification": {
    "dingtalk": {
      "webhook": "https://oapi.dingtalk.com/robot/send?access_token=xxx",
      "secret": "your_secret"
    }
  }
}
```

### 飞书机器人

```json
{
  "notification": {
    "feishu": {
      "webhook": "https://open.feishu.cn/open-apis/bot/v2/hook/xxx"
    }
  }
}
```

### 企业微信

```json
{
  "notification": {
    "wecom": {
      "webhook": "https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxx"
    }
  }
}
```

### 邮件

```json
{
  "notification": {
    "email": {
      "smtp_server": "smtp.qq.com",
      "smtp_port": 587,
      "username": "[REDACTED_SECRET]",
      "password": "your_auth_code",
      "to": ["[REDACTED_SECRET]"]
    }
  }
}
```

## 命令行工具

```bash
# 启动监控服务
python scripts/monitor_service.py start

# 执行一次监控
python scripts/monitor_service.py run
python scripts/monitor_service.py run --task task_id

# 生成监控报告
python scripts/monitor_service.py report
python scripts/monitor_service.py report --days 7 --output report.md

# 抓取单个页面
python scripts/scraper.py --url "https://..." --platform douyin --screenshot

# 测试异常检测
python scripts/detector.py

# 测试通知
python scripts/notifier.py
```

## 数据存储

```
assets/data/
├── raw/                    # 原始抓取数据
│   ├── task_001/
│   │   ├── 2024-01-15.jsonl
│   │   └── 2024-01-16.jsonl
│   └── task_002/
├── alerts/                 # 异常记录
│   └── 2024-01-16.jsonl
└── screenshots/            # 截图存档
    └── task_001_20240116_120000.png
```

## 异常检测类型

| 类型 | 说明 | 触发条件 |
|------|------|----------|
| **暴增 (SPIKE)** | 数据快速增长 | 增长率超过阈值 |
| **骤降 (DROP)** | 数据快速下降 | 下降率超过阈值 |
| **持续上升 (TREND_UP)** | 连续多次上升 | 连续N次增长 |
| **持续下降 (TREND_DOWN)** | 连续多次下降 | 连续N次下降 |
| **爆款 (BURST)** | 多指标同时爆发 | 多个指标同时超过阈值 |
| **阈值 (THRESHOLD)** | 超过绝对值阈值 | 达到设定的绝对值 |

## 文件结构

```
competitor-monitor/
├── SKILL.md                    # Skill主文件
├── README.md                   # 使用说明
├── scripts/
│   ├── monitor_service.py      # 监控服务主程序
│   ├── scraper.py              # 数据抓取模块
│   ├── detector.py             # 异常检测模块
│   └── notifier.py             # 通知模块
├── assets/
│   ├── config/
│   │   └── monitor_tasks.example.json
│   ├── data/                   # 数据存储
│   └── screenshots/            # 截图存储
└── references/
    └── PLATFORM_SELECTORS.md   # 平台选择器参考
```

## 注意事项

1. **合规性**：请遵守各平台的robots协议和使用条款
2. **频率控制**：避免过于频繁的抓取，建议间隔不小于1小时
3. **反爬处理**：部分平台可能有反爬机制
4. **数据隐私**：注意保护抓取的数据
5. **稳定性**：建议将监控服务部署在稳定的服务器上

## 许可证

MIT License


path: references/PLATFORM_SELECTORS.md
content: # 平台CSS选择器参考

本文档整理了各主流平台的CSS选择器，用于数据抓取。

## 抖音 (Douyin)

### 视频页面

| 数据项 | CSS选择器 |
|--------|-----------|
| 点赞数 | `[data-e2e="like-count"]` |
| 评论数 | `[data-e2e="comment-count"]` |
| 分享数 | `[data-e2e="share-count"]` |
| 播放量 | `[data-e2e="play-count"]` |
| 视频标题 | `[data-e2e="video-title"]` |
| 作者昵称 | `[data-e2e="user-name"]` |

### 用户主页

| 数据项 | CSS选择器 |
|--------|-----------|
| 获赞数 | `.like-num` |
| 关注数 | `.follow-num` |
| 粉丝数 | `.follower-num` |

## 小红书 (Xiaohongshu)

### 笔记页面

| 数据项 | CSS选择器 |
|--------|-----------|
| 点赞数 | `.like-wrapper .count` |
| 收藏数 | `.collect-wrapper .count` |
| 评论数 | `.comment-wrapper .count` |
| 笔记标题 | `.title` |
| 作者昵称 | `.author-name` |

### 用户主页

| 数据项 | CSS选择器 |
|--------|-----------|
| 获赞与收藏 | `.follows-num` |
| 关注数 | `.follow-num` |
| 粉丝数 | `.fans-num` |

## 微博 (Weibo)

### 单条微博

| 数据项 | CSS选择器 |
|--------|-----------|
| 转发数 | `.woo-repost-count` |
| 评论数 | `.woo-comment-count` |
| 点赞数 | `.woo-like-count` |
| 阅读量 | `.woo-read-count` |
| 微博内容 | `.weibo-text` |

### 用户主页

| 数据项 | CSS选择器 |
|--------|-----------|
| 关注数 | `.follow_num` |
| 粉丝数 | `.follower_num` |
| 微博数 | `.WB_frame_b .tab .current` |

## B站 (Bilibili)

### 视频页面

| 数据项 | CSS选择器 |
|--------|-----------|
| 播放量 | `.view-text` |
| 弹幕数 | `.dm-text` |
| 点赞数 | `.video-like .like-text` |
| 投币数 | `.video-coin .coin-text` |
| 收藏数 | `.video-fav .fav-text` |
| 分享数 | `.video-share .share-text` |
| 视频标题 | `.video-title` |
| UP主名称 | `.username` |

### 用户空间

| 数据项 | CSS选择器 |
|--------|-----------|
| 关注数 | `.n-followers` |
| 粉丝数 | `.n-fans` |
| 获赞数 | `.n-like` |

## 快手 (Kuaishou)

### 视频页面

| 数据项 | CSS选择器 |
|--------|-----------|
| 点赞数 | `.like-count` |
| 评论数 | `.comment-count` |
| 分享数 | `.share-count` |
| 播放量 | `.play-count` |
| 视频标题 | `.video-title` |

## 淘宝/天猫 (Taobao/Tmall)

### 商品详情页

| 数据项 | CSS选择器 |
|--------|-----------|
| 商品标题 | `.tb-detail-hd h1` |
| 价格 | `.tb-rmb-num` |
| 月销量 | `.sell-count` |
| 累计评价 | `.rate-count` |
| 店铺名称 | `.shop-name` |
| 店铺评分 | `.shop-score` |

## 京东 (JD)

### 商品详情页

| 数据项 | CSS选择器 |
|--------|-----------|
| 商品名称 | `.sku-name` |
| 京东价 | `.p-price .price` |
| 评价数 | `#comment-count` |
| 好评率 | `.percent-con` |
| 店铺名称 | `.shop-name` |

## 拼多多 (PDD)

### 商品详情页

| 数据项 | CSS选择器 |
|--------|-----------|
| 商品名称 | `.goods-name` |
| 价格 | `.price` |
| 已拼数量 | `.sold-num` |
| 店铺名称 | `.shop-name` |

## 选择器使用技巧

### 1. 多选择器备选

当单一选择器不稳定时，可以使用多个选择器：

```python
selectors = [
    '[data-e2e="like-count"]',
    '.video-count .like-count',
    'span[class*="like"]'
]
```

### 2. 等待元素加载

```python
# Playwright
page.wait_for_selector('[data-e2e="like-count"]', timeout=5000)
```

### 3. 处理动态内容

```python
# 等待页面完全加载
page.wait_for_load_state('networkidle')

# 等待JavaScript渲染完成
page.wait_for_timeout(2000)
```

### 4. 提取数字

```python
import re

def extract_number(text):
    # 处理"1.2万"格式
    match = re.search(r'(\d+\.?\d*)\s*万', text)
    if match:
        return int(float(match.group(1)) * 10000)
    
    # 处理普通数字
    match = re.search(r'\d+', text)
    if match:
        return int(match.group())
    
    return None
```

## 注意事项

1. **选择器可能变化**：平台更新时选择器可能改变，需要定期维护
2. **反爬机制**：频繁抓取可能触发反爬，建议控制频率
3. **动态加载**：部分数据通过JavaScript动态加载，需要等待渲染
4. **登录限制**：部分数据需要登录后才能查看

## 调试方法

### 使用浏览器开发者工具

1. 打开目标页面
2. 按F12打开开发者工具
3. 使用元素选择器工具选中目标元素
4. 右键选择"Copy > Copy selector"获取选择器

### 使用Playwright调试

```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()
    page.goto('https://...')
    
    # 暂停等待调试
    page.pause()
```


path: scripts/detector.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
异常检测模块
用于检测数据异常波动和爆款
"""

import json
import math
from datetime import datetime, timedelta
from typing import Dict, List, Optional, Any
from dataclasses import dataclass
from enum import Enum


class AlertLevel(Enum):
    """告警级别"""
    CRITICAL = "critical"  # 严重
    HIGH = "high"          # 高
    MEDIUM = "medium"      # 中
    LOW = "low"            # 低
    INFO = "info"          # 信息


class AlertType(Enum):
    """告警类型"""
    SPIKE = "spike"                    # 数据暴增
    DROP = "drop"                      # 数据骤降
    TREND_UP = "trend_up"              # 持续上升
    TREND_DOWN = "trend_down"          # 持续下降
    BURST = "burst"                    # 爆款检测
    THRESHOLD = "threshold"            # 阈值触发


@dataclass
class Alert:
    """告警对象"""
    task_id: str
    timestamp: str
    level: AlertLevel
    type: AlertType
    metric: str
    message: str
    details: Dict[str, Any]
    
    def to_dict(self) -> Dict:
        return {
            'task_id': self.task_id,
            'timestamp': self.timestamp,
            'level': self.level.value,
            'type': self.type.value,
            'metric': self.metric,
            'message': self.message,
            'details': self.details
        }


class AnomalyDetector:
    """异常检测器"""
    
    def __init__(self, config: Dict[str, Any] = None):
        """
        初始化检测器
        
        Args:
            config: 检测配置
        """
        self.config = config or {}
        self.history_data = []
        
    def load_history(self, data: List[Dict]):
        """加载历史数据"""
        self.history_data = sorted(data, key=lambda x: x.get('timestamp', ''))
    
    def detect(self, current_data: Dict[str, Any], task_config: Dict) -> List[Alert]:
        """
        检测异常
        
        Args:
            current_data: 当前数据
            task_config: 任务配置
            
        Returns:
            告警列表
        """
        alerts = []
        task_id = current_data.get('task_id', 'unknown')
        timestamp = current_data.get('timestamp', datetime.now().isoformat())
        
        # 获取阈值配置
        thresholds = task_config.get('alert_threshold', {})
        
        # 对每个指标进行检测
        for metric, current_value in current_data.get('data', {}).items():
            if not isinstance(current_value, (int, float)):
                continue
            
            metric_threshold = thresholds.get(metric, {})
            
            # 1. 增长率检测
            spike_alert = self._detect_spike(
                task_id, timestamp, metric, current_value, metric_threshold
            )
            if spike_alert:
                alerts.append(spike_alert)
            
            # 2. 持续趋势检测
            trend_alert = self._detect_trend(
                task_id, timestamp, metric, current_value, metric_threshold
            )
            if trend_alert:
                alerts.append(trend_alert)
            
            # 3. 绝对值阈值检测
            threshold_alert = self._detect_threshold(
                task_id, timestamp, metric, current_value, metric_threshold
            )
            if threshold_alert:
                alerts.append(threshold_alert)
        
        # 4. 爆款检测（综合多个指标）
        burst_alert = self._detect_burst(
            task_id, timestamp, current_data.get('data', {}), task_config
        )
        if burst_alert:
            alerts.append(burst_alert)
        
        return alerts
    
    def _detect_spike(self, task_id: str, timestamp: str, metric: str,
                      current_value: float, threshold: Dict) -> Optional[Alert]:
        """检测数据暴增/骤降"""
        increase_percent = threshold.get('increase_percent', 50)
        decrease_percent = threshold.get('decrease_percent', 50)
        min_value = threshold.get('min_value', 0)
        window = threshold.get('window', 3)  # 默认取最近3条数据
        
        # 获取历史数据
        if len(self.history_data) < window:
            return None
        
        # 计算历史平均值
        recent_data = self.history_data[-window:]
        historical_values = [
            d.get('data', {}).get(metric) 
            for d in recent_data 
            if d.get('data', {}).get(metric) is not None
        ]
        
        if not historical_values:
            return None
        
        avg_value = sum(historical_values) / len(historical_values)
        
        if avg_value == 0:
            return None
        
        change_percent = ((current_value - avg_value) / avg_value) * 100
        
        # 检测暴增
        if change_percent >= increase_percent and current_value >= min_value:
            return Alert(
                task_id=task_id,
                timestamp=timestamp,
                level=AlertLevel.HIGH if change_percent >= 100 else AlertLevel.MEDIUM,
                type=AlertType.SPIKE,
                metric=metric,
                message=f"{metric} 暴增 {change_percent:.1f}%",
                details={
                    'current': current_value,
                    'average': avg_value,
                    'change_percent': change_percent,
                    'threshold': increase_percent
                }
            )
        
        # 检测骤降
        if change_percent <= -decrease_percent:
            return Alert(
                task_id=task_id,
                timestamp=timestamp,
                level=AlertLevel.MEDIUM,
                type=AlertType.DROP,
                metric=metric,
                message=f"{metric} 骤降 {abs(change_percent):.1f}%",
                details={
                    'current': current_value,
                    'average': avg_value,
                    'change_percent': change_percent,
                    'threshold': decrease_percent
                }
            )
        
        return None
    
    def _detect_trend(self, task_id: str, timestamp: str, metric: str,
                      current_value: float, threshold: Dict) -> Optional[Alert]:
        """检测持续趋势"""
        trend_window = threshold.get('trend_window', 5)
        trend_threshold = threshold.get('trend_threshold', 3)  # 连续几次同向变化
        
        if len(self.history_data) < trend_window:
            return None
        
        # 获取最近的趋势窗口数据
        recent_data = self.history_data[-trend_window:] + [{'data': {metric: current_value}}]
        
        values = [
            d.get('data', {}).get(metric) 
            for d in recent_data 
            if d.get('data', {}).get(metric) is not None
        ]
        
        if len(values) < trend_threshold + 1:
            return None
        
        # 计算变化方向
        directions = []
        for i in range(1, len(values)):
            if values[i] > values[i-1]:
                directions.append(1)  # 上升
            elif values[i] < values[i-1]:
                directions.append(-1)  # 下降
            else:
                directions.append(0)  # 持平
        
        # 检测连续上升
        consecutive_up = 0
        for d in directions:
            if d == 1:
                consecutive_up += 1
                if consecutive_up >= trend_threshold:
                    return Alert(
                        task_id=task_id,
                        timestamp=timestamp,
                        level=AlertLevel.LOW,
                        type=AlertType.TREND_UP,
                        metric=metric,
                        message=f"{metric} 连续 {consecutive_up} 次上升",
                        details={
                            'current': current_value,
                            'consecutive_count': consecutive_up,
                            'values': values[-consecutive_up-1:]
                        }
                    )
            else:
                consecutive_up = 0
        
        # 检测连续下降
        consecutive_down = 0
        for d in directions:
            if d == -1:
                consecutive_down += 1
                if consecutive_down >= trend_threshold:
                    return Alert(
                        task_id=task_id,
                        timestamp=timestamp,
                        level=AlertLevel.LOW,
                        type=AlertType.TREND_DOWN,
                        metric=metric,
                        message=f"{metric} 连续 {consecutive_down} 次下降",
                        details={
                            'current': current_value,
                            'consecutive_count': consecutive_down,
                            'values': values[-consecutive_down-1:]
                        }
                    )
            else:
                consecutive_down = 0
        
        return None
    
    def _detect_threshold(self, task_id: str, timestamp: str, metric: str,
                          current_value: float, threshold: Dict) -> Optional[Alert]:
        """检测绝对值阈值"""
        min_threshold = threshold.get('min_threshold')
        max_threshold = threshold.get('max_threshold')
        
        # 检测超过上限
        if max_threshold and current_value >= max_threshold:
            return Alert(
                task_id=task_id,
                timestamp=timestamp,
                level=AlertLevel.MEDIUM,
                type=AlertType.THRESHOLD,
                metric=metric,
                message=f"{metric} 超过阈值 {current_value} >= {max_threshold}",
                details={
                    'current': current_value,
                    'threshold': max_threshold,
                    'type': 'max'
                }
            )
        
        # 检测低于下限
        if min_threshold and current_value <= min_threshold:
            return Alert(
                task_id=task_id,
                timestamp=timestamp,
                level=AlertLevel.LOW,
                type=AlertType.THRESHOLD,
                metric=metric,
                message=f"{metric} 低于阈值 {current_value} <= {min_threshold}",
                details={
                    'current': current_value,
                    'threshold': min_threshold,
                    'type': 'min'
                }
            )
        
        return None
    
    def _detect_burst(self, task_id: str, timestamp: str, data: Dict[str, float],
                      task_config: Dict) -> Optional[Alert]:
        """检测爆款（综合多个指标）"""
        burst_config = task_config.get('burst_detection', {})
        
        if not burst_config.get('enabled', False):
            return None
        
        min_metrics = burst_config.get('min_metrics', 2)  # 至少几个指标同时触发
        thresholds = burst_config.get('threshold', {})
        
        triggered_metrics = []
        
        for metric, threshold_value in thresholds.items():
            current_value = data.get(metric)
            if current_value and current_value >= threshold_value:
                triggered_metrics.append({
                    'metric': metric,
                    'value': current_value,
                    'threshold': threshold_value
                })
        
        if len(triggered_metrics) >= min_metrics:
            return Alert(
                task_id=task_id,
                timestamp=timestamp,
                level=AlertLevel.CRITICAL,
                type=AlertType.BURST,
                metric='multiple',
                message=f"🔥 检测到爆款！{len(triggered_metrics)} 个指标同时爆发",
                details={
                    'triggered_metrics': triggered_metrics,
                    'min_required': min_metrics,
                    'all_data': data
                }
            )
        
        return None
    
    def calculate_statistics(self, metric: str, window: int = 7) -> Dict:
        """计算统计数据"""
        if len(self.history_data) < 2:
            return {}
        
        values = [
            d.get('data', {}).get(metric) 
            for d in self.history_data[-window:]
            if d.get('data', {}).get(metric) is not None
        ]
        
        if not values:
            return {}
        
        n = len(values)
        mean = sum(values) / n
        variance = sum((x - mean) ** 2 for x in values) / n
        std_dev = math.sqrt(variance)
        
        return {
            'count': n,
            'mean': mean,
            'std_dev': std_dev,
            'min': min(values),
            'max': max(values),
            'latest': values[-1]
        }


def main():
    """测试入口"""
    # 模拟历史数据
    history = [
        {'timestamp': '2024-01-15T10:00:00', 'data': {'likes': 1000, 'comments': 100}},
        {'timestamp': '2024-01-15T12:00:00', 'data': {'likes': 1100, 'comments': 110}},
        {'timestamp': '2024-01-15T14:00:00', 'data': {'likes': 1050, 'comments': 105}},
    ]
    
    # 当前数据（模拟暴增）
    current = {
        'task_id': 'test_001',
        'timestamp': '2024-01-15T16:00:00',
        'data': {'likes': 5000, 'comments': 500}
    }
    
    # 任务配置
    config = {
        'alert_threshold': {
            'likes': {'increase_percent': 50, 'min_value': 1000},
            'comments': {'increase_percent': 50, 'min_value': 100}
        },
        'burst_detection': {
            'enabled': True,
            'min_metrics': 2,
            'threshold': {
                'likes': 3000,
                'comments': 300
            }
        }
    }
    
    # 检测
    detector = AnomalyDetector()
    detector.load_history(history)
    alerts = detector.detect(current, config)
    
    # 输出结果
    print(f"检测到 {len(alerts)} 个异常:")
    for alert in alerts:
        print(f"\n[{alert.level.value.upper()}] {alert.type.value}")
        print(f"  指标: {alert.metric}")
        print(f"  消息: {alert.message}")
        print(f"  详情: {json.dumps(alert.details, ensure_ascii=False)}")


if __name__ == '__main__':
    main()


path: scripts/monitor_service.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
监控服务主程序
定时执行监控任务，检测异常并发送通知
"""

import os
import sys
import json
import time
import signal
import argparse
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Any

# 添加脚本目录到路径
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

from scraper import DataScraper
from detector import AnomalyDetector, Alert
from notifier import NotificationManager

# 尝试导入schedule
try:
    import schedule
    HAS_SCHEDULE = True
except ImportError:
    HAS_SCHEDULE = False
    print("警告: 未安装schedule，请运行: pip install schedule")


class MonitorService:
    """监控服务"""
    
    def __init__(self, config_path: str):
        """
        初始化监控服务
        
        Args:
            config_path: 配置文件路径
        """
        self.config_path = config_path
        self.config = self._load_config()
        self.tasks = self.config.get('tasks', [])
        self.global_config = self.config.get('global', {})
        
        # 数据存储目录
        self.data_dir = Path(self.global_config.get('data_dir', '../assets/data'))
        self.data_dir.mkdir(parents=True, exist_ok=True)
        
        # 截图存储目录
        self.screenshot_dir = Path(self.global_config.get('screenshot_dir', '../assets/screenshots'))
        self.screenshot_dir.mkdir(parents=True, exist_ok=True)
        
        # 告警存储目录
        self.alert_dir = self.data_dir / 'alerts'
        self.alert_dir.mkdir(parents=True, exist_ok=True)
        
        # 运行状态
        self.running = False
        
        # 信号处理
        signal.signal(signal.SIGINT, self._signal_handler)
        signal.signal(signal.SIGTERM, self._signal_handler)
    
    def _load_config(self) -> Dict:
        """加载配置文件"""
        with open(self.config_path, 'r', encoding='utf-8') as f:
            return json.load(f)
    
    def _signal_handler(self, signum, frame):
        """信号处理"""
        print(f"\n收到信号 {signum}，正在停止监控服务...")
        self.running = False
    
    def _get_task_data_file(self, task_id: str) -> Path:
        """获取任务数据文件路径"""
        task_dir = self.data_dir / 'raw' / task_id
        task_dir.mkdir(parents=True, exist_ok=True)
        
        today = datetime.now().strftime('%Y-%m-%d')
        return task_dir / f'{today}.jsonl'
    
    def _load_task_history(self, task_id: str, days: int = 7) -> List[Dict]:
        """加载任务历史数据"""
        history = []
        task_dir = self.data_dir / 'raw' / task_id
        
        if not task_dir.exists():
            return history
        
        # 读取最近几天的数据
        for i in range(days):
            date_str = (datetime.now() - __import__('datetime').timedelta(days=i)).strftime('%Y-%m-%d')
            data_file = task_dir / f'{date_str}.jsonl'
            
            if data_file.exists():
                with open(data_file, 'r', encoding='utf-8') as f:
                    for line in f:
                        try:
                            history.append(json.loads(line.strip()))
                        except:
                            pass
        
        return sorted(history, key=lambda x: x.get('timestamp', ''))
    
    def _save_task_data(self, task_id: str, data: Dict):
        """保存任务数据"""
        data_file = self._get_task_data_file(task_id)
        
        with open(data_file, 'a', encoding='utf-8') as f:
            f.write(json.dumps(data, ensure_ascii=False) + '\n')
    
    def _save_alert(self, alert: Alert):
        """保存告警"""
        today = datetime.now().strftime('%Y-%m-%d')
        alert_file = self.alert_dir / f'{today}.jsonl'
        
        with open(alert_file, 'a', encoding='utf-8') as f:
            f.write(json.dumps(alert.to_dict(), ensure_ascii=False) + '\n')
    
    def execute_task(self, task: Dict) -> Dict:
        """
        执行单个监控任务
        
        Args:
            task: 任务配置
            
        Returns:
            执行结果
        """
        task_id = task['id']
        task_name = task['name']
        url = task['url']
        platform = task.get('platform')
        
        print(f"\n[{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}] 执行任务: {task_name}")
        
        result = {
            'task_id': task_id,
            'task_name': task_name,
            'timestamp': datetime.now().isoformat(),
            'success': False,
            'data': {},
            'alerts': [],
            'screenshot': None
        }
        
        try:
            # 抓取数据
            with DataScraper(headless=True) as scraper:
                scrape_result = scraper.scrape(url, platform=platform)
                
                if not scrape_result['success']:
                    print(f"抓取失败: {scrape_result.get('error')}")
                    return result
                
                result['data'] = scrape_result['data']
                result['success'] = True
                
                # 保存数据
                self._save_task_data(task_id, {
                    'task_id': task_id,
                    'timestamp': result['timestamp'],
                    'url': url,
                    'data': result['data']
                })
                
                print(f"抓取成功: {result['data']}")
                
                # 异常检测
                history = self._load_task_history(task_id)
                
                detector = AnomalyDetector()
                detector.load_history(history)
                
                alerts = detector.detect({
                    'task_id': task_id,
                    'timestamp': result['timestamp'],
                    'data': result['data']
                }, task)
                
                result['alerts'] = [a.to_dict() for a in alerts]
                
                # 处理告警
                if alerts:
                    print(f"检测到 {len(alerts)} 个异常")
                    
                    # 是否需要截图
                    screenshot_on_alert = task.get('screenshot_on_alert', True)
                    screenshot_path = None
                    
                    if screenshot_on_alert:
                        timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
                        screenshot_path = str(self.screenshot_dir / f'{task_id}_{timestamp}.png')
                        
                        # 重新访问页面截图
                        scrape_result = scraper.scrape(url, platform=platform, screenshot=True, screenshot_path=screenshot_path)
                        result['screenshot'] = screenshot_path
                        print(f"截图已保存: {screenshot_path}")
                    
                    # 发送通知
                    notification_config = task.get('notification', {})
                    if notification_config:
                        notifier = NotificationManager(notification_config)
                        
                        for alert in alerts:
                            self._save_alert(alert)
                            notifier.send_alert(alert.to_dict(), screenshot_path)
                
        except Exception as e:
            print(f"任务执行失败: {e}")
            result['error'] = str(e)
        
        return result
    
    def run_task(self, task_id: str = None):
        """
        运行任务
        
        Args:
            task_id: 指定任务ID，None则运行所有任务
        """
        if task_id:
            # 运行指定任务
            task = next((t for t in self.tasks if t['id'] == task_id), None)
            if task:
                self.execute_task(task)
            else:
                print(f"未找到任务: {task_id}")
        else:
            # 运行所有任务
            for task in self.tasks:
                self.execute_task(task)
                time.sleep(2)  # 避免请求过快
    
    def schedule_tasks(self):
        """设置定时任务"""
        if not HAS_SCHEDULE:
            print("错误: 未安装schedule模块")
            return
        
        for task in self.tasks:
            schedule_str = task.get('schedule', self.global_config.get('default_schedule', '0 */6 * * *'))
            
            # 解析cron格式（简化版，只支持部分格式）
            parts = schedule_str.split()
            
            if len(parts) == 5:
                minute, hour, day, month, weekday = parts
                
                # 构建schedule任务
                if hour.startswith('*/') and minute == '0':
                    # 每N小时
                    hours = int(hour.replace('*/', ''))
                    schedule.every(hours).hours.do(self.execute_task, task)
                    print(f"已设置定时任务: {task['name']} 每{hours}小时")
                    
                elif ',' in hour and minute == '0':
                    # 指定多个时间点
                    for h in hour.split(','):
                        schedule.every().day.at(f"{h}:00").do(self.execute_task, task)
                    print(f"已设置定时任务: {task['name']} 每天 {hour} 点")
                    
                elif hour == '*' and minute.startswith('*/'):
                    # 每N分钟
                    minutes = int(minute.replace('*/', ''))
                    schedule.every(minutes).minutes.do(self.execute_task, task)
                    print(f"已设置定时任务: {task['name']} 每{minutes}分钟")
                    
                else:
                    # 默认每6小时
                    schedule.every(6).hours.do(self.execute_task, task)
                    print(f"已设置定时任务: {task['name']} 每6小时")
    
    def start(self):
        """启动监控服务"""
        if not HAS_SCHEDULE:
            print("错误: 请先安装schedule模块: pip install schedule")
            return
        
        print("=" * 50)
        print("竞品监控服务启动")
        print("=" * 50)
        print(f"配置文件: {self.config_path}")
        print(f"监控任务数: {len(self.tasks)}")
        print(f"数据存储: {self.data_dir}")
        print(f"截图存储: {self.screenshot_dir}")
        print("=" * 50)
        
        # 设置定时任务
        self.schedule_tasks()
        
        # 立即执行一次
        print("\n立即执行首次监控...")
        self.run_task()
        
        # 启动定时循环
        self.running = True
        print("\n监控服务运行中，按 Ctrl+C 停止...")
        
        while self.running:
            schedule.run_pending()
            time.sleep(1)
        
        print("\n监控服务已停止")
    
    def generate_report(self, days: int = 7, output: str = None):
        """
        生成监控报告
        
        Args:
            days: 统计天数
            output: 输出文件路径
        """
        from datetime import timedelta
        
        # 统计告警
        alert_stats = {
            'critical': 0,
            'high': 0,
            'medium': 0,
            'low': 0,
            'total': 0
        }
        
        alerts = []
        for i in range(days):
            date_str = (datetime.now() - timedelta(days=i)).strftime('%Y-%m-%d')
            alert_file = self.alert_dir / f'{date_str}.jsonl'
            
            if alert_file.exists():
                with open(alert_file, 'r', encoding='utf-8') as f:
                    for line in f:
                        try:
                            alert = json.loads(line.strip())
                            alerts.append(alert)
                            level = alert.get('level', 'info')
                            if level in alert_stats:
                                alert_stats[level] += 1
                            alert_stats['total'] += 1
                        except:
                            pass
        
        # 生成报告
        report = f"""
# 竞品监控报告

**统计时间**: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}  
**统计周期**: 最近 {days} 天

## 告警统计

| 级别 | 数量 |
|------|------|
| 🔴 严重 | {alert_stats['critical']} |
| 🟠 高 | {alert_stats['high']} |
| 🟡 中 | {alert_stats['medium']} |
| 🟢 低 | {alert_stats['low']} |
| **总计** | **{alert_stats['total']}** |

## 最近告警

"""
        
        for alert in sorted(alerts, key=lambda x: x.get('timestamp', ''), reverse=True)[:10]:
            report += f"""
### {alert.get('message', '告警')}

- 时间: {alert.get('timestamp', 'N/A')}
- 任务: {alert.get('task_id', 'N/A')}
- 级别: {alert.get('level', 'N/A')}
- 详情: {json.dumps(alert.get('details', {}), ensure_ascii=False)}

"""
        
        if output:
            with open(output, 'w', encoding='utf-8') as f:
                f.write(report)
            print(f"报告已保存: {output}")
        else:
            print(report)
        
        return report


def main():
    """命令行入口"""
    parser = argparse.ArgumentParser(description='竞品监控服务')
    parser.add_argument('--config', '-c', default='../assets/config/monitor_tasks.json',
                       help='配置文件路径')
    parser.add_argument('action', choices=['start', 'run', 'report'],
                       help='操作: start(启动服务), run(执行一次), report(生成报告)')
    parser.add_argument('--task', '-t', help='指定任务ID')
    parser.add_argument('--days', '-d', type=int, default=7, help='报告统计天数')
    parser.add_argument('--output', '-o', help='报告输出路径')
    
    args = parser.parse_args()
    
    # 确保配置文件路径正确
    config_path = args.config
    if not os.path.isabs(config_path):
        script_dir = os.path.dirname(os.path.abspath(__file__))
        config_path = os.path.join(script_dir, config_path)
    
    service = MonitorService(config_path)
    
    if args.action == 'start':
        service.start()
    elif args.action == 'run':
        service.run_task(args.task)
    elif args.action == 'report':
        service.generate_report(args.days, args.output)


if __name__ == '__main__':
    main()


path: scripts/notifier.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
通知模块
支持多种通知方式：邮件、钉钉、飞书、企业微信
"""

import json
import requests
from typing import Dict, List, Optional, Any
from datetime import datetime


class Notifier:
    """通知器基类"""
    
    def __init__(self, config: Dict[str, Any]):
        self.config = config
    
    def send(self, message: str, **kwargs) -> bool:
        """发送通知"""
        raise NotImplementedError


class EmailNotifier(Notifier):
    """邮件通知"""
    
    def __init__(self, config: Dict[str, Any]):
        super().__init__(config)
        self.smtp_server = config.get('smtp_server', 'smtp.qq.com')
        self.smtp_port = config.get('smtp_port', 587)
        self.username = config.get('username')
        self.[REDACTED_SECRET]'password')
        self.to = config.get('to', [])
    
    def send(self, message: str, subject: str = None, **kwargs) -> bool:
        """发送邮件"""
        try:
            import smtplib
            from email.mime.text import MIMEText
            from email.mime.multipart import MIMEMultipart
            
            if not self.username or not self.[REDACTED_SECRET]"邮件配置不完整")
                return False
            
            msg = MIMEMultipart()
            msg['From'] = self.username
            msg['To'] = ', '.join(self.to)
            msg['Subject'] = subject or '竞品监控告警'
            
            msg.attach(MIMEText(message, 'html', 'utf-8'))
            
            # 添加附件
            if 'attachments' in kwargs:
                from email.mime.base import MIMEBase
                from email import encoders
                
                for attachment_path in kwargs['attachments']:
                    with open(attachment_path, 'rb') as f:
                        part = MIMEBase('application', 'octet-stream')
                        part.set_payload(f.read())
                        encoders.encode_base64(part)
                        part.add_header(
                            'Content-Disposition',
                            f'attachment; filename={attachment_path.split("/")[-1]}'
                        )
                        msg.attach(part)
            
            server = smtplib.SMTP(self.smtp_server, self.smtp_port)
            server.starttls()
            server.login(self.username, self.password)
            server.send_message(msg)
            server.quit()
            
            print(f"邮件已发送至: {self.to}")
            return True
            
        except Exception as e:
            print(f"邮件发送失败: {e}")
            return False


class DingTalkNotifier(Notifier):
    """钉钉机器人通知"""
    
    def __init__(self, config: Dict[str, Any]):
        super().__init__(config)
        self.webhook = config.get('webhook')
        self.[REDACTED_SECRET]'secret')
    
    def _generate_sign(self, timestamp: str) -> str:
        """生成签名"""
        import hmac
        import hashlib
        import base64
        
        string_to_sign = f'{timestamp}\n{self.secret}'
        hmac_code = hmac.new(
            self.secret.encode('utf-8'),
            string_to_sign.encode('utf-8'),
            digestmod=hashlib.sha256
        ).digest()
        return base64.b64encode(hmac_code).decode('utf-8')
    
    def send(self, message: str, title: str = None, **kwargs) -> bool:
        """发送钉钉消息"""
        try:
            if not self.webhook:
                print("钉钉webhook未配置")
                return False
            
            timestamp = str(int(datetime.now().timestamp() * 1000))
            
            url = self.webhook
            if self.secret:
                sign = self._generate_sign(timestamp)
                url = f'{self.webhook}&timestamp={timestamp}&sign={sign}'
            
            # 构建消息
            data = {
                'msgtype': 'markdown',
                'markdown': {
                    'title': title or '竞品监控告警',
                    'text': message
                }
            }
            
            response = requests.post(url, json=data, timeout=30)
            result = response.json()
            
            if result.get('errcode') == 0:
                print("钉钉消息发送成功")
                return True
            else:
                print(f"钉钉消息发送失败: {result}")
                return False
                
        except Exception as e:
            print(f"钉钉消息发送失败: {e}")
            return False


class FeishuNotifier(Notifier):
    """飞书机器人通知"""
    
    def __init__(self, config: Dict[str, Any]):
        super().__init__(config)
        self.webhook = config.get('webhook')
    
    def send(self, message: str, title: str = None, **kwargs) -> bool:
        """发送飞书消息"""
        try:
            if not self.webhook:
                print("飞书webhook未配置")
                return False
            
            data = {
                'msg_type': 'interactive',
                'card': {
                    'header': {
                        'title': {
                            'tag': 'plain_text',
                            'content': title or '竞品监控告警'
                        },
                        'template': 'red'
                    },
                    'elements': [
                        {
                            'tag': 'div',
                            'text': {
                                'tag': 'lark_md',
                                'content': message
                            }
                        }
                    ]
                }
            }
            
            response = requests.post(self.webhook, json=data, timeout=30)
            result = response.json()
            
            if result.get('code') == 0:
                print("飞书消息发送成功")
                return True
            else:
                print(f"飞书消息发送失败: {result}")
                return False
                
        except Exception as e:
            print(f"飞书消息发送失败: {e}")
            return False


class WeComNotifier(Notifier):
    """企业微信机器人通知"""
    
    def __init__(self, config: Dict[str, Any]):
        super().__init__(config)
        self.webhook = config.get('webhook')
    
    def send(self, message: str, title: str = None, **kwargs) -> bool:
        """发送企业微信消息"""
        try:
            if not self.webhook:
                print("企业微信webhook未配置")
                return False
            
            data = {
                'msgtype': 'markdown',
                'markdown': {
                    'content': f"**{title or '竞品监控告警'}**\n\n{message}"
                }
            }
            
            response = requests.post(self.webhook, json=data, timeout=30)
            result = response.json()
            
            if result.get('errcode') == 0:
                print("企业微信消息发送成功")
                return True
            else:
                print(f"企业微信消息发送失败: {result}")
                return False
                
        except Exception as e:
            print(f"企业微信消息发送失败: {e}")
            return False


class NotificationManager:
    """通知管理器"""
    
    def __init__(self, config: Dict[str, Any]):
        """
        初始化通知管理器
        
        Args:
            config: 通知配置
        """
        self.config = config
        self.notifiers = []
        
        # 初始化各通知渠道
        if 'email' in config:
            self.notifiers.append(EmailNotifier(config['email']))
        
        if 'dingtalk' in config:
            self.notifiers.append(DingTalkNotifier(config['dingtalk']))
        
        if 'feishu' in config:
            self.notifiers.append(FeishuNotifier(config['feishu']))
        
        if 'wecom' in config:
            self.notifiers.append(WeComNotifier(config['wecom']))
    
    def send_alert(self, alert: Dict, screenshot_path: str = None) -> List[bool]:
        """
        发送告警通知
        
        Args:
            alert: 告警信息
            screenshot_path: 截图路径
            
        Returns:
            各通知渠道发送结果
        """
        # 构建消息
        message = self._format_alert_message(alert)
        title = f"[{alert.get('level', 'INFO').upper()}] {alert.get('message', '监控告警')}"
        
        results = []
        for notifier in self.notifiers:
            kwargs = {}
            if screenshot_path:
                kwargs['attachments'] = [screenshot_path]
            
            result = notifier.send(message, title=title, **kwargs)
            results.append(result)
        
        return results
    
    def _format_alert_message(self, alert: Dict) -> str:
        """格式化告警消息"""
        level_emoji = {
            'critical': '🔴',
            'high': '🟠',
            'medium': '🟡',
            'low': '🟢',
            'info': '🔵'
        }
        
        level = alert.get('level', 'info')
        emoji = level_emoji.get(level, '⚪')
        
        message = f"""
{emoji} **{alert.get('message', '监控告警')}**

**告警详情：**
- 任务ID: {alert.get('task_id', 'N/A')}
- 告警类型: {alert.get('type', 'N/A')}
- 监控指标: {alert.get('metric', 'N/A')}
- 触发时间: {alert.get('timestamp', 'N/A')}

**数据详情：**
```json
{json.dumps(alert.get('details', {}), ensure_ascii=False, indent=2)}
```
        """
        
        return message.strip()
    
    def send_summary(self, summary: Dict) -> List[bool]:
        """
        发送汇总报告
        
        Args:
            summary: 汇总信息
            
        Returns:
            各通知渠道发送结果
        """
        message = f"""
📊 **竞品监控日报**

**统计时间：** {summary.get('date', 'N/A')}

**监控概况：**
- 监控任务数: {summary.get('total_tasks', 0)}
- 数据采集次数: {summary.get('total_scrapes', 0)}
- 异常告警数: {summary.get('total_alerts', 0)}

**异常分布：**
- 严重: {summary.get('critical_alerts', 0)}
- 高: {summary.get('high_alerts', 0)}
- 中: {summary.get('medium_alerts', 0)}
- 低: {summary.get('low_alerts', 0)}

**热门监控：**
{chr(10).join([f"- {item['task_name']}: {item['alert_count']} 次告警" for item in summary.get('top_tasks', [])])}
        """
        
        results = []
        for notifier in self.notifiers:
            result = notifier.send(message.strip(), title='竞品监控日报')
            results.append(result)
        
        return results


def main():
    """测试入口"""
    # 测试配置
    config = {
        'dingtalk': {
            'webhook': 'https://oapi.dingtalk.com/robot/send?access_token=your_token',
            'secret': 'your_secret'
        }
    }
    
    # 测试告警
    alert = {
        'task_id': 'test_001',
        'level': 'high',
        'type': 'spike',
        'metric': 'likes',
        'message': '点赞数暴增 350%',
        'timestamp': datetime.now().isoformat(),
        'details': {
            'current': 5000,
            'average': 1111,
            'change_percent': 350
        }
    }
    
    # 发送通知
    manager = NotificationManager(config)
    results = manager.send_alert(alert)
    
    print(f"通知发送结果: {results}")


if __name__ == '__main__':
    main()


path: scripts/scraper.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
竞品数据抓取模块
支持多平台数据抓取和截图
"""

import re
import json
import time
import argparse
from datetime import datetime
from pathlib import Path
from typing import Dict, Optional, Any
from urllib.parse import urlparse

# 尝试导入playwright
try:
    from playwright.sync_api import sync_playwright, TimeoutError as PlaywrightTimeout
    HAS_PLAYWRIGHT = True
except ImportError:
    HAS_PLAYWRIGHT = False
    print("警告: 未安装Playwright，请运行: pip install playwright && playwright install chromium")


class DataScraper:
    """数据抓取器"""
    
    # 平台预设选择器
    PLATFORM_SELECTORS = {
        'douyin': {
            'likes': '[data-e2e="like-count"], .video-count .like-count, span[class*="like"]',
            'comments': '[data-e2e="comment-count"], .video-count .comment-count',
            'shares': '[data-e2e="share-count"], .video-count .share-count',
            'views': '[data-e2e="play-count"], .video-count .play-count',
            'title': '.title, .video-title, h1'
        },
        'xiaohongshu': {
            'likes': '.like-wrapper .count, .interaction-item .like-count',
            'collections': '.collect-wrapper .count, .interaction-item .collect-count',
            'comments': '.comment-wrapper .count, .interaction-item .comment-count',
            'title': '.title, .note-title, h1'
        },
        'weibo': {
            'likes': '.woo-like-count, .toolbar .like-count',
            'comments': '.woo-comment-count, .toolbar .comment-count',
            'reposts': '.woo-repost-count, .toolbar .repost-count',
            'views': '.woo-read-count, .toolbar .read-count',
            'title': '.weibo-text, .wbpro-text, .detail-text'
        },
        'bilibili': {
            'likes': '.video-like .like-text, .like span',
            'coins': '.video-coin .coin-text, .coin span',
            'collections': '.video-fav .fav-text, .fav span',
            'views': '.view-text, .play-count',
            'comments': '.reply-item, .comment-count',
            'title': '.video-title, h1.title'
        },
        'kuaishou': {
            'likes': '.like-count, .video-like-count',
            'comments': '.comment-count, .video-comment-count',
            'views': '.play-count, .video-play-count',
            'title': '.video-title, h1'
        },
        'taobao': {
            'sell_count': '.sell-count, [data-spm="sellercount"]',
            'rate_count': '.rate-count, [data-spm="ratecount"]',
            'price': '.price, .tb-rmb-num',
            'title': '.tb-detail-hd h1, .item-title'
        },
        'jd': {
            'sell_count': '#comment-count, .comment-count',
            'price': '.price, .p-price .price',
            'title': '.sku-name, .product-intro h1'
        },
        'pdd': {
            'sell_count': '.sold-num, [data-testid="sold-num"]',
            'price': '.price, [data-testid="price"]',
            'title': '.goods-name, h1'
        }
    }
    
    def __init__(self, headless: bool = True, timeout: int = 30):
        self.headless = headless
        self.timeout = timeout
        self.browser = None
        self.context = None
        self.page = None
        
    def __enter__(self):
        """上下文管理器入口"""
        if not HAS_PLAYWRIGHT:
            raise ImportError("请先安装Playwright: pip install playwright && playwright install chromium")
        
        self.playwright = sync_playwright().start()
        self.browser = self.playwright.chromium.launch(headless=self.headless)
        self.context = self.browser.new_context(
            viewport={'width': 1920, 'height': 1080},
            user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
        )
        self.page = self.context.new_page()
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        """上下文管理器出口"""
        if self.context:
            self.context.close()
        if self.browser:
            self.browser.close()
        if hasattr(self, 'playwright'):
            self.playwright.stop()
    
    def detect_platform(self, url: str) -> Optional[str]:
        """从URL检测平台类型"""
        domain = urlparse(url).netloc.lower()
        
        platform_map = {
            'douyin': 'douyin',
            'iesdouyin': 'douyin',
            'xiaohongshu': 'xiaohongshu',
            'weibo': 'weibo',
            'bilibili': 'bilibili',
            'b23.tv': 'bilibili',
            'kuaishou': 'kuaishou',
            'taobao': 'taobao',
            'tmall': 'taobao',
            'jd': 'jd',
            'pinduoduo': 'pdd',
            'yangkeduo': 'pdd'
        }
        
        for key, platform in platform_map.items():
            if key in domain:
                return platform
        
        return None
    
    def scrape(self, url: str, platform: str = None, screenshot: bool = False,
               screenshot_path: str = None) -> Dict[str, Any]:
        """
        抓取页面数据
        
        Args:
            url: 页面URL
            platform: 平台类型，不指定则自动检测
            screenshot: 是否截图
            screenshot_path: 截图保存路径
            
        Returns:
            抓取结果字典
        """
        result = {
            'url': url,
            'timestamp': datetime.now().isoformat(),
            'platform': platform or self.detect_platform(url),
            'success': False,
            'data': {},
            'screenshot': None,
            'error': None
        }
        
        try:
            # 访问页面
            print(f"正在访问: {url}")
            self.page.goto(url, wait_until='networkidle', timeout=self.timeout * 1000)
            
            # 等待页面加载
            time.sleep(2)
            
            # 获取平台选择器
            platform = result['platform']
            if not platform or platform not in self.PLATFORM_SELECTORS:
                result['error'] = f"不支持的平台或未识别的URL: {url}"
                return result
            
            selectors = self.PLATFORM_SELECTORS[platform]
            
            # 抓取数据
            for metric, selector in selectors.items():
                try:
                    value = self._extract_number(selector)
                    if value is not None:
                        result['data'][metric] = value
                except Exception as e:
                    print(f"抓取 {metric} 失败: {e}")
            
            # 截图
            if screenshot:
                if screenshot_path is None:
                    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
                    screenshot_path = f"screenshot_{timestamp}.png"
                
                self.page.screenshot(path=screenshot_path, full_page=True)
                result['screenshot'] = screenshot_path
                print(f"截图已保存: {screenshot_path}")
            
            result['success'] = True
            print(f"抓取成功: {result['data']}")
            
        except PlaywrightTimeout:
            result['error'] = "页面加载超时"
        except Exception as e:
            result['error'] = str(e)
            print(f"抓取失败: {e}")
        
        return result
    
    def _extract_number(self, selector: str) -> Optional[int]:
        """从页面元素提取数字"""
        # 尝试多个选择器
        selectors = selector.split(',')
        
        for sel in selectors:
            sel = sel.strip()
            try:
                # 等待元素出现
                element = self.page.locator(sel).first
                if element.is_visible():
                    text = element.inner_text()
                    return self._parse_number(text)
            except:
                continue
        
        return None
    
    def _parse_number(self, text: str) -> Optional[int]:
        """解析数字文本"""
        if not text:
            return None
        
        # 移除空白字符
        text = text.strip()
        
        # 匹配数字（支持万、千、百万等单位）
        patterns = [
            r'(\d+\.?\d*)\s*万',  # 1.2万
            r'(\d+\.?\d*)\s*千',  # 1.5千
            r'(\d+\.?\d*)\s*百万',  # 1.5百万
            r'(\d+\.?\d*)\s*亿',  # 1.2亿
            r'(\d+)',  # 普通数字
        ]
        
        for pattern in patterns:
            match = re.search(pattern, text)
            if match:
                num = float(match.group(1))
                
                # 转换单位
                if '万' in text:
                    num *= 10000
                elif '千' in text:
                    num *= 1000
                elif '百万' in text:
                    num *= 1000000
                elif '亿' in text:
                    num *= 100000000
                
                return int(num)
        
        return None
    
    def scrape_batch(self, urls: list, screenshot: bool = False) -> list:
        """批量抓取"""
        results = []
        for url in urls:
            result = self.scrape(url, screenshot=screenshot)
            results.append(result)
            time.sleep(2)  # 避免请求过快
        return results


def main():
    """命令行入口"""
    parser = argparse.ArgumentParser(description='竞品数据抓取工具')
    parser.add_argument('--url', '-u', required=True, help='要抓取的页面URL')
    parser.add_argument('--platform', '-p', help='平台类型（自动检测）')
    parser.add_argument('--screenshot', '-s', action='store_true', help='是否截图')
    parser.add_argument('--output', '-o', help='输出文件路径')
    parser.add_argument('--headless', action='store_true', default=True, help='无头模式')
    
    args = parser.parse_args()
    
    with DataScraper(headless=args.headless) as scraper:
        result = scraper.scrape(
            url=args.url,
            platform=args.platform,
            screenshot=args.screenshot
        )
    
    # 输出结果
    output = json.dumps(result, ensure_ascii=False, indent=2)
    
    if args.output:
        with open(args.output, 'w', encoding='utf-8') as f:
            f.write(output)
        print(f"结果已保存: {args.output}")
    else:
        print(output)


if __name__ == '__main__':
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)