Skill: noah-1106/baidu-hot-monitor (version 1.0.0)

--- name: baidu-hot-cn description: "百度热榜监控 | Baidu Hot Topics Monitor. 获取百度热搜榜、搜索趋势、关键词热度 | Get Baidu trending searches, trends, keyword popularity. 触发词：百度、热搜、baidu." metadata: openclaw: emoji: "🔍" category: "search" tags: ["baidu", "hot", "trending", "china", "search-engine"] requires: bins: ["python3"] --- # 百度热榜监控 | Baidu Hot Topics Monitor 百度搜索热搜榜实时监控，支持搜索趋势分析、关键词热度追踪。 Real-time monitoring of Baidu search hot topics, supporting trend analysis and keyword popularity tracking. --- ## 功能 | Features ### 热榜获取 | Hot List Retrieval - **实时热搜 / Real-time Hot** - 获取当前百度热搜榜 Top 30 | Get current Baidu hot search Top 30 - **搜索指数 / Search Index** - 显示每条热搜的搜索量 | Show search volume for each hot topic - **分类标签 / Category Tags** - 娱乐、科技、社会、体育等 | Entertainment, Tech, Society, Sports, etc. ### 趋势分析 | Trend Analysis - **上升最快 / Rising Fastest** - 搜索量上升最快的关键词 | Keywords with fastest rising search volume - **地域分布 / Regional Distribution** - 不同地区的搜索热度 | Search heat by region - **时间趋势 / Time Trends** - 24小时搜索趋势 | 24-hour search trends ### 关键词监控 | Keyword Monitoring - **关键词追踪 / Keyword Tracking** - 监控特定关键词热度 | Monitor specific keyword popularity - **竞品监控 / Competitor Monitoring** - 监控竞品相关搜索 | Monitor competitor-related searches - **行业趋势 / Industry Trends** - 行业关键词热度变化 | Industry keyword heat changes --- ## 使用方式 | Usage ### 获取热搜榜 | Get Hot Search List ``` 获取百度热搜榜前 10 ``` 返回 / Returns: ```json [ {"rank": 1, "title": "春节档电影票房", "search_count": 1234567, "category": "娱乐"}, {"rank": 2, "title": "AI技术突破", "search_count": 987654, "category": "科技"}, {"rank": 3, "title": "春运最新消息", "search_count": 876543, "category": "社会"} ] ``` ### 关键词搜索 | Keyword Search ``` 查询 "AI" 在百度的搜索热度 ``` ### 趋势分析 | Trend Analysis ``` 分析百度热搜今天的科技类话题 ``` --- ## 数据来源 | Data Sources - 百度热搜榜 API（如可用）| Baidu Hot Search API (if available) - 模拟数据（API 不可用时）| Simulated data (when API unavailable) - 历史数据对比 | Historical data comparison --- ### 热榜列表 | Hot List ``` 🔍 百度热搜榜 Top 10 1. [娱乐] 春节档电影票房 - 123.5万搜索 2. [科技] AI技术突破 - 98.8万搜索 3. [社会] 春运最新消息 - 87.7万搜索 ... ``` ### 趋势报告 | Trend Report ``` 📊 百度热搜趋势分析 今日热点分类： - 娱乐：35% - 科技：28% - 社会：22% - 体育：15% 上升最快：AI技术突破 (+250%) ``` --- ## 数据采集与持久化 | Data Collection & Persistence 新增数据库存储和可视化功能，用于调研数据收集。 New database storage and visualization features for research data collection. ### 快速开始 | Quick Start ```bash # 1. 初始化数据库（首次使用）/ Initialize database (first time) cd scripts python3 init_db.py # 2. 采集数据并保存到数据库 / Collect data and save to database python3 save_to_db.py 50 # 3. 查询数据 / Query data python3 query.py today # 4. 生成 HTML 报告 / Generate HTML report python3 generate_html.py open ../data/index.html ``` ### 新增脚本说明 | New Scripts | 脚本 / Script | 功能 / Function | |--------------|-----------------| | `init_db.py` | 初始化 SQLite 数据库 / Initialize SQLite database | | `save_to_db.py` | 采集热搜并保存到数据库 / Collect hot search and save to DB | | `query.py` | 查询数据库内容 / Query database content | | `generate_html.py` | 生成可视化 HTML 报告 / Generate visual HTML report | ### 使用示例 | Usage Examples ```bash # 采集50条热搜 / Collect 50 hot items cd scripts python3 save_to_db.py 50 # 查看今天的热搜 / View today's hot items python3 query.py today # 查看分类统计 / View category statistics python3 query.py categories # 查看统计 / View statistics python3 query.py stats 7 ``` ### 数据存储位置 | Data Storage Location ``` data/ ├── baidu.db # SQLite 数据库 / Database └── index.html # HTML 报告（生成后）/ HTML report (generated) ``` --- 感谢原作者开源原技能baidu-hot-cn，本技能是根据作者原版技能更新和修改完成。

path: data/index.html
content: <!DOCTYPE html>
<html lang="zh-CN">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>百度热搜 - 数据可视化</title>
    <style>
        * { margin: 0; padding: 0; box-sizing: border-box; }
        body {
            font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "PingFang SC", sans-serif;
            background: #f5f5f5;
            line-height: 1.6;
        }
        
        .header {
            background: linear-gradient(135deg, #2932e1 0%, #1a1f9c 100%);
            color: white;
            padding: 25px 20px;
            position: sticky;
            top: 0;
            z-index: 100;
            box-shadow: 0 2px 8px rgba(0,0,0,0.1);
        }
        .header h1 { font-size: 24px; margin-bottom: 8px; display: flex; align-items: center; gap: 10px; }
        .header h1::before { content: "🔍"; }
        .header .meta { font-size: 13px; opacity: 0.9; }
        
        .filters {
            background: white;
            padding: 20px;
            border-bottom: 1px solid #e8e8e8;
            display: flex;
            gap: 15px;
            flex-wrap: wrap;
            align-items: flex-end;
        }
        
        .filter-group { display: flex; flex-direction: column; gap: 6px; }
        .filter-group label { font-size: 12px; color: #666; font-weight: 500; }
        .filter-group select, .filter-group input {
            padding: 8px 12px; border: 1px solid #ddd; border-radius: 6px;
            font-size: 14px; background: white; min-width: 120px;
        }
        
        .btn { padding: 8px 16px; border: none; border-radius: 6px;
            cursor: pointer; font-size: 14px; background: #f0f0f0; color: #666; }
        .btn:hover { background: #e0e0e0; }
        
        .stats {
            background: #f0f2ff;
            padding: 12px 20px;
            font-size: 13px;
            color: #666;
            border-bottom: 1px solid #e8e8e8;
            display: flex;
            justify-content: space-between;
        }
        .stats .highlight { color: #2932e1; font-weight: 600; }
        
        .content { max-width: 800px; margin: 0 auto; padding: 20px; }
        
        .hot-list { display: flex; flex-direction: column; gap: 10px; }
        
        .hot-item {
            display: flex;
            align-items: center;
            padding: 14px 18px;
            background: white;
            border-radius: 10px;
            box-shadow: 0 1px 4px rgba(0,0,0,0.05);
            transition: transform 0.1s, box-shadow 0.1s;
        }
        
        .hot-item:hover {
            transform: translateY(-2px);
            box-shadow: 0 4px 12px rgba(0,0,0,0.1);
        }
        
        .rank {
            width: 32px;
            height: 32px;
            border-radius: 8px;
            display: flex;
            align-items: center;
            justify-content: center;
            font-weight: 700;
            font-size: 15px;
            margin-right: 14px;
            flex-shrink: 0;
        }
        
        .rank.top1 { background: linear-gradient(135deg, #ff6b6b, #ee5a5a); color: white; }
        .rank.top2 { background: linear-gradient(135deg, #ffa94d, #ff8c42); color: white; }
        .rank.top3 { background: linear-gradient(135deg, #ffd43b, #fcc419); color: #333; }
        .rank.normal { background: #f0f0f0; color: #666; }
        
        .item-content { flex: 1; min-width: 0; }
        
        .item-title {
            font-size: 15px;
            color: #1a1a1a;
            margin-bottom: 4px;
            line-height: 1.5;
            word-break: break-word;
        }
        
        .item-meta { display: flex; align-items: center; gap: 10px; font-size: 12px; }
        
        .category-badge {
            padding: 2px 10px;
            border-radius: 12px;
            font-size: 11px;
            font-weight: 600;
            color: white;
        }
        
        .empty { text-align: center; padding: 80px 20px; color: #999; }
        .empty-icon { font-size: 48px; margin-bottom: 16px; }
        
        @media (max-width: 640px) {
            .content { padding: 12px; }
            .hot-item { padding: 12px 14px; }
        }
    </style>
</head>
<body>
    <div class="header">
        <h1>百度热搜数据可视化</h1>
        <div class="meta">
            更新时间: 2026-03-16 05:12 | 
            共 10 条记录
        </div>
    </div>
    
    <div class="filters">
        <div class="filter-group">
            <label>日期</label>
            <select id="dateSelect">
                <option value="">全部</option>
                <option value="2026-03-15">2026-03-15</option>
            </select>
        </div>
        
        <div class="filter-group">
            <label>分类</label>
            <select id="categorySelect">
                <option value="">全部</option>
                <option value="新">新</option>
                <option value="热">热</option>
                <option value="热议">热议</option>
                <option value="综合">综合</option>
            </select>
        </div>
        
        <div class="filter-group">
            <label>搜索</label>
            <input type="text" id="searchInput" placeholder="输入关键词...">
        </div>
        
        <button class="btn" onclick="resetFilters()">重置</button>
    </div>
    
    <div class="stats">
        <span>显示 <span class="highlight" id="showCount">10</span> / 10 条</span>
    </div>
    
    <div class="content">
        <div class="hot-list" id="hotList"></div>
    </div>

    <script>
        const items = [{"rank": 0, "title": "《推动海洋经济高质量发展》", "category": "综合", "label": "", "url": "https://www.baidu.com/s?wd=%E3%80%8A%E6%8E%A8%E5%8A%A8%E6%B5%B7%E6%B4%8B%E7%BB%8F%E6%B5%8E%E9%AB%98%E8%B4%A8%E9%87%8F%E5%8F%91%E5%B1%95%E3%80%8B", "date": "2026-03-15"}, {"rank": 1, "title": "315晚会", "category": "热议", "label": "热议", "url": "https://www.baidu.com/s?wd=315%E6%99%9A%E4%BC%9A", "date": "2026-03-15"}, {"rank": 2, "title": "315曝光网红鸡爪 生产车间令人作呕", "category": "热议", "label": "热议", "url": "https://www.baidu.com/s?wd=315%E6%9B%9D%E5%85%89%E7%BD%91%E7%BA%A2%E9%B8%A1%E7%88%AA%20%E7%94%9F%E4%BA%A7%E8%BD%A6%E9%97%B4%E4%BB%A4%E4%BA%BA%E4%BD%9C%E5%91%95", "date": "2026-03-15"}, {"rank": 3, "title": "以“315”为契机 护航“365”", "category": "综合", "label": "", "url": "https://www.baidu.com/s?wd=%E4%BB%A5%E2%80%9C315%E2%80%9D%E4%B8%BA%E5%A5%91%E6%9C%BA%20%E6%8A%A4%E8%88%AA%E2%80%9C365%E2%80%9D", "date": "2026-03-15"}, {"rank": 4, "title": "“有友鸡爪”客服回应", "category": "新", "label": "", "url": "https://www.baidu.com/s?wd=%E2%80%9C%E6%9C%89%E5%8F%8B%E9%B8%A1%E7%88%AA%E2%80%9D%E5%AE%A2%E6%9C%8D%E5%9B%9E%E5%BA%94", "date": "2026-03-15"}, {"rank": 5, "title": "哈啰回应被315晚会点名", "category": "热", "label": "", "url": "https://www.baidu.com/s?wd=%E5%93%88%E5%95%B0%E5%9B%9E%E5%BA%94%E8%A2%AB315%E6%99%9A%E4%BC%9A%E7%82%B9%E5%90%8D", "date": "2026-03-15"}, {"rank": 6, "title": "网警提醒看不见的虚假更要防", "category": "综合", "label": "", "url": "https://www.baidu.com/s?wd=%E7%BD%91%E8%AD%A6%E6%8F%90%E9%86%92%E7%9C%8B%E4%B8%8D%E8%A7%81%E7%9A%84%E8%99%9A%E5%81%87%E6%9B%B4%E8%A6%81%E9%98%B2", "date": "2026-03-15"}, {"rank": 7, "title": "315曝光的成都3家企业被责令停产停业", "category": "综合", "label": "", "url": "https://www.baidu.com/s?wd=315%E6%9B%9D%E5%85%89%E7%9A%84%E6%88%90%E9%83%BD3%E5%AE%B6%E4%BC%81%E4%B8%9A%E8%A2%AB%E8%B4%A3%E4%BB%A4%E5%81%9C%E4%BA%A7%E5%81%9C%E4%B8%9A", "date": "2026-03-15"}, {"rank": 8, "title": "美宜佳门店被查获非法卷烟139.99万支", "category": "综合", "label": "", "url": "https://www.baidu.com/s?wd=%E7%BE%8E%E5%AE%9C%E4%BD%B3%E9%97%A8%E5%BA%97%E8%A2%AB%E6%9F%A5%E8%8E%B7%E9%9D%9E%E6%B3%95%E5%8D%B7%E7%83%9F139.99%E4%B8%87%E6%94%AF", "date": "2026-03-15"}, {"rank": 9, "title": "口水娃：鸡爪生产过程未添加双氧水", "category": "新", "label": "", "url": "https://www.baidu.com/s?wd=%E5%8F%A3%E6%B0%B4%E5%A8%83%EF%BC%9A%E9%B8%A1%E7%88%AA%E7%94%9F%E4%BA%A7%E8%BF%87%E7%A8%8B%E6%9C%AA%E6%B7%BB%E5%8A%A0%E5%8F%8C%E6%B0%A7%E6%B0%B4", "date": "2026-03-15"}];
        const categoryColors = {"热": "#ff6b6b", "新": "#4ecdc4", "热议": "#ffa94d", "辟谣": "#a55eea", "综合": "#868e96"};
        
        function getRankClass(rank) {
            if (rank === 1) return 'top1';
            if (rank === 2) return 'top2';
            if (rank === 3) return 'top3';
            return 'normal';
        }
        
        function renderItems() {
            const dateSelect = document.getElementById('dateSelect').value;
            const categorySelect = document.getElementById('categorySelect').value;
            const search = document.getElementById('searchInput').value.toLowerCase().trim();
            
            let filtered = items.filter(item => {
                if (dateSelect && item.date !== dateSelect) return false;
                if (categorySelect && item.category !== categorySelect) return false;
                if (search && !item.title.toLowerCase().includes(search)) return false;
                return true;
            });
            
            document.getElementById('showCount').textContent = filtered.length;
            
            const container = document.getElementById('hotList');
            if (filtered.length === 0) {
                container.innerHTML = `
                    <div class="empty" style="grid-column: 1/-1;">
                        <div class="empty-icon">🔍</div>
                        <div>没有找到记录</div>
                    </div>
                `;
                return;
            }
            
            // 按日期分组
            const grouped = {};
            filtered.forEach(item => {
                if (!grouped[item.date]) grouped[item.date] = [];
                grouped[item.date].push(item);
            });
            
            let html = '';
            Object.keys(grouped).sort().reverse().forEach(date => {
                html += grouped[date].map(item => {
                    const bgColor = categoryColors[item.category] || '#868e96';
                    return `
                        <a href="${item.url}" class="hot-item" target="_blank" style="text-decoration: none; color: inherit;">
                            <div class="rank ${getRankClass(item.rank)}">${item.rank}</div>
                            <div class="item-content">
                                <div class="item-title">${item.title}</div>
                                <div class="item-meta">
                                    <span class="category-badge" style="background: ${bgColor}">${item.category}</span>
                                    <span style="color: #999;">${item.date}</span>
                                </div>
                            </div>
                        </a>
                    `;
                }).join('');
            });
            
            container.innerHTML = html;
        }
        
        function resetFilters() {
            document.getElementById('dateSelect').value = '';
            document.getElementById('categorySelect').value = '';
            document.getElementById('searchInput').value = '';
            renderItems();
        }
        
        document.getElementById('dateSelect').addEventListener('change', renderItems);
        document.getElementById('categorySelect').addEventListener('change', renderItems);
        document.getElementById('searchInput').addEventListener('input', renderItems);
        
        renderItems();
    </script>
</body>
</html>

path: scripts/baidu_hot.py
content: #!/usr/bin/env python3
"""
百度热搜榜获取脚本
Baidu Hot Topics Fetcher - 使用真实API
"""

import json
import sys
import urllib.request
from datetime import datetime

def get_baidu_hot(limit=50):
    """获取百度热搜榜 - 使用真实API"""
    try:
        # 百度热搜实时API
        url = 'https://top.baidu.com/api/board?platform=wise&tab=realtime'
        
        # 设置请求头
        headers = {
            'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36',
            'Accept': 'application/json, text/plain, */*',
            'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
            'Referer': 'https://top.baidu.com/'
        }
        
        req = urllib.request.Request(url, headers=headers)
        
        with urllib.request.urlopen(req, timeout=15) as response:
            data = json.loads(response.read().decode('utf-8'))
            
            # 解析数据结构: data.cards[0].content[0].content
            if not data.get('success') or 'data' not in data:
                raise Exception('API返回数据格式异常')
            
            cards = data['data'].get('cards', [])
            if not cards:
                raise Exception('无法获取热搜卡片数据')
            
            # 提取内容列表 - 需要嵌套解析
            hot_items = []
            for card in cards:
                if 'content' in card and isinstance(card['content'], list):
                    for content_item in card['content']:
                        if 'content' in content_item and isinstance(content_item['content'], list):
                            hot_items.extend(content_item['content'])
            
            if not hot_items:
                raise Exception('热搜内容为空')
            
            # 解析热搜条目
            hot_list = []
            for item in hot_items[:limit]:
                rank = item.get('index', 0)
                title = item.get('word', '').strip()
                hot_tag = item.get('hotTag', '0')  # 0=普通, 1=新, 3=热
                
                # 获取标签
                label = ''
                if item.get('labelTag'):
                    label = item['labelTag'].get('day', {}).get('text', '')
                
                # 热度标记转换
                if label:
                    category = label  # 优先使用标签（热议、辟谣等）
                elif hot_tag == '1':
                    category = '新'
                elif hot_tag == '3':
                    category = '热'
                else:
                    category = '综合'
                
                if title:  # 只添加有标题的条目
                    hot_list.append({
                        'rank': rank,
                        'title': title,
                        'category': category,
                        'search_count': '',
                        'hot_tag': hot_tag,
                        'label': label
                    })
            
            return {
                'status': 'success',
                'data': hot_list,
                'count': len(hot_list),
                'timestamp': datetime.now().isoformat(),
                'source': 'baidu_realtime_api'
            }
        
    except Exception as e:
        return {
            'status': 'error',
            'error': f'获取失败: {str(e)}',
            'data': [],
            'count': 0
        }


def format_output(data):
    """格式化输出"""
    if not data:
        return "❌ 未获取到数据"
    
    output = "🔍 百度热搜榜\n\n"
    for item in data:
        rank = item['rank']
        title = item['title']
        category = item['category']
        
        output += f"{rank}. [{category}] {title}\n"
    
    return output


def main():
    limit = 50
    if len(sys.argv) > 1:
        try:
            limit = int(sys.argv[1])
        except:
            pass
    
    result = get_baidu_hot(limit)
    
    if '--json' in sys.argv:
        print(json.dumps(result, ensure_ascii=False, indent=2))
    else:
        if result['status'] == 'success':
            print(format_output(result["data"]))
        else:
            print(f"❌ 获取失败: {result.get('error', '未知错误')}")
            sys.exit(1)


if __name__ == "__main__":
    main()


path: scripts/db.py
content: #!/usr/bin/env python3
"""
Baidu Hot - Database Operations
百度热搜数据库操作模块
"""

import sqlite3
import json
import hashlib
import urllib.parse
from datetime import datetime, timezone
from pathlib import Path

DB_PATH = Path(__file__).parent.parent / "data" / "baidu.db"

def get_db():
    """获取数据库连接"""
    return sqlite3.connect(DB_PATH)

def generate_id(title, date_str):
    """生成条目ID"""
    return hashlib.sha256((title + date_str).encode()).hexdigest()[:16]

def get_today_str():
    """获取今天的日期字符串"""
    return datetime.now(timezone.utc).strftime('%Y-%m-%d')

def save_hot_items(items):
    """
    保存热搜条目到数据库
    
    Args:
        items: 条目列表
    
    Returns:
        dict: {found, new, errors}
    """
    if not items:
        return {'found': 0, 'new': 0, 'errors': 0}
    
    conn = get_db()
    cursor = conn.cursor()
    
    today = get_today_str()
    now = int(datetime.now(timezone.utc).timestamp())
    
    found = len(items)
    new_count = 0
    errors = 0
    
    for item in items:
        try:
            title = item.get('title', '')
            if not title:
                continue
            
            # 生成唯一ID
            item_id = generate_id(title, today)
            rank = item.get('rank', 0)
            category = item.get('category', '综合')
            hot_tag = item.get('hot_tag', '0')
            label = item.get('label', '')
            
            # 检查是否已存在
            cursor.execute('SELECT id FROM hot_items WHERE id = ?', (item_id,))
            if cursor.fetchone():
                continue  # 已存在，跳过
            
            # 插入新记录
            url = f"https://www.baidu.com/s?wd={urllib.parse.quote(title)}"
            cursor.execute('''
                INSERT INTO hot_items 
                (id, platform, rank, title, category, hot_tag, label, url, fetched_at, fetch_date, raw_data)
                VALUES (?, 'baidu', ?, ?, ?, ?, ?, ?, ?, ?, ?)
            ''', (item_id, rank, title, category, hot_tag, label, url, now, today, json.dumps(item, ensure_ascii=False)))
            
            new_count += 1
            
        except Exception as e:
            print(f"   ⚠️ 保存失败: {e}")
            errors += 1
            continue
    
    conn.commit()
    conn.close()
    
    return {
        'found': found,
        'new': new_count,
        'errors': errors
    }

def log_fetch(found, new, status=0, error=None):
    """记录抓取日志"""
    conn = get_db()
    cursor = conn.cursor()
    
    now = int(datetime.now(timezone.utc).timestamp())
    
    cursor.execute('''
        INSERT INTO fetch_logs 
        (started_at, ended_at, found, new, status, error)
        VALUES (?, ?, ?, ?, ?, ?)
    ''', (now, now, found, new, status, error))
    
    conn.commit()
    conn.close()

def get_items_by_date(date_str=None, category=None, limit=100):
    """获取指定日期的热搜条目"""
    if date_str is None:
        date_str = get_today_str()
    
    conn = get_db()
    cursor = conn.cursor()
    
    if category:
        cursor.execute('''
            SELECT rank, title, category, label, fetched_at
            FROM hot_items
            WHERE fetch_date = ? AND category = ?
            ORDER BY rank ASC
            LIMIT ?
        ''', (date_str, category, limit))
    else:
        cursor.execute('''
            SELECT rank, title, category, label, fetched_at
            FROM hot_items
            WHERE fetch_date = ?
            ORDER BY rank ASC
            LIMIT ?
        ''', (date_str, limit))
    
    rows = cursor.fetchall()
    conn.close()
    
    return [{
        'rank': row[0],
        'title': row[1],
        'category': row[2],
        'label': row[3],
        'fetched_at': row[4]
    } for row in rows]

def get_stats(days=7):
    """获取最近N天的统计信息"""
    conn = get_db()
    cursor = conn.cursor()
    
    cursor.execute('''
        SELECT fetch_date, COUNT(*) as count, 
               COUNT(DISTINCT category) as categories
        FROM hot_items
        WHERE fetch_date >= date('now', '-{} days')
        GROUP BY fetch_date
        ORDER BY fetch_date DESC
    '''.format(days))
    
    rows = cursor.fetchall()
    conn.close()
    
    return [{
        'date': row[0],
        'count': row[1],
        'categories': row[2]
    } for row in rows]

def get_category_stats(date_str=None):
    """获取分类统计"""
    if date_str is None:
        date_str = get_today_str()
    
    conn = get_db()
    cursor = conn.cursor()
    
    cursor.execute('''
        SELECT category, COUNT(*) as count
        FROM hot_items
        WHERE fetch_date = ?
        GROUP BY category
        ORDER BY count DESC
    ''', (date_str,))
    
    rows = cursor.fetchall()
    conn.close()
    
    return [{
        'category': row[0],
        'count': row[1]
    } for row in rows]

if __name__ == '__main__':
    print(f"Database path: {DB_PATH}")


path: scripts/generate_html.py
content: #!/usr/bin/env python3
"""
Baidu Hot - Generate HTML Report
百度热搜HTML可视化报告
"""

import sqlite3
import json
from datetime import datetime, timezone
from pathlib import Path

DB_PATH = Path(__file__).parent.parent / "data" / "baidu.db"
OUTPUT_PATH = Path(__file__).parent.parent / "data" / "index.html"

# 分类颜色
CATEGORY_COLORS = {
    '热': '#ff6b6b',
    '新': '#4ecdc4',
    '热议': '#ffa94d',
    '辟谣': '#a55eea',
    '综合': '#868e96',
}

def generate_html():
    """生成HTML报告"""
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row
    cursor = conn.cursor()
    
    # 获取所有数据
    cursor.execute('''
        SELECT rank, title, category, label, url, fetch_date
        FROM hot_items
        ORDER BY fetch_date DESC, rank ASC
        LIMIT 1000
    ''')
    rows = cursor.fetchall()
    
    # 获取所有日期
    cursor.execute('SELECT DISTINCT fetch_date FROM hot_items ORDER BY fetch_date DESC')
    all_dates = [row[0] for row in cursor.fetchall()]
    
    # 获取所有分类
    cursor.execute('SELECT DISTINCT category FROM hot_items WHERE category IS NOT NULL')
    categories = [row[0] for row in cursor.fetchall()]
    
    conn.close()
    
    if not rows:
        print("❌ 数据库中没有数据")
        return
    
    # 转换数据
    items = []
    for row in rows:
        items.append({
            'rank': row['rank'],
            'title': row['title'],
            'category': row['category'] or '综合',
            'label': row['label'] or '',
            'url': row['url'] or '',
            'date': row['fetch_date']
        })
    
    max_date = all_dates[0] if all_dates else datetime.now().strftime('%Y-%m-%d')
    min_date = all_dates[-1] if all_dates else max_date
    
    # 生成HTML
    html = f'''<!DOCTYPE html>
<html lang="zh-CN">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>百度热搜 - 数据可视化</title>
    <style>
        * {{ margin: 0; padding: 0; box-sizing: border-box; }}
        body {{
            font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "PingFang SC", sans-serif;
            background: #f5f5f5;
            line-height: 1.6;
        }}
        
        .header {{
            background: linear-gradient(135deg, #2932e1 0%, #1a1f9c 100%);
            color: white;
            padding: 25px 20px;
            position: sticky;
            top: 0;
            z-index: 100;
            box-shadow: 0 2px 8px rgba(0,0,0,0.1);
        }}
        .header h1 {{ font-size: 24px; margin-bottom: 8px; display: flex; align-items: center; gap: 10px; }}
        .header h1::before {{ content: "🔍"; }}
        .header .meta {{ font-size: 13px; opacity: 0.9; }}
        
        .filters {{
            background: white;
            padding: 20px;
            border-bottom: 1px solid #e8e8e8;
            display: flex;
            gap: 15px;
            flex-wrap: wrap;
            align-items: flex-end;
        }}
        
        .filter-group {{ display: flex; flex-direction: column; gap: 6px; }}
        .filter-group label {{ font-size: 12px; color: #666; font-weight: 500; }}
        .filter-group select, .filter-group input {{
            padding: 8px 12px; border: 1px solid #ddd; border-radius: 6px;
            font-size: 14px; background: white; min-width: 120px;
        }}
        
        .btn {{ padding: 8px 16px; border: none; border-radius: 6px;
            cursor: pointer; font-size: 14px; background: #f0f0f0; color: #666; }}
        .btn:hover {{ background: #e0e0e0; }}
        
        .stats {{
            background: #f0f2ff;
            padding: 12px 20px;
            font-size: 13px;
            color: #666;
            border-bottom: 1px solid #e8e8e8;
            display: flex;
            justify-content: space-between;
        }}
        .stats .highlight {{ color: #2932e1; font-weight: 600; }}
        
        .content {{ max-width: 800px; margin: 0 auto; padding: 20px; }}
        
        .hot-list {{ display: flex; flex-direction: column; gap: 10px; }}
        
        .hot-item {{
            display: flex;
            align-items: center;
            padding: 14px 18px;
            background: white;
            border-radius: 10px;
            box-shadow: 0 1px 4px rgba(0,0,0,0.05);
            transition: transform 0.1s, box-shadow 0.1s;
        }}
        
        .hot-item:hover {{
            transform: translateY(-2px);
            box-shadow: 0 4px 12px rgba(0,0,0,0.1);
        }}
        
        .rank {{
            width: 32px;
            height: 32px;
            border-radius: 8px;
            display: flex;
            align-items: center;
            justify-content: center;
            font-weight: 700;
            font-size: 15px;
            margin-right: 14px;
            flex-shrink: 0;
        }}
        
        .rank.top1 {{ background: linear-gradient(135deg, #ff6b6b, #ee5a5a); color: white; }}
        .rank.top2 {{ background: linear-gradient(135deg, #ffa94d, #ff8c42); color: white; }}
        .rank.top3 {{ background: linear-gradient(135deg, #ffd43b, #fcc419); color: #333; }}
        .rank.normal {{ background: #f0f0f0; color: #666; }}
        
        .item-content {{ flex: 1; min-width: 0; }}
        
        .item-title {{
            font-size: 15px;
            color: #1a1a1a;
            margin-bottom: 4px;
            line-height: 1.5;
            word-break: break-word;
        }}
        
        .item-meta {{ display: flex; align-items: center; gap: 10px; font-size: 12px; }}
        
        .category-badge {{
            padding: 2px 10px;
            border-radius: 12px;
            font-size: 11px;
            font-weight: 600;
            color: white;
        }}
        
        .empty {{ text-align: center; padding: 80px 20px; color: #999; }}
        .empty-icon {{ font-size: 48px; margin-bottom: 16px; }}
        
        @media (max-width: 640px) {{
            .content {{ padding: 12px; }}
            .hot-item {{ padding: 12px 14px; }}
        }}
    </style>
</head>
<body>
    <div class="header">
        <h1>百度热搜数据可视化</h1>
        <div class="meta">
            更新时间: {datetime.now().strftime('%Y-%m-%d %H:%M')} | 
            共 {len(items)} 条记录
        </div>
    </div>
    
    <div class="filters">
        <div class="filter-group">
            <label>日期</label>
            <select id="dateSelect">
                <option value="">全部</option>
'''
    
    for date in all_dates[:30]:
        html += f'                <option value="{date}">{date}</option>\n'
    
    html += f'''            </select>
        </div>
        
        <div class="filter-group">
            <label>分类</label>
            <select id="categorySelect">
                <option value="">全部</option>
'''
    
    for cat in categories:
        html += f'                <option value="{cat}">{cat}</option>\n'
    
    html += f'''            </select>
        </div>
        
        <div class="filter-group">
            <label>搜索</label>
            <input type="text" id="searchInput" placeholder="输入关键词...">
        </div>
        
        <button class="btn" onclick="resetFilters()">重置</button>
    </div>
    
    <div class="stats">
        <span>显示 <span class="highlight" id="showCount">{len(items)}</span> / {len(items)} 条</span>
    </div>
    
    <div class="content">
        <div class="hot-list" id="hotList"></div>
    </div>

    <script>
        const items = {json.dumps(items, ensure_ascii=False)};
        const categoryColors = {json.dumps(CATEGORY_COLORS, ensure_ascii=False)};
        
        function getRankClass(rank) {{
            if (rank === 1) return 'top1';
            if (rank === 2) return 'top2';
            if (rank === 3) return 'top3';
            return 'normal';
        }}
        
        function renderItems() {{
            const dateSelect = document.getElementById('dateSelect').value;
            const categorySelect = document.getElementById('categorySelect').value;
            const search = document.getElementById('searchInput').value.toLowerCase().trim();
            
            let filtered = items.filter(item => {{
                if (dateSelect && item.date !== dateSelect) return false;
                if (categorySelect && item.category !== categorySelect) return false;
                if (search && !item.title.toLowerCase().includes(search)) return false;
                return true;
            }});
            
            document.getElementById('showCount').textContent = filtered.length;
            
            const container = document.getElementById('hotList');
            if (filtered.length === 0) {{
                container.innerHTML = `
                    <div class="empty" style="grid-column: 1/-1;">
                        <div class="empty-icon">🔍</div>
                        <div>没有找到记录</div>
                    </div>
                `;
                return;
            }}
            
            // 按日期分组
            const grouped = {{}};
            filtered.forEach(item => {{
                if (!grouped[item.date]) grouped[item.date] = [];
                grouped[item.date].push(item);
            }});
            
            let html = '';
            Object.keys(grouped).sort().reverse().forEach(date => {{
                html += grouped[date].map(item => {{
                    const bgColor = categoryColors[item.category] || '#868e96';
                    return `
                        <a href="${{item.url}}" class="hot-item" target="_blank" style="text-decoration: none; color: inherit;">
                            <div class="rank ${{getRankClass(item.rank)}}">${{item.rank}}</div>
                            <div class="item-content">
                                <div class="item-title">${{item.title}}</div>
                                <div class="item-meta">
                                    <span class="category-badge" style="background: ${{bgColor}}">${{item.category}}</span>
                                    <span style="color: #999;">${{item.date}}</span>
                                </div>
                            </div>
                        </a>
                    `;
                }}).join('');
            }});
            
            container.innerHTML = html;
        }}
        
        function resetFilters() {{
            document.getElementById('dateSelect').value = '';
            document.getElementById('categorySelect').value = '';
            document.getElementById('searchInput').value = '';
            renderItems();
        }}
        
        document.getElementById('dateSelect').addEventListener('change', renderItems);
        document.getElementById('categorySelect').addEventListener('change', renderItems);
        document.getElementById('searchInput').addEventListener('input', renderItems);
        
        renderItems();
    </script>
</body>
</html>'''
    
    OUTPUT_PATH.write_text(html, encoding='utf-8')
    print(f"✅ HTML报告已生成: {OUTPUT_PATH}")
    print(f"   共 {len(items)} 条记录")
    print(f"   日期: {min_date} ~ {max_date}")

if __name__ == '__main__':
    generate_html()


path: scripts/init_db.py
content: #!/usr/bin/env python3
"""
Baidu Hot - Database Initialization
百度热搜数据库初始化
"""

import sqlite3
from pathlib import Path

DB_PATH = Path(__file__).parent.parent / "data" / "baidu.db"

def init_db():
    """初始化数据库"""
    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
    
    conn = sqlite3.connect(DB_PATH)
    cursor = conn.cursor()
    
    # 热搜条目表
    cursor.execute('''
        CREATE TABLE IF NOT EXISTS hot_items (
            id TEXT PRIMARY KEY,              -- 条目ID: 标题+日期的哈希
            platform TEXT DEFAULT 'baidu',    -- 平台标识
            rank INTEGER,                     -- 排名
            title TEXT NOT NULL,              -- 标题
            category TEXT,                    -- 分类/标签: 热/新/热议/辟谣等
            hot_tag TEXT,                     -- 热度标记: 0/1/3
            label TEXT,                       -- 原始标签
            url TEXT,                         -- 链接(可选)
            fetched_at INTEGER DEFAULT (strftime('%s', 'now')), -- 抓取时间
            fetch_date TEXT,                  -- 抓取日期(YYYY-MM-DD)
            raw_data TEXT                     -- 原始JSON
        )
    ''')
    
    # 创建索引
    cursor.execute('CREATE INDEX IF NOT EXISTS idx_hot_items_date ON hot_items(fetch_date)')
    cursor.execute('CREATE INDEX IF NOT EXISTS idx_hot_items_rank ON hot_items(rank)')
    cursor.execute('CREATE INDEX IF NOT EXISTS idx_hot_items_category ON hot_items(category)')
    cursor.execute('CREATE INDEX IF NOT EXISTS idx_hot_items_fetched ON hot_items(fetched_at)')
    
    # 抓取日志表
    cursor.execute('''
        CREATE TABLE IF NOT EXISTS fetch_logs (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            started_at INTEGER DEFAULT (strftime('%s', 'now')),
            ended_at INTEGER,
            found INTEGER DEFAULT 0,
            new INTEGER DEFAULT 0,
            status INTEGER DEFAULT 0,
            error TEXT
        )
    ''')
    
    cursor.execute('CREATE INDEX IF NOT EXISTS idx_fetch_logs_date ON fetch_logs(started_at)')
    
    conn.commit()
    conn.close()
    
    print(f"✅ Baidu database initialized: {DB_PATH}")
    return DB_PATH

if __name__ == '__main__':
    init_db()


path: scripts/query.py
content: #!/usr/bin/env python3
"""
Baidu Hot - Query Tool
百度热搜数据查询工具
"""

import sqlite3
from datetime import datetime, timezone
from pathlib import Path
import sys

DB_PATH = Path(__file__).parent.parent / "data" / "baidu.db"

def query_today(limit=50):
    """查询今天的数据"""
    conn = sqlite3.connect(DB_PATH)
    cursor = conn.cursor()
    
    today = datetime.now(timezone.utc).strftime('%Y-%m-%d')
    
    cursor.execute('''
        SELECT rank, title, category, label
        FROM hot_items
        WHERE fetch_date = ?
        ORDER BY rank ASC
        LIMIT ?
    ''', (today, limit))
    
    rows = cursor.fetchall()
    conn.close()
    
    print(f"\n🔍 百度热搜 - {today}")
    print("=" * 80)
    
    for row in rows:
        rank, title, category, label = row
        label_str = f" [{category}]" if category else ""
        print(f"  {rank:2d}. {title}{label_str}")
    
    print(f"\n   共 {len(rows)} 条数据")

def query_by_date(date_str, limit=50):
    """查询指定日期的数据"""
    conn = sqlite3.connect(DB_PATH)
    cursor = conn.cursor()
    
    cursor.execute('''
        SELECT rank, title, category, label
        FROM hot_items
        WHERE fetch_date = ?
        ORDER BY rank ASC
        LIMIT ?
    ''', (date_str, limit))
    
    rows = cursor.fetchall()
    conn.close()
    
    if not rows:
        print(f"❌ 未找到 {date_str} 的数据")
        return
    
    print(f"\n🔍 百度热搜 - {date_str}")
    print("=" * 80)
    
    for row in rows:
        rank, title, category, label = row
        label_str = f" [{category}]" if category else ""
        print(f"  {rank:2d}. {title}{label_str}")
    
    print(f"\n   共 {len(rows)} 条数据")

def query_stats(days=7):
    """查询统计"""
    conn = sqlite3.connect(DB_PATH)
    cursor = conn.cursor()
    
    cursor.execute('''
        SELECT fetch_date, COUNT(*) as count
        FROM hot_items
        WHERE fetch_date >= date('now', '-{} days')
        GROUP BY fetch_date
        ORDER BY fetch_date DESC
    '''.format(days))
    
    rows = cursor.fetchall()
    conn.close()
    
    print(f"\n📊 最近{days}天统计")
    print("=" * 60)
    print(f"{'日期':<12} {'数量':<10}")
    print("-" * 60)
    
    for row in rows:
        date_str, count = row
        print(f"{date_str:<12} {count:<10}")

def query_categories(date_str=None):
    """查询分类统计"""
    if date_str is None:
        date_str = datetime.now(timezone.utc).strftime('%Y-%m-%d')
    
    conn = sqlite3.connect(DB_PATH)
    cursor = conn.cursor()
    
    cursor.execute('''
        SELECT category, COUNT(*) as count
        FROM hot_items
        WHERE fetch_date = ?
        GROUP BY category
        ORDER BY count DESC
    ''', (date_str,))
    
    rows = cursor.fetchall()
    conn.close()
    
    print(f"\n🏷️ 分类统计 - {date_str}")
    print("=" * 60)
    print(f"{'分类':<15} {'数量':<10}")
    print("-" * 60)
    
    for row in rows:
        category, count = row
        print(f"{category:<15} {count:<10}")

def query_logs(limit=10):
    """查询抓取日志"""
    conn = sqlite3.connect(DB_PATH)
    cursor = conn.cursor()
    
    cursor.execute('''
        SELECT started_at, found, new, status
        FROM fetch_logs
        ORDER BY started_at DESC
        LIMIT ?
    ''', (limit,))
    
    rows = cursor.fetchall()
    conn.close()
    
    print(f"\n📝 最近{limit}次抓取日志")
    print("=" * 60)
    print(f"{'时间':<20} {'发现':<8} {'新增':<8} {'状态':<8}")
    print("-" * 60)
    
    for row in rows:
        started, found, new, status = row
        time_str = datetime.fromtimestamp(started, tz=timezone.utc).strftime('%m-%d %H:%M:%S')
        status_str = '✅' if status == 0 else '⚠️' if status == 1 else '❌'
        print(f"{time_str:<20} {found:<8} {new:<8} {status_str:<8}")

def main():
    args = sys.argv[1:]
    
    if not args or args[0] == 'today':
        limit = int(args[1]) if len(args) > 1 else 50
        query_today(limit)
    elif args[0] == 'date':
        if len(args) < 2:
            print("Usage: query.py date YYYY-MM-DD [limit]")
            return
        date_str = args[1]
        limit = int(args[2]) if len(args) > 2 else 50
        query_by_date(date_str, limit)
    elif args[0] == 'stats':
        days = int(args[1]) if len(args) > 1 else 7
        query_stats(days)
    elif args[0] == 'categories':
        date_str = args[1] if len(args) > 1 else None
        query_categories(date_str)
    elif args[0] == 'logs':
        limit = int(args[1]) if len(args) > 1 else 10
        query_logs(limit)
    else:
        print("Usage: query.py [command] [args]")
        print("")
        print("Commands:")
        print("  today [limit]           查询今天的数据")
        print("  date YYYY-MM-DD [limit] 查询指定日期的数据")
        print("  stats [days]            统计")
        print("  categories [date]       分类统计")
        print("  logs [limit]            抓取日志")

if __name__ == '__main__':
    main()


path: scripts/save_to_db.py
content: #!/usr/bin/env python3
"""
Baidu Hot - Save to Database
将百度热搜保存到数据库
"""

import subprocess
import json
import sys
from pathlib import Path
from datetime import datetime, timezone

sys.path.insert(0, str(Path(__file__).parent))
from db import save_hot_items, log_fetch

def fetch_and_save(limit=50):
    """采集百度热搜并保存到数据库"""
    script_path = Path(__file__).parent / "baidu_hot.py"
    
    print("🔍 采集百度热搜...")
    
    try:
        # 调用原有脚本获取数据
        result = subprocess.run(
            [sys.executable, str(script_path), str(limit), '--json'],
            capture_output=True,
            text=True,
            timeout=30
        )
        
        if result.returncode != 0:
            print(f"❌ 脚本执行失败: {result.stderr}")
            log_fetch(0, 0, status=2, error=result.stderr)
            return None
        
        # 解析JSON
        data = json.loads(result.stdout)
        
        if data.get('status') != 'success':
            print(f"❌ 获取失败: {data.get('error', '未知错误')}")
            log_fetch(0, 0, status=2, error=data.get('error'))
            return None
        
        items = data.get('data', [])
        print(f"   获取到 {len(items)} 条热搜")
        
        # 保存到数据库
        result = save_hot_items(items)
        
        # 记录日志
        status = 0 if result['errors'] == 0 else 1
        log_fetch(result['found'], result['new'], status=status)
        
        return result
        
    except subprocess.TimeoutExpired:
        print("❌ 采集超时")
        log_fetch(0, 0, status=2, error='Timeout')
        return None
    except Exception as e:
        print(f"❌ 异常: {e}")
        log_fetch(0, 0, status=2, error=str(e))
        return None

def main():
    """主函数"""
    args = sys.argv[1:]
    limit = int(args[0]) if args else 50
    
    print("=" * 60)
    print(f"🔍 百度热搜采集 → 数据库")
    print(f"   数量: {limit}")
    print("=" * 60)
    
    # 确保数据库已初始化
    db_path = Path(__file__).parent.parent / "data" / "baidu.db"
    if not db_path.exists():
        print("\n🔄 初始化数据库...")
        from init_db import init_db
        init_db()
    
    # 采集数据
    result = fetch_and_save(limit)
    
    if result:
        print("\n" + "=" * 60)
        print("✅ 完成!")
        print(f"   数据库: {db_path}")
        print(f"   发现: {result['found']} 条")
        print(f"   新增: {result['new']} 条")
        if result['errors'] > 0:
            print(f"   错误: {result['errors']} 条")
        
        # 显示Top 5
        print("\n🔥 Top 5:")
        # 重新查询今天数据
        from db import get_items_by_date
        items = get_items_by_date(limit=5)
        for item in items:
            label_str = f" [{item['category']}]" if item['category'] else ""
            print(f"  {item['rank']}. {item['title'][:40]}...{label_str}")
    else:
        print("\n❌ 采集失败")
        sys.exit(1)

if __name__ == '__main__':
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)