OpenClaw/clawhub-security-signals huggingface dataset:
Skill: nihaovand/kite-agent-smart-wallet-permissionless-protocol-v2 (version 2.0.5)

# Kite AI Agent Smart Wallet Permissionless Protocol > 让用户通过Telegram控制加密货币钱包 / Telegram wallet control for Kite AI ![Version](https://img.shields.io/badge/version-2.0.4-blue) ![Kite AI](https://img.shields.io/badge/Kite-AI-purple) ![Telegram](https://img.shields.io/badge/Telegram-Bot-blue) ## 简介 / Introduction 这是一个让用户通过Telegram管理Kite AI链上钱包的协议。 A protocol for users to manage Kite AI wallet via Telegram. - 用户本地运行Bot / Bot runs locally - 私钥用户自己保管 / Private key stays with user - 完全去中心化 / Fully decentralized ## 特性 / Features - 📱 Telegram钱包控制 / Telegram wallet control - 🔐 智能钱包 / Smart wallet - 🔑 Session Keys / 授权密钥 - 💰 消费限额 / Spending limits - 🌍 中英双语 / Bilingual ## 快速开始 / Quick Start ### 1. 创建Telegram机器人 1. 打开Telegram → @BotFather 2. 发送 `/newbot` 3. 给机器人起名 4. 复制Token ### 2. 安装 ```bash git clone <repo> cd kite-wallet npm install ``` ### 3. 配置 ```env PRIVATE_KEY=你的私钥 TELEGRAM_BOT_TOKEN=你的Token ``` ### 4. 运行 ```bash node telegram-bot.js ``` ## 命令 / Commands | 中文 | English | 功能 Function | |------|---------|---------------| | /create | /create | 创建钱包 | | /wallet | /wallet | 查看地址 | | /balance | /balance | 查看余额 | | /session add | /session add | 添加授权 | | /limit set | /limit set | 设置限额 | | /send | /send | 转账 | ## 网络 / Network | 网络 Network | Chain ID | RPC | |-------------|----------|-----| | Testnet | 2368 | https://rpc-testnet.gokite.ai | ## 合约 / Contracts | 合约 Contract | 地址 Address | |--------------|-------------| | AgentSmartWalletFactory | [REDACTED_SECRET] | ## 相关链接 / Links - 🌐 Website: https://gokite.ai - 🔍 Explorer: https://testnet.kitescan.ai - 🚰 Faucet: https://faucet.gokite.ai - 📖 Docs: https://docs.gokite.ai ## 版本历史 / Version History - v2.0.4 - 中英双语 Bilingual - v2.0.3 - 用户手册 User guide - v2.0.2 - 用户本地运行 User runs locally - v1.0.0 - 初始版 Initial --- **作者 / Author**: VandNi **许可证 / License**: MIT


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)
Skill: buildingbrien/greenhouse-apply (version 1.0.0)

--- name: greenhouse-apply description: > Submit job applications on Greenhouse (job-boards.greenhouse.io). Handles the full flow: fill text fields, set React Select dropdowns, select phone country, upload resume, enter email verification code, and submit. Use when: applying to a job on Greenhouse, filling out a Greenhouse application form, submitting a resume to a company that uses Greenhouse ATS. --- # Greenhouse Application Skill Fill and submit Greenhouse job application forms via browser automation. ## Prerequisites - Browser tool available (OpenClaw browser or Chrome extension) - Resume PDF accessible on the filesystem - Gmail or email access to retrieve verification codes - User-provided: name, email, phone, resume path, answers to custom questions ## Workflow ### Phase 1: Open the Job Page Navigate to the Greenhouse job URL. ### Phase 2: Fill All Text Fields via JS Evaluate **Critical:** Do NOT use type on individual fields. Use a single evaluate call with native setters. ### Phase 3: Phone Country Dropdown The phone country selector is an intl-tel-input widget. It hijacks focus from other dropdowns. ### Phase 4: React Select Dropdowns Use Playwright click, type, press Enter on the combobox ref. ### Phase 5: Resume Upload Use the browser upload tool with the Attach button ref. ### Phase 6: Audit Before Submit Take a snapshot and verify EVERY field. ### Phase 7: Submit & Verification Code The code input is 8 individual inputs with IDs security-input-0 through security-input-7. DO NOT use Playwright type on snapshot refs for code boxes — use JS evaluate instead. ## Failure Modes & Recovery - Characters in wrong fields: Use JS getElementById instead of snapshot refs - Country dropdown stealing focus: Click document.body first to blur - Dropdown shows Select after setting: Use click-type-Enter pattern - Submit button stays disabled: Ensure input+change events fire with bubbles:true - Code expired: Re-submit to get fresh code ## Notes - Voluntary EEO fields are optional - The Why Company textarea is the highest-value field - Phone auto-formats after country selection - Greenhouse may throttle repeat submissions

path: references/field-mapping.md
content: # Greenhouse Field Mapping Reference

## Standard Fields
| Field | Type | Notes |
|-------|------|-------|
| First Name | text input | Required |
| Last Name | text input | Required |
| Email | text input | Required, triggers verification |
| Phone | text input | Digits only, auto-formats |
| Phone Country | intl-tel-input | Separate widget |
| Resume/CV | file upload | pdf/doc/docx/txt/rtf |
| Website | text input | Optional |
| LinkedIn | text input | Often required |

## Verification Code Inputs
IDs: security-input-0 through security-input-7 (maxLength=1 each)
Always use getElementById, never snapshot refs.

Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)
Skill: bibaofeng/aisa-tavily (version 1.0.0)

--- name: aisa-tavily description: 'Run web, multi-source, or last-30-days research through AIsa. Use when: the user needs search, synthesis, competitor scans, or trend discovery. Supports research-ready outputs and structured retrieval.' author: AIsa version: 1.0.0 license: Apache-2.0 user-invocable: true primaryEnv: AISA_API_KEY requires: bins: - node env: - AISA_API_KEY metadata: aisa: emoji: 🔎 requires: bins: - node env: - AISA_API_KEY primaryEnv: AISA_API_KEY compatibility: - openclaw - claude-code - hermes openclaw: emoji: 🔎 requires: bins: - node env: - AISA_API_KEY primaryEnv: AISA_API_KEY --- # AIsa Tavily Run web, multi-source, or last-30-days research through AIsa. Use when: the user needs search, synthesis, competitor scans, or trend discovery. Supports research-ready outputs and structured retrieval. ## When to use - The user needs web, multi-source, or last-30-days research. - The user wants competitor scans, trend discovery, or structured search output. - The user wants one skill to cover multiple retrieval surfaces. ## High-Intent Workflows - Search and summarize recent evidence. - Compare two tools or companies using recent signals. - Turn multi-source retrieval into a research brief. ## Setup - `AISA_API_KEY` is required for AIsa-backed API access. - Use repo-relative `scripts/` paths from the shipped package. - Prefer explicit CLI auth flags when a script exposes them. ## Example Requests - Research OpenAI Agents SDK over the last 30 days - Compare OpenClaw and Codex using recent public discussion - Search recent sentiment around a product launch ## Guardrails - Do not present test-only helpers as public features. - Do not claim sources that were not actually queried. - If some providers time out, report that honestly.


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)
Skill: pointgh/gisransformcoords (version 1.0.0)

--- name: gis-coord-transform description: GIS 坐标转换工具，支持点/线/面数据在 WGS84（经纬度）和 Web Mercator（墨卡托/EPSG:3857）之间转换。处理 GeoJSON 格式，支持批量转换。Use when: (1) 需要转换 GIS 数据坐标系，(2) WebGIS 项目需要墨卡托投影，(3) 处理 GPS 数据转地图显示，(4) 转换 Shapefile/GeoJSON 等矢量数据。 license: MIT --- # GIS 坐标转换 Skill 快速转换点、线、面数据的坐标系：**WGS84（经纬度）↔ Web Mercator（EPSG:3857）** ## 快速开始 ### 转换 GeoJSON 到墨卡托 ```bash python3 ~/.openclaw/workspace/skills/gis-coord-transform/scripts/transform_coords.py input.geojson -t mercator -o output.geojson --pretty ``` ### 转换墨卡托回经纬度 ```bash python3 ~/.openclaw/workspace/skills/gis-coord-transform/scripts/transform_coords.py input.geojson -t wgs84 -o output.geojson --pretty ``` ## 支持的几何类型 | 类型 | 说明 | |------|------| | Point | 点 | | LineString | 线 | | Polygon | 面 | | MultiPoint | 多点 | | MultiLineString | 多线 | | MultiPolygon | 多面 | | Feature | GeoJSON 要素 | | FeatureCollection | 要素集合 | ## 命令行参数 | 参数 | 说明 | |------|------| | `input` | 输入 GeoJSON 文件路径 | | `-o, --output` | 输出文件路径（默认输出到 stdout） | | `-t, --to` | 目标坐标系：`mercator` 或 `wgs84` | | `--pretty` | 美化输出 JSON | ## 使用示例 ### 示例 1：转换单个点 ```json // input.geojson { "type": "Point", "coordinates": [116.4074, 39.9042] } ``` ```bash python3 scripts/transform_coords.py input.geojson -t mercator ``` 输出（墨卡托坐标）： ```json { "type": "Point", "coordinates": [12958396.7, 4865942.3] } ``` ### 示例 2：转换 FeatureCollection ```bash python3 scripts/transform_coords.py cities.geojson -t mercator -o cities_mercator.geojson --pretty ``` ### 示例 3：批量转换 ```bash for file in *.geojson; do python3 scripts/transform_coords.py "$file" -t mercator -o "${file%.geojson}_mercator.geojson" done ``` ## 依赖安装 ```bash pip3 install pyproj ``` ## 坐标系说明 | 坐标系 | EPSG 代码 | 用途 | |--------|----------|------| | WGS84 | EPSG:4326 | GPS、经纬度、地理坐标 | | Web Mercator | EPSG:3857 | Web 地图（Google Maps、OpenStreetMap、Mapbox 等） | ## 常见问题 ### 为什么转换后坐标数字变大了？ Web Mercator 使用米为单位，而 WGS84 使用度。墨卡托坐标通常在百万级别。 ### 转换精度如何？ 使用 pyproj 库，精度非常高，适合专业 GIS 应用。 ### 支持其他坐标系吗？ 当前版本仅支持 WGS84 ↔ Web Mercator。如需其他坐标系，可扩展 `pyproj.Transformer`。 ## 扩展建议 需要添加其他坐标系转换时，修改脚本中的 Transformer 定义： ```python # 例如：WGS84 → CGCS2000 (EPSG:4490) transformer = Transformer.from_crs("EPSG:4326", "EPSG:4490", always_xy=True) ``` ## 相关文件 - `scripts/transform_coords.py` - 坐标转换主脚本


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)
Skill: bgoodwinstudio/google-alerts (version 1.0.3)

--- name: google-alerts description: "Monitor keywords via free web search. Use when: (1) Tracking brand or product mentions, (2) Monitoring industry keywords, (3) Researching competitors and market trends, (4) Daily competitive intel reports. Creates Hootsuite-style formatted output from search results." metadata: openclaw: requires: env: - GOOGLE_ALERT_FEED_ID bins: - bash - curl - python3 --- # Google Alerts Monitor Keyword monitoring with Hootsuite-style output. Free, no API keys required. ## What This Skill Does - Fetches Google Alerts RSS feeds for your keywords - Formats results into visual, social-media-style cards - Prints clean JSON output and formatted text for use by agents or cron jobs **What this skill does NOT do:** It does not save files, schedule cron jobs, or post to chat. Those integrations are handled by your agent prompt or external automation. ## Keywords Edit `references/keywords.md` with your own keywords. One per line. ## Search Script ```bash # Set your Google Alert feed ID (see setup-guide.md) export GOOGLE_ALERT_FEED_ID="your_feed_id_here" # Search a keyword bash scripts/search.sh "your keyword" 10 ``` Output: JSON array of results (title, link, description, published date) ## Formatter Script Takes JSON results and formats them into Hootsuite-style cards: ```bash bash scripts/search.sh "your keyword" 10 \ | python3 scripts/format-results.py --term "your keyword" --source google --count 10 ``` ## Output Format ``` ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 📊 YOUR KEYWORD — GOOGLE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 👤 Result Title · 3h ago "Description text preview truncated to 280 chars..." ↗️ https://example.com/link ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` ## Setup 1. Create Google Alerts at [google.com/alerts](https://www.google.com/alerts) 2. Get the RSS feed ID from each alert's feed URL 3. Set `GOOGLE_ALERT_FEED_ID` in your environment or edit `scripts/search.sh` 4. See `references/setup-guide.md` for details ## Scripts - `scripts/search.sh` — Fetch Google Alerts RSS, output JSON - `scripts/format-results.py` — Format JSON results as Hootsuite-style cards ## References - `references/keywords.md` — Keyword definitions - `references/setup-guide.md` — Detailed setup instructions


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)
Skill: opendolph/fast-response-optimizer (version 1.0.0)

--- name: fast-response-optimizer description: Response speed optimizer - implements reply-first-then-process, parallel tool calls, and memory file caching version: 1.0.0 author: opendolph source: https://github.com/opendolph/skills/tree/main/fast-response-optimizer --- # Fast Response Optimizer ⚡ > Reduce response time from 5-20 seconds to under 1 second --- ## Core Optimization Strategies ### 1. Reply First, Process Later **Rule:** - Receive request → Immediately reply "Received/Processing" → Execute in background → Push result when done **Implementation:** ```javascript // Quick reply await message.send("Received, processing..."); // Background execution const result = await processInBackground(task); // Push result await message.send(result); ``` ### 2. Parallel Tool Execution **Rule:** - Execute independent tool calls in parallel instead of waiting sequentially **Implementation:** ```javascript // Before (sequential) const result1 = await toolA(); const result2 = await toolB(); const result3 = await toolC(); // After (parallel) const [result1, result2, result3] = await Promise.all([ toolA(), toolB(), toolC() ]); ``` ### 3. Memory File Caching **Rule:** - Load cache every 1 minute via scheduled task - Reload cache if more than 1 minute since last message - Cache stored in SESSION-STATE.md **Implementation:** ```javascript // Cache structure const cache = { lastUpdate: timestamp, userProfile: {...}, memorySummary: {...}, skills: [...] }; // Check if refresh needed if (now - lastMessage > 60000 || now - cache.lastUpdate > 60000) { await refreshCache(); } ``` --- ## Trigger Conditions | Scenario | Trigger Method | |----------|----------------| | Auto-trigger | Execute on every message processing | | Scheduled refresh | Cron job every 1 minute | | Manual trigger | User inputs "refresh cache" | --- ## Cache Files ``` SESSION-STATE.md # Active state + cache ├── cache/ # Cache directory │ ├── user_profile.json # User profile cache │ ├── memory_summary.json # Memory summary cache │ └── skills_list.json # Skills list cache └── last_refresh.txt # Last refresh timestamp ``` --- ## Performance Goals | Metric | Before | After | |--------|--------|-------| | First Response | 5-20s | <1s | | Tool Calls | Sequential accumulation | Parallel execution | | File Reading | Read every time | Cache hit | | Cache Refresh | None | Every 1 min / >1 min | --- ## Usage ```bash # Initialize optimizer node skills/fast-response-optimizer/bootstrap.js # Manual cache refresh node skills/fast-response-optimizer/scripts/cache-manager.js refresh # Check cache status node skills/fast-response-optimizer/scripts/cache-manager.js status # Update message timestamp node skills/fast-response-optimizer/scripts/cache-manager.js message ``` --- *Make every response lightning fast ⚡*


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)
Skill: grahammiranda13/openclaw-ok-computer-skill (version 0.1.0)

--- name: ok-computer-swarm description: | Spawn multiple sub‑agents to perform concurrent research on a list of topics, inspired by Kimi.com’s OK Computer and Agent Swarm features【453334500861599†L40-L99】. Each sub‑agent performs a DuckDuckGo search for its assigned topic and returns the top results. When all sub‑agents finish, the skill aggregates their findings into a single structured report. Use this skill for broad, multi‑topic research where parallel exploration saves time. user-invocable: true metadata: moltbot: emoji: "🧠" requires: bins: ["python"] homepage: https://www.grahammiranda.com/ --- # OK Computer Swarm ## Overview This skill lets OpenClaw emulate the “100 sub‑agents” style of Kimi’s Agent Swarm【453334500861599†L40-L99】. When you need to research several topics at once, the skill spins up lightweight sub‑agents that fetch the top web results via DuckDuckGo. By running these tasks in parallel, the skill reduces overall waiting time and surfaces a diverse set of sources. ## Commands ### `/ok-computer-swarm search` Run concurrent searches for multiple topics. **Inputs** - `query` (string, repeated): One or more search phrases. You can provide multiple `query` flags to search many topics at once. At least one `query` is required. **Example** ```bash python scripts/swarm_search.py --query "Agent Swarm" --query "OpenClaw skills" ``` **Output** The script prints a JSON array where each element corresponds to a search query. Each element contains the original query and an array of result objects (title and URL). The format is easy for downstream agents to parse and can be further processed or summarised. ## When to use this skill Use `ok-computer-swarm` whenever you need to gather high‑level information on multiple topics concurrently. It is ideal for: - Broad research tasks that involve several different subjects. - Generating a starting point for more in‑depth analysis. - Situations where time is critical and sequential research would be too slow. ## Limitations - The skill uses DuckDuckGo’s free API; results may be less comprehensive than paid search APIs. - It performs minimal summarisation. Consider integrating additional summarisation or reading tools if you need deeper insights. ---


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)
Skill: mvogt99/think-block-leakage (version 1.0.0)

--- name: think-block-leakage description: Internal reasoning from <think> blocks leaks into the final user-facing reply instead of being stripped. emoji: 🧠 metadata: clawdis: os: [macos, linux, windows] --- # think-block-leakage The model's internal reasoning escapes into the reply the user sees. This usually means an unclosed `<think>` / `<thinking>` tag, or a reply that begins with planning prose instead of the answer itself. ## Symptoms - Reply contains literal `<think>`, `<thinking>`, or similar reasoning tags. - Reply opens with "Let me think...", "Okay, the user wants...", "First I'll need to...", or other planning preamble. - Reply is cut off mid-sentence and an opening reasoning tag has no matching close. - A `</think>` appears with only a few dozen characters of content after it. ## What to do - Inspect the raw LLM output for unmatched reasoning tags before returning it. Strip or redact any content inside reasoning tags. - If the provider supports a separate reasoning channel, emit reasoning there and keep it out of the reply body entirely. - If leakage is detected, regenerate the reply. Do not ship reasoning-as-answer. - If the reply starts with planning language, trim the preamble. The user should see the answer first. - For persistent leakage, tighten the system prompt to forbid meta-commentary in the reply body.


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)
Skill: willgary/recruitly-mcp (version 1.0.0)

--- name: recruitly-mcp description: "Search candidates, manage jobs, view pipelines, track billing and team performance in Recruitly CRM via MCP." homepage: https://recruitly.io metadata: openclaw: emoji: "🎯" requires: bins: - mcporter env: - RECRUITLY_TOKEN --- # Recruitly CRM Access your Recruitly recruitment CRM via MCP — search candidates, manage jobs, view pipelines, and track team performance from any messaging app. ## Setup ### 1. Get your token Sign in at [mcp.recruitly.dev](https://mcp.recruitly.dev) using your Recruitly account (Google, Microsoft, or email). Your OAuth token is issued automatically. ### 2. Configure mcporter Add to `config/mcporter.json`: ```json { "mcpServers": { "recruitly": { "baseUrl": "https://mcp.recruitly.dev/mcp", "description": "Recruitly CRM", "headers": { "Authorization": "Bearer ${RECRUITLY_TOKEN}" } } } } ``` ### 3. Set your token ```bash export RECRUITLY_TOKEN="your_oauth_token_here" ``` ## Available Tools ### Discovery | Tool | Description | |------|-------------| | **list_options** | Returns all field options (job statuses, candidate statuses, activity types) for your account. Call this first before filtering by status or type. | ### Search & List | Tool | Description | |------|-------------| | **search_candidates** | Search candidates by keyword, location, skills, or job title | | **search_contacts** | Search contacts (clients, hiring managers) by name, company, or email | | **search_companies** | Search companies by name, industry, or location | | **list_jobs** | List jobs with optional status and keyword filters | | **list_pipelines** | List hiring pipelines (deal flows) | ### Detail | Tool | Description | |------|-------------| | **get_record** | Get a specific candidate, contact, company, or job by ID | ## Usage Examples **Find candidates:** > "Find React developers in London" Calls `search_candidates` with query "React" and location "London". **Check open jobs:** > "Show me all open jobs" Calls `list_options` to get the exact status label for open jobs, then `list_jobs` with that status. **Pipeline overview:** > "What's in my pipeline?" Calls `list_pipelines` and summarises stage counts. **Company lookup:** > "Find me details on Acme Corp" Calls `search_companies` with query "Acme Corp", then `get_record` for the full company profile. **Weekly report:** > "Give me a summary of new candidates this week" Calls `search_candidates` with recent date filters and summarises by skills and location. ## Notes - All data respects your Recruitly account permissions and team visibility rules - Status and type values are tenant-configured (may be in any language) — always call `list_options` first - Search results are capped at 50 per request; use pagination for larger result sets - Works on Recruitly's free plan — no upgrade needed ## Links - [Recruitly](https://recruitly.io) - [MCP Server](https://recruitly.io/mcp)


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)
Skill: xujiangdong/file-report-skill-xjd (version 1.0.0)

--- name: file-report-skill description: 统计指定目录的文件类型和数量，生成Markdown格式报表 version: 1.0.0 \# 触发器配置 triggers: &nbsp; - "统计文件" &nbsp; - "文件报表" &nbsp; - "目录分析" \# 技能执行所需权限 requires: &nbsp; - filesystem.read &nbsp; - filesystem.write --- \# 文件统计报表技能 当用户需要统计一个文件夹里的文件情况时，请按以下流程操作： \## 执行步骤 1\. \*\*获取目标路径\*\*：从用户的对话中提取出要统计的目录路径。如果用户没有提供，请主动询问。 2\. \*\*统计文件信息\*\*： &nbsp; \* 读取目标目录下的所有文件（\*\*注意\*\*：不包含子目录，如需要递归统计请说明）。 &nbsp; \* 根据文件的扩展名（如 `.txt`, `.jpg`, `.pdf`）进行分类计数。 &nbsp; \* 对于没有扩展名的文件，归类为“无扩展名”。 3\. \*\*生成报表\*\*： &nbsp; \* 创建一个Markdown格式的报表。 &nbsp; \* 报表内容应包含统计的目录路径、统计时间、一个包含“文件类型”和“数量”的表格，以及文件总数。 4\. \*\*输出结果\*\*：将生成的Markdown报表内容直接通过对话返回给用户。 \## 使用示例 \*\*用户\*\*: 请帮我统计一下 `/home/user/documents` 文件夹里的文件情况。 \*\*你\*\*: (执行上述步骤后返回) 好的，已为您统计 `/home/user/documents` 目录下的文件： | 文件类型 | 数量 | |----------|------| | .txt | 15 | | .pdf | 8 | | .jpg | 23 | | 无扩展名 | 2 | \*\*总文件数\*\*：48份


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)
Skill: kaiyuelv/smart-crawler (version 1.0.0)

--- name: smart-crawler description: 智能爬虫工具 - 企业级数据采集与反爬虫处理 | Smart Web Crawler - Enterprise data collection with anti-detection homepage: https://github.com/openclaw/smart-crawler category: data-collection tags: ["crawler", "scraping", "data-collection", "playwright", "selenium", "automation"] --- # Smart Crawler - 智能爬虫工具 企业级数据采集解决方案，支持智能反爬虫处理、分布式爬取和数据清洗。 ## 核心功能 | 功能模块 | 说明 | |---------|------| | **智能爬虫引擎** | 基于 Playwright/Selenium 的动态渲染爬取 | | **反爬虫处理** | 自动切换 User-Agent、代理池、请求频率控制 | | **数据提取** | XPath/CSS Selector/Regex 多模式数据提取 | | **分布式支持** | Redis 队列支持的分布式爬取 | | **数据清洗** | 自动去重、格式标准化、敏感信息过滤 | ## 快速开始 ```python from scripts.crawler_engine import CrawlerEngine # 创建爬虫引擎 crawler = CrawlerEngine(use_proxy=True, headless=True) # 爬取网页 result = crawler.crawl('https://example.com', extract_rules={'title': '//h1/text()', 'content': '//div[@class="content"]//p/text()'}) print(result) ``` ## 安装 ```bash pip install -r requirements.txt playwright install ``` ## 项目结构 ``` smart-crawler/ ├── SKILL.md # Skill说明文档 ├── README.md # 完整文档 ├── requirements.txt # 依赖列表 ├── scripts/ # 核心模块 │ ├── crawler_engine.py # 爬虫引擎 │ ├── proxy_manager.py # 代理管理器 │ ├── data_extractor.py # 数据提取器 │ └── anti_detection.py # 反检测模块 ├── examples/ # 使用示例 │ └── basic_usage.py └── tests/ # 单元测试 └── test_crawler.py ``` ## 运行测试 ```bash cd tests python test_crawler.py ```


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)