Skill: shaharsha/tavily-search-pro (version 1.0.0)

--- name: tavily-search-pro slug: tavily-search-pro description: > Tavily AI search platform with 5 modes: Search (web/news/finance), Extract (URL content), Crawl (website crawling), Map (sitemap discovery), and Research (deep research with citations). Use for: web search with LLM answers, content extraction, site crawling, deep research. version: 1.0.0 author: Leo 🦁 tags: [search, tavily, web, news, finance, extract, crawl, research, api] metadata: {"clawdbot":{"emoji":"🔎","requires":{"env":["TAVILY_API_KEY"]},"primaryEnv":"TAVILY_API_KEY","install":[{"id":"pip","kind":"pip","package":"tavily-python","label":"Install dependencies (pip)"}]}} allowed-tools: [exec] --- # Tavily Search 🔎 AI-powered web search platform with 5 modes: Search, Extract, Crawl, Map, and Research. ## Requirements - `TAVILY_API_KEY` environment variable ## Configuration | Env Variable | Default | Description | |---|---|---| | `TAVILY_API_KEY` | — | **Required.** Tavily API key | Set in OpenClaw config: ```json { "env": { "TAVILY_API_KEY": "tvly-..." } } ``` ## Script Location ```bash python3 skills/tavily/lib/tavily_search.py <command> "query" [options] ``` --- ## Commands ### search — Web Search (Default) General-purpose web search with optional LLM-synthesized answer. ```bash python3 lib/tavily_search.py search "query" [options] ``` **Examples:** ```bash # Basic search python3 lib/tavily_search.py search "latest AI news" # With LLM answer python3 lib/tavily_search.py search "what is quantum computing" --answer # Advanced depth (better results, 2 credits) python3 lib/tavily_search.py search "climate change solutions" --depth advanced # Time-filtered python3 lib/tavily_search.py search "OpenAI announcements" --time week # Domain filtering python3 lib/tavily_search.py search "machine learning" --include-domains arxiv.org,nature.com # Country boost python3 lib/tavily_search.py search "tech startups" --country US # With raw content and images python3 lib/tavily_search.py search "solar energy" --raw --images -n 10 # JSON output python3 lib/tavily_search.py search "bitcoin price" --json ``` **Output format (text):** ``` Answer: <LLM-synthesized answer if --answer> Results: 1. Result Title https://example.com/article Content snippet from the page... 2. Another Result https://example.com/other Another snippet... ``` --- ### news — News Search Search optimized for news articles. Sets `topic=news`. ```bash python3 lib/tavily_search.py news "query" [options] ``` **Examples:** ```bash python3 lib/tavily_search.py news "AI regulation" python3 lib/tavily_search.py news "Israel tech" --time day --answer python3 lib/tavily_search.py news "stock market" --time week -n 10 ``` --- ### finance — Finance Search Search optimized for financial data and news. Sets `topic=finance`. ```bash python3 lib/tavily_search.py finance "query" [options] ``` **Examples:** ```bash python3 lib/tavily_search.py finance "NVIDIA stock analysis" python3 lib/tavily_search.py finance "cryptocurrency market trends" --time month python3 lib/tavily_search.py finance "S&P 500 forecast 2026" --answer ``` --- ### extract — Extract Content from URLs Extract readable content from one or more URLs. ```bash python3 lib/tavily_search.py extract URL [URL...] [options] ``` **Parameters:** - `urls`: One or more URLs to extract (positional args) - `--depth basic|advanced`: Extraction depth - `--format markdown|text`: Output format (default: markdown) - `--query "text"`: Rerank extracted chunks by relevance to query **Examples:** ```bash # Extract single URL python3 lib/tavily_search.py extract "https://example.com/article" # Extract multiple URLs python3 lib/tavily_search.py extract "https://url1.com" "https://url2.com" # Advanced extraction with relevance reranking python3 lib/tavily_search.py extract "https://arxiv.org/paper" --depth advanced --query "transformer architecture" # Text format output python3 lib/tavily_search.py extract "https://example.com" --format text ``` **Output format:** ``` URL: https://example.com/article ───────────────────────────────── <Extracted content in markdown/text> URL: https://another.com/page ───────────────────────────────── <Extracted content> ``` --- ### crawl — Crawl a Website Crawl a website starting from a root URL, following links. ```bash python3 lib/tavily_search.py crawl URL [options] ``` **Parameters:** - `url`: Root URL to start crawling - `--depth basic|advanced`: Crawl depth - `--max-depth N`: Maximum link depth to follow (default: 2) - `--max-breadth N`: Maximum pages per depth level (default: 10) - `--limit N`: Maximum total pages (default: 10) - `--instructions "text"`: Natural language crawl instructions - `--select-paths p1,p2`: Only crawl these path patterns - `--exclude-paths p1,p2`: Skip these path patterns - `--format markdown|text`: Output format **Examples:** ```bash # Basic crawl python3 lib/tavily_search.py crawl "https://docs.example.com" # Focused crawl with instructions python3 lib/tavily_search.py crawl "https://docs.python.org" --instructions "Find all asyncio documentation" --limit 20 # Crawl specific paths only python3 lib/tavily_search.py crawl "https://example.com" --select-paths "/blog,/docs" --max-depth 3 ``` **Output format:** ``` Crawled 5 pages from https://docs.example.com Page 1: https://docs.example.com/intro ───────────────────────────────── <Content> Page 2: https://docs.example.com/guide ───────────────────────────────── <Content> ``` --- ### map — Sitemap Discovery Discover all URLs on a website (sitemap). ```bash python3 lib/tavily_search.py map URL [options] ``` **Parameters:** - `url`: Root URL to map - `--max-depth N`: Depth to follow (default: 2) - `--max-breadth N`: Breadth per level (default: 20) - `--limit N`: Maximum URLs (default: 50) **Examples:** ```bash # Map a site python3 lib/tavily_search.py map "https://example.com" # Deep map python3 lib/tavily_search.py map "https://docs.python.org" --max-depth 3 --limit 100 ``` **Output format:** ``` Sitemap for https://example.com (42 URLs found): 1. https://example.com/ 2. https://example.com/about 3. https://example.com/blog ... ``` --- ### research — Deep Research Comprehensive AI-powered research on a topic with citations. ```bash python3 lib/tavily_search.py research "query" [options] ``` **Parameters:** - `query`: Research question - `--model mini|pro|auto`: Research model (default: auto) - `mini`: Faster, cheaper - `pro`: More thorough - `auto`: Let Tavily decide - `--json`: JSON output (supports structured output schema) **Examples:** ```bash # Basic research python3 lib/tavily_search.py research "Impact of AI on healthcare in 2026" # Pro model for thorough research python3 lib/tavily_search.py research "Comparison of quantum computing approaches" --model pro # JSON output python3 lib/tavily_search.py research "Electric vehicle market analysis" --json ``` **Output format:** ``` Research: Impact of AI on healthcare in 2026 <Comprehensive research report with citations> Sources: [1] https://source1.com [2] https://source2.com ... ``` --- ## Options Reference | Option | Applies To | Description | Default | |---|---|---|---| | `--depth basic\|advanced` | search, news, finance, extract | Search/extraction depth | basic | | `--time day\|week\|month\|year` | search, news, finance | Time range filter | none | | `-n NUM` | search, news, finance | Max results (0-20) | 5 | | `--answer` | search, news, finance | Include LLM answer | off | | `--raw` | search, news, finance | Include raw page content | off | | `--images` | search, news, finance | Include image URLs | off | | `--include-domains d1,d2` | search, news, finance | Only these domains | none | | `--exclude-domains d1,d2` | search, news, finance | Exclude these domains | none | | `--country XX` | search, news, finance | Boost country results | none | | `--json` | all | Structured JSON output | off | | `--format markdown\|text` | extract, crawl | Content format | markdown | | `--query "text"` | extract | Relevance reranking query | none | | `--model mini\|pro\|auto` | research | Research model | auto | | `--max-depth N` | crawl, map | Max link depth | 2 | | `--max-breadth N` | crawl, map | Max pages per level | 10/20 | | `--limit N` | crawl, map | Max total pages/URLs | 10/50 | | `--instructions "text"` | crawl | Natural language instructions | none | | `--select-paths p1,p2` | crawl | Include path patterns | none | | `--exclude-paths p1,p2` | crawl | Exclude path patterns | none | --- ## Error Handling - **Missing API key:** Clear error message with setup instructions. - **401 Unauthorized:** Invalid API key. - **429 Rate Limit:** Rate limit exceeded, try again later. - **Network errors:** Descriptive error with cause. - **No results:** Clean "No results found." message. - **Timeout:** 30-second timeout on all HTTP requests. --- ## Credits & Pricing | API | Basic | Advanced | |---|---|---| | Search | 1 credit | 2 credits | | Extract | 1 credit/URL | 2 credits/URL | | Crawl | 1 credit/page | 2 credits/page | | Map | 1 credit | 1 credit | | Research | Varies by model | - | --- ## Install ```bash bash skills/tavily/install.sh ```

path: install.sh
content: #!/usr/bin/env bash
# Tavily Search skill installer
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
echo "📦 Installing Tavily Search skill..."

# Install Python dependencies
pip install --break-system-packages --quiet tavily-python 2>/dev/null || {
    echo "⚠️  pip install failed, trying without --break-system-packages..."
    pip install --quiet tavily-python 2>/dev/null || {
        echo "❌ Failed to install tavily-python. Install manually: pip install tavily-python"
        exit 1
    }
}

# Verify API key
if [ -z "${TAVILY_API_KEY:-}" ]; then
    echo "⚠️  TAVILY_API_KEY not set. Set it in OpenClaw config before using."
else
    echo "✅ TAVILY_API_KEY found"
fi

# Quick smoke test
if python3 "$SCRIPT_DIR/lib/tavily_search.py" --help >/dev/null 2>&1; then
    echo "✅ Tavily Search skill ready."
else
    echo "⚠️  Smoke test failed - check Python dependencies."
    exit 1
fi


path: lib/tavily_search.py
content: #!/usr/bin/env python3
"""
Tavily Search v1.0 - AI-powered web search platform with 5 modes.
Author: Leo 🦁
Created: 2026-02-07

Commands:
- search: General web search with optional LLM answer
- news: News-optimized search (topic=news)
- finance: Finance-optimized search (topic=finance)
- extract: Extract content from URLs
- crawl: Crawl a website
- map: Discover sitemap URLs
- research: Deep AI research with citations

Environment Variables:
- TAVILY_API_KEY: Required. Tavily API key.
"""

import argparse
import json
import os
import sys
import urllib.request
import urllib.error
from typing import Any, Optional

# ─── Configuration ───────────────────────────────────────────────────────────

API_KEY: str = os.environ.get("TAVILY_API_KEY", "")
BASE_URL: str = "https://api.tavily.com"
REQUEST_TIMEOUT: int = 30
RESEARCH_TIMEOUT: int = 120  # Research can take longer


# ─── HTTP Helper ─────────────────────────────────────────────────────────────

def _api_request(
    endpoint: str,
    payload: dict[str, Any],
    timeout: int = REQUEST_TIMEOUT,
) -> dict[str, Any]:
    """
    Make a POST request to the Tavily API.

    Args:
        endpoint: API endpoint path (e.g., '/search').
        payload: JSON request body.
        timeout: Request timeout in seconds.

    Returns:
        Parsed JSON response.

    Raises:
        SystemExit: On API errors with descriptive messages.
    """
    url = f"{BASE_URL}{endpoint}"
    data = json.dumps(payload).encode("utf-8")
    req = urllib.request.Request(
        url,
        data=data,
        headers={
            "Content-Type": "application/json",
            "Authorization": f"Bearer {API_KEY}",
        },
        method="POST",
    )

    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            return json.loads(resp.read())
    except urllib.error.HTTPError as e:
        body = e.read().decode("utf-8", errors="replace")[:500]
        if e.code == 401:
            print("Error: Invalid TAVILY_API_KEY. Check your key at https://app.tavily.com", file=sys.stderr)
        elif e.code == 429:
            print("Error: Rate limit exceeded. Try again later.", file=sys.stderr)
        elif e.code == 400:
            # Try to extract error message from JSON response
            try:
                err_data = json.loads(body)
                msg = err_data.get("detail", err_data.get("message", body))
                print(f"Error: Bad request - {msg}", file=sys.stderr)
            except (json.JSONDecodeError, KeyError):
                print(f"Error: Bad request - {body}", file=sys.stderr)
        else:
            print(f"Error: Tavily API returned {e.code}: {body}", file=sys.stderr)
        sys.exit(1)
    except urllib.error.URLError as e:
        print(f"Error: Network error - {e.reason}", file=sys.stderr)
        sys.exit(1)
    except TimeoutError:
        print(f"Error: Request timed out after {timeout}s", file=sys.stderr)
        sys.exit(1)


# ─── Output Formatting ──────────────────────────────────────────────────────

def _format_search_results(data: dict[str, Any], as_json: bool = False) -> str:
    """Format search/news/finance results for display."""
    if as_json:
        return json.dumps(data, ensure_ascii=False, indent=2)

    lines: list[str] = []

    # LLM answer
    answer = data.get("answer")
    if answer:
        lines.append(f"Answer: {answer}")
        lines.append("")

    # Images
    images = data.get("images")
    if images:
        lines.append("Images:")
        for img in images:
            if isinstance(img, dict):
                lines.append(f"  - {img.get('url', img)}")
            else:
                lines.append(f"  - {img}")
        lines.append("")

    # Results
    results = data.get("results", [])
    if results:
        lines.append("Results:")
        for i, r in enumerate(results, 1):
            title = r.get("title", "Untitled")
            url = r.get("url", "")
            content = r.get("content", "")
            score = r.get("score")
            published = r.get("published_date", "")

            lines.append(f"  {i}. {title}")
            lines.append(f"     {url}")
            if published:
                lines.append(f"     Published: {published}")
            if score is not None:
                lines.append(f"     Score: {score:.4f}")
            if content:
                # Truncate long content to keep output readable
                snippet = content[:500].strip()
                if len(content) > 500:
                    snippet += "..."
                lines.append(f"     {snippet}")

            # Raw content (if requested)
            raw = r.get("raw_content")
            if raw:
                lines.append(f"     --- Raw Content ---")
                raw_snippet = raw[:1000].strip()
                if len(raw) > 1000:
                    raw_snippet += f"... [{len(raw)} chars total]"
                lines.append(f"     {raw_snippet}")

            lines.append("")
    elif not answer:
        lines.append("No results found.")

    return "\n".join(lines).rstrip()


def _format_extract_results(data: dict[str, Any], as_json: bool = False) -> str:
    """Format extract results for display."""
    if as_json:
        return json.dumps(data, ensure_ascii=False, indent=2)

    lines: list[str] = []
    results = data.get("results", [])

    if not results:
        return "No content extracted."

    for r in results:
        url = r.get("url", "Unknown URL")
        content = r.get("raw_content", "")
        lines.append(f"URL: {url}")
        lines.append("─" * 50)
        if content:
            lines.append(content.strip())
        else:
            lines.append("(No content extracted)")
        lines.append("")

    # Failed URLs
    failed = data.get("failed_results", [])
    if failed:
        lines.append("Failed URLs:")
        for f in failed:
            url = f.get("url", "Unknown")
            error = f.get("error", "Unknown error")
            lines.append(f"  ✗ {url}: {error}")

    return "\n".join(lines).rstrip()


def _format_crawl_results(data: dict[str, Any], as_json: bool = False) -> str:
    """Format crawl results for display."""
    if as_json:
        return json.dumps(data, ensure_ascii=False, indent=2)

    lines: list[str] = []
    results = data.get("results", [])
    base_url = data.get("base_url", "")

    lines.append(f"Crawled {len(results)} pages from {base_url}")
    lines.append("")

    for i, r in enumerate(results, 1):
        url = r.get("url", "Unknown URL")
        content = r.get("raw_content", "")
        lines.append(f"Page {i}: {url}")
        lines.append("─" * 50)
        if content:
            # Truncate very long pages
            snippet = content[:2000].strip()
            if len(content) > 2000:
                snippet += f"\n... [{len(content)} chars total]"
            lines.append(snippet)
        else:
            lines.append("(No content)")
        lines.append("")

    # Failed
    failed = data.get("failed_results", [])
    if failed:
        lines.append("Failed URLs:")
        for f in failed:
            url = f.get("url", "Unknown")
            error = f.get("error", "Unknown error")
            lines.append(f"  ✗ {url}: {error}")

    return "\n".join(lines).rstrip()


def _format_map_results(data: dict[str, Any], url: str, as_json: bool = False) -> str:
    """Format map/sitemap results for display."""
    if as_json:
        return json.dumps(data, ensure_ascii=False, indent=2)

    urls = data.get("results", [])
    lines: list[str] = []
    lines.append(f"Sitemap for {url} ({len(urls)} URLs found):")
    lines.append("")

    for i, u in enumerate(urls, 1):
        if isinstance(u, dict):
            lines.append(f"  {i}. {u.get('url', u)}")
        else:
            lines.append(f"  {i}. {u}")

    if not urls:
        lines.append("  No URLs discovered.")

    return "\n".join(lines).rstrip()


def _format_research_results(data: dict[str, Any], as_json: bool = False) -> str:
    """Format research results for display."""
    if as_json:
        return json.dumps(data, ensure_ascii=False, indent=2)

    lines: list[str] = []

    # Topic
    topic = data.get("topic") or data.get("query", "")
    if topic:
        lines.append(f"Research: {topic}")
        lines.append("")

    # Main content
    content = data.get("content") or data.get("output") or data.get("report", "")
    if content:
        lines.append(content.strip())
    else:
        lines.append("No research output returned.")

    # Sources
    sources = data.get("sources", [])
    if sources:
        lines.append("")
        lines.append("Sources:")
        for i, src in enumerate(sources, 1):
            if isinstance(src, dict):
                url = src.get("url", src.get("link", str(src)))
                title = src.get("title", "")
                if title:
                    lines.append(f"  [{i}] {title}")
                    lines.append(f"      {url}")
                else:
                    lines.append(f"  [{i}] {url}")
            else:
                lines.append(f"  [{i}] {src}")

    return "\n".join(lines).rstrip()


# ─── Commands ────────────────────────────────────────────────────────────────

def cmd_search(args: argparse.Namespace) -> str:
    """Execute search/news/finance command."""
    topic_map = {
        "search": "general",
        "news": "news",
        "finance": "finance",
    }

    payload: dict[str, Any] = {
        "query": args.query,
        "topic": topic_map.get(args.command, "general"),
        "search_depth": args.depth,
        "max_results": args.n,
    }

    if args.answer:
        payload["include_answer"] = True
    if args.raw:
        payload["include_raw_content"] = "markdown"
    if args.images:
        payload["include_images"] = True
    if args.time:
        payload["time_range"] = args.time
    if args.include_domains:
        payload["include_domains"] = [d.strip() for d in args.include_domains.split(",")]
    if args.exclude_domains:
        payload["exclude_domains"] = [d.strip() for d in args.exclude_domains.split(",")]
    if args.country:
        payload["country"] = args.country

    data = _api_request("/search", payload)
    return _format_search_results(data, as_json=args.as_json)


def cmd_extract(args: argparse.Namespace) -> str:
    """Execute extract command."""
    urls = args.urls
    if not urls:
        print("Error: At least one URL is required for extract.", file=sys.stderr)
        sys.exit(1)

    payload: dict[str, Any] = {
        "urls": urls if len(urls) > 1 else urls[0],
    }

    if args.depth and args.depth != "basic":
        payload["extract_depth"] = args.depth
    if hasattr(args, "format_type") and args.format_type:
        payload["format"] = args.format_type
    if hasattr(args, "query") and args.query:
        payload["query"] = args.query

    data = _api_request("/extract", payload)
    return _format_extract_results(data, as_json=args.as_json)


def cmd_crawl(args: argparse.Namespace) -> str:
    """Execute crawl command."""
    payload: dict[str, Any] = {
        "url": args.url,
    }

    if args.max_depth is not None:
        payload["max_depth"] = args.max_depth
    if args.max_breadth is not None:
        payload["max_breadth"] = args.max_breadth
    if args.limit is not None:
        payload["limit"] = args.limit
    if hasattr(args, "instructions") and args.instructions:
        payload["instructions"] = args.instructions
    if hasattr(args, "select_paths") and args.select_paths:
        payload["select_paths"] = [p.strip() for p in args.select_paths.split(",")]
    if hasattr(args, "exclude_paths") and args.exclude_paths:
        payload["exclude_paths"] = [p.strip() for p in args.exclude_paths.split(",")]
    if hasattr(args, "format_type") and args.format_type:
        payload["format"] = args.format_type

    data = _api_request("/crawl", payload, timeout=60)
    return _format_crawl_results(data, as_json=args.as_json)


def cmd_map(args: argparse.Namespace) -> str:
    """Execute map/sitemap command."""
    payload: dict[str, Any] = {
        "url": args.url,
    }

    if args.max_depth is not None:
        payload["max_depth"] = args.max_depth
    if args.max_breadth is not None:
        payload["max_breadth"] = args.max_breadth
    if args.limit is not None:
        payload["limit"] = args.limit

    data = _api_request("/map", payload)
    return _format_map_results(data, args.url, as_json=args.as_json)


def cmd_research(args: argparse.Namespace) -> str:
    """Execute research command."""
    payload: dict[str, Any] = {
        "input": args.query,
    }

    if args.model:
        payload["model"] = args.model

    data = _api_request("/research", payload, timeout=RESEARCH_TIMEOUT)
    return _format_research_results(data, as_json=args.as_json)


# ─── CLI ─────────────────────────────────────────────────────────────────────

def build_parser() -> argparse.ArgumentParser:
    """Build the argument parser with all subcommands."""
    parser = argparse.ArgumentParser(
        description="Tavily Search v1.0 - AI-powered web search platform",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
Examples:
  %(prog)s search "latest AI news" --answer
  %(prog)s news "tech industry" --time week
  %(prog)s finance "NVIDIA stock" --depth advanced
  %(prog)s extract "https://example.com/article"
  %(prog)s crawl "https://docs.example.com" --limit 20
  %(prog)s map "https://example.com"
  %(prog)s research "Impact of AI on healthcare"
        """,
    )

    subparsers = parser.add_subparsers(dest="command", help="Command to execute")

    # ── Common search options ──
    def add_search_options(sub: argparse.ArgumentParser) -> None:
        sub.add_argument("query", help="Search query")
        sub.add_argument("--depth", choices=["basic", "advanced"], default="basic",
                         help="Search depth (default: basic; advanced = 2 credits)")
        sub.add_argument("--time", choices=["day", "week", "month", "year", "d", "w", "m", "y"],
                         default=None, help="Time range filter")
        sub.add_argument("-n", type=int, default=5, help="Max results 0-20 (default: 5)")
        sub.add_argument("--answer", action="store_true", help="Include LLM-synthesized answer")
        sub.add_argument("--raw", action="store_true", help="Include raw page content")
        sub.add_argument("--images", action="store_true", help="Include image URLs")
        sub.add_argument("--include-domains", default=None,
                         help="Comma-separated domains to include")
        sub.add_argument("--exclude-domains", default=None,
                         help="Comma-separated domains to exclude")
        sub.add_argument("--country", default=None, help="Country code to boost (e.g., US, IL)")
        sub.add_argument("--json", action="store_true", dest="as_json", help="JSON output")

    # search
    p_search = subparsers.add_parser("search", help="Web search (general)")
    add_search_options(p_search)

    # news
    p_news = subparsers.add_parser("news", help="News search")
    add_search_options(p_news)

    # finance
    p_finance = subparsers.add_parser("finance", help="Finance search")
    add_search_options(p_finance)

    # extract
    p_extract = subparsers.add_parser("extract", help="Extract content from URLs")
    p_extract.add_argument("urls", nargs="+", help="URLs to extract content from")
    p_extract.add_argument("--depth", choices=["basic", "advanced"], default="basic",
                           help="Extraction depth")
    p_extract.add_argument("--format", dest="format_type", choices=["markdown", "text"],
                           default=None, help="Output format (default: markdown)")
    p_extract.add_argument("--query", default=None,
                           help="Query for relevance reranking of chunks")
    p_extract.add_argument("--json", action="store_true", dest="as_json", help="JSON output")

    # crawl
    p_crawl = subparsers.add_parser("crawl", help="Crawl a website")
    p_crawl.add_argument("url", help="Root URL to crawl")
    p_crawl.add_argument("--depth", choices=["basic", "advanced"], default=None,
                         help="Crawl depth")
    p_crawl.add_argument("--max-depth", type=int, default=None, help="Max link depth (default: 2)")
    p_crawl.add_argument("--max-breadth", type=int, default=None,
                         help="Max pages per level (default: 10)")
    p_crawl.add_argument("--limit", type=int, default=None, help="Max total pages (default: 10)")
    p_crawl.add_argument("--instructions", default=None,
                         help="Natural language crawl instructions")
    p_crawl.add_argument("--select-paths", default=None,
                         help="Comma-separated path patterns to include")
    p_crawl.add_argument("--exclude-paths", default=None,
                         help="Comma-separated path patterns to exclude")
    p_crawl.add_argument("--format", dest="format_type", choices=["markdown", "text"],
                         default=None, help="Output format")
    p_crawl.add_argument("--json", action="store_true", dest="as_json", help="JSON output")

    # map
    p_map = subparsers.add_parser("map", help="Discover sitemap URLs")
    p_map.add_argument("url", help="Root URL to map")
    p_map.add_argument("--max-depth", type=int, default=None, help="Max depth (default: 2)")
    p_map.add_argument("--max-breadth", type=int, default=None,
                       help="Max breadth per level (default: 20)")
    p_map.add_argument("--limit", type=int, default=None, help="Max URLs (default: 50)")
    p_map.add_argument("--json", action="store_true", dest="as_json", help="JSON output")

    # research
    p_research = subparsers.add_parser("research", help="Deep AI research")
    p_research.add_argument("query", help="Research question")
    p_research.add_argument("--model", choices=["mini", "pro", "auto"], default=None,
                            help="Research model (default: auto)")
    p_research.add_argument("--json", action="store_true", dest="as_json", help="JSON output")

    return parser


def main() -> None:
    """CLI entry point."""
    parser = build_parser()
    args = parser.parse_args()

    if not args.command:
        parser.print_help()
        sys.exit(1)

    if not [REDACTED_SECRET]"Error: TAVILY_API_KEY environment variable not set.", file=sys.stderr)
        print("Set it in OpenClaw config or export TAVILY_API_KEY=your_key", file=sys.stderr)
        sys.exit(1)

    try:
        if args.command in ("search", "news", "finance"):
            result = cmd_search(args)
        elif args.command == "extract":
            result = cmd_extract(args)
        elif args.command == "crawl":
            result = cmd_crawl(args)
        elif args.command == "map":
            result = cmd_map(args)
        elif args.command == "research":
            result = cmd_research(args)
        else:
            parser.print_help()
            sys.exit(1)

        print(result)
    except SystemExit:
        raise
    except Exception as e:
        print(f"Error: {e}", file=sys.stderr)
        sys.exit(1)


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)