Skill: robbyczgw-cla/youtube-apify-transcript (version 1.3.3)

--- name: youtube-apify-transcript version: 1.3.3 description: Fetch YouTube transcripts via APIFY API. Works from cloud IPs (Hetzner, AWS, etc.) by bypassing YouTube's bot detection. Features local caching (FREE repeat requests!) and batch mode. Requires APIFY_API_TOKEN and Python requests. No credit card required for the free tier. tags: [youtube, transcript, apify, video, subtitles, captions, cloud-ip, free-tier, web-scraping, caching, batch] metadata: {"openclaw":{"requires":{"bins":["python3"],"env":{"APIFY_API_TOKEN":"required","YT_TRANSCRIPT_CACHE_DIR":"optional - defaults to .cache/ in skill dir"}},"install":[{"id":"pip-requests","kind":"pip","package":"requests","bins":["python3"],"label":"Install Python requests"}]}} --- # youtube-apify-transcript Fetch YouTube transcripts via APIFY API (works from cloud IPs, bypasses YouTube bot detection). ## Why APIFY? YouTube blocks transcript requests from cloud IPs (AWS, GCP, etc.). APIFY runs the request through residential proxies, bypassing bot detection reliably. ## Free Tier - **$5/month free credits** (~714 videos) - No credit card required - Perfect for personal use ## Cost - **$0.007 per video** (less than 1 cent!) - Track usage at: https://console.apify.com/billing ## Links - 🔗 [APIFY Pricing](https://apify.com/pricing) - 🔑 [Get API Key](https://console.apify.com/account/integrations) - 🎬 [YouTube Transcripts Actor](https://apify.com/topaz_sharingan/Youtube-Transcript-Scraper-1) ## Setup 1. Create free APIFY account: https://apify.com/ 2. Get your API [REDACTED_SECRET] 3. Set environment variable: ```bash # Add to ~/.bashrc or ~/.zshrc export APIFY_API_TOKEN="apify_api_YOUR_TOKEN_HERE" # Install Python dependency pip install requests # Or use .env file (never commit this!) echo [REDACTED_SECRET] >> .env ``` ## Usage ### Basic Usage ```bash # Get transcript as text (uses cache by default) python3 scripts/fetch_transcript.py "https://www.youtube.com/watch?v=VIDEO_ID" # Short URL also works python3 scripts/fetch_transcript.py "https://youtu.be/VIDEO_ID" ``` ### Options ```bash # Output to file python3 scripts/fetch_transcript.py "URL" --output transcript.txt # JSON format (includes timestamps) python3 scripts/fetch_transcript.py "URL" --json # Both: JSON to file python3 scripts/fetch_transcript.py "URL" --json --output transcript.json # Specify language preference python3 scripts/fetch_transcript.py "URL" --lang de ``` ### Caching (saves money!) Transcripts are cached locally by default. Repeat requests for the same video cost $0. ```bash # First request: fetches from APIFY ($0.007) python3 scripts/fetch_transcript.py "URL" # Second request: uses cache (FREE!) python3 scripts/fetch_transcript.py "URL" # Output: [cached] Transcript for: VIDEO_ID # Bypass cache (force fresh fetch) python3 scripts/fetch_transcript.py "URL" --no-cache # View cache stats python3 scripts/fetch_transcript.py --cache-stats # Clear all cached transcripts python3 scripts/fetch_transcript.py --clear-cache ``` Cache location: `.cache/` in skill directory (override with `YT_TRANSCRIPT_CACHE_DIR` env var) ### Batch Mode Process multiple videos at once: ```bash # Create a file with URLs (one per line) cat > urls.txt << EOF https://youtube.com/watch?v=VIDEO1 https://youtu.be/VIDEO2 https://youtube.com/watch?v=VIDEO3 EOF # Process all URLs python3 scripts/fetch_transcript.py --batch urls.txt # Output: # [1/3] Fetching VIDEO1... # [2/3] [cached] VIDEO2 # [3/3] Fetching VIDEO3... # Batch complete: 2 fetched, 1 cached, 0 failed # [Cost: ~$0.014 for 2 API call(s)] # Batch with JSON output to file python3 scripts/fetch_transcript.py --batch urls.txt --json --output all_transcripts.json ``` ### Output Formats **Text (default):** ``` Hello and welcome to this video. Today we're going to talk about... ``` **JSON (--json):** ```json { "video_id": "dQw4w9WgXcQ", "title": "Video Title", "transcript": [ {"start": 0.0, "duration": 2.5, "text": "Hello and welcome"}, {"start": 2.5, "duration": 3.0, "text": "to this video"} ], "full_text": "Hello and welcome to this video..." } ``` ## Agent Instructions When the user asks to summarize a YouTube video, first fetch the transcript using the script, then summarize the transcript text directly using your own model capabilities. Do NOT use --summarize flag. ## Error Handling The script handles common errors: - Invalid YouTube URL - Video has no transcript - API quota exceeded - Network errors ## Metadata ```yaml metadata: clawdbot: emoji: "📹" requires: env: ["APIFY_API_TOKEN"] bins: ["python3"] python: packages: ["requests"] ```

path: CHANGELOG.md
content: # Changelog

## [1.3.3] - 2026-03-31

### Fixed
- Declared Python dependency `requests` explicitly in skill metadata and install hints
- Expanded package metadata so required env vars and optional cache env var are stated more explicitly for registry scanners
- Clarified setup docs around `APIFY_API_TOKEN` and Python dependency installation

## [1.3.2] - 2026-03-31

### Changed
- Removed --summarize / --summary-* flags from fetch_transcript.py: summarization now handled by the OpenClaw agent directly using the configured model, not via hardcoded claude CLI subprocess
- Removed `claude` CLI dependency from skill metadata
- Updated SKILL.md with agent-level summarization instructions

## [1.3.1] - 2026-03-31

### Fixed
- Declared `claude` CLI as required binary in metadata (`bins`) — summarization feature was invoking it without declaration, causing ClawHub review warnings
- Fixed model IDs in summarize function: removed `anthropic/` provider prefix (Claude CLI only accepts short IDs like `claude-haiku-4-5`)
- Updated skill description to mention optional AI summarization feature

## [1.3.0] - 2026-03-31

### Added
- Added `--summarize` / `-s` to generate transcript summaries with the Claude CLI.
- Added `--summary-model`, `--summary-style`, and `--summary-lang` flags for summary customization.
- Cached summaries in each transcript cache file under the `summaries` key and reused cached summaries when available.
- Added JSON output support for summaries via a top-level `summary` field.

## [1.1.3] - 2026-03-03

### Changed
- Improved auth/cache docs and synced metadata cleanup changes.


## [1.1.2] - 2026-02-11

### 🆕 Local Caching

- **FREE Repeat Requests:** Transcripts are cached locally — no API cost for re-fetching!
- **Cache Location:** `.cache/` in skill directory (configurable via `YT_TRANSCRIPT_CACHE_DIR`)
- **Cache Management:**
  - `--cache-stats` — View cache statistics
  - `--no-cache` — Bypass cache, fetch fresh
  - `--clear-cache` — Delete all cached transcripts

### 🆕 Batch Mode

- **Multiple Videos:** Process a list of URLs in one command
- **Usage:** `--batch urls.txt` where file contains one URL per line
- **Output:** Shows progress, cached vs fetched count, total cost estimate

### Changed

- **APIFY Auth:** Uses Bearer header instead of query string (more secure)
- **Cache Key:** Based on video ID for simple lookups

### Usage Examples

```bash
# Batch processing
python3 scripts/fetch_transcript.py --batch urls.txt

# View cache stats
python3 scripts/fetch_transcript.py --cache-stats

# Clear cache
python3 scripts/fetch_transcript.py --clear-cache

# Skip cache
python3 scripts/fetch_transcript.py "URL" --no-cache
```

## [1.0.7] - 2026-02-04

- Privacy cleanup: removed hardcoded paths and personal info from docs


path: README.md
content: # 📹 YouTube Transcript Fetcher (APIFY)

Fetch YouTube video transcripts from anywhere — even cloud servers where YouTube blocks direct access.

## ✨ Features

- Works from cloud IPs (AWS, GCP, VPS, etc.)
- Bypasses YouTube bot detection via APIFY proxies
- **Local caching** (v1.1.0+) — repeat requests are FREE
- **Batch mode** (v1.1.0+) — process multiple videos at once
- **Cache management** — `--cache-stats`, `--clear-cache`, `--no-cache`
- Text or JSON output with timestamps
- Language preference support
- Simple Python script, no SDK needed

## 💰 Free Tier

APIFY offers **$5/month free credits** — that's approximately **714 videos per month** at $0.007 each!

No credit card required. [Sign up here](https://apify.com/)

## 🚀 Quick Start

```bash
# 1. Set your API token
export APIFY_API_TOKEN="apify_api_YOUR_TOKEN"

# 2. Fetch a transcript
python3 scripts/fetch_transcript.py "https://youtube.com/watch?v=VIDEO_ID"
```

## 📖 Documentation

See [SKILL.md](SKILL.md) for full documentation, setup instructions, and usage examples.

## 🔗 Links

- [APIFY Free Tier](https://apify.com/pricing) - $5/month free
- [Get API Key](https://console.apify.com/account/integrations)
- [YouTube Transcripts Actor](https://apify.com/karamelo/youtube-transcripts)

## ⚙ Requirements

- Python 3.6+
- `requests` library (`pip install requests`)
- APIFY API token (free)

## 📄 License

MIT


path: package.json
content: {
  "name": "@openclaw/youtube-apify-transcript",
  "version": "1.3.3",
  "description": "Fetch YouTube transcripts via APIFY API. Works from cloud IPs (Hetzner, AWS, etc.) by bypassing YouTube's bot detection. Features local caching and batch mode. Requires APIFY_API_TOKEN and Python requests. Free tier includes $5/month credits (~714 videos).",
  "author": "robbyczgw-cla",
  "license": "MIT",
  "keywords": [
    "openclaw",
    "skill",
    "youtube",
    "transcript",
    "apify",
    "video",
    "subtitles",
    "captions",
    "cloud-ip",
    "free-tier",
    "web-scraping"
  ],
  "repository": {
    "type": "git",
    "url": "https://github.com/robbyczgw-cla/youtube-apify-transcript.git"
  },
  "homepage": "https://clawhub.ai/robbyczgw-cla/youtube-apify-transcript",
  "bugs": {
    "url": "https://github.com/robbyczgw-cla/youtube-apify-transcript/issues"
  },
  "openclaw": {
    "skill": true,
    "triggers": [
      "youtube transcript",
      "get transcript",
      "video transcript"
    ],
    "capabilities": [
      "transcript-extraction",
      "cloud-ip-bypass",
      "multi-language"
    ],
    "requirements": {
      "bins": [
        "python3"
      ],
      "python": {
        "packages": [
          "requests"
        ]
      },
      "env": {
        "APIFY_API_TOKEN": "required",
        "YT_TRANSCRIPT_CACHE_DIR": "optional"
      }
    },
    "install": [
      {
        "id": "pip-requests",
        "kind": "pip",
        "package": "requests",
        "bins": [
          "python3"
        ],
        "label": "Install Python requests"
      }
    ]
  },
  "files": [
    "SKILL.md",
    "README.md",
    ".env.example",
    "scripts/"
  ]
}

path: scripts/fetch_transcript.py
content: #!/usr/bin/env python3
"""
Fetch YouTube transcripts via APIFY API with local caching.

Usage:
    python3 fetch_transcript.py "https://youtube.com/watch?v=VIDEO_ID"
    python3 fetch_transcript.py "https://youtu.be/VIDEO_ID" --json
    python3 fetch_transcript.py "URL" --output transcript.txt
    python3 fetch_transcript.py --batch urls.txt
    python3 fetch_transcript.py --cache-stats
    python3 fetch_transcript.py --clear-cache

Requires:
    - APIFY_API_TOKEN environment variable
    - requests library (pip install requests)
"""

import argparse
import json
import os
import re
import sys
import time
from datetime import datetime
from pathlib import Path
from urllib.parse import parse_qs, urlparse

try:
    import requests
except ImportError:
    print("Error: 'requests' library not installed.", file=sys.stderr)
    print("Install with: pip install requests", file=sys.stderr)
    sys.exit(1)


DEFAULT_APIFY_ACTOR_ID = "topaz_sharingan~Youtube-Transcript-Scraper-1"
APIFY_API_BASE = "https://api.apify.com/v2"
CACHE_DIR = Path(os.environ.get("YT_TRANSCRIPT_CACHE_DIR", os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), ".cache")))

ACTOR_ALIASES = {
    # Common typo: store actor name is singular "transcript"
    "pintostudio~youtube-transcripts-scraper": "pintostudio~youtube-transcript-scraper",
}


def get_api_token():
    """Get APIFY API token from environment."""
    [REDACTED_SECRET]"APIFY_API_TOKEN")
    if not [REDACTED_SECRET]"Error: APIFY_API_TOKEN environment variable not set.", file=sys.stderr)
        print("\nSetup instructions:", file=sys.stderr)
        print("1. Create free account: https://apify.com/", file=sys.stderr)
        print("2. Get API [REDACTED_SECRET]", file=sys.stderr)
        print("3. Export: export APIFY_API_TOKEN='apify_api_YOUR_TOKEN'", file=sys.stderr)
        sys.exit(1)
    return token


def extract_video_id(url):
    """Extract YouTube video ID from various URL formats."""
    # Handle youtu.be short links
    if "youtu.be" in url:
        path = urlparse(url).path
        return path.lstrip("/").split("?")[0]
    
    # Handle youtube.com URLs
    parsed = urlparse(url)
    
    # /watch?v=VIDEO_ID
    if "v" in parse_qs(parsed.query):
        return parse_qs(parsed.query)["v"][0]
    
    # /embed/VIDEO_ID or /v/VIDEO_ID
    match = re.search(r"/(embed|v)/([a-zA-Z0-9_-]{11})", parsed.path)
    if match:
        return match.group(2)
    
    # /shorts/VIDEO_ID
    match = re.search(r"/shorts/([a-zA-Z0-9_-]{11})", parsed.path)
    if match:
        return match.group(1)
    
    # Maybe it's just the video ID
    if re.match(r"^[a-zA-Z0-9_-]{11}$", url):
        return url
    
    return None


# ============== CACHING FUNCTIONS ==============

def ensure_cache_dir():
    """Create cache directory if it doesn't exist."""
    CACHE_DIR.mkdir(parents=True, exist_ok=True)


def get_cache_path(video_id):
    """Get the cache file path for a video ID."""
    return CACHE_DIR / f"{video_id}.json"


def load_from_cache(video_id):
    """Load transcript from cache. Returns None if not cached."""
    cache_path = get_cache_path(video_id)
    if cache_path.exists():
        try:
            with open(cache_path, "r", encoding="utf-8") as f:
                return json.load(f)
        except (json.JSONDecodeError, IOError):
            return None
    return None


def write_cache_data(video_id, cache_data):
    """Write cache data to disk."""
    ensure_cache_dir()
    cache_path = get_cache_path(video_id)
    with open(cache_path, "w", encoding="utf-8") as f:
        json.dump(cache_data, f, ensure_ascii=False, indent=2)


def save_to_cache(video_id, result, language=None):
    """Save transcript to cache with metadata."""
    # Support both old (captions list) and new (text string) actor formats
    title = (result.get("title") or result.get("videoTitle") or "Unknown")
    raw_text = result.get("text", "")
    captions = result.get("captions") or ([{"text": raw_text}] if raw_text else [])

    cache_data = {
        "video_id": video_id,
        "title": title,
        "channel": result.get("channelName", ""),
        "language": language or result.get("language", "unknown"),
        "fetched_at": datetime.utcnow().isoformat() + "Z",
        "captions": captions,
        "full_text": raw_text,
        "_raw_result": result  # Keep original for full fidelity
    }

    write_cache_data(video_id, cache_data)


def clear_cache():
    """Delete all cached transcripts."""
    if not CACHE_DIR.exists():
        print("Cache directory does not exist.")
        return 0
    
    count = 0
    for cache_file in CACHE_DIR.glob("*.json"):
        cache_file.unlink()
        count += 1
    
    print(f"Cleared {count} cached transcript(s).")
    return count


def get_cache_stats():
    """Get cache statistics."""
    if not CACHE_DIR.exists():
        return {"count": 0, "total_size": 0, "videos": []}
    
    videos = []
    total_size = 0
    
    for cache_file in CACHE_DIR.glob("*.json"):
        size = cache_file.stat().st_size
        total_size += size
        
        try:
            with open(cache_file, "r", encoding="utf-8") as f:
                data = json.load(f)
                videos.append({
                    "video_id": data.get("video_id", cache_file.stem),
                    "title": data.get("title", "Unknown")[:50],
                    "fetched_at": data.get("fetched_at", "Unknown"),
                    "size": size
                })
        except (json.JSONDecodeError, IOError):
            videos.append({
                "video_id": cache_file.stem,
                "title": "(corrupt)",
                "fetched_at": "Unknown",
                "size": size
            })
    
    return {
        "count": len(videos),
        "total_size": total_size,
        "videos": sorted(videos, key=lambda x: x.get("fetched_at", ""), reverse=True)
    }


def print_cache_stats():
    """Print cache statistics to stderr."""
    stats = get_cache_stats()
    
    print(f"\n📦 Transcript Cache Stats", file=sys.stderr)
    print(f"   Location: {CACHE_DIR}", file=sys.stderr)
    print(f"   Cached videos: {stats['count']}", file=sys.stderr)
    print(f"   Total size: {stats['total_size']:,} bytes ({stats['total_size'] / 1024:.1f} KB)", file=sys.stderr)
    
    if stats['videos']:
        print(f"\n   Recent cached videos:", file=sys.stderr)
        for v in stats['videos'][:10]:
            print(f"   • {v['video_id']} - {v['title']}", file=sys.stderr)
        if len(stats['videos']) > 10:
            print(f"   ... and {len(stats['videos']) - 10} more", file=sys.stderr)


# ============== APIFY FUNCTIONS ==============

def run_apify_actor(video_url, api_token, language=None, actor_id=None):
    """Run the APIFY actor and return results."""
    actor_id = actor_id or DEFAULT_APIFY_ACTOR_ID
    actor_id = ACTOR_ALIASES.get(actor_id, actor_id)
    
    # Start the actor run
    run_url = f"{APIFY_API_BASE}/acts/{actor_id}/runs"
    
    # topaz_sharingan actor uses startUrls format
    if actor_id == "topaz_sharingan~Youtube-Transcript-Scraper-1":
        input_data = {"startUrls": [{"url": video_url}]}
    elif actor_id == "pintostudio~youtube-transcript-scraper":
        input_data = {"videoUrl": video_url}
    elif actor_id == "starvibe~youtube-video-transcript":
        input_data = {"youtube_url": video_url}
        if language:
            input_data["language"] = language
    elif actor_id == "karamelo~youtube-transcripts":
        input_data = {
            "urls": [video_url],
            "outputFormat": "captions"
        }
        if language:
            input_data["preferredLanguage"] = language
    else:
        input_data = {"startUrls": [{"url": video_url}]}
        if language:
            input_data["language"] = language
    
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {api_token}"
    }
    
    params = {}
    
    try:
        # Start the run
        response = requests.post(
            run_url,
            headers=headers,
            params=params,
            json=input_data,
            timeout=30
        )
        
        if response.status_code == 401:
            print("Error: Invalid API token.", file=sys.stderr)
            sys.exit(1)
        
        if response.status_code == 402:
            print("Error: APIFY quota exceeded. Check your billing:", file=sys.stderr)
            print("https://console.apify.com/billing", file=sys.stderr)
            sys.exit(1)
        
        if response.status_code >= 400:
            detail = response.text.strip()
            print(
                f"Error: APIFY returned HTTP {response.status_code} for actor '{actor_id}'.",
                file=sys.stderr
            )
            if detail:
                print(detail, file=sys.stderr)
            response.raise_for_status()

        response.raise_for_status()
        run_data = response.json()["data"]
        run_id = run_data["id"]
        
    except requests.exceptions.RequestException as e:
        print(f"Error starting APIFY actor: {e}", file=sys.stderr)
        sys.exit(1)
    
    # Wait for completion
    status_url = f"{APIFY_API_BASE}/actor-runs/{run_id}"
    max_wait = 120  # seconds
    start_time = time.time()
    
    while time.time() - start_time < max_wait:
        try:
            response = requests.get(status_url, params=params, timeout=10)
            response.raise_for_status()
            status_data = response.json()["data"]
            status = status_data["status"]
            
            if status == "SUCCEEDED":
                break
            elif status in ("FAILED", "ABORTED", "TIMED-OUT"):
                print(f"Error: APIFY actor {status.lower()}.", file=sys.stderr)
                sys.exit(1)
            
            time.sleep(2)
            
        except requests.exceptions.RequestException as e:
            print(f"Error checking status: {e}", file=sys.stderr)
            sys.exit(1)
    else:
        print("Error: Timeout waiting for APIFY actor.", file=sys.stderr)
        sys.exit(1)
    
    # Get results from dataset
    dataset_id = status_data["defaultDatasetId"]
    dataset_url = f"{APIFY_API_BASE}/datasets/{dataset_id}/items"
    
    try:
        response = requests.get(dataset_url, params=params, timeout=30)
        response.raise_for_status()
        results = response.json()
        
        if not results:
            print("Error: No transcript found for this video.", file=sys.stderr)
            print("The video might not have captions available.", file=sys.stderr)
            sys.exit(1)
        
        return results[0]
        
    except requests.exceptions.RequestException as e:
        print(f"Error fetching results: {e}", file=sys.stderr)
        sys.exit(1)


# ============== FORMATTING FUNCTIONS ==============

def format_transcript_text(result):
    """Format transcript as plain text."""
    # Check cache-level full_text first (topaz actor stores it directly)
    if result.get("full_text"):
        return result["full_text"]

    # Handle cached format vs raw APIFY format
    raw = result.get("_raw_result", result)

    # topaz / streamers actor: plain text field
    if raw.get("text"):
        return raw["text"]

    captions = result.get("captions", []) or raw.get("captions", [])

    # Alternative actor format (e.g. starvibe): transcript array of segments.
    if not captions and isinstance(result.get("transcript"), list):
        transcript_lines = []
        for segment in result.get("transcript", []):
            if isinstance(segment, dict):
                text = str(segment.get("text", "")).strip()
                if text:
                    transcript_lines.append(text)
        if transcript_lines:
            return "\n".join(transcript_lines)

    if not captions:
        # Maybe it's in a different format
        if "text" in result:
            return result["text"]
        return "No transcript content found."
    
    lines = []
    for caption in captions:
        if caption is None:
            continue
        # Handle both string and dict formats
        if isinstance(caption, str):
            text = caption.strip()
        elif isinstance(caption, dict):
            text = caption.get("text", "").strip()
        else:
            continue
        
        if text:
            lines.append(text)
    
    return "\n".join(lines)


def format_transcript_json(result, video_id):
    """Format transcript as JSON with metadata."""
    # Handle cached format vs raw APIFY format
    raw_result = result.get("_raw_result", result)
    captions = raw_result.get("captions", [])
    transcript_segments = raw_result.get("transcript", [])
    
    output = {
        "video_id": video_id,
        "title": result.get("title", raw_result.get("title", "Unknown")),
        "transcript": [],
        "full_text": ""
    }
    
    # Include cache info if available
    if "fetched_at" in result:
        output["fetched_at"] = result["fetched_at"]
        output["language"] = result.get("language", "unknown")
    
    texts = []

    if not captions and isinstance(transcript_segments, list):
        for segment in transcript_segments:
            if not isinstance(segment, dict):
                continue
            text = str(segment.get("text", "")).strip()
            if not text:
                continue
            texts.append(text)
            output["transcript"].append({
                "start": segment.get("start", 0),
                "duration": segment.get("duration", 0),
                "text": text
            })
        output["full_text"] = " ".join(texts)
        return output
    for caption in captions:
        if caption is None:
            continue
        # Handle both string and dict formats
        if isinstance(caption, str):
            text = caption.strip()
            if text:
                texts.append(text)
                output["transcript"].append({
                    "text": text
                })
        elif isinstance(caption, dict):
            text = caption.get("text", "").strip()
            if text:
                texts.append(text)
                output["transcript"].append({
                    "start": caption.get("start", 0),
                    "duration": caption.get("duration", 0),
                    "text": text
                })
        else:
            continue
    
    output["full_text"] = " ".join(texts)

    return output


# ============== BATCH PROCESSING ==============

def process_batch(batch_file, api_token, use_cache, language, output_json, actor_id=None):
    """Process a batch of URLs from a file."""
    if not os.path.exists(batch_file):
        print(f"Error: Batch file not found: {batch_file}", file=sys.stderr)
        sys.exit(1)
    
    with open(batch_file, "r", encoding="utf-8") as f:
        lines = f.readlines()
    
    urls = [line.strip() for line in lines if line.strip() and not line.strip().startswith("#")]
    
    if not urls:
        print("Error: No URLs found in batch file.", file=sys.stderr)
        sys.exit(1)
    
    print(f"Processing {len(urls)} URL(s)...\n", file=sys.stderr)
    
    fetched = 0
    cached = 0
    failed = 0
    results = []
    
    for i, url in enumerate(urls, 1):
        video_id = extract_video_id(url)
        if not video_id:
            print(f"[{i}/{len(urls)}] ❌ Invalid URL: {url}", file=sys.stderr)
            failed += 1
            continue
        
        # Check cache
        if use_cache:
            cache_data = load_from_cache(video_id)
            if cache_data:
                print(f"[{i}/{len(urls)}] [cached] {video_id}", file=sys.stderr)
                cached += 1
                results.append(cache_data)
                continue
        
        # Fetch from APIFY
        try:
            print(f"[{i}/{len(urls)}] Fetching {video_id}...", file=sys.stderr)
            video_url = f"https://www.youtube.com/watch?v={video_id}"
            result = run_apify_actor(video_url, api_token, language, actor_id)
            
            # Save to cache
            if use_cache:
                save_to_cache(video_id, result, language)
            
            fetched += 1
            results.append(result)
            
        except SystemExit:
            # run_apify_actor calls sys.exit on error, catch it for batch mode
            failed += 1
            continue
    
    # Print summary
    print(f"\n{'='*40}", file=sys.stderr)
    print(f"Batch complete: {fetched} fetched, {cached} cached, {failed} failed", file=sys.stderr)
    if fetched > 0:
        print(f"[Cost: ~${fetched * 0.007:.3f} for {fetched} API call(s)]", file=sys.stderr)
    
    return results


# ============== SINGLE VIDEO PROCESSING ==============

def process_single(url, api_token, use_cache, language, actor_id=None):
    """Process a single video URL. Returns (result, from_cache)."""
    video_id = extract_video_id(url)
    if not video_id:
        print(f"Error: Could not extract video ID from: {url}", file=sys.stderr)
        sys.exit(1)
    
    # Check cache first
    if use_cache:
        cache_data = load_from_cache(video_id)
        if cache_data:
            print(f"[cached] Transcript for: {video_id}", file=sys.stderr)
            return cache_data, True
    
    # Fetch from APIFY
    video_url = f"https://www.youtube.com/watch?v={video_id}"
    print(f"Fetching transcript for: {video_id}", file=sys.stderr)
    result = run_apify_actor(video_url, api_token, language, actor_id)
    
    # Save to cache
    if use_cache:
        save_to_cache(video_id, result, language)
    
    return result, False


def main():
    parser = argparse.ArgumentParser(
        description="Fetch YouTube transcripts via APIFY API with local caching"
    )
    parser.add_argument(
        "url",
        nargs="?",
        help="YouTube video URL or video ID"
    )
    parser.add_argument(
        "--output", "-o",
        help="Output file path (default: stdout)"
    )
    parser.add_argument(
        "--json", "-j",
        action="store_true",
        help="Output as JSON with timestamps"
    )
    parser.add_argument(
        "--lang", "-l",
        help="Preferred transcript language (e.g., 'en', 'de')"
    )
    parser.add_argument(
        "--actor",
        default=DEFAULT_APIFY_ACTOR_ID,
        help=f"APIFY actor ID (default: {DEFAULT_APIFY_ACTOR_ID})"
    )
    
    # Cache options
    parser.add_argument(
        "--no-cache",
        action="store_true",
        help="Bypass cache (always fetch from APIFY)"
    )
    parser.add_argument(
        "--clear-cache",
        action="store_true",
        help="Clear all cached transcripts and exit"
    )
    parser.add_argument(
        "--cache-stats",
        action="store_true",
        help="Show cache statistics and exit"
    )
    
    # Batch mode
    parser.add_argument(
        "--batch", "-b",
        metavar="FILE",
        help="Process URLs from a text file (one per line)"
    )
    
    args = parser.parse_args()
    
    # Handle cache management commands (no API token needed)
    if args.clear_cache:
        clear_cache()
        return
    
    if args.cache_stats:
        print_cache_stats()
        return
    
    # Need either URL or batch file
    if not args.url and not args.batch:
        parser.print_help()
        print("\nError: Provide a URL or use --batch <file>", file=sys.stderr)
        sys.exit(1)
    
    # Get API token
    api_token = get_api_token()
    use_cache = not args.no_cache
    
    # Batch mode
    if args.batch:
        results = process_batch(args.batch, api_token, use_cache, args.lang, args.json, args.actor)
        
        if args.json and results:
            # Output all results as JSON array
            formatted = [format_transcript_json(r, r.get("video_id", "unknown")) for r in results]
            output = json.dumps(formatted, indent=2, ensure_ascii=False)
            
            if args.output:
                with open(args.output, "w", encoding="utf-8") as f:
                    f.write(output)
                print(f"Results saved to: {args.output}", file=sys.stderr)
            else:
                print(output)
        
        return
    
    # Single URL mode
    result, from_cache = process_single(args.url, api_token, use_cache, args.lang, args.actor)
    video_id = extract_video_id(args.url)

    # Format output
    if args.json:
        output = json.dumps(format_transcript_json(result, video_id), indent=2, ensure_ascii=False)
    else:
        output = format_transcript_text(result)
    
    # Write output
    if args.output:
        with open(args.output, "w", encoding="utf-8") as f:
            f.write(output)
        print(f"Transcript saved to: {args.output}", file=sys.stderr)
    else:
        print(output)
    
    if not from_cache:
        print("\n[Cost: ~$0.007 per video]", file=sys.stderr)


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)