Skill: cyberpsychosissss/video-news-downloader (version 1.0.0)

--- name: video-news-downloader description: "Automated daily news video downloader with AI subtitle proofreading. Downloads CBS Evening News and BBC News at Ten from YouTube, extracts and proofreads subtitles using DeepSeek, serves videos via HTTP with embedded players. Use when: (1) Setting up automated daily news video downloads, (2) Downloading CBS/BBC news with subtitles, (3) Proofreading subtitle files with AI, (4) Creating local video streaming servers with web players, (5) Managing cron jobs for scheduled video updates." --- # Video News Downloader with AI Subtitle Proofreading Complete workflow for downloading daily news videos, processing subtitles, and serving them via HTTP with web players. ## Overview This skill automates: 1. **Video Download**: CBS Evening News + BBC News at Ten from YouTube 2. **Subtitle Processing**: Extract auto-captions and convert to VTT format 3. **AI Proofreading**: Use DeepSeek to fix speech recognition errors 4. **HTTP Streaming**: Serve videos with embedded web players 5. **Scheduled Updates**: Daily cron jobs at configurable times ## Quick Start ### 1. Download Latest News ```bash python3 scripts/video_download.py --cbs --bbc ``` ### 2. Proofread Subtitles ```bash python3 scripts/subtitle_proofreader.py /path/to/subtitle.vtt ``` Or use DeepSeek directly: > "校对字幕文件 /path/to/subtitle.vtt" ### 3. Start HTTP Servers ```bash bash scripts/setup_server.sh ``` ### 4. Setup Daily Cron Jobs ```bash bash scripts/setup_cron.sh ``` ## Commands ### Video Download Script **Download CBS only:** ```bash python3 scripts/video_download.py --cbs ``` **Download BBC only:** ```bash python3 scripts/video_download.py --bbc ``` **Download both:** ```bash python3 scripts/video_download.py --cbs --bbc ``` **With subtitle proofreading:** ```bash python3 scripts/video_download.py --cbs --bbc --proofread ``` ### Subtitle Proofreading **Proofread single file:** ```bash python3 scripts/subtitle_proofreader.py <vtt_file_path> ``` **Auto-proofread all news subtitles:** ```bash python3 scripts/subtitle_proofreader.py --all ``` ### Server Management **Start servers:** ```bash bash scripts/setup_server.sh start ``` **Check status:** ```bash bash scripts/setup_server.sh status ``` **Stop servers:** ```bash bash scripts/setup_server.sh stop ``` ## File Structure ``` /workspace/ ├── cbs-live-local/ │ ├── cbs_latest.mp4 │ ├── cbs_latest.en.vtt # Original subtitle │ ├── cbs_latest.en.vtt-backup # Backup │ ├── cbs_latest-corrected.txt # DeepSeek corrected text │ └── cbs_latest-corrections.md # Error list │ ├── bbc-news-live/ │ ├── bbc_news_latest.mp4 │ ├── bbc_news_latest.en.vtt │ ├── bbc_news_latest.en.vtt-backup │ ├── bbc_news_latest-corrected.txt │ └── bbc_news_latest-corrections.md │ └── temp/ # Temporary download files ``` ## HTTP Endpoints | Endpoint | Description | |----------|-------------| | http://IP:8093/ | CBS Evening News player | | http://IP:8093/cbs_latest.mp4 | CBS video direct | | http://IP:8095/ | BBC News at Ten player | | http://IP:8095/bbc_news_latest.mp4 | BBC video direct | ## Cron Jobs ### Default Schedule (Beijing Time) | Time | Task | |------|------| | 20:00 | Download latest CBS + BBC videos | | 20:30 | DeepSeek proofread subtitles | ### Manual Cron Setup See [references/cron-setup.md](references/cron-setup.md) for detailed cron configuration. ## DeepSeek Proofreading ### What Gets Fixed - Speech recognition errors (e.g., "noraster" → "nor'easter") - Name errors (e.g., "trunk" → "Trump") - Location name errors - Professional terminology errors - Obvious spelling mistakes ### Output Files For each subtitle file, generates: 1. `-backup.vtt` - Original subtitle (never modified) 2. `-corrected.txt` - AI-corrected plain text 3. `-corrections.md` - List of corrections made ## Troubleshooting ### Video Download Fails - Check yt-dlp is installed: `yt-dlp --version` - Check YouTube URL is accessible - Try manual download first ### Subtitle Extraction Fails - Some videos don't have auto-captions - Check if `--list-subs` shows available languages ### Server Won't Start - Check ports 8093/8095 are free: `lsof -i :8093` - Check Python http.server is available ### Proofreading Issues - Ensure DeepSeek model is available - Check subtitle file exists and is valid VTT format ## See Also - [references/workflow.md](references/workflow.md) - Detailed workflow documentation - [references/cron-setup.md](references/cron-setup.md) - Cron job configuration guide

path: references/cron-setup.md
content: # Cron Job Setup Guide

## Default Schedule

| Beijing Time | UTC Time | Task |
|--------------|----------|------|
| 20:00 | 12:00 | Download CBS + BBC videos |
| 20:30 | 12:30 | DeepSeek proofread subtitles |

## Installation

### Using setup_cron.sh Script

```bash
bash scripts/setup_cron.sh install
```

### Manual Cron Setup

```bash
# Edit crontab
crontab -e

# Add these lines:
# Video download at 20:00 Beijing Time (12:00 UTC)
0 12 * * * cd /root/.openclaw/workspace/skills/video-news-downloader && python3 scripts/video_download.py --cbs --bbc --proofread >> /root/.openclaw/workspace/logs/video-download.log 2>&1

# Subtitle proofreading at 20:30 Beijing Time (12:30 UTC)
30 12 * * * cd /root/.openclaw/workspace/skills/video-news-downloader && python3 scripts/subtitle_proofreader.py --all >> /root/.openclaw/workspace/logs/proofread.log 2>&1
```

## OpenClaw Cron Integration

For OpenClaw-managed cron jobs:

```json
{
  "name": "Daily Video Download",
  "schedule": {
    "kind": "cron",
    "expr": "0 12 * * *",
    "tz": "UTC"
  },
  "payload": {
    "kind": "agentTurn",
    "message": "Download latest CBS and BBC news videos",
    "timeoutSeconds": 300
  }
}
```

## Time Zone Conversion

| Beijing | UTC | US Eastern | US Pacific |
|---------|-----|------------|------------|
| 08:00 | 00:00 | 20:00 (prev) | 17:00 (prev) |
| 12:00 | 04:00 | 00:00 | 21:00 (prev) |
| 20:00 | 12:00 | 08:00 | 05:00 |
| 22:00 | 14:00 | 10:00 | 07:00 |

## Managing Cron Jobs

### List Current Jobs

```bash
crontab -l
```

### Remove All Video Jobs

```bash
bash scripts/setup_cron.sh remove
```

### Check Job Status

```bash
bash scripts/setup_cron.sh status
```

### View Execution Logs

```bash
# Video download logs
tail -f /root/.openclaw/workspace/logs/video-download.log

# Proofreading logs
tail -f /root/.openclaw/workspace/logs/proofread.log
```

## Troubleshooting

### Jobs Not Running

1. Check cron service: `systemctl status cron`
2. Check syntax: `crontab -l | crontab -`
3. Check logs: `grep CRON /var/log/syslog`

### Permission Issues

Ensure scripts are executable:
```bash
chmod +x scripts/*.py scripts/*.sh
```

### Path Issues

Always use full paths in cron:
```bash
# Good
/usr/bin/python3 /full/path/to/script.py

# Bad (may fail)
python3 script.py
```


path: references/workflow.md
content: # Video News Downloader - Detailed Workflow

## Complete Workflow Diagram

```
┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│   Cron Job      │────▶│  Video Download  │────▶│  HTTP Server    │
│  (Daily 20:00)  │     │  (yt-dlp)        │     │  (Python)       │
└─────────────────┘     └──────────────────┘     └─────────────────┘
                               │
                               ▼
                        ┌──────────────────┐
                        │ Subtitle Extract │
                        │ (VTT format)     │
                        └──────────────────┘
                               │
                               ▼
                        ┌──────────────────┐
                        │ AI Proofreading  │────▶┌─────────────────┐
                        │ (DeepSeek)       │     │ Corrected Files │
                        └──────────────────┘     └─────────────────┘
```

## Step-by-Step Process

### 1. Video Download (20:00 Beijing Time)

**CBS Evening News:**
- Source: YouTube Playlist
- URL: https://www.youtube.com/playlist?list=PLotzEBRQdc0cMXjf4FKw6_1Pu8in6rzBQ
- Quality: 360p MP4 (format 18)
- Output: `cbs-live-local/cbs_latest.mp4`

**BBC News at Ten:**
- Source: YouTube Search
- Query: "BBC News at Ten"
- Quality: 360p MP4
- Output: `bbc-news-live/bbc_news_latest.mp4`

### 2. Subtitle Extraction

**Automatic Captions:**
- Language: English (en)
- Format: WebVTT (.vtt)
- Source: YouTube auto-generated
- Backup: Original saved as `-backup.vtt`

**Common Issues:**
- Some videos lack auto-captions
- Live streams may have delayed captions
- Music/background noise affects accuracy

### 3. HTTP Server (Always Running)

**Ports:**
- 8093: CBS Evening News
- 8095: BBC News at Ten

**Features:**
- Embedded HTML5 video player
- Subtitle track support (VTT)
- Direct MP4 download links
- CORS enabled for cross-origin

**Player Features:**
- Auto-play disabled
- Subtitle toggle (CC button)
- Full-screen support
- Responsive design

### 4. AI Proofreading (20:30 Beijing Time)

**Process:**
1. Extract plain text from VTT
2. Send to DeepSeek with proofreading prompt
3. Receive corrections list
4. Generate corrected text file

**Error Types Detected:**

| Type | Example | Correction |
|------|---------|------------|
| Speech Recognition | noraster | nor'easter |
| Name Errors | trunk | Trump |
| Location | bucking ham | Buckingham |
| Terminology | AI words | proper terms |
| Spelling | obviosly | obviously |

**Output Files:**

For each subtitle:
- `-backup.vtt`: Original (never modified)
- `-corrected.txt`: AI-corrected plain text
- `-corrections.md`: List of changes made

## Directory Structure

```
/workspace/
├── cbs-live-local/              # CBS content
│   ├── cbs_latest.mp4          # Video file
│   ├── cbs_latest.en.vtt       # Current subtitle
│   ├── cbs_latest.en.vtt-backup
│   ├── cbs_latest-corrected.txt
│   ├── cbs_latest-corrections.md
│   └── index.html              # Web player
│
├── bbc-news-live/               # BBC content
│   ├── bbc_news_latest.mp4
│   ├── bbc_news_latest.en.vtt
│   ├── bbc_news_latest.en.vtt-backup
│   ├── bbc_news_latest-corrected.txt
│   ├── bbc_news_latest-corrections.md
│   └── index.html
│
├── temp/                        # Temporary downloads
│   └── (auto-cleaned daily)
│
└── logs/                        # Download logs
    ├── video-download.log
    └── proofread.log
```

## Manual Operations

### Force Download Now

```bash
cd /root/.openclaw/workspace/skills/video-news-downloader
python3 scripts/video_download.py --cbs --bbc
```

### Force Proofread Now

```bash
python3 scripts/subtitle_proofreader.py --all
```

### Check Server Status

```bash
bash scripts/setup_server.sh status
```

### Restart Servers

```bash
bash scripts/setup_server.sh restart
```

### View Logs

```bash
tail -f /root/.openclaw/workspace/logs/video-download.log
tail -f /root/.openclaw/workspace/logs/proofread.log
```

## Troubleshooting

### Video Download Fails

**Symptom:** yt-dlp returns error

**Solutions:**
1. Check YouTube URL is accessible
2. Try manual download: `yt-dlp -f 18 "URL"`
3. Check disk space: `df -h`
4. Update yt-dlp: `pip install -U yt-dlp`

### Subtitle Not Found

**Symptom:** No .vtt file generated

**Solutions:**
1. Check if video has auto-captions: `yt-dlp --list-subs URL`
2. Some videos don't have English captions
3. Try different video source

### Server Won't Start

**Symptom:** Port already in use

**Solutions:**
1. Find process: `lsof -i :8093`
2. Kill process: `kill $(lsof -t -i :8093)`
3. Use different port

### Proofreading Takes Too Long

**Symptom:** DeepSeek times out

**Solutions:**
1. Subtitle may be too long (limit to 6000 chars)
2. Split into chunks
3. Use lighter model

## Customization

### Change Download Quality

Edit `scripts/video_download.py`:
```python
"-f", "22"  # 720p instead of 360p
```

### Change Schedule

Edit cron jobs:
```bash
# Beijing Time 22:00
0 14 * * * python3 video_download.py --cbs --bbc
```

### Add More News Sources

1. Add new directory in workspace
2. Create download function in video_download.py
3. Add to cron job
4. Create index.html player

## API Reference

### Video Download Script

```python
python3 video_download.py [options]

Options:
  --cbs          Download CBS Evening News
  --bbc          Download BBC News at Ten
  --proofread    Auto-proofread after download
  --cleanup N    Keep files for N days (default: 2)
```

### Subtitle Proofreader

```python
python3 subtitle_proofreader.py [file|--all]

Arguments:
  file           Single VTT file to proofread
  --all          Proofread all news subtitles
```

### Server Control

```bash
bash setup_server.sh [command]

Commands:
  start    Start both servers
  stop     Stop both servers
  restart  Restart both servers
  status   Check server status
```


path: scripts/setup_cron.sh
content: #!/bin/bash
# Cron job setup for automated video downloads
# Usage: setup_cron.sh [install|remove|status]

SKILL_DIR="/root/.openclaw/workspace/skills/video-news-downloader"
WORKSPACE="/root/.openclaw/workspace"

# Beijing Time 20:00 = UTC 12:00
VIDEO_CRON="0 12 * * * cd $SKILL_DIR && python3 scripts/video_download.py --cbs --bbc --proofread >> $WORKSPACE/logs/video-download.log 2>&1"

# Beijing Time 20:30 = UTC 12:30  
PROOFREAD_CRON="30 12 * * * cd $SKILL_DIR && python3 scripts/subtitle_proofreader.py --all >> $WORKSPACE/logs/proofread.log 2>&1"

# Ensure log directory exists
mkdir -p "$WORKSPACE/logs"

case "${1:-status}" in
    install)
        echo "Installing video download cron jobs..."
        
        # Add video download job
        (crontab -l 2>/dev/null | grep -v "video_download.py"; echo "$VIDEO_CRON") | crontab -
        
        # Add proofreading job  
        (crontab -l 2>/dev/null | grep -v "subtitle_proofreader.py"; echo "$PROOFREAD_CRON") | crontab -
        
        echo "✅ Cron jobs installed!"
        echo ""
        echo "Schedule (Beijing Time):"
        echo "  20:00 - Download latest CBS + BBC videos"
        echo "  20:30 - DeepSeek proofread subtitles"
        echo ""
        crontab -l | grep -E "(video_download|subtitle_proofreader)"
        ;;
        
    remove)
        echo "Removing video download cron jobs..."
        crontab -l 2>/dev/null | grep -v "video_download.py" | grep -v "subtitle_proofreader.py" | crontab -
        echo "✅ Cron jobs removed!"
        ;;
        
    status)
        echo "Current video download cron jobs:"
        echo "================================="
        crontab -l 2>/dev/null | grep -E "(video_download|subtitle_proofreader)" || echo "No jobs found"
        ;;
        
    *)
        echo "Usage: $0 [install|remove|status]"
        exit 1
        ;;
esac


path: scripts/setup_server.sh
content: #!/bin/bash
# HTTP Server Setup for Video Streaming
# Usage: setup_server.sh [start|stop|status|restart]

CBS_DIR="/root/.openclaw/workspace/cbs-live-local"
BBC_DIR="/root/.openclaw/workspace/bbc-news-live"
CBS_PORT=8093
BBC_PORT=8095

# Colors for output
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

get_server_pid() {
    local port=$1
    lsof -t -i :$port 2>/dev/null
}

start_server() {
    local dir=$1
    local port=$2
    local name=$3
    
    # Check if already running
    pid=$(get_server_pid $port)
    if [ -n "$pid" ]; then
        echo -e "${YELLOW}$name is already running on port $port (PID: $pid)${NC}"
        return 0
    fi
    
    # Check directory exists
    if [ ! -d "$dir" ]; then
        echo -e "${RED}Directory does not exist: $dir${NC}"
        return 1
    fi
    
    # Start server
    cd "$dir" && nohup python3 -m http.server $port --bind 0.0.0.0 > /dev/null 2>&1 &
    sleep 1
    
    # Verify started
    pid=$(get_server_pid $port)
    if [ -n "$pid" ]; then
        echo -e "${GREEN}✅ $name started on port $port (PID: $pid)${NC}"
        return 0
    else
        echo -e "${RED}❌ Failed to start $name${NC}"
        return 1
    fi
}

stop_server() {
    local port=$1
    local name=$2
    
    pid=$(get_server_pid $port)
    if [ -n "$pid" ]; then
        kill $pid 2>/dev/null
        sleep 1
        if [ -z "$(get_server_pid $port)" ]; then
            echo -e "${GREEN}✅ $name stopped (port $port)${NC}"
        else
            echo -e "${RED}❌ Failed to stop $name${NC}"
        fi
    else
        echo -e "${YELLOW}$name is not running (port $port)${NC}"
    fi
}

check_status() {
    local port=$1
    local name=$2
    
    pid=$(get_server_pid $port)
    if [ -n "$pid" ]; then
        echo -e "${GREEN}✅ $name: Running on port $port (PID: $pid)${NC}"
        return 0
    else
        echo -e "${RED}❌ $name: Not running (port $port)${NC}"
        return 1
    fi
}

get_ip() {
    hostname -I 2>/dev/null | awk '{print $1}' || echo "localhost"
}

show_urls() {
    ip=$(get_ip)
    echo ""
    echo "📺 Video Streaming URLs:"
    echo "======================="
    echo ""
    echo "🇺🇸 CBS Evening News:"
    echo "   Player: http://$ip:$CBS_PORT/"
    echo "   Video:  http://$ip:$CBS_PORT/cbs_latest.mp4"
    echo ""
    echo "🇬🇧 BBC News at Ten:"
    echo "   Player: http://$ip:$BBC_PORT/"
    echo "   Video:  http://$ip:$BBC_PORT/bbc_news_latest.mp4"
    echo ""
}

# Main command handling
case "${1:-status}" in
    start)
        echo "Starting video streaming servers..."
        start_server "$CBS_DIR" $CBS_PORT "CBS Evening News"
        start_server "$BBC_DIR" $BBC_PORT "BBC News at Ten"
        show_urls
        ;;
    stop)
        echo "Stopping video streaming servers..."
        stop_server $CBS_PORT "CBS Evening News"
        stop_server $BBC_PORT "BBC News at Ten"
        ;;
    restart)
        echo "Restarting video streaming servers..."
        stop_server $CBS_PORT "CBS Evening News"
        stop_server $BBC_PORT "BBC News at Ten"
        sleep 1
        start_server "$CBS_DIR" $CBS_PORT "CBS Evening News"
        start_server "$BBC_DIR" $BBC_PORT "BBC News at Ten"
        show_urls
        ;;
    status)
        echo "Video streaming server status:"
        echo "=============================="
        check_status $CBS_PORT "CBS Evening News"
        check_status $BBC_PORT "BBC News at Ten"
        show_urls
        ;;
    *)
        echo "Usage: $0 [start|stop|restart|status]"
        exit 1
        ;;
esac


path: scripts/subtitle_proofreader.py
content: #!/usr/bin/env python3
"""
Subtitle Proofreader - AI-powered subtitle correction using DeepSeek
Usage: python3 subtitle_proofreader.py <vtt_file> | --all
"""

import argparse
import os
import re
import sys
from pathlib import Path

def extract_text_from_vtt(vtt_path):
    """Extract plain text from VTT file"""
    with open(vtt_path, 'r', encoding='utf-8') as f:
        content = f.read()
    
    lines = []
    for line in content.split('\n'):
        line = line.strip()
        # Skip non-text lines
        if (not line or 
            line.startswith('WEBVTT') or 
            line.startswith('Kind:') or 
            line.startswith('Language:') or
            re.match(r'^\d{2}:\d{2}:\d{2}', line) or
            line.startswith('align:') or
            line.startswith('position:')):
            continue
        # Clean tags
        clean = re.sub(r'<\d{2}:\d{2}:\d{2}\.\d{3}>', '', line)
        clean = re.sub(r'</?c>', '', clean)
        clean = re.sub(r'<[^\u003e]+\u003e', '', clean)
        if clean.strip():
            lines.append(clean.strip())
    
    return ' '.join(lines)

def create_backup(vtt_path):
    """Create backup of original subtitle"""
    backup_path = str(vtt_path).replace('.vtt', '-backup.vtt')
    if not os.path.exists(backup_path):
        with open(vtt_path, 'r', encoding='utf-8') as f:
            content = f.read()
        with open(backup_path, 'w', encoding='utf-8') as f:
            f.write(content)
        print(f"✅ Backed up: {backup_path}")
    return backup_path

def generate_deepseek_prompt(text, source_name=""):
    """Generate DeepSeek proofreading prompt"""
    return f"""You are a professional subtitle proofreading expert. Please proofread the following news subtitle text and correct obvious errors.

Common error types to fix:
1. Speech recognition errors: e.g., "noraster" → "nor'easter" (northeast storm)
2. Name errors: e.g., "trunk" → "Trump"
3. Location name errors: e.g., "bucking ham" → "Buckingham"
4. Professional terminology errors
5. Obvious spelling mistakes

Important guidelines:
- Only fix OBVIOUS errors, keep colloquial expressions
- Preserve capitalization and punctuation style
- Don't over-correct natural speech patterns
- Keep the text as close to original as possible while fixing clear mistakes

Source: {source_name}

Text to proofread:
{text[:6000]}

Please respond in this format:

## Corrections Found
1. Original: "xxx" → Corrected: "yyy" (Error type)
2. ...

## Corrected Full Text
[Full corrected text here]

Be concise. Only list actual errors found."""

def proofread_with_deepseek(text, source_name=""):
    """
    Prepare DeepSeek proofreading task
    Returns: (prompt, word_count)
    """
    prompt = generate_deepseek_prompt(text, source_name)
    return prompt, len(text.split())

def save_results(vtt_path, corrections, corrected_text):
    """Save proofreading results"""
    output_dir = os.path.dirname(vtt_path)
    base_name = os.path.basename(vtt_path).replace('.vtt', '')
    
    # Save corrected text
    corrected_path = os.path.join(output_dir, f"{base_name}-corrected.txt")
    with open(corrected_path, 'w', encoding='utf-8') as f:
        f.write("# AI-Corrected Subtitle Text\n\n")
        f.write(corrected_text)
    
    # Save corrections list
    corrections_path = os.path.join(output_dir, f"{base_name}-corrections.md")
    with open(corrections_path, 'w', encoding='utf-8') as f:
        f.write("# Subtitle Proofreading Report\n\n")
        f.write(corrections)
    
    return corrected_path, corrections_path

def proofread_file(vtt_path):
    """Proofread a single subtitle file"""
    if not os.path.exists(vtt_path):
        print(f"❌ File not found: {vtt_path}")
        return False
    
    print(f"📄 Processing: {vtt_path}")
    
    # Create backup
    create_backup(vtt_path)
    
    # Extract text
    print("📝 Extracting text...")
    text = extract_text_from_vtt(vtt_path)
    word_count = len(text.split())
    print(f"   Extracted {word_count} words")
    
    # Generate DeepSeek prompt
    source_name = os.path.basename(vtt_path)
    prompt, _ = proofread_with_deepseek(text, source_name)
    
    # Save prompt for manual processing or API call
    prompt_path = vtt_path.replace('.vtt', '-proofread-task.txt')
    with open(prompt_path, 'w', encoding='utf-8') as f:
        f.write(prompt)
    
    print(f"\n✅ Proofreading task prepared!")
    print(f"   Task file: {prompt_path}")
    print(f"\nNext steps:")
    print(f"  1. Send the task file content to DeepSeek model")
    print(f"  2. Or tell me: '校对字幕 {vtt_path}'")
    
    return True

def proofread_all():
    """Proofread all news subtitles"""
    workspace = "/root/.openclaw/workspace"
    
    files_to_proofread = [
        f"{workspace}/cbs-live-local/cbs_latest.en.vtt",
        f"{workspace}/bbc-news-live/bbc_news_latest.en.vtt"
    ]
    
    success_count = 0
    for vtt_file in files_to_proofread:
        if os.path.exists(vtt_file):
            if proofread_file(vtt_file):
                success_count += 1
        else:
            print(f"⚠️ Skipping (not found): {vtt_file}")
    
    print(f"\n{'='*50}")
    print(f"Prepared {success_count} files for proofreading")
    print('='*50)
    
    return success_count > 0

def main():
    parser = argparse.ArgumentParser(description='Proofread subtitle files')
    parser.add_argument('file', nargs='?', help='VTT file to proofread')
    parser.add_argument('--all', action='store_true', help='Proofread all news subtitles')
    
    args = parser.parse_args()
    
    if args.all:
        return 0 if proofread_all() else 1
    elif args.file:
        return 0 if proofread_file(args.file) else 1
    else:
        parser.print_help()
        return 1

if __name__ == '__main__':
    sys.exit(main())


path: scripts/video_download.py
content: #!/usr/bin/env python3
"""
Video News Downloader - Download CBS Evening News and BBC News at Ten
Usage: python3 video_download.py [--cbs] [--bbc] [--proofread]
"""

import argparse
import os
import subprocess
import sys
from datetime import datetime
from pathlib import Path

# Configuration
WORKSPACE = "/root/.openclaw/workspace"
CBS_DIR = f"{WORKSPACE}/cbs-live-local"
BBC_DIR = f"{WORKSPACE}/bbc-news-live"
TEMP_DIR = f"{WORKSPACE}/temp"

CBS_PLAYLIST = "https://www.youtube.com/playlist?list=PLotzEBRQdc0cMXjf4FKw6_1Pu8in6rzBQ"
BBC_SEARCH = "BBC News at Ten"

def ensure_dirs():
    """Ensure all required directories exist"""
    for d in [CBS_DIR, BBC_DIR, TEMP_DIR]:
        os.makedirs(d, exist_ok=True)

def download_cbs():
    """Download latest CBS Evening News"""
    print("📺 Downloading CBS Evening News...")
    
    # Get latest video URL from playlist
    cmd = [
        "yt-dlp", "--flat-playlist", "--playlist-end", "1",
        "--print", "%(url)s", CBS_PLAYLIST
    ]
    try:
        result = subprocess.run(cmd, capture_output=True, text=True, timeout=60)
        video_url = result.stdout.strip()
        if not video_url:
            print("❌ Could not get CBS video URL")
            return False
    except Exception as e:
        print(f"❌ Error getting CBS URL: {e}")
        return False
    
    # Download video + auto subtitles
    output_template = f"{CBS_DIR}/cbs_latest.%(ext)s"
    
    cmd = [
        "yt-dlp",
        "-f", "18",  # 360p MP4 for smaller size
        "--write-auto-subs",
        "--sub-langs", "en",
        "--convert-subs", "vtt",
        "-o", output_template,
        video_url
    ]
    
    try:
        result = subprocess.run(cmd, capture_output=True, text=True, timeout=300)
        if result.returncode == 0:
            print("✅ CBS download complete")
            # Backup original subtitle if exists
            backup_subtitle(f"{CBS_DIR}/cbs_latest.en.vtt")
            return True
        else:
            print(f"❌ CBS download failed: {result.stderr}")
            return False
    except Exception as e:
        print(f"❌ Error downloading CBS: {e}")
        return False

def download_bbc():
    """Download latest BBC News at Ten"""
    print("📺 Downloading BBC News at Ten...")
    
    # Search for latest BBC News at Ten
    cmd = [
        "yt-dlp", "--default-search", "ytsearch1",
        "--print", "%(url)s", f"ytsearch1:{BBC_SEARCH}"
    ]
    try:
        result = subprocess.run(cmd, capture_output=True, text=True, timeout=60)
        video_url = result.stdout.strip()
        if not video_url:
            print("❌ Could not get BBC video URL")
            return False
    except Exception as e:
        print(f"❌ Error getting BBC URL: {e}")
        return False
    
    # Save source URL
    with open(f"{BBC_DIR}/source_url.txt", 'w') as f:
        f.write(video_url)
    
    # Download video + auto subtitles
    output_template = f"{BBC_DIR}/bbc_news_latest.%(ext)s"
    
    cmd = [
        "yt-dlp",
        "-f", "18",
        "--write-auto-subs",
        "--sub-langs", "en",
        "--convert-subs", "vtt",
        "-o", output_template,
        video_url
    ]
    
    try:
        result = subprocess.run(cmd, capture_output=True, text=True, timeout=300)
        if result.returncode == 0:
            print("✅ BBC download complete")
            # Backup original subtitle
            backup_subtitle(f"{BBC_DIR}/bbc_news_latest.en.vtt")
            return True
        else:
            print(f"❌ BBC download failed: {result.stderr}")
            return False
    except Exception as e:
        print(f"❌ Error downloading BBC: {e}")
        return False

def backup_subtitle(vtt_path):
    """Create backup of original subtitle"""
    backup_path = vtt_path.replace('.vtt', '-backup.vtt')
    if os.path.exists(vtt_path) and not os.path.exists(backup_path):
        import shutil
        shutil.copy(vtt_path, backup_path)
        print(f"   📄 Backed up subtitle: {backup_path}")

def cleanup_old_files(days=2):
    """Remove video files older than specified days"""
    print(f"🗑️ Cleaning up files older than {days} days...")
    
    import time
    now = time.time()
    cutoff = now - (days * 24 * 3600)
    
    patterns = ['bbc_*.mp4', 'bbc_*.vtt', 'cbs_*.mp4', 'cbs_*.vtt']
    
    for pattern in patterns:
        for f in Path(WORKSPACE).glob(pattern):
            if f.stat().st_mtime < cutoff:
                try:
                    f.unlink()
                    print(f"   🗑️ Deleted: {f.name}")
                except Exception as e:
                    print(f"   ⚠️ Could not delete {f}: {e}")

def main():
    parser = argparse.ArgumentParser(description='Download news videos')
    parser.add_argument('--cbs', action='store_true', help='Download CBS Evening News')
    parser.add_argument('--bbc', action='store_true', help='Download BBC News at Ten')
    parser.add_argument('--proofread', action='store_true', help='Proofread subtitles after download')
    parser.add_argument('--cleanup', type=int, default=2, help='Days to keep old files (default: 2)')
    
    args = parser.parse_args()
    
    if not args.cbs and not args.bbc:
        print("Please specify --cbs and/or --bbc")
        parser.print_help()
        sys.exit(1)
    
    ensure_dirs()
    results = []
    
    if args.cbs:
        results.append(('CBS', download_cbs()))
    
    if args.bbc:
        results.append(('BBC', download_bbc()))
    
    # Cleanup old files
    cleanup_old_files(args.cleanup)
    
    # Summary
    print("\n" + "="*50)
    print("Download Summary:")
    for name, success in results:
        status = "✅" if success else "❌"
        print(f"   {status} {name}")
    print("="*50)
    
    # Trigger proofreading if requested
    if args.proofread:
        print("\n🤖 Starting subtitle proofreading...")
        import subprocess
        if args.cbs and os.path.exists(f"{CBS_DIR}/cbs_latest.en.vtt"):
            subprocess.run([sys.executable, "scripts/subtitle_proofreader.py", 
                          f"{CBS_DIR}/cbs_latest.en.vtt"])
        if args.bbc and os.path.exists(f"{BBC_DIR}/bbc_news_latest.en.vtt"):
            subprocess.run([sys.executable, "scripts/subtitle_proofreader.py",
                          f"{BBC_DIR}/bbc_news_latest.en.vtt"])
    
    return 0 if all(r[1] for r in results) else 1

if __name__ == '__main__':
    sys.exit(main())


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)