Skill: shawnshenopeninterx/vea (version 1.1.2)

--- name: vea description: Video Editing Agent (VEA) for automated video processing, highlight generation, and editing. Use when asked to index videos, create highlight reels, generate narration, add subtitles, select background music, or perform any video editing task. Supports long-form video comprehension and AI-powered clip selection. metadata: {"openclaw": {"requires": {"env": ["MEMORIES_API_KEY"], "bins": ["ffmpeg"]}, "primaryEnv": "MEMORIES_API_KEY", "homepage": "https://github.com/Memories-ai-labs/vea-open-source"}} --- ## Installation VEA is open source! Get it from GitHub: ```bash # Clone the repo git clone https://github.com/Memories-ai-labs/vea-open-source.git cd vea-open-source # Install uv package manager curl -LsSf https://astral.sh/uv/install.sh | sh # Install dependencies uv sync source .venv/bin/activate # Copy config and add your API keys cp config.example.json config.json ``` **📄 Paper**: https://arxiv.org/abs/2509.16811 **💻 Code**: https://github.com/Memories-ai-labs/vea-open-source ## Requirements - **Python 3.11+** - **FFmpeg** - Must be installed on system - **uv** - Package manager (installed above) - **API Keys** (in `config.json`): - `MEMORIES_API_KEY` (required) - Video indexing & comprehension - Get at https://memories.ai/app/service/key - `GOOGLE_API_KEY` (required) - Script generation - Google Cloud Console - `ELEVENLABS_API_KEY` (required) - TTS narration & subtitles - `SOUNDSTRIPE_KEY` (optional) - Background music selection ### Install FFmpeg | OS | Command | |----|---------| | Ubuntu/Debian | `sudo apt install ffmpeg` | | macOS | `brew install ffmpeg` | | Windows | Download from ffmpeg.org | ## Start Server ```bash gcloud auth application-default login # Authenticate GCP source .venv/bin/activate python -m src.app ``` Server runs at `http://localhost:8000` ### Privacy Note - Videos processed locally by VEA server - Video frames sent to Memories.ai for AI comprehension - ElevenLabs receives text for TTS narration - All intermediate files stored locally in `data/outputs/` # Video Editing Agent (VEA) Local video editing service at `http://localhost:8000`. Runs from `~/vea`. ## ⚠️ User Interaction Flow (MUST FOLLOW) **Before processing any video edit request, show config options and wait for confirmation:** ``` 📹 VEA Video Edit Configuration 🎬 Source Video: [video path/name] 📝 Edit Request: [user's prompt] Please confirm the following settings: ┌─────────────────┬────────┬─────────────────────────┐ │ Setting │ Value │ Description │ ├─────────────────┼────────┼─────────────────────────┤ │ 🔊 Original Audio │ ❌ OFF │ Keep original video sound │ │ 🎤 Narration │ ✅ ON │ AI-generated voiceover │ │ 🎵 Background Music │ ✅ ON │ Auto-select from Soundstripe │ │ 📝 Subtitles │ ✅ ON │ Auto-generate and burn-in │ │ 📐 Aspect Ratio │ 16:9 │ 16:9 / 9:16 vertical / 1:1 │ │ 🎼 Snap to Beat │ ❌ OFF │ Sync cuts to music beats │ └─────────────────┴────────┴─────────────────────────┘ Reply "confirm" to start editing, or tell me which settings to adjust. ``` **Default Settings:** - `original_audio`: false (mute original, use narration instead) - `narration`: true (enable AI voiceover) - `music`: true (enable background music) - `subtitles`: true (enable subtitles) - `aspect_ratio`: 1.78 (16:9 landscape) - `snap_to_beat`: false (no beat sync) **Aspect Ratio Options:** - `16:9` (1.78) — Landscape, YouTube - `9:16` (0.5625) — Vertical, TikTok/Reels - `1:1` (1.0) — Square, Instagram ## Quick Start ```bash # Start VEA server (use tmux for long tasks) cd ~/vea && source .venv/bin/activate && python src/app.py ``` ## Core Workflows ### 1. Index a Video (Required First Step) Before any editing, index the video to enable AI comprehension: ```bash curl -X POST "http://localhost:8000/video-edit/v1/index" \ -H "Content-Type: application/json" \ -d '{"blob_path": "data/videos/PROJECT_NAME/video.mp4"}' ``` Creates `~/vea/data/indexing/PROJECT_NAME/media_indexing.json`. ### 2. Generate Highlight Reel ```bash curl -X POST "http://localhost:8000/video-edit/v1/flexible_respond" \ -H "Content-Type: application/json" \ -d '{ "blob_path": "data/videos/PROJECT_NAME/video.mp4", "prompt": "Create a 1-minute highlight reel of the best moments", "video_response": true, "original_audio": false, "music": true, "narration": true, "aspect_ratio": 1.78, "subtitles": true }' ``` **Parameters:** - `video_response: true` — Generate video output (vs text-only) - `original_audio: false` — Mute original audio, use narration - `music: true` — Add background music (requires Soundstripe API) - `narration: true` — Generate AI voiceover (ElevenLabs) - `subtitles: true` — Burn subtitles into video - `aspect_ratio` — 1.78 (16:9), 1.0 (square), 0.5625 (9:16 vertical) ### 3. Manual Video Assembly For more control, use the helper scripts: ```bash # Add background music to existing video python ~/vea/scripts/add_soundstripe_music.py # Generate video with subtitles python ~/vea/scripts/add_music_subtitles.py ``` ## Directory Structure ``` ~/vea/ ├── data/ │ ├── videos/PROJECT_NAME/ # Source videos │ ├── indexing/PROJECT_NAME/ # media_indexing.json │ └── outputs/PROJECT_NAME/ # Final outputs │ ├── PROJECT_NAME.mp4 # Final video │ ├── clip_plan.json # Clip timestamps + narration │ ├── narrations/ # TTS audio files │ ├── subtitles/ # SRT files │ └── music/ # Background music ├── config.json # API keys configuration └── src/app.py # FastAPI server ``` ## API Keys (in config.json) | Key | Service | Purpose | Required | |-----|---------|---------|----------| | `MEMORIES_API_KEY` | Memories.ai | Video indexing & comprehension | ✅ Yes | | `GOOGLE_API_KEY` | Gemini | Script generation | ✅ Yes | | `ELEVENLABS_API_KEY` | ElevenLabs | TTS narration, STT subtitles | ✅ Yes | | `SOUNDSTRIPE_KEY` | Soundstripe | Background music selection | Optional | ## Common Issues **"ViNet assets not found"** — Dynamic cropping disabled. Set `enable_dynamic_cropping: false` in config.json. **Subprocess fails from API but works manually** — Run server in tmux to preserve environment. **Music download 401/403** — Check Soundstripe API key validity. **Clip timestamps wrong** — Ensure `original_audio: true` to enable timestamp refinement via transcription. ## Manual Music Addition When Soundstripe fails, manually download and mix: ```bash # Download from Soundstripe API SOUNDSTRIPE_KEY=$(jq -r '.api_keys.SOUNDSTRIPE_KEY' ~/vea/config.json) curl -s "https://api.soundstripe.com/v1/songs/TRACK_ID" \ -H "Authorization: Token $SOUNDSTRIPE_KEY" | jq '.included[0].attributes.versions.mp3' # Mix with ffmpeg (15-20% music volume) ffmpeg -y -i video.mp4 -i music.mp3 \ -filter_complex "[1:a]volume=0.18,afade=t=out:st=70:d=4[m];[0:a][m]amix=inputs=2:duration=first[a]" \ -map 0:v -map "[a]" -c:v copy -c:a aac output.mp4 ``` ## References - [API Documentation](references/api.md) — Full endpoint specs - [Config Schema](references/config.md) — Configuration options


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)