Skill: shaharsha/elevenlabs-tts (version 2.4.0)

--- name: elevenlabs-tts description: ElevenLabs TTS - the best ElevenLabs integration for OpenClaw. ElevenLabs Text-to-Speech with emotional audio tags, ElevenLabs voice synthesis for WhatsApp, ElevenLabs multilingual support. Generate realistic AI voices using ElevenLabs API. tags: [elevenlabs, tts, voice, text-to-speech, audio, speech, whatsapp, multilingual, ai-voice] metadata: {"clawdbot":{"emoji":"🎙️","requires":{"env":["ELEVENLABS_API_KEY"],"system":["ffmpeg"],"tools":["ffmpeg"]},"primaryEnv":"ELEVENLABS_API_KEY","source":"https://clawhub.com/skills/elevenlabs-tts","version":"2.3.0"}} allowed-tools: [exec, tts, message] --- # ElevenLabs TTS (Text-to-Speech) Generate expressive voice messages using ElevenLabs v3 with audio tags. ## Prerequisites - **ElevenLabs API Key** (`ELEVENLABS_API_KEY`): Required. Get one at [elevenlabs.io](https://elevenlabs.io) → Profile → API Keys. Configure in `openclaw.json` under `messages.tts.elevenlabs.apiKey`. - **ffmpeg**: Required for audio format conversion (MP3 → Opus for WhatsApp compatibility). Must be installed and available on PATH. ## Quick Start Examples **Storytelling (emotional journey):** ``` [soft] It started like any other day... [pause] But something felt different. [nervous] My hands were shaking as I opened the envelope. [gasps] I got in! [excited] I actually got in! [laughs] [happy] This changes everything! ``` **Horror/Suspense (building dread):** ``` [whispers] The house has been empty for years... [pause] At least, that's what they told me. [nervous] But I keep hearing footsteps. [scared] They're getting closer. [gasps] [panicking] The door— it's opening by itself! ``` **Conversation with reactions:** ``` [curious] So what happened at the meeting? [pause] [surprised] Wait, they fired him?! [gasps] [sad] That's terrible... [sighs] He had a family. [thoughtful] I wonder what he'll do now. ``` **Hebrew (romantic moment):** ``` [soft] היא עמדה שם, מול השקיעה... [pause] הלב שלי פעם כל כך חזק. [nervous] לא ידעתי מה להגיד. [hesitates] אני... [breathes] [tender] את יודעת שאני אוהב אותך, נכון? ``` **Spanish (celebration to reflection):** ``` [excited] ¡Lo logramos! [laughs] [happy] No puedo creerlo... [pause] [thoughtful] Fueron tantos años de trabajo. [emotional] [soft] Gracias a todos los que creyeron en mí. [sighs] [content] Valió la pena cada momento. ``` ## Configuration (OpenClaw) In `openclaw.json`, configure TTS under `messages.tts`: ```json { "messages": { "tts": { "provider": "elevenlabs", "elevenlabs": { "apiKey": "sk_your_api_key_here", "voiceId": "pNInz6obpgDQGcFmaJgB", "modelId": "eleven_v3", "languageCode": "en", "voiceSettings": { "stability": 0.5, "similarityBoost": 0.75, "style": 0, "useSpeakerBoost": true, "speed": 1 } } } } } ``` **Getting your API Key:** 1. Go to https://elevenlabs.io 2. Sign up/login 3. Click profile → API Keys 4. Copy your key ## Recommended Voices for v3 These premade voices are optimized for v3 and work well with audio tags: | Voice | ID | Gender | Accent | Best For | |-------|-----|--------|--------|----------| | **Adam** | `pNInz6obpgDQGcFmaJgB` | Male | American | Deep narration, general use | | **Rachel** | `21m00Tcm4TlvDq8ikWAM` | Female | American | Calm narration, conversational | | **Brian** | `nPczCjzI2devNBz1zQrb` | Male | American | Deep narration, podcasts | | **Charlotte** | `XB0fDUnXU5powFXDhCwa` | Female | English-Swedish | Expressive, video games | | **George** | `JBFqnCBsd6RMkjVDRZzb` | Male | British | Raspy narration, storytelling | **Finding more voices:** - Browse: https://elevenlabs.io/voice-library - v3-optimized collection: https://elevenlabs.io/app/voice-library/collections/aF6JALq9R6tXwCczjhKH - API: `GET https://api.elevenlabs.io/v1/voices` **Voice selection tips:** - Use IVC (Instant Voice Clone) or premade voices - PVC not optimized for v3 yet - Match voice character to your use case (whispering voice won't shout well) - For expressive IVCs, include varied emotional tones in training samples ## Model Settings - **Model**: `eleven_v3` (alpha) - ONLY model supporting audio tags - **Languages**: 70+ supported with full audio tag control ### Stability Modes | Mode | Stability | Description | |------|-----------|-------------| | **Creative** | 0.3-0.5 | More emotional/expressive, may hallucinate | | **Natural** | 0.5-0.7 | Balanced, closest to original voice | | **Robust** | 0.7-1.0 | Highly stable, less responsive to tags | For audio tags, use **Creative** (0.5) or **Natural**. Higher stability reduces tag responsiveness. ### Speed Control Range: 0.7 (slow) to 1.2 (fast), default 1.0 Extreme values affect quality. For pacing, prefer audio tags like `[rushed]` or `[drawn out]`. ## Critical Rules ### Length Limits - **Optimal**: <800 characters per segment (best quality) - **Maximum**: 10,000 characters (API hard limit) - **Quality degrades** with longer text - voice becomes inconsistent ### Audio Tags - Best Practices for Natural Sound **How many tags to use:** - 1-2 tags per sentence or phrase (not more!) - Tags persist until the next tag - no need to repeat - Overusing tags sounds unnatural and robotic **Where to place tags:** - At emotional transition points - Before key dramatic moments - When energy/pace changes **Context matters:** - Write text that *matches* the tag emotion - Longer text with context = better interpretation - Example: `[nervous] I... I'm not sure about this. What if it doesn't work?` works better than `[nervous] Hello.` **Combine tags for nuance:** - `[nervously][whispers]` = nervous whispering - `[excited][laughs]` = excited laughter - Keep combinations to 2 tags max **Regenerate for best results:** - v3 is non-deterministic - same text = different outputs - Generate 3+ versions, pick the best - Small text tweaks can improve results **Match tag to voice:** - Don't use `[shouts]` on a whispering voice - Don't use `[whispers]` on a loud/energetic voice - Test tags with your chosen voice ### SSML Not Supported v3 does NOT support SSML break tags. Use audio tags and punctuation instead. ### Punctuation Effects (use with tags!) Punctuation enhances audio tags: - **Ellipses (...)** → dramatic pauses: `[nervous] I... I don't know...` - **CAPS** → emphasis: `[excited] That's AMAZING!` - **Dashes (—)** → interruptions: `[explaining] So what you do is— [interrupting] Wait!` - **Question marks** → uncertainty: `[nervous] Are you sure about this?` - **Exclamation!** → energy boost: `[happy] We did it!` Combine tags + punctuation for maximum effect: ``` [tired] It was a long day... [sighs] Nobody listens anymore. ``` ## WhatsApp Voice Messages ### Complete Workflow 1. **Generate** with `tts` tool (returns Opus in `/tmp/openclaw/tts-*/`) 2. **Copy** to workspace (message tool only allows workspace paths) 3. **Send** with `message` tool 4. **Cleanup** - delete the workspace copy ### Step-by-Step **1. Generate TTS (add [pause] at end to prevent cutoff):** ``` tts text="[excited] This is amazing! [pause]" channel=whatsapp ``` **2. Find the LATEST file (⚠️ CRITICAL - always use the newest file!):** ```bash find /tmp/openclaw/tts-* /tmp/tts-* -name "*.opus" -o -name "*.mp3" -o -name "*.ogg" 2>/dev/null | xargs ls -t | head -1 ``` The `tts` tool now outputs to `/tmp/openclaw/tts-*/` (NOT `/tmp/tts-*/`). Old files may exist in `/tmp/tts-*/` from previous sessions - **never use those!** **3. If file is MP3, convert to Opus:** ```bash ffmpeg -i /path/to/voice.mp3 -c:a libopus -b:a 64k -vbr on -application voip /path/to/voice.ogg ``` If already `.opus`, skip this step. **4. Copy to workspace and send:** ```bash cp /tmp/openclaw/tts-xxx/voice.opus ~/. openclaw/workspace/voice-temp.ogg ``` ``` message action=send channel=whatsapp target="+972..." filePath="/root/.openclaw/workspace/voice-temp.ogg" asVoice=true message=" " ``` **5. Cleanup:** ```bash rm /root/.openclaw/workspace/voice-temp.ogg ``` WhatsApp requires a non-empty message body to send voice notes. Use a single space as the message. ### Why Opus? | Format | iOS | Android | Transcribe | |--------|-----|---------|------------| | MP3 | ✅ Works | ❌ May fail | ❌ No | | Opus (.ogg) | ✅ Works | ✅ Works | ✅ Yes | **Always convert to Opus** - it's the only format that: - Works on all devices (iOS + Android) - Supports WhatsApp's transcribe button ### Audio Cutoff Fix ElevenLabs sometimes cuts off the last word. **Always add `[pause]` or `...` at the end:** ``` [excited] This is amazing! [pause] ``` ## Long-Form Audio (Podcasts) For content >800 chars: 1. Split into short segments (<800 chars each) 2. Generate each with `tts` tool 3. Concatenate with ffmpeg: ```bash cat > list.txt << EOF file '/path/file1.mp3' file '/path/file2.mp3' EOF ffmpeg -f concat -safe 0 -i list.txt -c copy final.mp3 ``` 4. Convert to Opus for WhatsApp 5. Send as single voice message **Important**: Don't mention "part 2" or "chapter" - keep it seamless. ## Multi-Speaker Dialogue v3 can handle multiple characters in one generation: ``` Jessica: [whispers] Did you hear that? Chris: [interrupting] —I heard it too! Jessica: [panicking] We need to hide! ``` **Dialogue tags**: `[interrupting]`, `[overlapping]`, `[cuts in]`, `[interjecting]` ## Audio Tags Quick Reference | Category | Tags | When to Use | |----------|------|-------------| | **Emotions** | [excited], [happy], [sad], [angry], [nervous], [curious] | Main emotional state - use 1 per section | | **Delivery** | [whispers], [shouts], [soft], [rushed], [drawn out] | Volume/speed changes | | **Reactions** | [laughs], [sighs], [gasps], [clears throat], [gulps] | Natural human moments - sprinkle sparingly | | **Pacing** | [pause], [hesitates], [stammers], [breathes] | Dramatic timing | | **Character** | [French accent], [British accent], [robotic tone] | Character voice shifts | | **Dialogue** | [interrupting], [overlapping], [cuts in] | Multi-speaker conversations | **Most effective tags** (reliable results): - Emotions: `[excited]`, `[nervous]`, `[sad]`, `[happy]` - Reactions: `[laughs]`, `[sighs]`, `[whispers]` - Pacing: `[pause]` **Less reliable** (test and regenerate): - Sound effects: `[explosion]`, `[gunshot]` - Accents: results vary by voice **Full tag list**: See [references/audio-tags.md](references/audio-tags.md) ## Troubleshooting **Tags read aloud?** - Verify using `eleven_v3` model - Use IVC/premade voices, not PVC - Simplify tags (no "tone" suffix) - Increase text length (250+ chars) **Voice inconsistent?** - Segment is too long - split at <800 chars - Regenerate (v3 is non-deterministic) - Try lower stability setting **WhatsApp won't play?** - Convert to Opus format (see above) **No emotion despite tags?** - Voice may not match tag style - Try Creative stability mode (0.5) - Add more context around the tag

path: references/audio-tags.md
content: # Audio Tags Reference

Complete guide to ElevenLabs v3 audio tags.

## Prerequisites

- **Model**: `eleven_v3` (alpha) - ONLY this model supports audio tags
- **Voice Type**: IVC (Instant Voice Clone) or designed voices - PVC not optimized yet
- **Prompt Length**: 250+ characters for consistent results
- **Stability**: Creative or Natural mode (Robust reduces tag responsiveness)

## Core Principle

Write NATURAL sentences that tags modify, NOT explanations.

❌ WRONG: `[excited] אני מתרגש!`
✅ RIGHT: `[excited] זה ממש מדהים מה שעשינו היום!`

---

## Tag Categories

### Emotions (High Reliability)

| Tag | Description |
|-----|-------------|
| `[excited]` | Energy, enthusiasm |
| `[happy]` | Joy, cheerfulness |
| `[happily]` | Speaking with happiness |
| `[sad]` | Sadness, melancholy |
| `[sorrowful]` | Deep sadness |
| `[angry]` | Anger, intensity |
| `[curious]` | Curiosity, interest |
| `[nervous]` | Nervousness, anxiety |
| `[sarcastic]` | Sarcasm, irony |
| `[tired]` | Fatigue, weariness |
| `[serious]` | Seriousness |
| `[confident]` | Confidence |
| `[frustrated]` | Frustration |
| `[mischievous]` | Playful mischief |
| `[awe]` | Wonder, amazement |
| `[resigned]` | Acceptance, giving up |
| `[flustered]` | Confused embarrassment |
| `[casual]` | Relaxed, informal |
| `[annoyed]` | Irritation |

### Delivery & Volume (High Reliability)

| Tag | Description |
|-----|-------------|
| `[whispers]` | Quiet, intimate |
| `[shouts]` | Loud, intense |
| `[dramatic tone]` | Theatrical |
| `[dramatic]` | Dramatic delivery |
| `[matter-of-fact]` | Plain, factual |
| `[whiny]` | Complaining tone |
| `[flatly]` | No emotion |
| `[quietly]` | Soft voice |
| `[suspiciously]` | Suspicious tone |

### Pacing & Timing (High Reliability)

| Tag | Description |
|-----|-------------|
| `[pause]` | Brief silence |
| `[breathes]` | Breathing sound |
| `[continues after a beat]` | Pause then continue |
| `[rushed]` | Fast, urgent |
| `[slows down]` | Decreasing speed |
| `[deliberate]` | Careful, intentional |
| `[rapid-fire]` | Very fast |
| `[drawn out]` | Stretched, slow |
| `[stammers]` | Stuttering |
| `[hesitates]` | Uncertainty |
| `[timidly]` | Shy, tentative |
| `[repeats]` | Repetition |

### Emphasis (Medium Reliability)

| Tag | Description |
|-----|-------------|
| `[emphasized]` | Strong emphasis |
| `[stress on next word]` | Emphasize following word |
| `[understated]` | Downplayed delivery |

### Reactions & Sounds (Very High Reliability)

| Tag | Description |
|-----|-------------|
| `[laughs]` | Laughter |
| `[laughs softly]` | Gentle laugh |
| `[laughs harder]` | Increasing laughter |
| `[starts laughing]` | Beginning to laugh |
| `[nervous laugh]` | Anxious laughter |
| `[giggles]` | Small laugh |
| `[wheezing]` | Breathless laugh |
| `[sighs]` | Exhale of emotion |
| `[sigh]` | Single sigh |
| `[gasps]` | Sharp intake |
| `[exhales]` | Breathing out |
| `[clears throat]` | Throat clearing |
| `[gulps]` | Swallowing |
| `[swallows]` | Swallowing sound |
| `[snorts]` | Snorting sound |
| `[crying]` | Sobbing |

### Character & Accents (Medium Reliability)

| Tag | Description |
|-----|-------------|
| `[French accent]` | French accent |
| `[American accent]` | American accent |
| `[British accent]` | British accent |
| `[Australian accent]` | Australian accent |
| `[Southern US accent]` | Southern American |
| `[strong X accent]` | Replace X with accent |
| `[pirate voice]` | Pirate character |
| `[evil scientist voice]` | Mad scientist |
| `[childlike tone]` | Child-like voice |
| `[robotic tone]` | Robot voice |
| `[deep voice]` | Lower pitch |

### Narrative & Genre (Medium Reliability)

| Tag | Description |
|-----|-------------|
| `[storytelling tone]` | Narrator voice |
| `[voice-over style]` | Documentary style |
| `[fantasy narrator]` | Epic fantasy |
| `[sci-fi AI voice]` | Futuristic AI |
| `[classic film noir]` | 1940s detective |
| `[epic build-up]` | Building intensity |
| `[narrative flourish]` | Dramatic narration |

### Multi-Speaker Dialogue

| Tag | Description |
|-----|-------------|
| `[interrupting]` | Cutting off speaker |
| `[overlapping]` | Speaking over |
| `[cuts in]` | Interjecting |
| `[interjecting]` | Jumping in |
| `[fast-paced]` | Quick exchange |

### Sound Effects (Low-Medium Reliability)

| Tag | Description |
|-----|-------------|
| `[gunshot]` | Gun sound |
| `[clapping]` | Applause |
| `[applause]` | Audience clapping |
| `[explosion]` | Blast sound |
| `[thunder]` | Thunder |

### Experimental (Test First)

| Tag | Description |
|-----|-------------|
| `[sings]` | Singing |
| `[woo]` | Exclamation |
| `[fart]` | Sound effect |
| `[panicked]` | Panic |
| `[trembling]` | Shaking voice |

---

## Usage Guidelines

### ✅ DO:
- Use simple tags: `[excited]` not `[excited tone]`
- Write natural sentences that work without tags
- Use 2-4 tags per paragraph max
- Place tags at sentence start or key moment
- Match tags to voice character
- Test and regenerate (v3 is non-deterministic)
- Combine tags: `[whispering][pause] Did you hear that?`

### ❌ DON'T:
- Don't add "tone" suffix: `[serious tone]` ❌
- Don't overload with tags
- Don't explain what the tag does
- Don't use incompatible combos (whisper voice + shout tag)
- Don't expect consistency (regenerate if needed)

---

## Examples

### Emotional Monologue
```
[sighs] I've been thinking about what you said. [pause] 
And you're right. [sadly] I should have listened earlier.
[determined] But I'm going to fix this. Starting now.
```

### Multi-Character Dialogue
```
Sarah: [whispers] I think someone's coming.
Mike: [interrupting] —I heard it too! [panicked] Hide!
Sarah: [annoyed] I was TRYING to tell you that!
```

### Comedic Timing
```
[confident] So I walked up to the boss and said... 
[pause] [nervous laugh] Actually, I didn't say anything. 
[sighs] I just stood there. [laughs] Classic me.
```

### Accent Performance
```
[British accent] Terribly sorry, but I must insist.
[switches to Southern US accent] Well now, that's mighty kind of y'all.
[French accent] Mon ami, you simply must try ze croissant!
```

---

## Troubleshooting

**Tags being read aloud?**
- Check you're using `eleven_v3` (not turbo_v3 or v2.5)
- Use IVC/designed voices, not PVC
- Simplify tags (remove "tone", "sound", etc.)
- Increase prompt length (250+ chars)

**Tags not working?**
- Generate multiple times (v3 is variable)
- Use Creative or Natural stability (not Robust)
- Add surrounding context text
- Try different tag placement
- Voice may not match tag style

**Multi-speaker not distinct?**
- Add character cues: `[deep voice]`, `[higher pitch]`
- Use accent tags for differentiation
- Add emotional contrast between speakers


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (MEDIUM)