Skill: cmfinlan/audio-speaker-tools (version 1.0.0)

--- name: audio-speaker-tools description: "Speaker separation, voice comparison, and audio processing tools. Use when working with multi-speaker audio, voice cloning, or speaker verification tasks including: (1) separating speakers from audio files via Demucs and pyannote diarization, (2) comparing voice samples for speaker verification or voice clone quality assessment using Resemblyzer, (3) extracting audio segments, (4) preparing samples for ElevenLabs voice cloning, or (5) validating speaker diarization results." --- # Audio Speaker Tools Tools for speaker separation, voice comparison, and audio processing using Demucs, pyannote, and Resemblyzer. ## Overview This skill provides three main workflows: 1. **Speaker separation** - Extract per-speaker audio from multi-speaker recordings 2. **Voice comparison** - Measure speaker similarity between two audio files 3. **Audio processing** - Segment extraction and voice isolation ## Prerequisites ### Setup Virtual Environment Run once to create the venv and install dependencies: ```bash bash scripts/setup_venv.sh ``` Default venv location: `./.venv` **Requirements:** - Python 3.9+ - ffmpeg (`brew install ffmpeg`) - HuggingFace token (set as env var `HF_TOKEN`) ## Scripts ### 1. Speaker Separation: `diarize_and_slice_mps.py` Separate speakers from multi-speaker audio: ```bash # Basic usage HF_TOKEN=<your-hf-token> \ /path/to/venv/bin/python scripts/diarize_and_slice_mps.py \ --input audio.mp3 \ --outdir /path/to/output \ --prefix MyShow # With speaker constraints HF_TOKEN=$TOKEN python scripts/diarize_and_slice_mps.py \ --input audio.mp3 \ --outdir ./out \ --min-speakers 2 \ --max-speakers 5 \ --pad-ms 100 ``` **Process:** 1. Converts input to 16kHz mono WAV 2. Runs Demucs vocal/background separation (optional, for cleaner input) 3. Runs pyannote speaker diarization (MPS-accelerated) 4. Extracts concatenated per-speaker WAV files **Output:** - `<prefix>_speaker1.wav`, `<prefix>_speaker2.wav`, etc. (one per detected speaker) - `diarization.rttm` (time-stamped speaker segments) - `segments.jsonl` (JSON segments metadata) - `meta.json` (pipeline info and speaker index) **Important:** - **Always pass HF token via `HF_TOKEN` env var**, never as CLI arg - **MPS first, CPU fallback** - Script prefers Metal GPU, falls back to CPU if unavailable - Default output: `./separated/` ### 2. Voice Comparison: `compare_voices.py` Measure similarity between two voice samples using Resemblyzer: ```bash # Basic comparison python scripts/compare_voices.py \ --audio1 sample1.wav \ --audio2 sample2.wav # JSON output python scripts/compare_voices.py \ --audio1 reference.wav \ --audio2 clone.wav \ --threshold 0.85 \ --json # Exit code = 0 if pass, 1 if fail ``` **Scores:** - `< 0.75` = Different speakers - `0.75-0.84` = Likely same speaker - `0.85+` = Excellent match (ideal for voice cloning validation) **Use cases:** - Voice clone quality assessment (compare clone vs. original) - Speaker verification (authenticate speaker identity) - Validate speaker separation (confirm separated speakers are distinct) **See:** `references/scoring-guide.md` for detailed interpretation ### 3. Audio Trimming Use `ffmpeg` directly for segment extraction: ```bash # Extract 10-second segment starting at 5 seconds ffmpeg -i input.mp3 -ss 5 -t 10 -c copy output.mp3 # Extract vocals only with Demucs (before diarization) demucs --two-stems vocals --out ./separated input.mp3 ``` ## Workflows ### Workflow 1: Extract Clean Voice Sample for Cloning **Goal:** Get a clean, single-speaker sample for ElevenLabs voice cloning ```bash # 1. Separate speakers HF_TOKEN=<your-hf-token> python scripts/diarize_and_slice_mps.py \ --input podcast.mp3 --outdir ./out --prefix Podcast # 2. Review speaker files (out/Podcast_speaker1.wav, etc.) # 3. Select best sample (5-30s, clean speech) ffmpeg -i out/Podcast_speaker2.wav -ss 10 -t 20 -c copy sample.wav # 4. Upload to ElevenLabs as instant voice clone ``` **See:** `references/elevenlabs-cloning.md` for best practices ### Workflow 2: Validate Voice Clone Quality **Goal:** Measure how well a cloned voice matches the original ```bash # 1. Generate test audio with ElevenLabs clone # (done via ElevenLabs web UI or API) # 2. Compare clone vs. reference python scripts/compare_voices.py \ --audio1 original_sample.wav \ --audio2 elevenlabs_clone.wav \ --threshold 0.85 \ --json # 3. Interpret score: # 0.85+ = excellent, publish-ready # 0.80-0.84 = acceptable, may need tweaking # < 0.80 = poor, try different sample or settings ``` **See:** `references/scoring-guide.md` for troubleshooting low scores ### Workflow 3: Multi-Speaker Conversation Analysis **Goal:** Separate and identify speakers in a conversation ```bash # 1. Run diarization HF_TOKEN=$TOKEN python scripts/diarize_and_slice_mps.py \ --input meeting.mp3 --outdir ./out --prefix Meeting # 2. Check detected speakers (meta.json) cat out/meta.json # 3. Compare speaker pairs to confirm separation python scripts/compare_voices.py \ --audio1 out/Meeting_speaker1.wav \ --audio2 out/Meeting_speaker2.wav # Expected: < 0.75 if separation worked correctly ``` ## Technical Notes ### Device Acceleration - **pyannote diarization:** MPS (Metal) by default, CPU fallback - **Resemblyzer:** CPU only (no GPU acceleration) - **Demucs:** MPS by default when available To force CPU for diarization: `--device cpu` ### Audio Formats - **Input:** Any format supported by ffmpeg (wav, mp3, flac, m4a, etc.) - **Processing:** Internally converted to 16kHz mono WAV for diarization - **Output:** WAV format (44.1kHz stereo preserved from source) ### HuggingFace Token - **Required for:** pyannote speaker diarization - **Access:** Must accept gated repo `pyannote/speaker-diarization-3.1` on HF - **Storage:** Any secure secrets manager - **Usage:** Always pass via `HF_TOKEN` env var, never CLI arg ### Sample Quality Tips - **Shorter is better:** 5-30s clean samples often score higher than 60+ second samples - **Clean audio:** Remove background noise with Demucs `--two-stems vocals` - **Single speaker:** Ensure isolated voice, not mixed conversation - **Good recording:** Studio mic > phone mic for voice comparison accuracy ## References - **elevenlabs-cloning.md** - Best practices for ElevenLabs instant voice cloning (model settings, sample selection, proven configurations) - **scoring-guide.md** - How to interpret Resemblyzer similarity scores (thresholds, use cases, troubleshooting) ## Common Issues ### "Missing HF token" error - Export token before running: `export HF_TOKEN=<your-token>` - Or pass inline: `HF_TOKEN=<your-token> python script.py ...` ### Low voice comparison scores for same speaker - Try shorter, cleaner samples (5-30s) - Use Demucs to isolate vocals: `demucs --two-stems vocals input.mp3` - Ensure consistent recording quality (same mic, environment) - See `references/scoring-guide.md` troubleshooting section ### Diarization not detecting all speakers - Adjust `--min-speakers` and `--max-speakers` flags - Check audio quality (clear speech, minimal overlap) - Try longer audio (30+ seconds) for better speaker modeling ### MPS/Metal acceleration not working - Ensure PyTorch with MPS support: `python -c "import torch; print(torch.backends.mps.is_available())"` - Fallback to CPU: `--device cpu` - Re-run `setup_venv.sh` to reinstall PyTorch

path: references/elevenlabs-cloning.md
content: # ElevenLabs Instant Voice Cloning (IVC) Best Practices

## Quick Reference

**Best settings for different voice types:**

- **Theatrical/dynamic voices**: `stability=85, similarity_boost=5`
- **Standard voices**: `stability=83, similarity_boost=10`

**Model versions:**
- **v2 (Multilingual)** > v3 for voice cloning quality
- v3 is newer but v2 produces better clones for most voices

**Sample quality:**
- **Clean, short samples** (5-30 seconds) often score better than longer ones
- Remove background noise, music, and other speakers
- Single-speaker, clear speech preferred

## Model Settings

### Stability (0-100)

Controls consistency vs. expressiveness:

- **Low (0-40)**: More variable, expressive, emotional (can drift from voice)
- **Medium (40-70)**: Balanced
- **High (70-100)**: Very consistent, less expressive (safer for cloning)

**For voice cloning:** Use high stability (80-85) to stay close to the reference voice.

### Similarity Boost (0-100)

Controls how closely the output matches the reference voice:

- **Low (0-30)**: More creative interpretation, may drift
- **Medium (40-70)**: Balanced
- **High (70-100)**: Strict adherence to reference voice

**For voice cloning:** Use moderate-to-low similarity boost (5-15) to avoid over-fitting.

### Style Exaggeration (v2 only, 0-100)

Amplifies the speaking style from the reference:

- **Low (0-30)**: Subtle style matching
- **Medium (40-70)**: Balanced
- **High (70-100)**: Exaggerated style (can sound unnatural)

**For most clones:** Use low style exaggeration (5-10) to avoid over-dramatization.

## Proven Settings

### Theatrical/Dynamic Voice
```
model: eleven_multilingual_v2
stability: 85
similarity_boost: 5
style_exaggeration: 5
```

### Standard Professional Voice
```
model: eleven_multilingual_v2
stability: 83
similarity_boost: 10
style_exaggeration: 5
```

## Sample Selection

**What makes a good voice sample:**

1. **Clean audio** - No background noise, music, or other speakers
2. **Short duration** - 5-30 seconds is often better than 60+ seconds
3. **Single speaker** - Isolated voice, not from a conversation
4. **Natural speech** - Not singing, shouting, or whispered
5. **Good recording quality** - Clear, well-recorded audio

**Pro tip:** Use `compare_voices.py` to score cloned voice against the original reference. Scores of 0.85+ indicate excellent cloning quality.

## Voice Comparison Workflow

1. **Separate speakers** from source audio using `diarize_and_slice_mps.py`
2. **Select best sample** (clean, short, single-speaker segments)
3. **Upload to ElevenLabs** as instant voice clone
4. **Generate test audio** with the clone
5. **Compare** clone vs. original using `compare_voices.py`
6. **Iterate** sample selection or settings if score < 0.85

## Model Version Comparison

| Feature | v2 (Multilingual) | v3 |
|---------|-------------------|-----|
| Clone quality | ⭐⭐⭐⭐⭐ Excellent | ⭐⭐⭐ Good |
| Style matching | ⭐⭐⭐⭐ Very good | ⭐⭐⭐⭐ Very good |
| Consistency | ⭐⭐⭐⭐ Reliable | ⭐⭐⭐⭐⭐ Excellent |
| Multilingual | ✓ 29 languages | ✓ 32 languages |
| Best for | Voice cloning, character voices | Consistency, production |

**Bottom line:** Use v2 for voice cloning tasks.

## Common Issues

### Clone sounds generic/flat
- **Cause:** Similarity boost too high, or stability too low
- **Fix:** Lower similarity boost to 5-10, raise stability to 80-85

### Clone drifts from reference
- **Cause:** Stability too low
- **Fix:** Raise stability to 80+

### Clone sounds over-dramatic
- **Cause:** Style exaggeration too high
- **Fix:** Lower style exaggeration to 5-10

### Clone sounds robotic
- **Cause:** Stability too high
- **Fix:** Lower stability to 75-80 for more natural variation

## Related Tools

- **compare_voices.py** - Score clone quality vs. reference (Resemblyzer)
- **diarize_and_slice_mps.py** - Extract per-speaker samples from multi-speaker audio


path: references/scoring-guide.md
content: # Resemblyzer Voice Similarity Scoring Guide

## Quick Reference

**Similarity score interpretation:**

- **< 0.75** = Different speakers
- **0.75 - 0.84** = Likely same speaker
- **0.85+** = Excellent match (same speaker)

## How Resemblyzer Works

Resemblyzer computes speaker embeddings (d-vectors) using a deep neural network trained on speaker verification. It:

1. Extracts audio features from each sample
2. Computes a 256-dimensional embedding vector representing the speaker's voice
3. Calculates cosine similarity between the two embeddings

**Range:** 0.0 (completely different) to 1.0 (identical)

## Score Interpretation

### 0.00 - 0.50: Clearly Different Speakers
- **What it means:** The voices are very different
- **Use case:** Confirming speaker separation worked correctly
- **Example:** Male vs. female voice, different accents, different vocal qualities

### 0.50 - 0.74: Different Speakers (Possible Similarities)
- **What it means:** Some vocal similarities but likely different people
- **Use case:** May indicate same gender/age range but different speakers
- **Example:** Two male voices with similar pitch, two female voices with similar tone

### 0.75 - 0.84: Likely Same Speaker
- **What it means:** High probability of same speaker, acceptable match
- **Use case:** Speaker verification, voice authentication
- **Example:** Same person recorded at different times or with different microphones
- **Action:** Acceptable for most verification tasks, but not ideal for cloning

### 0.85 - 0.95: Excellent Match
- **What it means:** Very high confidence, same speaker
- **Use case:** High-quality voice cloning verification, speaker identification
- **Example:** Clean samples from the same speaker with consistent recording quality
- **Action:** Ideal threshold for voice cloning quality assessment

### 0.95 - 1.00: Near Identical
- **What it means:** Extremely similar or identical recordings
- **Use case:** Confirming duplicate audio, very high-quality clones
- **Example:** Same audio file, slight variations in processing
- **Note:** Scores this high are rare unless comparing very similar recordings

## Factors Affecting Scores

### Positive Factors (Increase Score)
- **Clean audio** - No background noise or music
- **Short, focused samples** - 5-30 seconds of clear speech
- **Consistent recording quality** - Same microphone, environment
- **Similar emotional tone** - Neutral speech vs. neutral speech
- **Same language** - Model performs best within same language

### Negative Factors (Decrease Score)
- **Background noise or music** - Interferes with voice features
- **Long, varied samples** - Introduces more variability
- **Different recording conditions** - Phone vs. studio mic
- **Different emotional states** - Shouting vs. whispering
- **Different languages** - May reduce similarity even for same speaker
- **Multiple speakers** - If not properly isolated

## Practical Thresholds

### Voice Authentication (Security)
- **Strict:** 0.85+ (low false positive rate)
- **Balanced:** 0.80+ (moderate security)
- **Lenient:** 0.75+ (higher false positives, fewer false negatives)

### Voice Cloning Quality Assessment
- **Excellent clone:** 0.85+ (publish-ready)
- **Good clone:** 0.80-0.84 (acceptable, may need refinement)
- **Poor clone:** < 0.80 (needs better sample or settings)

### Speaker Diarization Validation
- **Confirming separation worked:** < 0.75 between different speaker segments
- **Confirming speaker continuity:** 0.75+ across segments of same speaker

## Common Use Cases

### 1. Voice Clone Quality Check
```bash
compare_voices.py --audio1 original_sample.wav --audio2 cloned_output.wav --threshold 0.85
```

**Expected:**
- Good clone: 0.85+
- Poor clone: < 0.80

### 2. Speaker Verification
```bash
compare_voices.py --audio1 enrollment.wav --audio2 test.wav --threshold 0.75
```

**Expected:**
- Same speaker: 0.75+
- Different speaker: < 0.75

### 3. Validating Speaker Separation
```bash
compare_voices.py --audio1 speaker1.wav --audio2 speaker2.wav --threshold 0.75
```

**Expected:**
- Successful separation: < 0.75 (confirms different speakers)
- Failed separation: 0.75+ (may indicate mixed speakers)

## Troubleshooting Low Scores

### Same speaker scoring < 0.75?

**Possible causes:**
- **Different recording quality** - Check microphones, environment
- **Background noise** - Use Demucs to isolate vocals
- **Different emotional tone** - Compare similar speaking styles
- **Poor audio quality** - Use cleaner, shorter samples

**Solutions:**
1. Pre-process audio with Demucs `--two-stems vocals`
2. Trim to shorter, cleaner segments (5-30s)
3. Ensure 16kHz+ sample rate
4. Remove silence/padding from samples

### Different speakers scoring > 0.75?

**Possible causes:**
- **Similar vocal characteristics** - Same gender, age, accent
- **Mixed/contaminated samples** - Multiple speakers in one file
- **Very short samples** - Not enough data for distinction

**Solutions:**
1. Use longer samples (15-30s) for better distinction
2. Re-run speaker diarization to ensure clean separation
3. Manually verify samples contain only target speaker
4. Try different segments from each speaker

## Technical Notes

- **Model:** Resemblyzer uses a GE2E (Generalized End-to-End) loss-trained encoder
- **Embedding size:** 256-dimensional d-vector
- **Metric:** Cosine similarity (dot product of normalized vectors)
- **Processing:** Audio is converted to mel-spectrograms before embedding
- **Speed:** CPU-based inference (typically 1-3 seconds per comparison)

## Related Scripts

- **compare_voices.py** - Main comparison tool
- **diarize_and_slice_mps.py** - Extract per-speaker samples for comparison


path: scripts/compare_voices.py
content: #!/usr/bin/env python3
"""Compare two audio files using Resemblyzer d-vector cosine similarity.

Computes speaker embeddings for two audio files and reports:
- Cosine similarity score
- Pass/fail verdict (based on configurable threshold)

Example:
  python compare_voices.py --audio1 sample1.wav --audio2 sample2.wav --threshold 0.75

Scoring guide:
  < 0.75 = Different speakers
  0.75-0.84 = Likely same speaker
  0.85+ = Excellent match (same speaker)

Notes:
- Works best with clean, short audio samples (5-30s)
- Resemblyzer uses CPU by default
- Audio format: any format supported by librosa (wav, mp3, flac, etc.)
"""

from __future__ import annotations

import argparse
import sys
from pathlib import Path

import numpy as np
from resemblyzer import VoiceEncoder, preprocess_wav


def compute_embedding(audio_path: str, encoder: VoiceEncoder) -> np.ndarray:
    """Load audio and compute d-vector embedding."""
    wav = preprocess_wav(Path(audio_path))
    embed = encoder.embed_utterance(wav)
    return embed


def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    """Compute cosine similarity between two vectors."""
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))


def main() -> None:
    ap = argparse.ArgumentParser(description="Compare two voice samples using Resemblyzer")
    ap.add_argument("--audio1", required=True, help="First audio file")
    ap.add_argument("--audio2", required=True, help="Second audio file")
    ap.add_argument("--threshold", type=float, default=0.75, help="Similarity threshold for pass/fail (default: 0.75)")
    ap.add_argument("--json", action="store_true", help="Output JSON format")
    args = ap.parse_args()

    if not Path(args.audio1).exists():
        print(f"Error: {args.audio1} does not exist", file=sys.stderr)
        sys.exit(1)
    if not Path(args.audio2).exists():
        print(f"Error: {args.audio2} does not exist", file=sys.stderr)
        sys.exit(1)

    # Initialize encoder (CPU)
    encoder = VoiceEncoder()

    # Compute embeddings
    embed1 = compute_embedding(args.audio1, encoder)
    embed2 = compute_embedding(args.audio2, encoder)

    # Compute similarity
    similarity = cosine_similarity(embed1, embed2)

    # Determine verdict
    if similarity >= 0.85:
        verdict = "EXCELLENT_MATCH"
        verdict_text = "Excellent match (same speaker)"
    elif similarity >= args.threshold:
        verdict = "LIKELY_SAME"
        verdict_text = "Likely same speaker"
    else:
        verdict = "DIFFERENT"
        verdict_text = "Different speakers"

    # Output
    if args.json:
        import json

        result = {
            "audio1": args.audio1,
            "audio2": args.audio2,
            "similarity": round(similarity, 4),
            "threshold": args.threshold,
            "verdict": verdict,
            "verdict_text": verdict_text,
            "pass": similarity >= args.threshold,
        }
        print(json.dumps(result, indent=2))
    else:
        print(f"Audio 1: {args.audio1}")
        print(f"Audio 2: {args.audio2}")
        print(f"Similarity: {similarity:.4f}")
        print(f"Threshold: {args.threshold:.2f}")
        print(f"Verdict: {verdict_text}")
        print(f"Pass: {'YES' if similarity >= args.threshold else 'NO'}")

    # Exit code based on pass/fail
    sys.exit(0 if similarity >= args.threshold else 1)


if __name__ == "__main__":
    main()


path: scripts/diarize_and_slice_mps.py
content: #!/usr/bin/env python3
"""Diarize an audio file with pyannote and export one concatenated WAV per speaker.

This variant *forces* the pyannote pipeline onto a specified torch device
(default: MPS) and errors if the device is unavailable.

Exports into outdir:
- diarization.rttm
- segments.jsonl
- meta.json
- <prefix>_speaker<N>.wav (concatenated segments)

Example:
  python diarize_and_slice_mps.py --input vocals.wav --outdir out --token $HF_TOKEN --prefix MyShow

Notes:
- Converts input to 16kHz mono WAV for pyannote.
- If you want to prevent silent CPU fallback for unsupported MPS ops, run with:
    PYTORCH_ENABLE_MPS_FALLBACK=0
"""

from __future__ import annotations

import argparse
import json
import os
import subprocess
from collections import defaultdict

from pydub import AudioSegment


def ffmpeg_convert_to_wav_mono_16k(src: str, dst: str) -> None:
    cmd = [
        "ffmpeg",
        "-y",
        "-i",
        src,
        "-ac",
        "1",
        "-ar",
        "16000",
        "-vn",
        dst,
    ]
    subprocess.run(cmd, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)


def _ensure_torchaudio_compat_shim() -> None:
    """Keep older deps happy on newer torchaudio."""
    try:
        import torchaudio  # type: ignore

        if not hasattr(torchaudio, "list_audio_backends"):

            def _list_audio_backends() -> list[str]:
                return ["ffmpeg"]

            torchaudio.list_audio_backends = _list_audio_backends  # type: ignore[attr-defined]
    except Exception:
        pass


def main() -> None:
    ap = argparse.ArgumentParser()
    ap.add_argument("--input", required=True, help="Input audio file (wav preferred)")
    ap.add_argument("--outdir", required=True)
    ap.add_argument("--token", default=os.environ.get("HF_TOKEN") or os.environ.get("HUGGINGFACE_TOKEN"))
    ap.add_argument("--prefix", default=None, help="Prefix for exported files (e.g., Podcast)")
    ap.add_argument("--min-speakers", type=int, default=None)
    ap.add_argument("--max-speakers", type=int, default=None)
    ap.add_argument("--pad-ms", type=int, default=80, help="Pad each segment on both sides (ms)")
    ap.add_argument("--device", default="mps", help="Torch device to run on (default: mps)")
    args = ap.parse_args()

    if not args.token:
        raise SystemExit("Missing HF token. Provide --token or set HF_TOKEN.")

    os.makedirs(args.outdir, exist_ok=True)

    base = os.path.splitext(os.path.basename(args.input))[0]
    prefix = args.prefix or base

    wav16 = os.path.join(args.outdir, f"{prefix}_vocals_16k_mono.wav")
    ffmpeg_convert_to_wav_mono_16k(args.input, wav16)

    _ensure_torchaudio_compat_shim()

    import torch
    from pyannote.audio import Pipeline

    device = torch.device(args.device)
    if device.type == "mps":
        if not torch.backends.mps.is_built() or not torch.backends.mps.is_available():
            raise SystemExit("MPS requested but torch.backends.mps.is_available() is False")

    pipeline = Pipeline.from_pretrained(
        "pyannote/speaker-diarization-3.1",
        [REDACTED_SECRET],
    )

    # Move pipeline to requested device.
    try:
        pipeline.to(device)
    except Exception as e:
        raise SystemExit(f"Failed to move pyannote pipeline to device={device}: {e}")

    diarize_output = pipeline(
        {"audio": wav16},
        min_speakers=args.min_speakers,
        max_speakers=args.max_speakers,
    )

    # pyannote.audio 3.1 returns a `DiarizeOutput` that contains
    # `speaker_diarization` (pyannote.core.Annotation).
    annotation = (
        getattr(diarize_output, "speaker_diarization", None)
        or getattr(diarize_output, "exclusive_speaker_diarization", None)
        or diarize_output
    )

    # Save RTTM
    rttm_path = os.path.join(args.outdir, "diarization.rttm")
    with open(rttm_path, "w", encoding="utf-8") as f:
        for segment, track, speaker in annotation.itertracks(yield_label=True):
            f.write(
                f"SPEAKER {prefix} 1 {segment.start:.3f} {segment.duration:.3f} <NA> <NA> {speaker} <NA> <NA>\n"
            )

    # Save segments JSONL
    seg_path = os.path.join(args.outdir, "segments.jsonl")
    speakers = []
    segments_by_speaker = defaultdict(list)
    with open(seg_path, "w", encoding="utf-8") as f:
        for segment, _, speaker in annotation.itertracks(yield_label=True):
            speakers.append(speaker)
            rec = {
                "speaker": speaker,
                "start": float(segment.start),
                "end": float(segment.end),
                "duration": float(segment.end - segment.start),
            }
            segments_by_speaker[speaker].append(rec)
            f.write(json.dumps(rec) + "\n")

    unique_speakers = sorted(set(speakers))
    speaker_to_index = {spk: i + 1 for i, spk in enumerate(unique_speakers)}

    # Concatenate segments per speaker.
    audio = AudioSegment.from_wav(wav16)
    for spk in unique_speakers:
        out = AudioSegment.silent(duration=0, frame_rate=audio.frame_rate)
        for rec in segments_by_speaker[spk]:
            s = max(0, int(rec["start"] * 1000) - args.pad_ms)
            e = min(len(audio), int(rec["end"] * 1000) + args.pad_ms)
            out += audio[s:e]
        out_path = os.path.join(args.outdir, f"{prefix}_speaker{speaker_to_index[spk]}.wav")
        out.export(out_path, format="wav")

    meta_path = os.path.join(args.outdir, "meta.json")
    meta = {
        "input": args.input,
        "converted": wav16,
        "pipeline": "pyannote/speaker-diarization-3.1",
        "torch": getattr(torch, "__version__", None),
        "device": str(device),
        "mps_available": bool(getattr(torch.backends, "mps", None) and torch.backends.mps.is_available()),
        "num_speakers": len(unique_speakers),
        "speakers": unique_speakers,
        "speaker_index": speaker_to_index,
        "rttm": rttm_path,
        "segments": seg_path,
        "pad_ms": args.pad_ms,
    }
    with open(meta_path, "w", encoding="utf-8") as f:
        json.dump(meta, f, indent=2)

    print(json.dumps({"num_speakers": len(unique_speakers), "outdir": args.outdir}, indent=2))


if __name__ == "__main__":
    main()


path: scripts/setup_venv.sh
content: #!/usr/bin/env bash
# Create and configure the audio-speaker-tools virtual environment.
#
# This script creates a venv with all required dependencies for:
# - Speaker diarization (pyannote, demucs)
# - Voice comparison (resemblyzer)
# - Audio processing (pydub, ffmpeg)
#
# Usage:
#   bash setup_venv.sh [venv_path]
#
# Default venv path: ./.venv
#
# Requirements:
# - Python 3.9+
# - ffmpeg (brew install ffmpeg)

set -euo pipefail

VENV_PATH="${1:-./.venv}"

echo "Setting up audio-speaker-tools venv at: $VENV_PATH"

# Check for Python 3
if ! command -v python3 &> /dev/null; then
    echo "Error: python3 not found in PATH" >&2
    exit 1
fi

# Check for ffmpeg
if ! command -v ffmpeg &> /dev/null; then
    echo "Error: ffmpeg not found. Install with: brew install ffmpeg" >&2
    exit 1
fi

# Create venv if it doesn't exist
if [ ! -d "$VENV_PATH" ]; then
    echo "Creating venv..."
    python3 -m venv "$VENV_PATH"
else
    echo "Venv already exists at $VENV_PATH"
fi

# Activate venv
source "$VENV_PATH/bin/activate"

# Upgrade pip
echo "Upgrading pip..."
pip install --quiet --upgrade pip

# Install PyTorch with MPS support (Metal Performance Shaders for Apple Silicon)
echo "Installing PyTorch with MPS support..."
pip install --quiet torch torchvision torchaudio

# Install audio processing libraries
echo "Installing audio processing libraries..."
pip install --quiet \
    demucs \
    pyannote.audio \
    pydub \
    resemblyzer \
    librosa

echo ""
echo "✓ Setup complete!"
echo ""
echo "Venv path: $VENV_PATH"
echo ""
echo "To activate:"
echo "  source $VENV_PATH/bin/activate"
echo ""
echo "To use with scripts:"
echo "  $VENV_PATH/bin/python scripts/diarize_and_slice_mps.py --help"
echo "  $VENV_PATH/bin/python scripts/compare_voices.py --help"
echo ""
echo "Required env var for diarization:"
echo "  export HF_TOKEN=<your-token>"


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)