Skill: xiaoyaner0201/qwen3-tts-voicedesign (version 1.0.0)

--- name: qwen3-tts description: Text-to-speech with Qwen3-TTS VoiceDesign. Design custom voices via natural language descriptions + seed-based timbre fixation. Includes OpenAI-compatible API server, one-click setup, and batch seed exploration tools. Use when generating speech, designing voices, or adding TTS to OpenClaw. --- # Qwen3-TTS VoiceDesign Text → Speech with natural language voice descriptions + seed-based timbre fixation. ## Quick Start ```bash # Generate speech (uses server defaults) TTS_URL=http://your-server:8881 scripts/say.sh "Hello world!" # Save to file scripts/say.sh "Save this" output.mp3 # Batch compare seeds (voice exploration) scripts/batch_seeds.sh "Hello world!" 42 123 201 456 789 /tmp/seeds ``` ## Environment Variables All config via env vars — text is the only required argument: | Variable | Default | Description | |----------|---------|-------------| | `TTS_URL` | `http://localhost:8881` | Server base URL (client side) | | `TTS_SEED` | `4096` | Random seed → controls timbre | | `TTS_INSTRUCT` | (generic female voice) | Voice description prompt | | `TTS_MODEL_PATH` | `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | Model weights path | | `TTS_PORT` | `8881` | Server listen port | | `TTS_HOST` | `0.0.0.0` | Server bind address | | `TTS_FORMAT` | `mp3` | Output format: `mp3` / `wav` | Server reads from `.env` file in its directory. Client scripts read from shell env. ## Voice Description Example ``` 30岁男性播音员，声音低沉磁性， 语速稳重从容，咬字清晰标准， 像新闻联播主播的专业感，又带一点温暖。 ``` > **Tip:** Once you've found your perfect voice (description + seed), set them as server defaults in `.env`. Then client calls only need to pass `text`. ## API ### OpenAI-Compatible ```bash curl -X POST $TTS_URL/v1/audio/speech \ -H "Content-Type: application/json" \ -d '{"input": "Hello!"}' -o speech.mp3 ``` ### Custom (seed + instruct override) ```bash curl -X POST $TTS_URL/tts \ -H "Content-Type: application/json" \ -d '{"text": "Hello!", "seed": 201, "instruct": "温柔女生"}' -o speech.mp3 ``` ### GET (quick test) ```bash curl "$TTS_URL/tts?text=Hello&seed=201" -o test.mp3 ``` ## Seed Mechanics Same `(description + seed)` → same timbre. Different seeds → completely different voices. ⚠️ **Seeds are purely random** — seed 42 and 43 can sound completely different. Finding a voice = opening blind boxes. **Workflow:** fix description → batch 30-40 seeds → listen → shortlist 2-3 → compare across scenarios → pick. ## Deploy Your Own ```bash # One-click setup (Python 3.10+ and CUDA GPU required) bash scripts/setup.sh ./my-tts # Configure voice in .env echo 'TTS_SEED=201' >> ./my-tts/.env echo 'TTS_INSTRUCT=Your voice description here' >> ./my-tts/.env # Start server bash scripts/setup.sh start ./my-tts ``` Setup installs: `qwen-tts`, `soundfile`, `pydub`, `uvicorn`, `fastapi`, `torch` (CUDA). Downloads VoiceDesign model (~3.5GB) via ModelScope (China) or HuggingFace. **Requirements:** CUDA GPU with 4GB+ VRAM, Python 3.10+, ~4GB disk. ## Scripts | Script | Purpose | |--------|---------| | `scripts/say.sh` | Generate speech — `say.sh "text" [output.mp3]` | | `scripts/batch_seeds.sh` | Compare seeds — `batch_seeds.sh "text" seed1 seed2 ...` | | `scripts/tts_server.py` | FastAPI server (fully env-configurable) | | `scripts/setup.sh` | One-click deploy (venv + deps + model download) | ## OpenClaw Integration In `openclaw.json`: ```json { "env": { "OPENAI_TTS_BASE_URL": "http://<your-server>:8881/v1" }, "messages": { "tts": { "provider": "openai", "openai": { "apiKey": "dummy", "model": "qwen3-tts", "voice": "default" }, "timeoutMs": 120000 } } } ``` ## Server Management ```bash # Health check curl -s $TTS_URL/health # Start (foreground) python tts_server.py # Start (background, Linux/macOS) nohup python tts_server.py > server.log 2>&1 & # Auto-restart (Windows — scheduled task + guard script) # Create tts_guard.bat: # @echo off # :loop # python tts_server.py # timeout /t 10 # goto loop # Register: schtasks /create /tn "TTS-Guard" /tr "tts_guard.bat" /sc onlogon /rl highest # Auto-restart (Linux — systemd) # See setup.sh output for systemd unit template # Stop # Linux/macOS: kill $(lsof -ti:8881) # Windows: for /f "tokens=5" %a in ('netstat -aon ^| findstr :8881') do taskkill /PID %a /F ``` ## Troubleshooting - **Connection refused** → Server not running; start it - **30s+ first request** → Cold start (model loading ~60s); subsequent requests 10-15s - **Behind proxy** → Set `NO_PROXY=<server_ip>` on client side - **Windows firewall** → `netsh advfirewall firewall add rule name="TTS" dir=in action=allow protocol=TCP localport=8881` - **No flash-attn on Windows** → Expected; falls back to PyTorch SDPA (slower but works) - **PowerShell corrupts Chinese** → Edit `.env`/config via Python or SCP, not PowerShell `Set-Content` - **Process dies on SSH disconnect** → Use scheduled task (Windows) or systemd (Linux) instead of foreground ## Voice Design Tips Describe like casting a voice actor: - **Age/gender**: "18岁女大学生" / "30岁男性播音员" - **Texture**: "柔和温暖" / "清脆明亮" / "低沉磁性" - **Emotion**: "轻柔细腻" / "活泼开朗" - **Accent**: "南方口音软糯" / "台湾腔" / "东北大碴子味" - **Metaphor**: "像棉花糖" / "像播音主持" (helps the model capture feeling) ⚠️ **Timbre ≠ description.** Description controls style/emotion; seed controls timbre. Don't put personality traits ("灵动俏皮") in description — that's the seed's job.

path: scripts/batch_seeds.sh
content: #!/usr/bin/env bash
# Generate TTS samples with multiple seeds for voice comparison
# Usage: bash batch_seeds.sh [text] [seeds...] [output_dir]
# Example: bash batch_seeds.sh "你好呀！今天天气真好。" 100 200 300 400 500 /tmp/seeds

set -euo pipefail

HOST="${TTS_HOST:-localhost}"
PORT="${TTS_PORT:-8881}"
TEXT="${1:-你好呀！今天天气真好，想出去走走呢。}"
shift 2>/dev/null || true

# Collect seeds (default: 10 random-ish seeds)
SEEDS=()
OUT_DIR=""
while [[ $# -gt 0 ]]; do
  if [[ $# -eq 1 && "$1" == /* ]]; then
    OUT_DIR="$1"
    shift
  else
    SEEDS+=("$1")
    shift
  fi
done

[[ ${#SEEDS[@]} -eq 0 ]] && SEEDS=(42 123 201 456 789 1024 2048 4096 8888 20260201)
[[ -z "$OUT_DIR" ]] && OUT_DIR="/tmp/tts_seeds_$(date +%Y%m%d_%H%M%S)"

mkdir -p "$OUT_DIR"
echo "Generating ${#SEEDS[@]} samples → $OUT_DIR"
echo "Text: $TEXT"
echo "---"

for seed in "${SEEDS[@]}"; do
  out="$OUT_DIR/seed_${seed}.mp3"
  printf "Seed %-10s → " "$seed"
  t0=$(date +%s)
  http_code=$(curl -s -o "$out" -w "%{http_code}" \
    -X POST "http://${HOST}:${PORT}/tts" \
    -H "Content-Type: application/json" \
    -d "{\"text\": \"${TEXT}\", \"seed\": ${seed}, \"format\": \"mp3\"}")
  t1=$(date +%s)
  size=$(stat -f%z "$out" 2>/dev/null || stat -c%s "$out" 2>/dev/null || echo "?")
  echo "${http_code} ${size}B $((t1-t0))s → $out"
done

echo "---"
echo "Done! Listen and compare: open $OUT_DIR"


path: scripts/say.sh
content: #!/usr/bin/env bash
# Quick TTS: generate speech from text
#
# Server defaults are authoritative (seed + instruct are baked in).
# Only override TTS_URL if pointing to a different server.
#
# Environment variables (all optional):
#   TTS_URL       Server base URL (default: http://localhost:8881)
#   TTS_FORMAT    Output format: mp3/wav (default: mp3)
#
# Usage:
#   say.sh "你好呀！"                    → plays audio (macOS) or saves to /tmp
#   say.sh "你好呀！" output.mp3         → saves to output.mp3
#   say.sh -                             → read text from stdin

set -euo pipefail

TTS_URL="${TTS_URL:-http://localhost:8881}"
TTS_FORMAT="${TTS_FORMAT:-mp3}"

# Parse args
TEXT="${1:-}"
OUTPUT="${2:-}"

if [[ "$TEXT" == "-" ]]; then
    TEXT=$(cat)
elif [[ -z "$TEXT" ]]; then
    echo "Usage: say.sh <text> [output_file]" >&2
    echo "  or:  echo 'text' | say.sh -" >&2
    exit 1
fi

# Build JSON body — only text and format, let server handle seed+instruct
BODY=$(python3 -c "import json,sys; print(json.dumps({'text': sys.argv[1], 'format': '$TTS_FORMAT'}))" "$TEXT")

# Generate
if [[ -z "$OUTPUT" ]]; then
    OUTPUT="/tmp/tts_$(date +%s).${TTS_FORMAT}"
    AUTO_PLAY=1
else
    AUTO_PLAY=0
fi

curl -s -o "$OUTPUT" \
    -X POST "${TTS_URL}/tts" \
    -H "Content-Type: application/json" \
    -d "$BODY"

size=$(stat -f%z "$OUTPUT" 2>/dev/null || stat -c%s "$OUTPUT" 2>/dev/null || echo "?")
echo "$OUTPUT (${size}B)"

# Auto-play on macOS
if [[ "$AUTO_PLAY" == "1" ]] && command -v afplay &>/dev/null; then
    afplay "$OUTPUT" &
fi


path: scripts/setup.sh
content: #!/usr/bin/env bash
# Qwen3-TTS VoiceDesign — One-click setup
# Tested on: Windows (Git Bash/CMD), Linux, macOS
# Requirements: Python 3.10+, CUDA GPU (4GB+ VRAM), ~4GB disk for model
#
# Usage:
#   bash setup.sh [install_dir]
#   # Default install_dir: ./qwen3-tts
#
# After setup, start the server:
#   bash setup.sh start [install_dir]

set -euo pipefail

ACTION="${1:-install}"
INSTALL_DIR="${2:-${1:-qwen3-tts}}"

# If first arg looks like a path, treat as install
if [[ "$ACTION" == "start" ]]; then
    INSTALL_DIR="${2:-qwen3-tts}"
else
    INSTALL_DIR="${1:-qwen3-tts}"
    ACTION="install"
fi

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
VENV_DIR="$INSTALL_DIR/.venv"

# ---------- Detect OS ----------
detect_python() {
    for cmd in python3 python; do
        if command -v "$cmd" &>/dev/null; then
            echo "$cmd"
            return
        fi
    done
    echo "ERROR: Python not found. Install Python 3.10+" >&2
    exit 1
}

# ---------- Install ----------
do_install() {
    echo "=== Qwen3-TTS VoiceDesign Setup ==="
    echo "Install dir: $INSTALL_DIR"
    echo ""

    PYTHON=$(detect_python)
    PY_VER=$($PYTHON --version 2>&1)
    echo "Python: $PYTHON ($PY_VER)"

    # Create install dir
    mkdir -p "$INSTALL_DIR"

    # Create venv
    if [[ ! -d "$VENV_DIR" ]]; then
        echo "Creating virtual environment..."
        $PYTHON -m venv "$VENV_DIR"
    fi

    # Activate venv
    if [[ -f "$VENV_DIR/bin/activate" ]]; then
        source "$VENV_DIR/bin/activate"
    elif [[ -f "$VENV_DIR/Scripts/activate" ]]; then
        source "$VENV_DIR/Scripts/activate"
    fi

    # Install dependencies
    echo "Installing dependencies..."
    pip install --upgrade pip
    pip install qwen-tts soundfile pydub uvicorn fastapi numpy

    # Install PyTorch with CUDA (if not already)
    if ! python -c "import torch; assert torch.cuda.is_available()" 2>/dev/null; then
        echo "Installing PyTorch with CUDA..."
        pip install torch --index-url https://download.pytorch.org/whl/cu128
    fi

    # Download model (via modelscope for China, huggingface otherwise)
    echo ""
    echo "Downloading VoiceDesign model (~3.5GB)..."
    echo "Trying ModelScope first (faster in China)..."
    if pip install modelscope 2>/dev/null && python -c "
from modelscope import snapshot_download
path = snapshot_download('Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign')
print(f'Model downloaded to: {path}')
" 2>/dev/null; then
        echo "Model downloaded via ModelScope ✓"
    else
        echo "ModelScope failed, trying HuggingFace..."
        python -c "
from qwen_tts import Qwen3TTSModel
m = Qwen3TTSModel.from_pretrained('Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign')
print('Model downloaded via HuggingFace ✓')
del m
"
    fi

    # Copy server script
    cp "$SCRIPT_DIR/tts_server.py" "$INSTALL_DIR/tts_server.py"
    cp "$SCRIPT_DIR/batch_seeds.sh" "$INSTALL_DIR/batch_seeds.sh" 2>/dev/null || true

    # Create .env template
    if [[ ! -f "$INSTALL_DIR/.env" ]]; then
        cat > "$INSTALL_DIR/.env" << 'ENVEOF'
# Qwen3-TTS Configuration
# Uncomment and edit as needed

# TTS_SEED=4096
# TTS_INSTRUCT=18岁温柔女大学生，声音柔和温暖...
# TTS_MODEL_PATH=Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
# TTS_PORT=8881
# TTS_HOST=0.0.0.0
ENVEOF
    fi

    echo ""
    echo "=== Setup complete! ==="
    echo ""
    echo "To start the server:"
    echo "  cd $INSTALL_DIR"
    echo "  source .venv/bin/activate   # or .venv\\Scripts\\activate on Windows"
    echo "  python tts_server.py"
    echo ""
    echo "Or configure via environment:"
    echo "  TTS_SEED=201 TTS_PORT=8881 python tts_server.py"
    echo ""
    echo "Test:"
    echo "  curl 'http://localhost:8881/tts?text=你好' -o test.mp3"
}

# ---------- Start ----------
do_start() {
    if [[ -f "$VENV_DIR/bin/activate" ]]; then
        source "$VENV_DIR/bin/activate"
    elif [[ -f "$VENV_DIR/Scripts/activate" ]]; then
        source "$VENV_DIR/Scripts/activate"
    else
        echo "ERROR: venv not found at $VENV_DIR. Run setup first." >&2
        exit 1
    fi

    # Load .env if exists
    if [[ -f "$INSTALL_DIR/.env" ]]; then
        set -a
        source "$INSTALL_DIR/.env"
        set +a
    fi

    echo "Starting Qwen3-TTS server on port ${TTS_PORT:-8881}..."
    python "$INSTALL_DIR/tts_server.py"
}

# ---------- Main ----------
case "$ACTION" in
    install) do_install ;;
    start)   do_start ;;
    *)       do_install ;;
esac


path: scripts/tts_server.py
content: #!/usr/bin/env python3
"""
Qwen3-TTS VoiceDesign Server — OpenAI-compatible TTS API

Environment variables:
  TTS_SEED        Default random seed for voice timbre (default: 4096)
  TTS_INSTRUCT    Default voice description prompt
  TTS_MODEL_PATH  Path to VoiceDesign model weights
  TTS_PORT        Server port (default: 8881)
  TTS_HOST        Bind address (default: 0.0.0.0)

Endpoints:
  POST /v1/audio/speech   OpenAI-compatible (input, model, voice, response_format, speed)
  POST /tts               Custom (text, seed, instruct, format)
  GET  /tts               Quick test (?text=...&seed=...&format=...)
  GET  /health            Health check
  GET  /v1/models         Model list
"""
import os, io, time

# Clear proxy env (avoid corporate proxy interference)
for k in ("HTTP_PROXY", "HTTPS_PROXY", "http_proxy", "https_proxy", "ALL_PROXY"):
    os.environ.pop(k, None)
os.environ.setdefault("HF_HUB_OFFLINE", "1")
os.environ.setdefault("TRANSFORMERS_OFFLINE", "1")

import torch, numpy as np, random
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from typing import Optional
import soundfile as sf
import uvicorn

# ---------- Config from env ----------
DEFAULT_SEED = int(os.environ.get("TTS_SEED", "4096"))
DEFAULT_INSTRUCT = os.environ.get("TTS_INSTRUCT", (
    "A warm and gentle female voice, soft and clear, "
    "with a natural, unhurried pace. Pleasant and inviting."
))
MODEL_PATH = os.environ.get("TTS_MODEL_PATH", "Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign")
PORT = int(os.environ.get("TTS_PORT", "8881"))
HOST = os.environ.get("TTS_HOST", "0.0.0.0")

app = FastAPI(title="Qwen3-TTS VoiceDesign Server")
model = None


def load_model():
    global model
    if model is None:
        from qwen_tts import Qwen3TTSModel
        print(f"Loading VoiceDesign model from {MODEL_PATH}...")
        model = Qwen3TTSModel.from_pretrained(
            MODEL_PATH, device_map="cuda", torch_dtype="auto"
        )
        print(f"Model loaded! Default seed={DEFAULT_SEED}")
    return model


# ---------- Request schemas ----------
class TTSRequest(BaseModel):
    text: str
    seed: Optional[int] = None
    instruct: Optional[str] = None
    format: Optional[str] = "mp3"

class OpenAITTSRequest(BaseModel):
    model: str = "qwen3-tts"
    input: str
    voice: str = "default"
    response_format: str = "mp3"
    speed: float = 1.0


# ---------- Endpoints ----------
@app.post("/v1/audio/speech")
async def openai_tts(req: OpenAITTSRequest):
    return await generate(req.input, DEFAULT_SEED, None, req.response_format)

@app.post("/tts")
async def tts(req: TTSRequest):
    return await generate(req.text, req.seed, req.instruct, req.format)

@app.get("/tts")
async def tts_get(text: str, seed: int = DEFAULT_SEED, format: str = "mp3"):
    return await generate(text, seed, None, format)


async def generate(text: str, seed: Optional[int], instruct: Optional[str], fmt: str):
    m = load_model()
    inst = instruct or DEFAULT_INSTRUCT
    s = seed if seed is not None else DEFAULT_SEED

    torch.manual_seed(s)
    torch.cuda.manual_seed_all(s)
    np.random.seed(s)
    random.seed(s)

    t0 = time.time()
    wavs, sr = m.generate_voice_design(text=text, instruct=inst)
    wav = wavs[0]
    elapsed = time.time() - t0
    print(f"Generated: {len(wav)/sr:.1f}s audio in {elapsed:.1f}s (seed={s})")

    buf = io.BytesIO()
    if fmt == "wav":
        sf.write(buf, wav, sr, format="WAV")
        media = "audio/wav"
    else:
        sf.write(buf, wav, sr, format="WAV")
        buf.seek(0)
        from pydub import AudioSegment
        audio = AudioSegment.from_wav(buf)
        buf = io.BytesIO()
        audio.export(buf, format="mp3")
        media = "audio/mpeg"

    buf.seek(0)
    return StreamingResponse(buf, media_type=media,
        headers={"X-Generation-Time": f"{elapsed:.2f}s", "X-Seed": str(s)})


@app.get("/v1/models")
def models():
    return {"object": "list", "data": [
        {"id": "qwen3-tts", "object": "model", "created": 1738368000, "owned_by": "local"},
    ]}

@app.get("/health")
def health():
    return {
        "status": "ok",
        "seed": DEFAULT_SEED,
        "model": MODEL_PATH.split("/")[-1] if "/" in MODEL_PATH else MODEL_PATH,
        "port": PORT,
    }


if __name__ == "__main__":
    load_model()
    uvicorn.run(app, host=HOST, port=PORT, log_level="info")


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (MEDIUM)