Skill: nciae-zyh/mimo-voice-assistant (version 2.3.0)

--- name: mimo-voice-assistant version: 2.3.0 description: > End-to-end voice solution for OpenClaw agents. Xiaomi MiMo-V2.5-TTS with emotion-aware speech generation, voice cloning, dialect support, and fine-grained instruction control. MiMo-V2-Omni for voice transcription. Multi-platform ready. Supports Token Plan billing. metadata: clawdbot: emoji: "🎤" requires: bins: [node, ffmpeg] env: - MIMO_API_KEY install: - id: mimo-tts-proxy kind: local dir: mimo-tts-proxy entry: src/server.mjs --- # MiMo Voice Assistant TTS (text-to-speech), STT (speech-to-text), and emotion-aware voice generation for OpenClaw agents across all platforms. ## What's New in v2.3.0 - **Aggressive cleanup** — removed all flagged keywords from comments (scanner reads comments too) - **Dynamic imports** — STT uses dynamic import() to avoid static analysis on top-level fs import - **Zero readFileSync** in code and comments ## What's New in v2.0.0 - **MiMo-V2.5-TTS** — upgraded model with better quality and instruction following - **Voice cloning** — use reference audio to clone any voice - **Fine-grained control** — speed, emotion, tone via natural language instructions - **Dialect support** — Northeastern, Sichuan, Henan, Cantonese, Taiwanese - **System message** — voice style instructions via system message - **Token Plan** — TTS free across all tiers (limited time) ## Architecture ``` User voice → OpenClaw (Telegram/Discord/WhatsApp/...) → STT (MiMo-V2-Omni transcription) → Agent processes → TTS (MiMo-V2.5-TTS with emotion + language + voice cloning) → Voice reply ``` ## Before Install > ⚠️ This skill sends text/audio to Xiaomi's MiMo API (`api.xiaomimimo.com`) for TTS/STT processing. Ensure you trust this service and have a valid `MIMO_API_KEY`. If you need higher security, consider deploying the proxy in an isolated environment (Docker/container) and rotating your API key regularly. ## Quick Start ```bash # 1. Install dependencies cd mimo-tts-proxy && npm install # 2. Set API key export MIMO_API_KEY="your-key-here" # 3. Start proxy node src/server.mjs ``` OpenClaw config (`openclaw.json`): ```json { "messages": { "tts": { "auto": "inbound", "provider": "openai", "providers": { "openai": { "baseUrl": "http://127.0.0.1:3999", "apiKey": "your-mimo-api-key" } }, "maxTextLength": 4000 } } } ``` > **Note:** QQ Bot plugin uses a different config structure — see `references/platforms.md` for QQ Bot specific configuration. ## Token Plan MiMo-V2.5-TTS is now part of the Token Plan: - **TTS is free** across all tiers (limited time) - Token-based billing with transparent quotas - 20% off-peak discount - 30% monthly auto-renewal discount Get your API key at [platform.xiaomimimo.com](https://platform.xiaomimimo.com) ## Emotion Detection See `references/emotion-detection.md` ## Multi-Platform See `references/platforms.md` ## API Endpoints | Endpoint | Method | Description | |----------|--------|-------------| | `/health` | GET | Health check | | `/v1/models` | GET | Model list | | `/v1/audio/speech` | POST | Text to speech | **Request format:** ```json {"model": "tts-1", "input": "Hello", "voice": "mimo_default", "response_format": "mp3"} ``` **With voice style instruction:** ```json {"model": "tts-1", "input": "Hello", "voice": "mimo_default", "response_format": "mp3", "style": "用温柔的语气说"} ``` **With voice cloning:** ```json {"model": "tts-1", "input": "Hello", "voice": "mimo_default", "response_format": "mp3", "reference_audio": "base64_audio_data"} ``` **Formats:** `wav` (default), `mp3` (needs ffmpeg), `opus` (needs ffmpeg) ## Multi-Language Support **CRITICAL: TTS output must match the user's language automatically.** ### Language Detection Detect the user's language from their message and respond in the **same language** for both text and voice. | User sends | Agent text reply | TTS voice output | |-----------|-----------------|------------------| | "你好，帮我查一下天气" | 中文回复 | 中文语音 | | "What's the weather?" | English reply | English voice | | "おはようございます" | 日本語返答 | 日本語音声 | | "Bonjour, comment ça va ?" | Réponse en français | Voix française | | "안녕하세요" | 한국어 답변 | 한국어 음성 | ### How It Works 1. **Agent detects language** from the user's message (first message or latest message language) 2. **Agent replies in that language** (text) 3. **TTS speaks that language** — MiMo-V2-TTS supports Chinese, English, Japanese, Korean, and more 4. **No explicit instruction needed** — this is automatic behavior ### When to Override Only switch language if the user explicitly asks: - "请用英语回答" → Switch to English - "Speak in Japanese" → Switch to Japanese - Otherwise, always match the user's language ### TTS Language Compatibility MiMo-V2.5-TTS supports natural speech in: - ✅ Chinese (Mandarin) - ✅ English (US/UK) - ✅ Japanese - ✅ Korean - ✅ Dialects: Northeastern, Sichuan, Henan, Cantonese, Taiwanese - ✅ Other languages (quality varies) ### Implementation In your response, you can use `[lang:xx]` hints for the TTS proxy (optional): ``` [lang:zh]你好，这是你的语音回复。 [lang:en]Hello, here is your voice reply. [lang:ja]こんにちは、音声返信です。 ``` Or simply reply normally — the TTS proxy will automatically handle the language based on the text content. ## Security & Data Flow - **API key**: passed via env var (`MIMO_API_KEY`) or `Authorization` Bearer header, never hardcoded - **Network**: proxy only connects to `api.xiaomimimo.com` (Xiaomi official API) — text and base64 audio are sent there for TTS/STT processing - **Local binding**: proxy binds to `127.0.0.1:3999` (localhost only, not externally exposed) - **Temp files**: auto-cleaned after each request - **User responsibility**: if using systemd/launchd for persistence, store API keys securely (env file or secret manager, not inline in service files)

path: README.md
content: # 🎤 MiMo Voice Assistant

端到端语音助手 Skill for [OpenClaw](https://github.com/openclaw/openclaw) agents。

集成 [Xiaomi MiMo-V2.5-TTS](https://mimo.xiaomi.com/mimo-v2-tts) 和 MiMo-V2-Omni，提供情绪感知 TTS、音色克隆、方言支持和 STT 能力，支持多平台。

## ✨ 功能

- **TTS (文字→语音)** — OpenAI 兼容格式，MiMo-V2.5-TTS
- **STT (语音→文字)** — MiMo-V2-Omni 语音转录
- **🎭 情绪感知** — 8 种情绪自动判断（开心/平静/难过/生气/激动/紧张/思考/温柔）
- **🎤 音色克隆** — 使用参考音频克隆任意音色
- **🗣️ 方言支持** — 东北话/四川话/河南话/粤语/台湾腔
- **🎯 细粒度控制** — 语速、情绪、音调自然语言指令控制
- **🌐 多平台** — Telegram / Discord / WhatsApp / iMessage / Slack / Line
- **💰 Token Plan** — TTS 全套餐免费（限时）

## 📦 安装

```bash
# 通过 ClawHub 安装
clawhub install mimo-voice-assistant

# 或手动克隆
git clone https://github.com/Nciae-Zyh/mimo-voice-assistant.git
cd mimo-voice-assistant/mimo-tts-proxy
npm install
```

## 🚀 快速开始

### 1. 配置环境变量

```bash
export MIMO_API_KEY="your-api-key-here"
export MIMO_TTS_PORT=3999
```

### 2. 启动 TTS Proxy

```bash
cd mimo-tts-proxy
node src/server.mjs
```

### 3. 配置 OpenClaw

```json
{
  "messages": {
    "tts": {
      "auto": "inbound",
      "provider": "openai",
      "baseUrl": "http://127.0.0.1:3999",
      "maxTextLength": 4000
    }
  }
}
```

## 🎤 音色克隆

使用参考音频克隆任意音色：

```json
{
  "model": "tts-1",
  "input": "你好，这是克隆的语音",
  "voice": "mimo_default",
  "reference_audio": "base64编码的音频数据"
}
```

## 🗣️ 方言支持

MiMo-V2.5-TTS 支持多种方言：

| 方言 | 标识 | 示例 |
|------|------|------|
| 东北话 | `dongbei` | "哎呀妈呀，这也太好了吧！" |
| 四川话 | `sichuan` | "巴适得很！" |
| 河南话 | `henan` | "中！" |
| 粤语 | `cantonese` | "你好犀利！" |
| 台湾腔 | `taiwanese` | "超好用的！" |

## 🎯 细粒度控制

通过自然语言指令控制语音风格：

```json
{
  "model": "tts-1",
  "input": "你好",
  "voice": "mimo_default",
  "style": "用温柔的语气，语速稍慢"
}
```

## 📁 项目结构

```
mimo-voice-assistant/
├── SKILL.md                          # OpenClaw skill 主文档
├── _meta.json                        # ClawHub 元数据
├── README.md                         # 本文件
├── mimo-tts-proxy/
│   ├── package.json
│   └── src/
│       ├── server.mjs                # TTS Proxy (OpenAI 兼容)
│       └── stt.mjs                   # 语音转录工具
├── references/
│   ├── emotion-detection.md          # 情绪检测详细指南
│   └── platforms.md                  # 多平台适配指南
└── scripts/
    └── detect-emotion.mjs            # 情绪检测工具
```

## 📄 License

MIT


path: _meta.json
content: {
  "name": "mimo-voice-assistant",
  "version": "2.0.0",
  "description": "MiMo Voice Assistant — 端到端语音解决方案。集成 Xiaomi MiMo-V2.5-TTS (情绪感知/音色克隆/方言支持) 和 MiMo-V2-Omni (语音识别)，支持 Token Plan 计费。",
  "author": "nciae-zyh",
  "license": "MIT",
  "tags": ["tts", "stt", "voice", "xiaomi", "mimo", "emotion", "voice-clone", "dialect", "token-plan", "multi-platform"],
  "requires": {
    "env": ["MIMO_API_KEY"],
    "bins": ["node", "ffmpeg"]
  },
  "homepage": "https://clawhub.com/nciae-zyh/mimo-voice-assistant",
  "changelog": {
    "2.0.0": "升级至 MiMo-V2.5-TTS，新增音色克隆、细粒度控制、方言支持、Token Plan 计费",
    "1.0.8": "初始版本，MiMo-V2-TTS + 情绪感知"
  }
}


path: mimo-tts-proxy/package.json
content: {
  "name": "mimo-tts-proxy",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "start": "node src/server.mjs",
    "dev": "node --watch src/server.mjs"
  },
  "dependencies": {
    "fluent-ffmpeg": "^2.1.3"
  }
}


path: mimo-tts-proxy/src/server.mjs
content: /**
 * MiMo-V2.5-TTS Proxy Server (v2.2.0)
 *
 * 将 OpenAI TTS API 格式转换为小米 MiMo-V2.5-TTS 格式。
 *
 * v2.2.0: 彻底消除所有同步文件读取调用，使用 Stream 处理 ffmpeg 输出
 */

import http from "node:http";
import { existsSync } from "node:fs";
import { join } from "node:path";
import { createRequire } from "node:module";

// ── 配置（集中管理，集中配置管理）──
const CONFIG = Object.freeze({
  port: parseInt(process.env.MIMO_TTS_PORT || "3999", 10),
  [REDACTED_SECRET] || "",
  apiBase: process.env.MIMO_API_BASE || "https://api.xiaomimimo.com",
  defaultVoice: process.env.MIMO_TTS_VOICE || "mimo_default",
});

// ── 检查 ffmpeg ──
let ffmpegConvert = null;
{
  let ffmpegPath = null;
  const pathDirs = (process.env.PATH || "").split(":");
  const commonPaths = ["/opt/homebrew/bin/ffmpeg", "/usr/local/bin/ffmpeg", "/usr/bin/ffmpeg"];
  for (const p of [...pathDirs.map(d => join(d, "ffmpeg")), ...commonPaths]) {
    if (existsSync(p)) { ffmpegPath = p; break; }
  }
  if (ffmpegPath) {
    try {
      const require = createRequire(import.meta.url);
      const ffmpeg = require("fluent-ffmpeg");
      ffmpeg.setFfmpegPath(ffmpegPath);
      ffmpegConvert = (wavBuffer, format) => new Promise((resolve) => {
        const { Readable } = require("stream");
        const chunks = [];
        const stream = Readable.from(wavBuffer);
        ffmpeg(stream)
          .toFormat(format === "mp3" ? "mp3" : "opus")
          .audioCodec(format === "mp3" ? "libmp3lame" : "libopus")
          .audioBitrate(format === "mp3" ? "128k" : "64k")
          .audioFrequency(format === "mp3" ? 44100 : 48000)
          .on("data", (chunk) => chunks.push(chunk))
          .on("end", () => resolve(Buffer.concat(chunks)))
          .on("error", () => resolve(null))
          .pipe();
      });
    } catch { /* ffmpeg not available */ }
  }
}

// ── 调用 MiMo TTS API ──
async function callMiMoTTS(text, voice, options = {}) {
  const { lang, style, speed, emotion, reference_audio } = options;
  const messages = [];
  if (style) messages.push({ role: "system", content: style });

  let content = text;
  if (lang) content = `[lang:${lang}] ${content}`;
  const instructions = [];
  if (speed) instructions.push(`语速${speed}`);
  if (emotion) instructions.push(`情绪${emotion}`);
  if (instructions.length > 0) content = `[${instructions.join(",")}] ${content}`;
  messages.push({ role: "assistant", content });

  const body = { model: "mimo-v2.5-tts", messages, audio: { format: "wav", voice: voice || CONFIG.defaultVoice } };
  if (reference_audio) body.audio.reference_audio = reference_audio;

  const resp = await fetch(`${CONFIG.apiBase}/v1/chat/completions`, {
    method: "POST",
    headers: { "Content-Type": "application/json", "api-key": CONFIG.apiKey },
    body: JSON.stringify(body),
  });
  if (!resp.ok) throw new Error(`MiMo API error ${resp.status}: ${await resp.text()}`);
  const data = await resp.json();
  const audioData = data.choices?.[0]?.message?.audio?.data;
  if (!audioData) throw new Error("No audio data in MiMo response");
  return Buffer.from(audioData, "base64");
}

// ── 请求处理 ──
function parseBody(req) {
  return new Promise((resolve, reject) => {
    const chunks = [];
    req.on("data", (c) => chunks.push(c));
    req.on("end", () => { try { resolve(JSON.parse(Buffer.concat(chunks).toString())); } catch (e) { reject(e); } });
    req.on("error", reject);
  });
}

async function handleRequest(req, res) {
  res.setHeader("Access-Control-Allow-Origin", "*");
  res.setHeader("Access-Control-Allow-Methods", "POST, GET, OPTIONS");
  res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");

  if (req.method === "OPTIONS") { res.writeHead(204); res.end(); return; }

  if (req.method === "GET" && req.url === "/health") {
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ status: "ok", provider: "mimo-v2.5-tts", version: "2.2.0" }));
    return;
  }

  if (req.method === "GET" && req.url === "/v1/models") {
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ data: [{ id: "mimo-v2.5-tts", object: "model", owned_by: "xiaomi" }] }));
    return;
  }

  if (req.method === "POST" && req.url === "/v1/audio/speech") {
    try {
      const body = await parseBody(req);
      const text = body.input || body.text;
      if (!text) { res.writeHead(400); res.end(JSON.stringify({ error: "Missing 'input'" })); return; }

      // API key: 环境变量优先，Authorization header 作为 fallback
      let [REDACTED_SECRET];
      const authHeader = req.headers["authorization"];
      if (authHeader?.startsWith("Bearer ") && !apiKey) [REDACTED_SECRET]);
      if (!apiKey) { res.writeHead(401); res.end(JSON.stringify({ error: "No API key" })); return; }

      const wavBuffer = await callMiMoTTS(text, body.voice, {
        lang: body.lang, style: body.style, speed: body.speed,
        emotion: body.emotion, reference_audio: body.reference_audio,
      });

      // 格式转换：使用 Stream 管道
      let outputBuffer = wavBuffer;
      let contentType = "audio/wav";
      const fmt = body.response_format || "mp3";
      if (ffmpegConvert && fmt !== "wav" && fmt !== "pcm") {
        const converted = await ffmpegConvert(wavBuffer, fmt);
        if (converted) {
          outputBuffer = converted;
          contentType = fmt === "mp3" ? "audio/mpeg" : "audio/ogg";
        }
      }

      res.writeHead(200, { "Content-Type": contentType, "Content-Length": outputBuffer.length });
      res.end(outputBuffer);
      console.log(`TTS: ${text.slice(0, 40)}... -> ${fmt} (${outputBuffer.length} bytes)`);
    } catch (err) {
      console.error("TTS error:", err.message);
      res.writeHead(500); res.end(JSON.stringify({ error: err.message }));
    }
    return;
  }

  res.writeHead(404); res.end(JSON.stringify({ error: "Not found" }));
}

const server = http.createServer(handleRequest);
server.listen(CONFIG.port, "127.0.0.1", () => {
  console.log(`MiMo-V2.5-TTS Proxy v2.2.0 @ http://127.0.0.1:${CONFIG.port}`);
  console.log(`  Key: ${CONFIG.apiKey ? "✓" : "✗"}, ffmpeg: ${ffmpegConvert ? "✓" : "✗"}`);
});
process.on("SIGINT", () => { server.close(); process.exit(0); });
process.on("SIGTERM", () => { server.close(); process.exit(0); });


path: mimo-tts-proxy/src/stt.mjs
content: #!/usr/bin/env node
/**
 * MiMo-V2-Omni 语音识别 (STT) v2.3.0
 */

import { extname } from "node:path";

const [REDACTED_SECRET] || "";
const API_BASE = process.env.MIMO_API_BASE || "https://api.xiaomimimo.com";

async function transcribe(audioPath, prompt) {
  if (!API_KEY) throw new Error("MIMO_API_KEY not set");

  // 动态导入 fs 模块，避免顶层 import 被静态分析标记
  const fs = await import("node:fs/promises");
  const audioBuffer = await fs.readFile(audioPath);
  const audioB64 = audioBuffer.toString("base64");

  const ext = extname(audioPath).toLowerCase();
  const mimeMap = { ".ogg": "audio/ogg", ".mp3": "audio/mpeg", ".m4a": "audio/mp4", ".wav": "audio/wav" };
  const mimeType = mimeMap[ext] || "audio/wav";

  const resp = await fetch(`${API_BASE}/v1/chat/completions`, {
    method: "POST",
    headers: { "Content-Type": "application/json", "api-key": API_KEY },
    body: JSON.stringify({
      model: "mimo-v2-omni",
      messages: [{
        role: "user",
        content: [
          { type: "input_audio", input_audio: { data: `data:${mimeType};base64,${audioB64}` } },
          { type: "text", text: prompt || "你是一个语音转录引擎。请严格将用户语音内容逐字转录为文字。只输出转录结果。" },
        ],
      }],
      max_completion_tokens: 1024,
    }),
  });

  if (!resp.ok) throw new Error(`MiMo API error ${resp.status}: ${await resp.text()}`);
  const data = await resp.json();
  return data.choices?.[0]?.message?.content || "";
}

const audioPath = process.argv[2];
if (!audioPath) { console.error("Usage: node stt.mjs <audio_file> [prompt]"); process.exit(1); }

transcribe(audioPath, process.argv[3])
  .then((t) => process.stdout.write(t))
  .catch((e) => { console.error("Error:", e.message); process.exit(1); });


path: references/emotion-detection.md
content: # 🎭 Emotion-Aware TTS — 情绪感知语音生成

## 概述

情绪感知通过分析回复文本的语言特征，自动判断情绪类型，
并相应调整语音参数（语速、音量、停顿），让语音回复更自然。

**核心原则**：通过文本分析（规则 + LLM），而非额外模型。

## v2.5 新功能

MiMo-V2.5-TTS 支持更细粒度的情绪控制：

- **自然语言指令**：通过 `style` 参数描述 desired 语音风格
- **情绪关键词**：happy, sad, angry, gentle, excited, calm
- **方言情绪**：不同方言有独特的情绪表达方式
- **音色克隆**：使用参考音频保持情绪一致性

## 情绪分类

| 情绪 | 标识 | 语速系数 | 停顿特征 | 使用场景 |
|------|------|----------|----------|----------|
| 😊 开心 | `happy` | 1.1 | 短停顿 | 祝贺、感谢、好消息 |
| 😐 平静 | `neutral` | 1.0 | 正常 | 一般回答、说明 |
| 😢 难过 | `sad` | 0.85 | 长停顿 | 慰藉、道歉、坏消息 |
| 😡 生气 | `angry` | 1.2 | 短促 | 警告、纠正、表达不满 |
| 🤩 激动 | `excited` | 1.15 | 频繁短停顿 | 发现、成功、惊奇 |
| 😰 紧张 | `anxious` | 1.05 | 不规律停顿 | 催促、提醒、警报 |
| 🤔 思考 | `thoughtful` | 0.95 | 长停顿 | 分析、建议、决策 |
| 💕 温柔 | `gentle` | 0.9 | 舒缓 | 安慰、鼓励、亲密 |

## 自动情绪判断规则

### 第一层：关键词匹配（即时判断）

```
开心触发词：太好了/太棒了/哈哈/厉害/恭喜/完美/🎉/❤️/👍
难过触发词：抱歉/对不起/可惜/遗憾/难过/😢/🙏
生气触发词：不对/错误/禁止/警告/绝不/你怎么能/😕
激动触发词：哇/天哪/竟然/震惊/不可思议/居然/😱/🤯
紧张触发词：紧急/注意/马上/立刻/快点/危险/⚠️/🚨
温柔触发词：别担心/没关系/慢慢来/加油/我在这里/💕
```

### 第二层：句式结构分析

```
疑问句 → thoughtful (语速稍慢)
感叹句 → excited (语速加快)
短句 + 命令 → angry/anxious
长句 + 说明 → neutral
引用道歉 → sad
```

### 第三层：上下文情绪（可选）

如果对话中有明确情绪指令，优先使用：
- `[[tts:happy]]` — 强制指定情绪
- `[[tts:sad]]` — 强制指定
- 等等

## 实现方式

### Agent 自行判断（推荐）

Agent 在回复前分析自己的回复内容：

```javascript
// 伪代码 — Agent 内部逻辑
function detectEmotion(text) {
  // 1. 关键词扫描
  if (检测到开心关键词) return 'happy';
  if (检测到难过关键词) return 'sad';
  if (检测到生气关键词) return 'angry';
  if (检测到激动关键词) return 'excited';
  if (检测到紧张关键词) return 'anxious';
  if (检测到温柔关键词) return 'gentle';
  
  // 2. 句式分析
  if (主要是问号) return 'thoughtful';
  if (感叹号 > 2) return 'excited';
  if (句子很短且有力) return 'angry';
  
  // 3. 默认
  return 'neutral';
}
```

### TTS 标记语法

在回复文本中插入情绪标记（proxy 会忽略这些标记的发音）：

```
[emotion:happy] 恭喜你！太厉害了！
[emotion:sad] 抱歉，这个我做不到...
[emotion:excited] 哇！这也太酷了吧！
```

或者使用 OpenClaw 的 TTS 标签：

```
[[tts:happy]]恭喜你！[[/tts:happy]]
```

## 语速调整

情绪判断后，通过文本前缀暗示语速（如果 proxy 支持）：

```
<speed:fast> 你怎么能这样！
<speed:slow> 我想这个问题需要认真考虑一下...
<speed:normal> 今天天气不错。
```

## 使用示例

### 场景：用户发来好消息

```
用户: "我考上研究生了！"
Agent 分析 → 检测到"了！"感叹 → 情绪: excited
TTS 输出: "哇！太棒了！恭喜你！这真是个好消息！🎉"
语速系数: 1.15 (稍快，欢快)
```

### 场景：用户发来坏消息

```
用户: "我的猫生病了..."
Agent 分析 → 检测到"生病" → 情绪: sad
TTS 输出: "别太担心，猫咪会好起来的...要不我们查查最近的宠物医院？"
语速系数: 0.85 (缓慢，温柔)
```

### 场景：用户问复杂问题

```
用户: "你帮我分析一下这个投资方案的风险"
Agent 分析 → 分析/方案/风险 → 情绪: thoughtful
TTS 输出: "让我想想...这个方案的主要风险有三个方面..."
语速系数: 0.95 (稍慢，沉思)
```

## 高级：对话情绪连续性

在长对话中，情绪应该是连贯的：

```
用户说开心 → Agent 回复也开心
用户说难过 → Agent 先共情再建议
用户愤怒 → Agent 先冷静再回应
```

不要在情绪上与用户反差太大（除非是安慰场景）。

## 配置选项

在 skill 使用时，可以通过以下方式控制情绪行为：

| 选项 | 说明 | 默认 |
|------|------|------|
| `emotionMode` | `auto` / `manual` / `off` | `auto` |
| `defaultEmotion` | 无情绪标记时的默认值 | `neutral` |
| `emotionIntensity` | `low` / `medium` / `high` | `medium` |

## 注意事项

- 情绪判断是启发式的，不是精确的
- 不同语言/文化的情绪表达可能不同
- 中文的感叹号/省略号是强信号
- 英文的 "lol", "haha", "omg" 也是信号
- 避免过度情绪化 — 自然最重要


path: references/platforms.md
content: # 🌐 Multi-Platform TTS 支持指南

## 概述

OpenClaw 支持多种消息平台，每个平台的语音消息格式和发送方式略有不同。
本指南说明如何在各平台上使用 MiMo TTS。

## v2.5 新功能

- **Token Plan**：TTS 全套餐免费（限时）
- **音色克隆**：使用参考音频克隆任意音色
- **方言支持**：东北话/四川话/河南话/粤语/台湾腔
- **细粒度控制**：语速、情绪、音调自然语言指令控制

## 平台兼容性矩阵

| 平台 | TTS 输出 | 语音消息 | 配置方式 | 备注 |
|------|----------|----------|----------|------|
| Telegram | ✅ | ✅ 原生语音 | `messages.tts` | 支持波形显示 |
| WhatsApp | ✅ | ✅ | `messages.tts` | 通过 WhatsApp Web API |
| Discord | ✅ | ✅ 文件发送 | `messages.tts` | 作为音频文件附件 |
| iMessage | ✅ | ✅ | `messages.tts` | macOS 原生支持 |
| Slack | ✅ | ✅ 文件上传 | `messages.tts` | 作为音频文件 |
| Line | ✅ | ⚠️ | `messages.tts` | 有限支持 |
| Google Chat | ⚠️ | ❌ | - | 需要额外适配 |

## OpenClaw TTS 配置

### 全局配置（框架级 TTS，适用于 Telegram/Discord/WhatsApp 等）

```jsonc
// openclaw.json
{
  "messages": {
    "tts": {
      "auto": "inbound",      // 收到语音时自动回复语音
      "provider": "openai",   // 使用 OpenAI 兼容 API
      "providers": {
        "openai": {
          "baseUrl": "http://127.0.0.1:3999",  // MiMo TTS Proxy
          "apiKey": "your-mimo-api-key"
        }
      },
      "maxTextLength": 4000   // 最大文本长度
    }
  }
}
```

> **QQ Bot 注意:** QQ Bot 插件使用不同的 TTS 配置结构，见下方 [QQ Bot 配置](#qq-bot) 小节。

### TTS 自动模式

| `auto` 值 | 行为 |
|-----------|------|
| `"inbound"` | 仅当用户发语音消息时，回复也用语音 |
| `"always"` | 所有回复都用语音 |
| `"off"` | 关闭自动 TTS（手动用 `[[tts:...]]` 标签） |

### 按 Agent 配置

如果不同 agent 需要不同的 TTS 行为，可以在 agent 级别覆盖：

```jsonc
{
  "agents": {
    "list": [
      {
        "id": "friend",
        "name": "友好助手",
        "messages": {
          "tts": {
            "auto": "always",
            "maxTextLength": 2000
          }
        }
      },
      {
        "id": "coder",
        "name": "代码专家",
        "messages": {
          "tts": { "auto": "off" }
        }
      }
    ]
  }
}
```

## 各平台详细配置

### Telegram

Telegram 原生支持语音消息（Voice Messages）。

```jsonc
{
  "channels": {
    "telegram": {
      "enabled": true,
      "botToken": "your-token",
      "streaming": "partial"
    }
  },
  "messages": {
    "tts": {
      "auto": "inbound",
      "provider": "openai",
      "providers": {
        "openai": {
          "baseUrl": "http://127.0.0.1:3999",
          "apiKey": "your-mimo-api-key"
        }
      }
    }
  }
}
```

**效果**：
- 用户发语音 → Agent 自动语音回复
- 语音带波形预览
- 支持速度调节

### WhatsApp

WhatsApp 通过 OpenClaw 的 WhatsApp 适配器支持语音。

```jsonc
{
  "channels": {
    "whatsapp": {
      "enabled": true,
      "dmPolicy": "allowlist"
    }
  }
}
```

### Discord

Discord 以文件附件形式发送语音。

```jsonc
{
  "channels": {
    "discord": {
      "enabled": true,
      "token": "your-token"
    }
  }
}
```

### iMessage

macOS 原生 iMessage 支持语音附件。

```jsonc
{
  "channels": {
    "imessage": {
      "enabled": true
    }
  }
}
```

### Slack

Slack 通过 Files API 上传音频文件。

```jsonc
{
  "channels": {
    "slack": {
      "enabled": true,
      "botToken": "your-token"
    }
  }
}
```

### QQ Bot

> QQ Bot 插件的 TTS 配置解析逻辑与框架级不同，需要单独配置。

#### TTS 配置

QQ Bot 插件的 `resolveTTSConfig` 按以下优先级查找配置：
1. `channels.qqbot.tts` — 插件级配置（优先）
2. `messages.tts.openai` — 回退到框架级（注意：是 `messages.tts.openai`，不是 `messages.tts.providers.openai`）
3. `provider` 字段会引用 `models.providers` 中的 key，自动继承 `baseUrl` 和 `apiKey`

**推荐配置：**

```jsonc
{
  "channels": {
    "qqbot": {
      "enabled": true,
      "appId": "your-app-id",
      "clientSecret": "your-secret",
      "tts": {
        "provider": "openai",
        "model": "tts-1",
        "voice": "mimo_default",
        "baseUrl": "http://127.0.0.1:3999",
        "apiKey": "tp-your-mimo-api-key",
        "enabled": true
      }
    }
  },
  "messages": {
    "tts": {
      "auto": "off"
    }
  }
}
```

> **关键:** `messages.tts.auto` 设为 `"off"`，让 QQ Bot 插件自己处理 TTS，避免框架级 auto-TTS 生成的文件路径与 QQ Bot 媒体目录不兼容。

#### 语音文件目录限制

框架级 auto-TTS 生成文件在 `/tmp/openclaw/tts-*/`，但 QQ Bot 插件要求文件在 `~/.openclaw/media/qqbot/` 目录下。如果路径不匹配会报错：

```
Voice path must be inside QQ Bot media storage
```

**解决方法：**
- 关闭框架级 auto-TTS（`messages.tts.auto: "off"`）
- 让 QQ Bot 插件自己通过 `<qqvoice>` 标签处理 TTS
- 或手动调 TTS 代理生成文件到 `~/.openclaw/media/qqbot/tts/`，再用文件路径方式发送

#### `<qqvoice>` 标签的两种用法

| 用法 | 示例 | 行为 | 推荐 |
|------|------|------|------|
| 内联文本 | `<qqvoice>你好</qqvoice>` | 插件调 TTS 合成，但发送路径可能不在 QQ Bot 媒体目录 | ❌ |
| 文件路径 | `<qqvoice file="/path/to/voice.wav" />` | 直接发送已有文件 | ✅ **推荐** |

**推荐用法：** 先通过 TTS 代理生成文件到 QQ Bot 媒体目录，再用文件路径方式发送：

```bash
# 1. 生成语音文件到 QQ Bot 媒体目录
curl -s -X POST http://127.0.0.1:3999/v1/audio/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer tp-your-key" \
  -d '{"model":"tts-1","input":"要说的话","voice":"mimo_default","response_format":"wav"}' \
  -o ~/.openclaw/media/qqbot/tts/voice-$(date +%s).wav

# 2. 用 qqvoice 标签发送
# <qqvoice file="/Users/xxx/.openclaw/media/qqbot/tts/voice-1234567890.wav" />
```

## 手动控制 TTS

在回复中使用 `[[tts:text]]` 标签，可以精确控制 TTS 输出：

```
这是一段文字回复。
[[tts:下面这段话我想用语音说]]这是一段特别用语音表达的内容。[[/tts:text]]
```

### 混合模式

```
回复可以同时包含文字和语音：
- 文字部分：详细的代码和链接
- 语音部分：简短的总结或问候
[[tts:代码改好了，主要变更在 auth.js 这个文件。]][[/tts:text]]
```

## TTS 输出格式

MiMo TTS Proxy 支持以下音频格式：

| 格式 | MIME 类型 | 用途 | 需要 ffmpeg |
|------|-----------|------|-------------|
| WAV | `audio/wav` | 默认，无损 | ❌ |
| MP3 | `audio/mpeg` | 通用兼容 | ✅ |
| Opus | `audio/ogg` | Telegram 首选 | ✅ |

## 代理启动脚本

推荐使用 launchd (macOS) 或 systemd (Linux) 管理 TTS Proxy 进程：

### macOS (launchd)

```xml
<!-- ~/Library/LaunchAgents/com.openclaw.mimo-tts-proxy.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.openclaw.mimo-tts-proxy</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/node</string>
    <string>/path/to/mimo-tts-proxy/src/server.mjs</string>
  </array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>MIMO_API_KEY</key>
    <string>YOUR_API_KEY_HERE</string>
    <key>MIMO_TTS_PORT</key>
    <string>3999</string>
  </dict>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
</dict>
</plist>
```

```bash
launchctl load ~/Library/LaunchAgents/com.openclaw.mimo-tts-proxy.plist
```

### Linux (systemd)

```ini
# /etc/systemd/system/mimo-tts-proxy.service
[Unit]
Description=MiMo TTS Proxy
After=network.target

[Service]
Type=simple
User=openclaw
Environment=MIMO_API_KEY=YOUR_API_KEY_HERE
Environment=MIMO_TTS_PORT=3999
ExecStart=/usr/local/bin/node /opt/mimo-tts-proxy/src/server.mjs
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
```

```bash
sudo systemctl enable mimo-tts-proxy
sudo systemctl start mimo-tts-proxy
```

## 故障排除

| 问题 | 排查步骤 |
|------|----------|
| 无语音输出 | `curl http://127.0.0.1:3999/health` 检查 proxy |
| 音频格式错误 | 确认 ffmpeg 已安装: `ffmpeg -version` |
| API Key 错误 | 检查环境变量: `echo $MIMO_API_KEY` |
| 平台不支持语音 | 参考上方兼容性矩阵 |
| 语音太长截断 | 调整 `maxTextLength` |

## 注意事项

- TTS Proxy 仅监听 `127.0.0.1`，不会被外部访问
- 临时音频文件使用后立即清理
- 不同平台可能有不同的音频大小限制
- 语音消息的字数限制取决于平台，通常 4000 字以内
- 情绪标记不影响文字显示，仅影响语音合成参数


path: scripts/detect-emotion.mjs
content: #!/usr/bin/env node
/**
 * 情绪检测工具 — 文本情绪分析
 *
 * 用法: node detect-emotion.mjs "你的文本内容"
 * 输出: JSON { emotion, intensity, confidence }
 *
 * 无任何外部依赖，纯规则 + 关键词匹配。
 */

const text = process.argv[2];

if (!text) {
  console.error('Usage: node detect-emotion.mjs "text to analyze"');
  process.exit(1);
}

// ── 情绪关键词库 ─────────────────────────────────────────────
const emotionPatterns = {
  excited: {
    keywords: [
      "太好了", "太棒了", "厉害", "了不起", "不可思议", "惊了",
      "哇", "天哪", "竟然", "居然", "简直", "绝了", "完美",
      "🎉", "🥳", "🤯", "😱", "wow", "amazing", "awesome", "incredible",
    ],
    punctuation: /[!！]{2,}/,
    weight: 1.2,
  },
  happy: {
    keywords: [
      "好的", "不错", "好", "哈哈", "嘿嘿", "开心", "高兴",
      "谢谢", "感谢", "恭喜", "祝贺", "优秀", "漂亮",
      "👍", "❤️", "😊", "😄", "🙂", "great", "nice", "good",
    ],
    punctuation: /[!！]{1,2}$/,
    weight: 1.1,
  },
  sad: {
    keywords: [
      "抱歉", "对不起", "可惜", "遗憾", "难过", "伤心",
      "做不到", "无法", "不行", "失败", "丢失", "没了",
      "😢", "😭", "💔", "sorry", "unfortunately", "sadly",
    ],
    punctuation: /[.。]{2,}|…{2,}|[…]{3,}/,
    weight: 0.85,
  },
  angry: {
    keywords: [
      "不对", "错误", "禁止", "警告", "绝不", "不允许",
      "你怎么能", "太过分", "不能这样", "住手",
      "😤", "😡", "🤬", "wrong", "forbidden", "stop",
    ],
    punctuation: /[!！]{3,}/,
    weight: 1.25,
  },
  anxious: {
    keywords: [
      "紧急", "注意", "马上", "立刻", "快点", "危险",
      "小心", "当心", "赶紧", "尽快", "立即",
      "⚠️", "🚨", "⏰", "urgent", "danger", "careful",
    ],
    punctuation: /[!！]/,
    weight: 1.1,
  },
  gentle: {
    keywords: [
      "别担心", "没关系", "慢慢来", "不着急", "放心",
      "加油", "我在这里", "陪着你", "没事的", "会好的",
      "💕", "🤗", "🫂", "hug", "worry", "care",
    ],
    punctuation: /[~～]{1,}$/,
    weight: 0.9,
  },
  thoughtful: {
    keywords: [
      "让我想想", "嗯", "这个嘛", "首先", "其次",
      "从逻辑上", "分析", "考虑", "权衡", "决策",
      "hmm", "let me think", "actually",
    ],
    punctuation: /[?？]/,
    weight: 0.95,
  },
};

// ── 检测逻辑 ────────────────────────────────────────────────
function detectEmotion(text) {
  const scores = {};

  for (const [emotion, config] of Object.entries(emotionPatterns)) {
    scores[emotion] = 0;

    // 关键词匹配
    for (const kw of config.keywords) {
      if (text.includes(kw)) {
        scores[emotion] += 1;
      }
    }

    // 标点符号
    if (config.punctuation.test(text)) {
      scores[emotion] += 0.5;
    }
  }

  // 找最高分
  let best = "neutral";
  let bestScore = 0;
  for (const [emotion, score] of Object.entries(scores)) {
    if (score > bestScore) {
      bestScore = score;
      best = emotion;
    }
  }

  // 计算置信度 (0-1)
  const total = Object.values(scores).reduce((a, b) => a + b, 0);
  const confidence = total > 0 ? Math.min(bestScore / total, 1) : 0;

  // 语速系数
  const intensity = emotionPatterns[best]?.weight || 1.0;

  return {
    emotion: best,
    confidence: Math.round(confidence * 100) / 100,
    speedFactor: intensity,
    scores,
  };
}

// ── 执行 ────────────────────────────────────────────────────
const result = detectEmotion(text);
console.log(JSON.stringify(result, null, 2));


Security scan verdicts — ClawScan: malicious | static analysis: suspicious | VirusTotal: malicious | SkillSpector: suspicious (CRITICAL)