Skill: q1lin570/audio-transcribe-summarize (version 1.0.1)

--- name: audio-transcribe-summarize description: Transcribe audio/video files to text and generate structured summaries using SenseAudio ASR API. Use when the user asks to transcribe, summarize, or take notes from audio files, video files, recordings, meetings, lectures, podcasts, or interviews. --- # Audio/Video Transcription & Summarization Transcribe audio/video files using the SenseASR API (`api.senseaudio.cn`), then summarize the content into structured notes. **`{baseDir}`** refers to this skill's directory. ## Prerequisites - Environment variable `SENSEAUDIO_API_KEY` configured (get your key at https://senseaudio.cn/platform/api-key) - Python 3.8+ with `requests` installed - For large files (>10MB): `ffmpeg` installed for splitting（macOS: `brew install ffmpeg`，Windows: [ffmpeg.org](https://ffmpeg.org/download.html) 下载并加入 PATH，Linux: `apt install ffmpeg`） ## Quick Start 1. Run the transcription script: ```bash python {baseDir}/scripts/transcribe.py <audio_file> [--model sense-asr-pro] [--language zh] [--speakers] [--sentiment] [--translate en] ``` 2. The script outputs a transcript `.txt` file alongside the source file 3. Read the transcript and generate a summary (see Summary Format below) ## Workflow ### Step 1: Assess the Audio File Check file size and format: - Supported formats: wav, mp3, ogg, flac, aac, m4a, mp4 - Max file size per request: 10MB - If file > 10MB, the script auto-splits using ffmpeg ### Step 2: Choose the Right Model | Model | Use When | |-------|----------| | `sense-asr-lite` | Quick batch transcription, simple audio, cost-sensitive | | `sense-asr` | General transcription, need speaker separation or timestamps | | `sense-asr-pro` | High accuracy needed: meetings, interviews, complex audio | | `sense-asr-deepthink` | Noisy audio, dialects, heavy jargon, speech-to-clean-text | Default to `sense-asr-pro` for best quality. ### Step 3: Transcribe Run the transcription script. Key options: ```bash # Basic transcription python {baseDir}/scripts/transcribe.py recording.mp3 # Meeting with multiple speakers + emotion python {baseDir}/scripts/transcribe.py meeting.wav \ --model sense-asr-pro \ --speakers --max-speakers 4 \ --sentiment \ --timestamps segment # Transcribe and translate to English python {baseDir}/scripts/transcribe.py lecture.mp3 \ --model sense-asr \ --translate en ``` ### Step 4: Summarize After transcription, read the transcript file and produce a summary using the format below. ## Summary Format Generate summaries in this structure: ```markdown # [Title - inferred from content] **Source**: filename.mp3 **Duration**: X min Y sec **Date**: YYYY-MM-DD **Speakers**: [if speaker diarization was used] ## Key Points - Point 1 - Point 2 - ... ## Detailed Summary [2-4 paragraph summary of the content organized by topic/chronology] ## Action Items - [ ] Action item 1 (assigned to Speaker X, if applicable) - [ ] Action item 2 ## Notable Quotes > "Direct quote from transcript" — Speaker X, [timestamp if available] ## Full Transcript <details> <summary>Click to expand full transcript</summary> [Full transcript text here, with speaker labels and timestamps if available] </details> ``` Adapt the template based on content type: - **Meeting**: emphasize action items, decisions, speaker contributions - **Lecture/Talk**: emphasize key concepts, learning points, structure - **Interview**: emphasize Q&A pairs, key responses - **Podcast**: emphasize topics discussed, interesting insights ## API Reference For full SenseASR API parameters and response formats, see [api-reference.md](api-reference.md).

path: api-reference.md
content: # SenseASR API Reference

## Endpoint

```
POST https://api.senseaudio.cn/v1/audio/transcriptions
Content-Type: multipart/form-data
Authorization: Bearer $API_KEY
```

## Models

| Model | Positioning | Best For |
|-------|------------|----------|
| `sense-asr-lite` | Ultra-fast | Batch transcription, 30+ languages, hotword boost |
| `sense-asr` | Standard | General use, subtitles, speaker diarization |
| `sense-asr-pro` | Professional | Meetings, interviews, high-accuracy scenarios |
| `sense-asr-deepthink` | Deep understanding | Dialect/jargon, speech cleanup, intelligent correction |

## Feature Matrix

| Feature | lite | asr | asr-pro | deepthink |
|---------|------|-----|---------|-----------|
| Basic recognition | Y | Y | Y | Y |
| Streaming | - | Y | Y | Y |
| Speaker diarization | - | Y | Y | - |
| Sentiment analysis | - | Y | Y | - |
| Word timestamps | - | Y | Y | - |
| Segment timestamps | - | Y | Y | - |
| Translation | - | Y | Y | Y |
| Hotword boost | Y | - | - | - |
| ITN | Y | Y | Y | - |
| Smart editing | - | - | - | Y |

## Required Parameters

| Param | Type | Description |
|-------|------|-------------|
| `file` | file | Audio file (wav/mp3/ogg/flac/aac/m4a/mp4), max 10MB |
| `model` | string | Model name (see above) |

## Optional Parameters

| Param | Type | Default | Description |
|-------|------|---------|-------------|
| `language` | string | auto-detect | ISO-639 code: `zh`, `en`, `ja`, etc. |
| `response_format` | string | `json` | `json` / `text` / `verbose_json` |
| `stream` | boolean | false | Enable SSE streaming (not lite) |
| `enable_itn` | boolean | true | Inverse text normalization (not deepthink) |
| `enable_punctuation` | boolean | false | Auto punctuation (asr/pro only) |
| `enable_speaker_diarization` | boolean | false | Speaker separation (asr/pro only) |
| `max_speakers` | integer | - | Max speakers 1-20 (asr-pro only) |
| `enable_sentiment` | boolean | false | Emotion detection (asr/pro only) |
| `timestamp_granularities[]` | array | - | `word` and/or `segment` (asr/pro only) |
| `target_language` | string | - | Translation target language (not lite) |
| `hotwords` | string | - | Comma-separated hotwords (lite only) |

## Response Formats

### JSON (default)
```json
{ "text": "recognized text" }
```

### Verbose JSON
```json
{
  "text": "full text",
  "duration": 4.153,
  "audio_info": { "duration": 4153, "format": "wav" },
  "segments": [
    {
      "id": 0, "start": 0.0, "end": 2.0,
      "text": "segment text",
      "speaker": "speaker_0",
      "sentiment": "Happy",
      "translation": "translated text"
    }
  ],
  "words": [
    { "word": "字", "start": 0.27, "end": 0.51 }
  ]
}
```

### Streaming (SSE)
```
data: {"delta": {"text": "incremental"}, "finish_reason": null}
data: {"delta": {"text": "."}, "finish_reason": "stop", "audio_info": {...}}
data: [DONE]
```

## Language Codes

Common codes: `zh` (Chinese), `en` (English), `ja` (Japanese), `ko` (Korean), `yue` (Cantonese), `fr` (French), `de` (German), `es` (Spanish), `ru` (Russian), `ar` (Arabic), `vi` (Vietnamese), `th` (Thai), `id` (Indonesian), `pt` (Portuguese), `it` (Italian).

## Error Codes

| HTTP | Code | Description |
|------|------|-------------|
| 400 | `invalid` | Bad parameters |
| 429 | `rate_limit_error` | Rate limited |
| 500 | `internal_error` | Server error |

## Constraints

- Max file size: 10MB per request
- Recommended: 16kHz sample rate, mono channel
- For files >10MB, split with ffmpeg before uploading


path: scripts/transcribe.py
content: #!/usr/bin/env python3
"""
Transcribe audio/video files using SenseAudio ASR API.
Handles files >10MB by auto-splitting with ffmpeg.

Usage:
    python transcribe.py <audio_file> [options]

Options:
    --model MODEL           ASR model (default: sense-asr-pro)
    --language LANG         Language code, e.g. zh, en, ja (default: auto-detect)
    --speakers              Enable speaker diarization
    --max-speakers N        Max speakers (1-20, requires --speakers, asr-pro only)
    --sentiment             Enable sentiment/emotion analysis
    --timestamps TYPE       Timestamp granularity: word, segment, or both
    --translate LANG        Translate to target language code
    --hotwords WORDS        Comma-separated hotwords (sense-asr-lite only)
    --output FILE           Output file path (default: <input>.transcript.txt)
    --format FORMAT         Response format: json, text, verbose_json (default: verbose_json)

Requires:
    - SENSEAUDIO_API_KEY environment variable
    - requests: pip install requests
    - ffmpeg (only for files >10MB)
"""

import argparse
import json
import math
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path

try:
    import requests
except ImportError:
    print("Error: 'requests' package required. Install with: pip install requests")
    sys.exit(1)

API_URL = "https://api.senseaudio.cn/v1/audio/transcriptions"
MAX_FILE_SIZE = 10 * 1024 * 1024  # 10MB

_bin_cache: dict[str, str | None] = {}

# IDE/GUI 启动的进程常缺少 Homebrew 等路径，需主动搜索
_EXTRA_SEARCH_PATHS = [
    "/opt/homebrew/bin",        # macOS Homebrew (Apple Silicon)
    "/usr/local/bin",           # macOS Homebrew (Intel) / Linux common
    "/usr/bin",                 # Linux system
    "/snap/bin",                # Linux snap
]


def find_bin(name: str) -> str:
    """Find executable by name, searching PATH + common install locations."""
    if name in _bin_cache:
        return _bin_cache[name]

    found = shutil.which(name)
    if not found:
        for d in _EXTRA_SEARCH_PATHS:
            candidate = os.path.join(d, name)
            if os.path.isfile(candidate) and os.access(candidate, os.X_OK):
                found = candidate
                break
    if not found:
        # Windows: also try common locations
        if sys.platform == "win32":
            for win_dir in [
                os.path.join(os.environ.get("LOCALAPPDATA", ""), "Programs", "ffmpeg", "bin"),
                os.path.join(os.environ.get("ProgramFiles", ""), "ffmpeg", "bin"),
                r"C:\ffmpeg\bin",
            ]:
                candidate = os.path.join(win_dir, f"{name}.exe")
                if os.path.isfile(candidate):
                    found = candidate
                    break

    _bin_cache[name] = found
    return found


def get_api_key():
    """Load API key from environment variable."""
    key = os.environ.get("SENSEAUDIO_API_KEY")
    if not key:
        print("Error: SENSEAUDIO_API_KEY not found in environment variables.")
        sys.exit(1)
    return key


def get_audio_duration(filepath):
    """Get audio duration in seconds using ffprobe."""
    ffprobe = find_bin("ffprobe")
    if not ffprobe:
        return None
    try:
        result = subprocess.run(
            [ffprobe, "-v", "quiet", "-print_format", "json", "-show_format", filepath],
            capture_output=True, text=True
        )
        info = json.loads(result.stdout)
        return float(info["format"]["duration"])
    except Exception:
        return None


def split_audio(filepath, chunk_duration=300):
    """Split audio into chunks using ffmpeg. Returns list of chunk file paths."""
    duration = get_audio_duration(filepath)
    ffmpeg = find_bin("ffmpeg")
    if duration is None or not ffmpeg:
        hint = ""
        if not find_bin("ffprobe"):
            hint = ("\nHint: ffmpeg may be installed but not in PATH. "
                    "Try running from a terminal, or set the full path in your PATH env var.")
        print(f"Error: ffmpeg/ffprobe not found.{hint}")
        sys.exit(1)

    num_chunks = math.ceil(duration / chunk_duration)
    tmp_dir = tempfile.mkdtemp(prefix="senseaudio_")
    ext = Path(filepath).suffix
    chunks = []

    print(f"Splitting {duration:.0f}s audio into {num_chunks} chunks of ~{chunk_duration}s each...")

    for i in range(num_chunks):
        start = i * chunk_duration
        chunk_path = os.path.join(tmp_dir, f"chunk_{i:04d}{ext}")
        subprocess.run(
            [ffmpeg, "-y", "-i", filepath, "-ss", str(start), "-t", str(chunk_duration),
             "-acodec", "copy", "-vn", chunk_path],
            capture_output=True
        )
        if os.path.exists(chunk_path) and os.path.getsize(chunk_path) > 0:
            chunks.append(chunk_path)

    return chunks, tmp_dir


def transcribe_file(filepath, args, api_key):
    """Transcribe a single audio file via SenseASR API."""
    headers = {"Authorization": f"Bearer {api_key}"}

    files = {"file": (os.path.basename(filepath), open(filepath, "rb"))}

    data = {"model": args.model}

    resp_format = getattr(args, "format", "verbose_json")
    data["response_format"] = resp_format

    if args.language:
        data["language"] = args.language
    if args.speakers:
        data["enable_speaker_diarization"] = "true"
    if args.max_speakers:
        data["max_speakers"] = str(args.max_speakers)
    if args.sentiment:
        data["enable_sentiment"] = "true"
    if args.translate:
        data["target_language"] = args.translate
    if args.hotwords:
        data["hotwords"] = args.hotwords

    if args.timestamps:
        if args.timestamps == "both":
            data["timestamp_granularities[]"] = ["word", "segment"]
        else:
            data["timestamp_granularities[]"] = args.timestamps

    response = requests.post(API_URL, headers=headers, files=files, data=data, timeout=300)

    if response.status_code != 200:
        print(f"API Error ({response.status_code}): {response.text}")
        return None

    return response.json() if resp_format != "text" else {"text": response.text}


def format_verbose_result(result):
    """Format verbose_json result into readable text."""
    lines = []
    segments = result.get("segments") or []

    if segments:
        for seg in segments:
            parts = []
            if seg.get("start") is not None:
                start = seg["start"]
                end = seg.get("end", start)
                parts.append(f"[{start:.1f}s - {end:.1f}s]")
            if seg.get("speaker"):
                parts.append(f"({seg['speaker']})")
            if seg.get("sentiment"):
                parts.append(f"[{seg['sentiment']}]")

            prefix = " ".join(parts)
            text = seg.get("text", "")
            translation = seg.get("translation")

            if prefix:
                lines.append(f"{prefix} {text}")
            else:
                lines.append(text)

            if translation:
                lines.append(f"  -> {translation}")
            lines.append("")
    else:
        lines.append(result.get("text", ""))

    return "\n".join(lines)


def main():
    parser = argparse.ArgumentParser(description="Transcribe audio using SenseAudio ASR API")
    parser.add_argument("audio_file", help="Path to audio/video file")
    parser.add_argument("--model", default="sense-asr-pro",
                        choices=["sense-asr-lite", "sense-asr", "sense-asr-pro", "sense-asr-deepthink"])
    parser.add_argument("--language", default=None)
    parser.add_argument("--speakers", action="store_true")
    parser.add_argument("--max-speakers", type=int, default=None)
    parser.add_argument("--sentiment", action="store_true")
    parser.add_argument("--timestamps", choices=["word", "segment", "both"], default=None)
    parser.add_argument("--translate", default=None)
    parser.add_argument("--hotwords", default=None)
    parser.add_argument("--output", default=None)
    parser.add_argument("--format", default="verbose_json",
                        choices=["json", "text", "verbose_json"])

    args = parser.parse_args()
    audio_path = args.audio_file

    if not os.path.exists(audio_path):
        print(f"Error: File not found: {audio_path}")
        sys.exit(1)

    [REDACTED_SECRET])
    file_size = os.path.getsize(audio_path)

    output_path = args.output or str(Path(audio_path).with_suffix(".transcript.txt"))

    if file_size <= MAX_FILE_SIZE:
        print(f"Transcribing {audio_path} ({file_size / 1024 / 1024:.1f}MB) with {args.model}...")
        result = transcribe_file(audio_path, args, api_key)
        if result is None:
            sys.exit(1)
        results = [result]
    else:
        print(f"File {file_size / 1024 / 1024:.1f}MB exceeds 10MB limit. Splitting...")
        chunks, tmp_dir = split_audio(audio_path)
        results = []
        for i, chunk in enumerate(chunks):
            print(f"Transcribing chunk {i + 1}/{len(chunks)}...")
            result = transcribe_file(chunk, args, api_key)
            if result:
                results.append(result)
            os.remove(chunk)
        os.rmdir(tmp_dir)

    full_text_parts = []
    raw_results = []

    for result in results:
        if args.format == "verbose_json":
            full_text_parts.append(format_verbose_result(result))
        else:
            full_text_parts.append(result.get("text", ""))
        raw_results.append(result)

    full_transcript = "\n".join(full_text_parts).strip()

    with open(output_path, "w", encoding="utf-8") as f:
        f.write(full_transcript)

    json_path = str(Path(output_path).with_suffix(".json"))
    with open(json_path, "w", encoding="utf-8") as f:
        json.dump(raw_results if len(raw_results) > 1 else raw_results[0], f, ensure_ascii=False, indent=2)

    duration_info = ""
    if raw_results and isinstance(raw_results[0], dict):
        dur = raw_results[0].get("duration") or (raw_results[0].get("audio_info") or {}).get("duration")
        if dur:
            total_sec = dur / 1000 if dur > 1000 else dur
            duration_info = f" ({total_sec:.0f}s)"

    print(f"\nDone! Transcript saved to: {output_path}{duration_info}")
    print(f"Raw JSON saved to: {json_path}")
    print(f"\nPreview (first 500 chars):\n{full_transcript[:500]}")


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)