Skill: lzfxxx/ppt-audio-to-video (version 0.1.0)

--- name: ppt-audio-to-video description: Convert narration audio plus slide decks into a narrated video. Use when the user has an audio-only `mp4/m4a/mp3/wav` and a `ppt/pptx/pdf` deck, and needs slide images, transcript extraction, slide timing planning, or final `mp4` rendering with `whisper-cpp` and `ffmpeg`. --- # PPT Audio To Video Use this skill when the source video has narration audio but no usable slide visuals, and the final deliverable should be a slide-based lecture video. Resolve bundled scripts relative to this skill directory. If the runtime has already opened this `SKILL.md`, prefer paths like `scripts/extract_slide_outline.py` and `scripts/render_from_timing_csv.py` instead of machine-specific absolute paths. ## Core workflow 1. Inventory inputs. - Confirm which of these exist: audio-only `mp4/m4a/mp3/wav`, `ppt/pptx`, `pdf`, and any pre-rendered slide images. - Prefer an existing `pdf` or image directory for rendering. Treat `pptx` as the source of slide text and as a fallback for export. 2. Prepare tools. - Required for deterministic steps: `ffmpeg`, `ffprobe`, `pdftoppm`. - Required for transcription: `whisper-cli` from `whisper-cpp` plus a multilingual model such as `ggml-small.bin`. - If only `pptx` exists and no `pdf/images` exist, prefer `Keynote` or `PowerPoint` export on macOS. Use `soffice` only as fallback because profile or rendering issues are common. 3. Produce slide images. - If `pdf` exists, render it to images: ```bash pdftoppm -png -r 200 "$PDF" "$OUTDIR/slide" ``` - If only `pptx` exists, export to `pdf` or slide images with `Keynote` or `PowerPoint`, then continue from `pdf`. - Keep slide filenames ordered and stable, such as `slide-01.png`, `slide-02.png`, ... 4. Extract slide text. - Run: ```bash python3 scripts/extract_slide_outline.py \ --pptx "$PPTX" \ --out "$WORKDIR/slide_outline.csv" ``` - Use the output to identify slide titles, distinctive keywords, and section changes. 5. Extract clean audio for ASR. - For audio-only `mp4`, extract mono `wav`: ```bash ffmpeg -y -i "$AUDIO_MP4" -ar 16000 -ac 1 -c:a pcm_s16le "$WORKDIR/audio.wav" ``` - If the source is already `wav/mp3/m4a`, convert to the same mono `wav` form if needed. 6. Transcribe with `whisper-cli`. - Example: ```bash whisper-cli -ng \ -m "$MODEL" \ -f "$WORKDIR/audio.wav" \ -l zh \ -ocsv -osrt -of "$WORKDIR/transcript" ``` - Prefer `transcript.csv` for downstream parsing. `transcript.srt` is useful for manual review. - If GPU allocation fails on macOS, retry with `-ng` to force CPU mode. 7. Build `slide_timings.csv`. - Do not average slide durations unless the user explicitly asks for it. - Read the transcript and slide outline together, then create a monotonic timing plan by topic changes, section boundaries, and unique keywords. - Use this schema: ```csv slide,start_sec,end_sec,duration_sec,reason 1,0.000,15.000,15.000,opening title and agenda 2,15.000,100.000,85.000,architecture overview starts here ``` - Keep slide numbers sequential and ensure `duration_sec = end_sec - start_sec`. - Validate that the last `end_sec` matches the audio duration or is within a small tolerance. 8. Render the final video. - Run: ```bash python3 scripts/render_from_timing_csv.py \ --images "$SLIDE_IMAGES_DIR" \ --timings "$WORKDIR/slide_timings.csv" \ --audio "$WORKDIR/audio.wav" \ --output "$OUT_VIDEO" ``` - The script generates an `ffconcat` file, validates timing continuity, and calls `ffmpeg` to encode the final `mp4`. 9. Verify and iterate. - Check output duration with `ffprobe`. - If a slide cuts too early or too late, edit only the affected rows in `slide_timings.csv` and rerun the render script. - Keep the transcript, outline, and timing CSV as reproducible working files. ## Heuristics for timing alignment - Use section-divider slides briefly. These slides usually hold for 5-20 seconds. - Use the first segment that clearly switches topic as the next slide start. - Prefer exact topic transitions over title-word matching. ASR often distorts proper nouns and product names. - Let the model infer timings, but keep the render step deterministic through `slide_timings.csv`. - When confidence is low, produce a first-cut video and tell the user which slide boundaries likely need review. ## Common commands Install dependencies on macOS if missing: ```bash brew install ffmpeg poppler whisper-cpp ``` Typical multilingual model download: ```bash mkdir -p .models curl -L 'https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin' -o .models/ggml-small.bin ``` ## Bundled scripts - `scripts/extract_slide_outline.py` Extract slide text from `pptx` into CSV or JSON for timing analysis. - `scripts/render_from_timing_csv.py` Validate a timing CSV, generate an `ffconcat`, and render the final video with `ffmpeg`.

path: scripts/extract_slide_outline.py
content: #!/usr/bin/env python3
import argparse
import csv
import json
from pathlib import Path

from pptx import Presentation


def normalize_shape_text(text: str) -> str:
    lines = [line.strip() for line in text.splitlines()]
    lines = [line for line in lines if line]
    return " | ".join(lines)


def extract_outline(pptx_path: Path):
    prs = Presentation(str(pptx_path))
    slides = []
    for index, slide in enumerate(prs.slides, start=1):
        texts = []
        for shape in slide.shapes:
            if hasattr(shape, "text") and shape.text:
                normalized = normalize_shape_text(shape.text)
                if normalized:
                    texts.append(normalized)
        slides.append(
            {
                "slide": index,
                "text": " || ".join(texts),
            }
        )
    return slides


def write_csv(slides, output: Path):
    with output.open("w", encoding="utf-8", newline="") as fh:
        writer = csv.DictWriter(fh, fieldnames=["slide", "text"])
        writer.writeheader()
        writer.writerows(slides)


def write_json(slides, output: Path):
    with output.open("w", encoding="utf-8") as fh:
        json.dump(slides, fh, ensure_ascii=False, indent=2)


def main():
    parser = argparse.ArgumentParser(description="Extract slide text outline from a PPTX file.")
    parser.add_argument("--pptx", required=True, help="Path to the source PPTX file.")
    parser.add_argument("--out", required=True, help="Path to the output CSV or JSON file.")
    args = parser.parse_args()

    pptx_path = Path(args.pptx).expanduser().resolve()
    output = Path(args.out).expanduser().resolve()

    if not pptx_path.exists():
        raise SystemExit(f"PPTX not found: {pptx_path}")

    slides = extract_outline(pptx_path)
    output.parent.mkdir(parents=True, exist_ok=True)

    if output.suffix.lower() == ".json":
        write_json(slides, output)
    else:
        write_csv(slides, output)

    print(f"OUTLINE_FILE={output}")
    print(f"SLIDE_COUNT={len(slides)}")


if __name__ == "__main__":
    main()


path: scripts/render_from_timing_csv.py
content: #!/usr/bin/env python3
import argparse
import csv
import subprocess
from pathlib import Path


def read_timings(path: Path):
    with path.open("r", encoding="utf-8", newline="") as fh:
        rows = list(csv.DictReader(fh))

    required = {"slide", "start_sec", "end_sec", "duration_sec"}
    missing = required - set(rows[0].keys() if rows else [])
    if missing:
        raise SystemExit(f"Timing CSV missing columns: {sorted(missing)}")

    normalized = []
    for row in rows:
        slide = int(row["slide"])
        start = float(row["start_sec"])
        end = float(row["end_sec"])
        duration = float(row["duration_sec"])
        if end < start:
            raise SystemExit(f"Invalid timing row for slide {slide}: end before start")
        if abs((end - start) - duration) > 0.05:
            raise SystemExit(
                f"Invalid timing row for slide {slide}: duration does not match end-start"
            )
        normalized.append(
            {
                "slide": slide,
                "start_sec": start,
                "end_sec": end,
                "duration_sec": duration,
            }
        )
    return normalized


def find_slide_images(images_dir: Path):
    files = sorted(
        path
        for path in images_dir.iterdir()
        if path.is_file() and path.suffix.lower() in {".png", ".jpg", ".jpeg"}
    )
    if not files:
        raise SystemExit(f"No slide images found in {images_dir}")
    return files


def validate_sequence(timings, images):
    if len(timings) != len(images):
        raise SystemExit(
            f"Slide/image count mismatch: {len(timings)} timing rows vs {len(images)} images"
        )

    expected_slide = 1
    previous_end = None
    for row in timings:
        if row["slide"] != expected_slide:
            raise SystemExit(
                f"Timing CSV must use sequential slide numbers starting at 1; got {row['slide']}"
            )
        if previous_end is not None and abs(row["start_sec"] - previous_end) > 0.05:
            raise SystemExit(
                f"Gap or overlap near slide {row['slide']}: start={row['start_sec']}, previous_end={previous_end}"
            )
        previous_end = row["end_sec"]
        expected_slide += 1


def write_ffconcat(images, timings, output: Path):
    lines = ["ffconcat version 1.0"]
    for image, row in zip(images, timings):
        lines.append(f"file '{image.resolve()}'")
        lines.append(f"duration {row['duration_sec']:.3f}")
    lines.append(f"file '{images[-1].resolve()}'")
    output.write_text("\n".join(lines) + "\n", encoding="utf-8")


def run_ffmpeg(ffconcat_path: Path, audio_path: Path, output_path: Path, overwrite: bool):
    cmd = [
        "ffmpeg",
        "-y" if overwrite else "-n",
        "-safe",
        "0",
        "-f",
        "concat",
        "-i",
        str(ffconcat_path),
        "-i",
        str(audio_path),
        "-vf",
        "scale=1920:1080:flags=lanczos,fps=30,format=yuv420p",
        "-c:v",
        "libx264",
        "-preset",
        "medium",
        "-crf",
        "20",
        "-c:a",
        "aac",
        "-b:a",
        "128k",
        "-movflags",
        "+faststart",
        "-shortest",
        str(output_path),
    ]
    subprocess.run(cmd, check=True)


def main():
    parser = argparse.ArgumentParser(
        description="Render a slide-based video from images, audio, and a timing CSV."
    )
    parser.add_argument("--images", required=True, help="Directory containing slide images.")
    parser.add_argument("--timings", required=True, help="Timing CSV path.")
    parser.add_argument("--audio", required=True, help="Audio file path.")
    parser.add_argument("--output", required=True, help="Output MP4 path.")
    parser.add_argument(
        "--ffconcat-out",
        help="Optional output path for the generated ffconcat file. Defaults next to the timing CSV.",
    )
    parser.add_argument(
        "--no-overwrite",
        action="store_true",
        help="Fail if the output file already exists.",
    )
    args = parser.parse_args()

    images_dir = Path(args.images).expanduser().resolve()
    timings_path = Path(args.timings).expanduser().resolve()
    audio_path = Path(args.audio).expanduser().resolve()
    output_path = Path(args.output).expanduser().resolve()
    ffconcat_path = (
        Path(args.ffconcat_out).expanduser().resolve()
        if args.ffconcat_out
        else timings_path.with_suffix(".ffconcat")
    )

    if not images_dir.exists():
        raise SystemExit(f"Images directory not found: {images_dir}")
    if not timings_path.exists():
        raise SystemExit(f"Timing CSV not found: {timings_path}")
    if not audio_path.exists():
        raise SystemExit(f"Audio file not found: {audio_path}")

    timings = read_timings(timings_path)
    images = find_slide_images(images_dir)
    validate_sequence(timings, images)

    ffconcat_path.parent.mkdir(parents=True, exist_ok=True)
    output_path.parent.mkdir(parents=True, exist_ok=True)
    write_ffconcat(images, timings, ffconcat_path)
    run_ffmpeg(ffconcat_path, audio_path, output_path, overwrite=not args.no_overwrite)

    print(f"FFCONCAT_FILE={ffconcat_path}")
    print(f"OUTPUT_VIDEO={output_path}")
    print(f"SLIDE_COUNT={len(images)}")


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)