supermind/OGG-Comfy

ComfyUI-OGG

OGG TTS (0.8B, block diffusion, 8 languages, 24 emotion and vocal tags) as self-contained ComfyUI nodes: the model, the Fish codec it speaks in, voice cloning, decode, an editor and Qwen3-ASR for transcripts. No other node pack is needed (it can be installed next to ComfyUI-Drama3: ids and types differ).

Load OGG TTS ───────────────────────────────┐
Load OGG Codec ─┬─► OGG Encode Voice ──► voice ─► OGG TTS Generate ─► codes ─► OGG Decode / OGG Editor ─► audio
Load Audio ─────┘        ▲ Qwen3-ASR Transcribe

Install

1. cd ComfyUI/custom_nodes && git clone <this repo>, restart ComfyUI. Nothing to pip install (uses the transformers ComfyUI requires). Needs ComfyUI ≥ 0.40.

2. Files:

   File	Folder
   OGG TTS model.safetensors from FashionFlora/ogg-tts (rename it, e.g. ogg_tts.safetensors)	models/diffusion_models
   drama3_codec.safetensors from Drama-3-ComfyUI	models/vae
   qwen3_asr_1.7b_bf16.safetensors or _0.6b_ from Qwen3-ASR-ComfyUI (only for transcripts and the editor's word timing)	models/asr

   OmniVoice's config.json and tokenizer files go in this pack's omnivoice/ folder; Load OGG TTS fetches them there on first use if they are missing (see omnivoice/README.md).

3. Open Templates → ComfyUI-OGG: ogg_text_to_speech, ogg_voice_clone, ogg_dialogue and ogg_voice_design.

Nodes

Node
Load OGG TTS	any file of models/diffusion_models (names with "ogg" first); dtype float32 (as the authors run it) or bfloat16 (half the memory).
Load OGG Codec	models/vae, names with "codec" first.
OGG Encode Voice / Save / Load	a clip and its exact transcript → a voice; saved voices (output/ogg_voices) are a few KB and load faster. ComfyUI-Drama3 voice files load too.
OGG TTS Generate	text, language, optional voice → codes. split_lines: one take per line, each hearing the two lines before and after it. Inline tags: see the node's description. Advanced: speaker (a trained speaker, or random), quality (a trained recording), acoustic-decoder sampling.
OGG TTS Dialogue	a script Name: [tag] text, up to 6 named speakers, each with a voice (name_N + voice_N), optional: a speaker without one speaks in a trained identity of the model (speaker_N, or one fixed by the name). Modes: line by line (every line from its speaker's voice, hearing the neighbouring lines as context) and one pass (the whole scene and all voices at once: the model's dialogue mode, one take).
OGG Voice from Take	a generated take (one of its lines) → a voice: fixes a voice found by chance, to use for any text.
Save / Load OGG Codes	keep a take (output/ogg_codes) to decode or edit again without generating.
OGG Decode	codes → 44.1 kHz audio; takes joined with 0.3 s gaps. shorten_long_pauses (off by default) trims as Fish's Drama 3 Space does.
OGG Editor	decode and edit on a timeline: captions follow the voice, cut, add silence, name voices and set volume.
Load Qwen3-ASR / Transcribe (OGG)	speech to text in 30 languages; word timing for the editor with the ForcedAligner file.

Defaults follow the reference script: steps 6, cfg 1.5 (1–1.5 natural tempo, 2 faster), temperature 0.8.

What was checked, and what was not

Checked on CPU with ComfyUI replaced by stand-ins (tests/: python test_pipeline.py, test_codec.py, test_dialogue.py): all modules import; the codec's real architecture (random weights) encodes 1 s to [10, 22] codes and decodes to the right length; the ported OGG model loads a state dict, and generation runs end to end (no voice, voice, context lines, cfg 0); voice and codes save/load round-trip; Decode and the Editor's analysis and plan run on takes; script parsing, both dialogue modes (including the turn markers the model is given) and the missing-voice errors; all three example workflows match the nodes' inputs one to one.

Not checked: the real weights, a GPU, or a live ComfyUI (so not the editor's web UI, nor audio quality). Most likely first-contact problems: (a) the transformers version (KV-cache calls are the reference code's), (b) how ComfyUI's ModelPatcher moves a plain module, (c) the checkpoint's exact key set, (d) the OmniVoice config/tokenizer files' layout.

The example voice-clone workflow has no clip: pick your own in Load Audio.

Shaping a voice

OGG takes no description of a voice in words (no "deep, tired, middle-aged"); it has no input for it. What does set the voice and its manner:

Lever	What it is
a reference clip	the strongest: voice, manner and mood of the clip carry over
speaker	no clip: spk_0 ... spk_3156 (3157 anonymous trained speakers, no gender or age listed), ru_x ... (the unknown speaker of a language), or random (picked by the seed; the console prints which)
quality	q0 ... q598: a book or a game the model heard, with its room and, to a degree, its manner of delivery
inline tags, cfg, temperature	delivery inside a line; cfg 2 is faster and more pointed

To get a voice you like: audition, then fix it. Use the ogg_voice_design workflow: speaker is random, every run is another speaker, each run saves its take and its voice (a clip of that take with its words) under matching numbers. Keep the good ones, and use them through Load OGG Voice in Generate or Dialogue; the speaker then stays the same for any text.

If the tags seem to have no effect: the checkpoint (120k of 400k steps) learned tags from labels of what was clearly audible, spans with tags were a minority, and in a quarter of the samples they were stripped, so they are hints. A neutral reference clip pulls the delivery toward neutral; try the same line without a voice (speaker = random), in English, with [whispering] or [shouting] (the easiest to hear), and a higher cfg (2-3). To see whether a tag changes anything at all, run the same seed with and without it: if the audio is identical, tell the pack's author.

Tags and take length: the model was trained on spans of 1-5 consecutive sentences, up to 25 s, with a tag placed before the words it applies to (and again where it changes; at most 4 per sentence, at most 2 side by side), and in a quarter of the samples the tags were removed, so they are hints, not commands. split_lines on gives every line its own short take, which is what it learned; with it off a long text is one take far longer than any training span, and tags deep inside it are easily lost. Keep a take to a few sentences, a tag right before its words. Reference clips have no limit in this pack; the model's training prompts were up to about 8 s, so a much longer clip is outside what it saw.

Dialogue: line by line follows the authors' documented way (their --before/--after) and is the one to use. one pass is experimental: it follows their training code (train_omnifish.dialogue_spans: voices named S1, S2..., all text first, then the clips one after another, instruct dialogue), but their README does not mention it, and in that code dialogue scenes are trained only when a data-check file exists (speaker_mismatch.parquet), so the released weights may never have learned it. The advanced dialogue_layout switches to the context layout of the authors' inference script. Neither is tested on the real weights, and the editor has no per-line timing for one pass.

Not in v1: streaming, continuation.

License

GPL-3.0 (it includes code from ComfyUI-Drama3), with parts under Apache-2.0 and the Fish Audio Community License; see NOTICE. Do not clone the voice of a real person without their consent.