Instructions to use xumo/reader-speech-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use xumo/reader-speech-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf xumo/reader-speech-models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf xumo/reader-speech-models:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf xumo/reader-speech-models:Q4_K_M
Use Docker
docker model run hf.co/xumo/reader-speech-models:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use xumo/reader-speech-models with Ollama:
ollama run hf.co/xumo/reader-speech-models:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use xumo/reader-speech-models with Docker Model Runner:
docker model run hf.co/xumo/reader-speech-models:Q4_K_M
- Lemonade
How to use xumo/reader-speech-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull xumo/reader-speech-models:Q4_K_M
Run and chat with the model
lemonade run user.reader-speech-models-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Reader accepted local speech artifacts
Reader-converted artifacts, not an official Qwen release. Files retain the bytes
accepted by Reader; models.lock.json pins size and SHA-256. Bind downloads to the
full publication commit, never a moving main branch.
Included artifacts
asr-0.6b-f16asr-1.7b-f16asr-1.7b-q4-k-masr-1.7b-q8-0aligner-q8-0mmproj-asr-0.6b-reader-compatmmproj-asr-1.7b-reader-compatmmproj-forcedaligner-reader-compatqwen3-tts-0.6b-base-f16synthetic-heartsynthetic-george
Sources and conversion
Qwen Base revision: 5d83992436eae1d760afd27aff78a71d676296fc.
Converter: ggml-org/llama.cpp 67107ac34f62882d8b53ea41ad68010ea635b9be.
Reader packaging source commit: 14d2029ba099c97ea4b2544bef0c58de4f11ed79 (may precede its source-repository push).
The complete recipe and reconstruction evidence are included in this model repo.
Pinned public converter:
https://github.com/ggml-org/llama.cpp/blob/67107ac34f62882d8b53ea41ad68010ea635b9be/convert_hf_to_gguf.py
The Qwen TTS GGUF pair includes the talker and projector/codec. The generated
reader-model.json preserves the original source-file receipt; raw safetensors
listed in that receipt are source inputs and are not required for inference.
Rebuilding is a separate operation; consult conversion.lock.json and rebuild.json.
Use the locked Python/package versions, install the converter's pinned gguf-py,
download all raw files from the pinned Base revision, and preserve the source
directory basename recorded by the recipe (it can affect GGUF naming metadata).
Run the two argument sets in the recipe through the pinned converter, providing
the raw model directory as positional input and fresh output paths. Keep the
speech_tokenizer subdirectory beside the raw model. Reconstruction is CPU-only;
the accepted record uses OMP/MKL/OpenBLAS thread counts of 4 and offline HF mode.
Synthetic Heart/George WAVs, when included, are reference audio resources, not model weights or user recordings. Their Kokoro model revision and source voice are recorded in each artifact origin. Preserve the supplied WAV bytes to retain the accepted speaker conditioning; generating new reference WAVs creates new assets.
Runtime compatibility and limits
Use the Reader runtime matching the locked upstream revision. Reader validates Qwen synthesis for English/Chinese; other upstream languages are not claimed by this release. Validated host: Linux x64, GTX 1080 Ti, CUDA SM61. Model files and platform runtime binaries have separate lifecycles. This repository contains no runtime binary, personal recordings, account data, or historical ASR/FA files with unverified provenance.
Attribution
Qwen3-TTS: https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base Kokoro reference source: https://huggingface.co/onnx-community/Kokoro-82M-v1.0-ONNX Upstream model cards mark Apache-2.0. See LICENSE. Reader changes: conversion to F16 GGUF and explicit packaging/identity metadata. The publication commit is not a model retraining or a new quality claim.
ASR/FA conversion provenance
The accepted text GGUF and Reader-compatible projectors were reconstructed exactly
from the official revisions and raw-file hashes in asr-conversion.lock.json.
asr-rebuild.json compares all eight outputs to their accepted SHA values.
To reconstruct: obtain public ggml-org/llama.cpp commit
66d65ec29ba7c440cbc31b6f63b74a17b536ba65, apply the included
legacy-asr-conversion.patch, and check the four source hashes in
legacy-asr-converter.lock.json. This freezes conversion-only sources; it does
not restore an old inference backend. Use the locked Python environment and the
patched source's gguf-py. Convert each official raw model to F16 text GGUF and an
F16 mmproj. The original model folder basename affects metadata and must match.
Build the CPU llama-quantize tool from current upstream commit
67107ac34f62882d8b53ea41ad68010ea635b9be. Quantize ASR 1.7B F16 to Q8_0/Q4_K_M
and FA F16 to Q8_0 with four threads. Apply the included standalone Reader
projector converter (conversion/) with --encoder-compat, using gguf-py from
that current pinned commit. No tensors are retrained; compare final SHA values
against the model lock before use.
The Reader-compatible mmproj files require Reader's encoder/aligner compatibility adapter. Ordinary upstream projectors are not interchangeable with these accepted files. See the per-artifact origins for source revisions and compatibility tags. Converter code is covered separately by LICENSE.llama.cpp; model provenance and Apache-2.0 attribution remain in LICENSE and the official Qwen model cards.
- Downloads last month
- 214
4-bit
8-bit
16-bit
Model tree for xumo/reader-speech-models
Base model
Qwen/Qwen3-ASR-0.6B