GigaChat Audio MLX — BF16

Runtime and quick start · Full results · Upstream model · Original paper

Native BF16 MLX conversion of ai-sage/GigaChat3.1-Audio-10B-A1.8B for Apple Silicon. It accepts audio plus a text instruction and produces text: audio QA, transcription, translation, emotion analysis, temporal grounding, and timestamped summaries.

This is an independent port, not an official release by the upstream authors.

The original model and paper are by Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin. The upstream model card declares MIT. See the runtime license and third-party notices.

What do I need to run it?

Model	Model size	Free disk to set aside	Measured whole-process memory	5:17 audio on M4 Pro
BF16	22.54 GB	about 25 GB	22.84 GiB	10.41 s · 30.5× realtime

The free-disk figure is a rounded practical reserve; the CLI dry run reports the exact value. Memory is measured use, not a minimum-memory guarantee. This release is verified only on a 48 GB Mac; 16/24/32 GB support is not yet claimed.

Why MLX matters on this Mac

Same Apple M4 Pro / 48 GB. Original is a stopped memory-safety smoke, not a completed speed benchmark; MLX values are medians of five clean runs.

Runtime	Audio	Outcome	Whole-process footprint
Original PyTorch BF16 on MPS	9.84 s	No answer after 277 s; stopped	at least 51.92 GiB
MLX BF16	317.27 s	Completed in 10.41 s	22.84 GiB
MLX Q8/BF16	317.27 s	Completed in 6.90 s	14.80 GiB

The pinned upstream Transformers path is demonstrated for CUDA. On MPS it exceeded this Mac's safe memory budget and nearly filled system swap during the short smoke. The 5-minute Original run was not started, so no speedup ratio is claimed. MLX is what makes the model usable on the tested Mac.

Quick start

brew install uv
uv tool install gigachat-audio-mlx
gigachat-audio-mlx doctor

gigachat-audio-mlx download --variant bf16 --dry-run

gigachat-audio-mlx download --variant bf16 --yes

The dry run reports the exact download size, cache hits, free disk space, and destination before any network transfer. The runtime never executes remote model code.

gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768

Exact task examples

The text instruction is part of every input. It must not be omitted when interpreting an answer.

Russian speech transcription

Prompt:

Transcribe the Russian speech exactly. Return only the transcript.

Output:

Всё во Вселенной состоит из материи. Вся материя состоит из мельчайших частиц, называемых атомами.

Temporal grounding

Prompt:

Укажи таймкод начала и конца речи в аудио.

Output:

Таймкоды начала и конца речи в аудио:
- Начало: 00:00
- Конец: 00:20

Artifact

Field	Value
Format	sharded safetensors, data-only
Runtime payload	22,539,048,633 bytes
Precision	BF16
Source revision	`bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb`
Runtime	`gigachat-audio-mlx==0.1.4`
Verified target	Apple M4 Pro / 48 GB / macOS 15.7.7
Weight files commit	`7484ac64355654f899fefa4e45637dc53a0ba40c`
Manifest SHA-256	`3a845299123f35ba65313fb19e5f030516dc84d53b791541c7657dac91a0f8a3`

The model repository contains no executable Python. model_manifest.json pins the source lineage and SHA-256 of every data file. Loading fails on a missing, changed, or unexpected artifact.

Results available today

The BF16 port is the accepted native MLX baseline. In the frozen 50-item parity gallery, its decoded text exactly matched the original PyTorch BF16 output on 41/50 inputs. This is an implementation-parity diagnostic, not an aggregate quality score.

What changes with Q8/BF16

Lower is better in every row. Both models used the same MLX workloads.

Metric	MLX BF16	MLX Q8/BF16	Change
Model size, decimal GB	22.54	13.75	−39.0%
Whole-process memory, 5-minute profile	22.84 GiB	14.80 GiB	−35.2%
End-to-end time, 5-minute profile	10.41 s	6.90 s	−33.7%
Responses with a factual error, lower is better	26/64 (40.63%)	29/64 (45.31%)	+3 responses (+4.69 pp)

The 5-minute workload contains 317.27 seconds of audio. Resource values are the median of five clean processes on the verified M4 Pro / 48 GB machine.

Q8/BF16 is the smaller, faster option; BF16 provides the best measured answer quality. In the blinded review, Q8/BF16 had factual errors in 3 more cases.

From Original to MLX

Check	Original PyTorch BF16	MLX BF16	MLX Q8/BF16
Model size, decimal GB	23.73	22.54	13.75
Exact decoded-text agreement	Reference	41/50 vs Original	45/50 vs MLX BF16

The post-publication Original PyTorch/MPS resource check did not complete even the 9.84-second smoke on this 48 GB Mac. After 277 seconds its process footprint had reached at least 51.92 GiB and system swap was nearly full, so it was stopped before the 5-minute run. Original therefore remains N/A in the direct Mac RAM and speed table; no latency is inferred from a failed, swap-dominated run.

Results reported by the Original authors

These values come from the upstream model card; we did not rerun them.

Task and dataset	Metric	Original reported result
Audio QA · MMAU	Accuracy ↑	62.2
Audio QA (Russian) · RuBQ	Accuracy ↑	60.0
Emotion · Dusha podcast	Accuracy ↑	92.4
ASR (Russian) · FLEURS ru	WER ↓	4.4
Translation · FLEURS ru→en	BLEU ↑	33.4
Temporal localization · 20–60 min	mIoU ↑	48.3
Timed summary · 20–60 min	Overall ↑	67.9

Full public release evidence separates ASR, translation, emotion, temporal localization, timed descriptions/summaries, factual review, memory, and speed. It does not collapse these tasks into one number. See the full results table.

Limitations

Verified only on Apple M4 Pro with 48 GB unified memory and macOS 15.7.7.
No claim is made for 16/24/32 GB machines or every M1–M5 generation.
No 90-minute-audio support claim is made in this release.
Generated facts, emotion labels, summaries, and timestamps can be wrong and must be checked against the source audio.
Long summaries may require a larger output-token budget, which also changes the memory estimate.
No diarization, streaming microphone mode, speech synthesis, training, or fine-tuning is included.
Do not use the output as the sole basis for medical, legal, employment, safety, or surveillance decisions.

Users are responsible for consent, privacy, copyright, and dataset terms for audio they process.

License and attribution

Upstream model: https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B
Upstream revision: bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb
Paper: GigaChat Audio: Time-aware Large Audio Language Model
Original authors: Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin
Upstream license declaration: MIT in the upstream model card
Runtime and MIT license: https://github.com/ai-babai/gigachat-audio-mlx
Model collection: https://huggingface.co/collections/ai-babai/gigachat-audio-mlx-6a5cab53ded991903b664437

Please cite both the original paper above and this project through CITATION.cff. See the upstream model card and THIRD_PARTY_NOTICES.md for full attribution and provenance.