aufklarer/LocalVQE-v1.4-AEC-200K-CoreML

LocalVQE v1.4-AEC 200K Core ML

Stateful FP16 Core ML export of the learned residual echo mask from LocalVQE v1.4-AEC. It is designed for local, streaming acoustic echo cancellation on Apple hardware.

This is an echo-only hybrid. It is not the older LocalVQE v1.3 joint noise-and-echo enhancer, and the Core ML model is not a standalone waveform model. A host implementation must run delay estimation, adaptive echo filtering, the short controller network, FFT, and overlap-add around it.

Model

Property	Value
Neural mask	200,199 parameters, FP16 Core ML ML Program
Adaptive controller	2,742 parameters, F32 host implementation
Streaming state	225,408 bytes FP16
Sample rate	16 kHz mono
FFT / hop	512 / 256 samples
Frame interval	16 ms
Deployment target	macOS 15 or iOS 18 and newer
Compiled bundle	707,015 bytes

Files

File	Size	Purpose
LocalVQEAECResidualMask.mlmodelc/	707,015 bytes	Compiled stateful Core ML residual mask
LocalVQEAECFrontend.npz	20,826 bytes	Adaptive-controller weights and host DSP constants
manifest.json	about 2 KB	Source identity, state shapes, and conversion parity
config.json	about 1 KB	Runtime contract and download-tracking metadata
checksums.json	about 1 KB	SHA-256 checksums for every other published file

No source GGUF is duplicated here. The verified upstream source remains in the LocalVQE repository.

Performance

Measured on an Apple M5 Pro. Each frame represents 16 ms of audio. Lower latency and real-time factor are better; higher audio-to-wall-time is better.

Stage	Mean latency	p99	Real-time factor	Audio / wall time
Core ML neural mask	0.177 ms	0.209 ms	0.0111	90.4x
Host adaptive filter	0.236 ms	0.398 ms	0.0148	67.8x
Complete hybrid, summed estimate	0.413 ms	0.607 ms	0.0258	38.7x
Upstream full GGML F32 on CPU	0.737 ms	1.268 ms	0.0461	21.7x

The complete-hybrid number conservatively adds separately measured host and Core ML stages. It is not yet an integrated Swift measurement.

The Core ML mask had 1.356% relative spectral error against the reconstructed PyTorch reference on the public ten-second double-talk example. Its output was 37.68 dB SI-SDR relative to the PyTorch waveform.

Echo quality

Microsoft AECMOS v4 scores the same public double-talk recording on a one-to-five scale. Higher is better for both columns.

Output	Echo MOS	Other-degradation MOS
Microphone passthrough	3.4555	3.7498
Adaptive filter only	4.1239	3.5080
Full hybrid, PyTorch mask	4.0473	3.7020
Full hybrid, Core ML mask	4.0471	3.6997

On this recording, the adaptive filter alone removes slightly more echo but damages near-end speech more. The learned mask restores 0.194 MOS of speech quality while giving back 0.077 echo MOS. Core ML changes the two PyTorch scores by only 0.0002 and 0.0022.

Download

hf download aufklarer/LocalVQE-v1.4-AEC-200K-CoreML \
  --local-dir LocalVQE-v1.4-AEC-200K-CoreML

Python usage

The two inputs are spectra produced by the host adaptive-filter front end: the residual microphone signal e and estimated echo yhat. Both have shape (1, 2, 1, 256), containing real and imaginary values for FFT bins 1 through 256. Reset the Core ML state at the beginning of every audio session.

from pathlib import Path

import coremltools as ct
import numpy as np
from huggingface_hub import snapshot_download

root = Path(snapshot_download(
    "aufklarer/LocalVQE-v1.4-AEC-200K-CoreML"
))
model = ct.models.CompiledMLModel(
    str(root / "LocalVQEAECResidualMask.mlmodelc"),
    compute_units=ct.ComputeUnit.ALL,
)
state = model.make_state()

# Supply one spectrum from each host-DSP stream every 16 ms.
residual_spectrum = np.zeros((1, 2, 1, 256), dtype=np.float16)
echo_estimate_spectrum = np.zeros((1, 2, 1, 256), dtype=np.float16)
enhanced_spectrum = model.predict(
    {
        "mic_spectrum": residual_spectrum,
        "reference_spectrum": echo_estimate_spectrum,
    },
    state=state,
)["enhanced_spectrum"]

The adaptive front end follows upstream LocalVQE v1.4: GCC-PHAT delay acquisition, a partitioned-block frequency-domain Kalman filter, and its v2xp controller. LocalVQEAECFrontend.npz contains the controller tensors and exact DSP constants required by a native implementation.

Limitations

• This export removes playback echo. It is not a general noise suppressor.
• Quality was measured on one public double-talk example. Evaluate a larger AEC corpus and recordings from the intended capture hardware before deployment.
• The host adaptive filter is required; passing microphone and playback spectra directly to the Core ML mask does not implement the published system.
• The complete native Swift pipeline and its memory use have not yet been measured.

Source and license

Converted from the F32 localvqe-v1.4-aec-200K release, run v1.4.r005, with source SHA-256 b6e43138588a83bfe903ab5e143b4020b91c1e1629f5a575ac5855ff0003c731.

• LocalVQE source and documentation
• Upstream model files
• Microsoft AECMOS

The upstream implementation and weights are Apache-2.0 licensed.

Links

• speech-swift — Apple SDK
• Documentation — installation and CLI
• soniqo.audio — website
• Blog — updates and technical notes