Duckets/Hy-Memory-30B-A3B-MLX-4bit

Hy-Memory-30B-A3B — MLX 4-bit

A straight 4-bit MLX quantization of Tencent's Hy-Memory-30B-A3B for Apple Silicon. (8-bit and mixed-precision MLX builds already existed — this is the plain 4-bit one.)

What Hy-Memory is

Hy-Memory is a 30B-total / ~3B-active MoE (48 layers, 128 experts, 262K context, architecture hy_v3) built by Tencent for one specialized job: writing an AI agent's long-term memory — extracting durable, self-contained memory statements from conversations. On Tencent's cross-harness evaluation (Mem0 / MemOS / Hy-Memory / EverMemOS) it outscored Claude Opus 5 and Qwen3.6-35B-A3B as a drop-in memory writer (macro average 55.49).

This quant

• Converted from the BF16 originals with mlx-lm 0.32.0: affine quantization, 4 bits, group size 64
• 16.9 GB on disk, 17.1 GB peak memory at inference — comfortable on a 32 GB Mac, fast on anything bigger (125 tok/s generation measured on an M5 Max)
• Ships with hy_v3.py, the mlx-lm model module for this architecture (upstream mlx-lm doesn't carry hy_v3 yet), plus the original chat template and tokenizer

Use it

LM Studio: download the folder into your LM Studio models directory and load it — it identifies as a standard MLX model.

mlx-lm:

pip install mlx-lm
# make sure hy_v3.py from this repo sits in mlx_lm/models/ (or alongside your loader)
python -m mlx_lm generate --model Duckets/Hy-Memory-30B-A3B-MLX-4bit \
  --prompt "User: I switched from drip coffee to a double espresso every morning. Extract the memory statements as a JSON list."

Feed it a conversation chunk; it returns clean, self-contained memory statements as JSON — ready to embed into your agent's memory bank.

Why I made it

I run a local agent-memory system (DuckBot RAG) and shadow-tested Hy-Memory against my previous extractor on my real memory files the day Tencent released it (2026-10-16): 107 facts extracted vs 40, duplicate rate 0.24 vs 0.65. The GGUF build worked, but MLX is the native fast path on Apple Silicon — and nobody had published a straight 4-bit MLX build. Now somebody has.

Credits & license

All model credit to Tencent — this is a quantization of their Apache-2.0 release, redistributed under the same license. The hy_v3.py module is adapted from Tencent's transformers implementation of the architecture.