infosave/Hy-Memory-30B-A3B-cmf

Tencent Hy-Memory-30B-A3B — CMF

tencent/Hy-Memory-30B-A3B is Tencent Hunyuan's compact long-term-memory writer (~3 B active parameters of a 30 B hy_v3 MoE): it decides what to retain, how to represent it and how new evidence updates a persistent memory bank. It is trained to replace the general-purpose LLM inside Mem0, MemOS and Hy-Memory (chat, working and coding scenarios) and transfers to unseen frameworks such as EverMemOS — in the upstream cross-harness matrix it averages 55.49, ahead of Claude Opus 5 (54.62) and DeepSeek V4.1 Flash (53.64). See the tech report, arXiv:2610.22196.

This repository packages it as a single CMF file for cortiq — a Rust inference engine with no Python and no ML framework (NVIDIA/AMD/Intel via Vulkan, Apple silicon via Metal, CPU).

Quick start

cargo install cortiq-cli
hf download infosave/Hy-Memory-30B-A3B-cmf hy-memory-30b-a3b-q4tp.cmf --local-dir .
cortiq run hy-memory-30b-a3b-q4tp.cmf \
  --prompt "Summarize in one sentence what an LLM memory writer does in Mem0." \
  --no-think --max-tokens 256

Requires cortiq 0.7.0 or newer (the hy_v3 architecture; this file was built and measured with 0.8.31). The model reasons aloud before answering — raise --max-tokens (256–512); with a tight budget it is cut mid-thought instead of stopping. finish: stop shows the model ending on its own EOS.

Files

model	file	source	bits	size	wikitext-2 ppl¹
Hy-Memory-30B-A3B	hy-memory-30b-a3b-q4tp.cmf	bf16 checkpoint	4.17	15,807,584,068 B (15.81 GB / 14.72 GiB)	9.17 (4096) · 12.02 (12×512)

¹ Teacher-forced perplexity on wikitext-2 test with cortiq ppl, exact attention, on the RTX 2000 Ada. Two numbers, two protocols: the first 4096 tokens (9.168) and the account-standard twelve 512-token windows (12.021). For orientation, the sibling Hy-MT2-30B-A3B card reports 9.88 under the same 12×512 protocol — a general-purpose translation/chat model. Hy-Memory is a fine-tune specialized for structured memory rollouts, so a higher free-prose perplexity alongside correct task behaviour is expected, not a conversion defect.

SHA-256:

• hy-memory-30b-a3b-q4tp.cmf — 3ebac5a4778e2c28671857c8da41b7e22f91227bbb4870322293909546d9baa3

sha256sum -c hy-memory-30b-a3b-q4tp.cmf.sha256
cortiq verify hy-memory-30b-a3b-q4tp.cmf   # 18 673 tensors, all hashes match
cortiq info   hy-memory-30b-a3b-q4tp.cmf

cortiq info: hy_v3 | 48 layers | hidden 2048 | 30.06 B params | 18 673 tensors (18 386 q4tp + 240 f16 + 47 f32) | vocab embedded (120 832) | chat template embedded (11 258 chars, 2 stop ids).

What the runtime had to learn

Hy-Memory rides the same hy_v3 block the runtime already carries for Hy-MT2-30B-A3B — that support is what makes this a drop-in conversion:

• 48 layers: layer 0 dense (6 912 wide), 47 sparse layers of 128 experts, top-8 by sigmoid score plus expert_bias (bias for the choice, unbiased scores for the weights), route_norm renormalization and router_scaling_factor = 2.826 — identical constants to Hy-MT2;
• one always-on shared expert of width 768, stored by the vendor as mlp.shared_mlp.* (the converter maps it onto the canonical layout);
• per-head q/k RMSNorm (qk_norm), HunYuan tokenizer (120 832 vocab, non-BPE markup) and RoPE theta 11 158 840 — now nested under rope_parameters in the vendor config; the converter reads both spellings.

One vendor quirk is worth documenting: config.json declares moe_intermediate_size: 1536, while both the routed experts and the shared expert are physically 768-wide (gate/up [768, 2048], down [2048, 768]). The converter encodes per-expert tensors by their actual safetensors shapes, so the header carries consistent geometry and the arithmetic below confirms it end to end.

New versus Hy-MT2: attention_bias/mlp_bias are explicitly false, the config adds mlp_layer_types (dense/sparse per layer — the same layer-0 dense pattern) and num_nextn_predict_layers: 0 (no MTP draft head), and enable_moe_fp32_combine is set — none of which change the block.

Measured

Hardware: NVIDIA RTX 2000 Ada (16 GB), Vulkan driver 595.91.07, Cortiq v0.8.31, single stream. The 15.8 GB file is larger than the card's 13.82 GB weight budget, so the tail of the MoE stays host-resident and is fetched as needed (8 588 MB initial upload recorded by the bench).

cortiq bench --core --ignore-eos --tokens 120:

metric	value
decode, steady state	17.1 tok/s
prefill (41-token prompt)	7.6 tok/s
graph capture	119 / 120 tokens (prefix)
initial weight upload	8 588 MB

Interactive cortiq run (same GPU): first token 2.8–5.1 s, decode 8.3–12.1 tok/s depending on prompt length. On CPU-only hosts expect the offload path to dominate — this model is built for a GPU or a big-memory box.

Quality checks

Greedy --no-think battery with --max-tokens 256+, run on the GPU; every answer ended with finish: stop:

check	expected	result
17 * 23, just the number	391	pass (reasoned, then 391)
1234 + 5678, just the number	6912	pass
capital of France, one word	Paris	pass
start of WWII (Russian), one number	1939	pass
exactly two sentences about a lighthouse keeper	2 sentences	pass (obeyed after a planning aside)
one sentence on the Mem0 memory writer	1 sentence	pass — content correct (extract + persist + retrieve)

The model always thinks aloud first; that is its training (memory-writer rollouts), not a defect. Give it tokens.

Prompting

The vendor chat template is embedded (11 258 chars, HunYuan <｜hy_User｜> / <｜hy_Assistant｜> markup, 2 stop ids), so cortiq run and cortiq serve apply it automatically; --raw bypasses it for completion-style probing. There is no system prompt in the upstream template.

For the model's actual job, wrap your task the way a memory harness would:

cortiq run hy-memory-30b-a3b-q4tp.cmf --no-think --max-tokens 512 --prompt \
"Conversation: user says 'I live in Almaty and I'm allergic to penicillin.'
Decide what to write to long-term memory. Output the memory entries as JSON."

Server

cortiq serve hy-memory-30b-a3b-q4tp.cmf --port 8080

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "hy-memory-30b-a3b-q4tp", "temperature": 0.7,
       "messages": [{"role": "user", "content": "Remember that I prefer metric units."}]}'

/v1/chat/completions, /v1/completions, /v1/models and a web dashboard are live on the same port.

Reproduce

cortiq convert --model tencent/Hy-Memory-30B-A3B --quant q4tp \
  --output hy-memory-30b-a3b-q4tp.cmf     # streamed shard-by-shard from HF
cortiq verify hy-memory-30b-a3b-q4tp.cmf

The conversion is a straight q4tp quantization of the bf16 safetensors — no calibration data, no requantization of a lower-precision file.

Scope

This is the text-generation memory-writer path. The CMF does not claim support for the upstream Python harnesses (Mem0/MemOS integrations, rollout tooling), LoRA adapters, or bit-exact numerical parity with the PyTorch implementation; those are harness-side concerns.

License and provenance

Upstream weights are Apache-2.0; this CMF is a quantized repackage and retains upstream license and attribution. Model behaviour, training data disclosures and use restrictions are documented in the upstream card and the tech report.