skt/A.X-K2-ALM

A.X K2 ALM — Audio Language Model

🤗 Models | 📄 Technical Report

Model Summary

A.X K2 ALM (Audio Language Model) is a member of the A.X K2 family that adds a speech modality to a frozen in-house LLM backbone — A.X K2 Light (20B-A2.7B MoE) — to enable natural voice-based conversation. It integrates speech understanding, voice activity detection (VAD), and speech generation within a single unified, streaming architecture built from four components: a chunk-wise Conformer speech encoder, an adapter (MLP projector + Q-Former), a lightweight VAD module, and a multi-token-prediction (MTP) speech decoder.

Motivated by production settings where the deployed LLM may be audio-unaware and its existing text-response quality must be preserved, we keep the LLM entirely frozen and train only the speech-side components — avoiding the catastrophic forgetting and text-quality degradation commonly induced by end-to-end multimodal training, and letting the same recipe transfer readily to other LLMs. On our Korean KVoiceBench, A.X K2 ALM reaches 95% of Qwen3-Omni-30B-A3B's response quality at roughly two-thirds the parameters.

_{A.X K2 ALM architecture. Speech encoder → adapter → frozen LLM → speech decoder; the VAD reads the adapter and LLM representations.}

Key Features

• A Korean ALM Built Entirely from Scratch Every speech-side component — the streaming speech encoder, the adapter, the VAD heads, and the speech decoder with its neural audio codec — is built and trained in-house from scratch rather than adapted from open-source models, on top of the frozen A.X K2 Light backbone. This lets the frame rate, streaming behavior, and language coverage be tuned end-to-end for Korean voice services.

• Self-Supervised (SSL) Streaming Speech Encoder A 300M chunk-wise Conformer encoder, self-supervised pre-trained from scratch on 400K hours of unlabeled speech with multi-codebook Random Quantization and fine-tuned as a 341M streaming transducer, outperforms Qwen3-Omni on Korean ASR benchmarks at roughly half its audio-encoder size — even under a low-latency streaming constraint.

• Efficient Streaming Speech Decoder We introduce a 0.88B multi-token speech decoder that integrates a text–speech language model with a depth-transformer to hierarchically generate semantic tokens and residual acoustic codes in real time. Operating at roughly half the parameter footprint of current SOTA baselines, our model achieves superior pronunciation accuracy with the lowest CER while maintaining high speech intelligibility comparable to leading systems, ensuring precise delivery of generated text responses in real-time ALMs.

• Forgetting-Free Modality Extension The LLM stays fully frozen; a two-stage adapter — speech alignment (syllable-CTC / char2token bridge) then self-distillation dialogue training — transfers the backbone's response behavior to the audio path without changing its text-input behavior.

• Integrated Context-Aware VAD The external VAD is replaced by lightweight six-class heads on the model's own representations — an adapter-based head detects onset, an LLM-based head verifies with context — enabling barge-in and far fewer false triggers on non-speech and mid-utterance pauses.

Model Details

Function	Component	Configuration	Params
Speech Understanding	Speech Encoder	Chunk-wise Conformer	300M
	Adapter — MLP Projector	2-layer, GELU	5.77M
	Adapter — char-CTC head	linear, 2048 → 2667	5.46M
	Adapter — Q-Former	4L / 16H / d=2048 / FFN 8192	201.4M
	LLM (frozen)	A.X K2 Light (in-house MoE)	20B-A2.7B
Speech Generation	Conditioning Projector	FFN stack (2048 → 1024, 2 paths)	71.3M
	Multi-Token Predictor	Text–Speech Language Model	630M
		Depth Transformer	110M
	Speaker Encoder	Transformer	100M
	Audio Codec Decoder	Transformer + ConvNet	40M
Total	A.X K2 ALM		≈22B-A4B

• VAD heads: six-class classifiers over the adapter output and LLM hidden states; parameter counts are negligible.
• Languages: Korean
• Streaming: all input/output paths are designed for streaming operation for low-latency live voice interaction.
• Full details: see the technical report for the architecture and training recipe.

Evaluation Results

Speech understanding is evaluated on KVoiceBench, our own Korean spoken-query benchmark suite built in-house by localizing VoiceBench with human review and adding Korean-native professional-knowledge items, covering speech recognition (STT), spoken QA (Speech-QA), and instruction following (Speech-IF). Open-ended (OPEN) categories are scored by an LLM judge and knowledge (QA) categories by correctness. See the technical report for KVoiceBench construction details. Bold = best among compared models.

Task	Benchmark	Metric	A.X K2 ALM (22B-A4B)	Qwen2.5-Omni (7B)	Qwen3-Omni (30B-A3B)	HyperCLOVA X 8B Omni
STT	KsponSpeech (eval-clean)	CER ↓	9.00	18.96	8.46	10.22
	KsponSpeech (eval-other)	CER ↓	9.12	22.72	7.91	10.15
Speech-QA	AdvBench	OPEN ↑	92.67	92.61	89.55	81.87
	AlpacaEval	OPEN ↑	69.70	52.87	71.14	51.81
	AlpacaEval-Full	OPEN ↑	70.75	52.45	72.21	52.43
	BBH	QA ↑	44.18	52.66	57.22	43.80
	CommonEval	OPEN ↑	70.20	55.41	68.25	55.49
	MMSU	QA ↑	43.67	32.08	43.48	32.08
	OpenBookQA	QA ↑	76.62	76.12	91.54	70.90
	SD-QA	QA ↑	46.87	25.89	48.23	27.52
	WildVoice	OPEN ↑	66.88	46.33	66.25	45.77
Speech-IF	IFEval	OPEN ↑	61.20	44.88	67.61	35.21
Avg.	Speech-QA / IF	↑	64.27	53.13	67.55	49.69
Speech Generation	KVoiceBench (Subset-200)	CER ↓	1.86	–	26.37	4.16
		UTMOS ↑	2.871	–	3.178	3.171
		STOI ↑	0.990	–	0.999	0.996

Averaged over the ten KVoiceBench subsets, A.X K2 ALM scores 64.27 — 95% of Qwen3-Omni-30B-A3B (67.55) at roughly two-thirds of its parameters — and outperforms Qwen2.5-Omni-7B and HyperCLOVA X 8B Omni by 21% and 29% (relative).

Demos

1. Natural Voice Conversation A fluid, natural spoken dialogue with the model.

2. Context-Aware Turn-Taking The context-aware VAD ignores listener back-channels (e.g., "음", "아") and waits through mid-sentence hesitations — so it neither falsely stops nor cuts in, responding only once the user has actually finished.

Availability

Model weights are planned for public release (coming soon). For now, this repository serves as the model card and hosts the demo videos above and the technical report for reference.

License

Released under the h-research license.

Citation

If you use A.X K2 ALM in your research, please cite the technical report:

@techreport{axk2alm2026,
  title       = {A.X K2 ALM (Audio Language Model)},
  author      = {SKT A.X Team},
  year        = {2026},
  institution = {SK Telecom}
}

Contact

For A.X models for business applications or scalable deployment, please contact a.x@sk.com.