Jour/hy-mt2-7b-discord-lora

Hy-MT2-7B — Discord LoRA

A small LoRA (rank 8, 42 MB) for Tencent's Hy-MT2-7B (Apache-2.0; not redistributed here). It teaches the model to keep markdown, links, mentions, emojis, code and [markNN] tags in Discord messages while translating informal chat text.

It was trained with QLoRA (the bf16 model loaded in 4-bit nf4, LoRA on all linear layers, rank 8, alpha 16, 2 epochs, ~2.7k pairs) and is meant to be applied at runtime on the official Q4_K_M GGUF tencent/Hy-MT2-7B-GGUF. The adapter was also checked against that GGUF with llama-server --lora.

Files

• r8/hy-mt2-7b-discord-lora-r8-f16.gguf (42 MB): GGUF adapter for llama-server --lora.
• r8/adapter_model.safetensors and r8/adapter_config.json: PEFT adapter, for transformers + peft.

Usage

llama-server -m Hy-MT2-7B-Q4_K_M.gguf --lora r8/hy-mt2-7b-discord-lora-r8-f16.gguf -ngl 99 -c 4096 --jinja

Official Hy-MT2 prompt, no system prompt: Translate the following text into French. Note that you should only output the translated result without any additional explanation:\n\n{text}. (Chinese prompt when Chinese is the source or the target.)

Results (143 unseen pairs, 108 with protected elements)

"Formatting preserved" = every protected span (tags, links, mentions, emojis, code) is identical in the output and the markdown marker and line break counts match. Greedy decoding, llama-server on an RTX 4070, official Q4_K_M GGUF, context 4096.

variant	formatting preserved (all / with protected elements)	chrF	generation
Hy-MT2-7B Q4_K_M, no LoRA	98.6 % / 98.1 %	67.6	~82 tok/s
+ this LoRA	99.3 % / 99.1 %	73.6	~69 tok/s
The 7B already preserves formatting well without the adapter; the LoRA mainly improves translation quality on informal Discord text (chrF +6.0) and removes a few remaining formatting errors. It costs ~15 % generation speed (unmerged adapter). VRAM for the server: ~5 GB.

Limits

Synthetic training data (Claude Haiku), ~2.7k pairs, 13 languages, skewed towards English and French, not human-reviewed; small test set (the same one as the 1.8B adapter, Jour/hy-mt2-1.8b-discord-1.25bit-lora). Keep a formatting check (tag masking) in your application. Adapter licensed Apache-2.0 (license file included); no real user data was used.