Hy-MT2-1.8B-StreamRevise-v4 (LoRA)
A LoRA adapter for tencent/Hy-MT2-1.8B, trained for
live subtitle translation with changing ASR hypotheses.
On every ASR update, the caller sends the current source hypothesis together with the model's last translation of the utterance. The model can preserve a still-correct prefix, extend it as more speech arrives, or revise it when the recognized meaning changes. The caller may also provide recent bilingual context so names, word senses, and conversational references remain consistent.
For a ready-to-run 4-bit model, use Hy-MT2-1.8B-StreamRevise-v4-GGUF.
中文简介：面向实时字幕的流式翻译 LoRA。每次 ASR 更新时，把当前识别文本和模型上一版译文一起 传入；模型会在语义不变时尽量保留已经显示的正确前缀，在新增内容或识别纠正时继续扩展或改写。 还可以传入最近几句原文与译文，帮助保持人名、词义和上下文一致。想直接本地运行可使用 GGUF 版。
Prompt format
The adapter is trained on the layouts below. Use an explicit English source and target language name such
as Japanese, Chinese, or English, and use greedy decoding.
First chunk of a new utterance
Translate the following text from {SOURCE_LANGUAGE} into {TARGET_LANGUAGE}. Note that you should only output the translated result without any additional explanation:
{CURRENT_SOURCE}
Subsequent updates
[Background Information]
Recent utterances and their translations:
{SOURCE_SENTENCE_1} → {TRANSLATION_1}
{SOURCE_SENTENCE_2} → {TRANSLATION_2}
...
Previous version of the current source:
{PREVIOUS_SOURCE}
Previous translation of the current source:
{PREVIOUS_TRANSLATION}
When the source meaning has not changed, preserve the still-correct prefix of the previous translation whenever possible. When content is added or corrected, accuracy and completeness take priority.
Please translate the following text from {SOURCE_LANGUAGE} into {TARGET_LANGUAGE}, taking the provided background information into consideration.
[Source Text]
{CURRENT_SOURCE}
The background blocks are optional, separated by one blank line, and kept in this order: recent context, previous source, previous translation, stability rule. Source-only history is also supported with this block:
Recent source utterances:
{SOURCE_SENTENCE_1}
{SOURCE_SENTENCE_2}
...
Use at most the most recent 10 utterances. Bilingual history is recommended when the application has the translations available. Do not mix bilingual and source-only lines inside the same history block.
Chat template
Use one user turn. apply_chat_template handles the wrapper; for a raw completion endpoint the exact form is:
<｜hy_begin▁of▁sentence｜><｜hy_User｜>{PROMPT}<｜hy_Assistant｜>
There is no trailing newline. Generation stops on token 120020.
Copy-paste renderer
LANGUAGE_NAMES = {
"zh": "Chinese",
"en": "English",
"ja": "Japanese",
}
STABILITY_RULE = (
"When the source meaning has not changed, preserve the still-correct prefix "
"of the previous translation whenever possible. When content is added or "
"corrected, accuracy and completeness take priority."
)
def render_prompt(*, source_language, target_language, current_source,
history=(), previous_source="", previous_translation="",
stability_rule=True):
source = LANGUAGE_NAMES.get(source_language, source_language)
target = LANGUAGE_NAMES.get(target_language, target_language)
if not history and not previous_source and not previous_translation:
return (
f"Translate the following text from {source} into {target}. Note that you "
"should only output the translated result without any additional "
f"explanation:\n\n{current_source}"
)
blocks = []
if history:
bilingual = isinstance(history[0], (tuple, list))
if bilingual:
lines = [f"{src} → {tgt}" for src, tgt in history[-10:]]
blocks.append("Recent utterances and their translations:\n" + "\n".join(lines))
else:
blocks.append("Recent source utterances:\n" + "\n".join(history[-10:]))
if previous_source:
blocks.append("Previous version of the current source:\n" + previous_source)
if previous_translation:
blocks.append("Previous translation of the current source:\n" + previous_translation)
if stability_rule:
blocks.append(STABILITY_RULE)
return (
"[Background Information]\n" + "\n\n".join(blocks)
+ f"\n\nPlease translate the following text from {source} into {target}, "
"taking the provided background information into consideration."
+ "\n\n[Source Text]\n" + current_source
)
Streaming example
An actual greedy Q4_K_M run on an English-to-Chinese ASR trajectory:
| step | current_source |
previous_translation |
output |
|---|---|---|---|
| 1 | It's too small, |
(none) | 它太小了， |
| 2 | It's too small, won't go into supernova, |
它太小了， |
它太小了，不会变成超新星， |
| 3 | It's too small, won't go into supernova, besides I won't be around to see it. |
它太小了，不会变成超新星， |
它太小了，不会变成超新星，而且我也不会在身边看到它。 |
Very short fragments can intentionally produce an empty string when there is not enough information to translate safely. In that case, keep displaying the last non-empty translation or wait for the next ASR update.
Usage
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
BASE = "tencent/Hy-MT2-1.8B"
ADAPTER = "febilly/Hy-MT2-1.8B-StreamRevise-v4-LoRA"
tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
BASE, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, ADAPTER).eval()
def translate(prompt: str) -> str:
text = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
output = model.generate(
**inputs,
do_sample=False,
max_new_tokens=256,
eos_token_id=120020,
pad_token_id=tokenizer.pad_token_id,
)
return tokenizer.decode(
output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True
).strip()
prompt = render_prompt(
source_language="en",
target_language="zh",
current_source="It's too small, won't go into supernova.",
history=[("That star is faint.", "那颗恒星很暗。")],
previous_source="It's too small,",
previous_translation="它太小了，",
)
print(translate(prompt))
Decoding: use greedy
Set do_sample=False (equivalently temperature=0). Consecutive ASR updates produce nearly identical
prompts, so sampling creates unrelated output changes that appear as subtitle flicker. The evaluation below
uses greedy decoding.
Driving it as a stream
The model is stateless. The caller owns the current revision chain:
history = []
previous_source = ""
previous_translation = ""
for current_source in asr_updates():
output = translate(render_prompt(
source_language="ja",
target_language="zh",
current_source=current_source,
history=history,
previous_source=previous_source,
previous_translation=previous_translation,
))
if output:
display(output)
previous_translation = output
previous_source = current_source
if utterance_finished:
if previous_translation:
history.append((current_source, previous_translation))
history = history[-10:]
previous_source = ""
previous_translation = ""
Enable prefix KV caching when your inference backend supports it. Requests remain independently retryable and load-balanceable because no state is stored by the model.
Evaluation
The published Q4_K_M build was evaluated with greedy decoding. The gate combines three independent tracks: general translation quality, real streaming ASR behavior, and targeted regression probes.
FLORES-200 devtest
Six Chinese/English/Japanese directions, 1,012 examples per direction. The metric is corpus chrF with
word_order=0; the overall corpus score is 46.393.
| direction | chrF |
|---|---|
| Japanese → Chinese | 27.303 |
| Chinese → Japanese | 32.953 |
| English → Chinese | 38.202 |
| Chinese → English | 57.369 |
| Japanese → English | 54.671 |
| English → Japanese | 40.643 |
FLORES is English-pivoted, so absolute scores for Chinese↔Japanese include reference divergence. Use these numbers as reproducible test results, not as a direct ranking of language-pair difficulty.
Real streaming replay
500 utterance trajectories / 792 ASR states:
| metric | result |
|---|---|
| source-language leakage | 1.01% |
| true source-copy rate | 0.00% |
| empty-output rate for source fragments longer than 4 characters | 0.35% |
| punctuation-insensitive prefix retention | 0.950 |
| punctuation-insensitive characters erased per transition | 0.272 |
| punctuation-insensitive rewrite rate | 6.8% |
| final output/source length ratio, median | 0.795 |
The targeted no-copy, no-added-brackets, and word-sense probes passed 24/24 cases. Streaming metrics are application-oriented heuristics and are not directly comparable with simultaneous-MT paper benchmarks.
Training
| item | value |
|---|---|
| Base | tencent/Hy-MT2-1.8B (HunYuanDenseV1, 32 layers, hidden size 2048) |
| Method | LoRA, r=64, alpha=128, dropout=0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable parameters | 77,594,624 |
| Training set | 48,778 examples |
| Schedule | 1 epoch, 1,525 optimizer steps, per-device batch size 8, effective global batch size 32 |
| Optimizer schedule | peak learning rate 1e-4, warmup, cosine decay, seed 42 |
| Precision | bf16 |
Data construction
The training set contains 36,611 streaming-revision states and 12,167 standalone translation anchors, split nearly evenly across all six directions between Chinese, English, and Japanese.
Streaming examples are built as trajectories of realistic ASR states. About 80% of transitions are prefix extensions, with the rest covering punctuation changes and substantive re-recognition. The data includes punctuation-triggered updates, pause-triggered multi-character updates, and very short fragments. Recent context is represented by bilingual source/translation pairs in most contextual samples, with source-only history retained as a supported input form.
Target construction uses two stages: a final translation is produced independently for the complete source, then intermediate translations are generated for partial ASR states. A separate future-content leakage check regenerates suspect intermediate states without access to the final translation. Standalone anchors are generated independently without streaming background so ordinary one-shot translation remains supported.
The training dataset is not released.
Limitations
- Stability is a tendency, not a guarantee. A recognition correction can require rewriting the whole line.
- Very short fragments can return an empty string. Keep the prior non-empty subtitle until more source text arrives.
- Prompt format matters. Off-format prompts can reduce both translation quality and stability.
- Core language coverage is Chinese, English, and Japanese. Other directions were not trained or evaluated.
- Sentence-scoped revision. The model revises only the current utterance; finalized earlier subtitles are not revisited.
- Greedy decoding is assumed.
- Garbled or highly incomplete ASR text can still produce mistranslations or hallucinations.
- The adapter inherits the base model's biases and limitations.
License
Apache 2.0, the same license as tencent/Hy-MT2-1.8B.
- Downloads last month
- 9