MGTV-AI/Mango-MT-9B

Mango-MT: Advancing Audiovisual Translation with a 9B Multilingual Model and Expert-Annotated Benchmark

Introduction

Global long-form video expansion requires robust multilingual subtitle translation, but generic MT fails at fragmented lines, timeline constraints and plot context for mass production. We open-sourced Mango-MT with 9B parameter size, an 11-language audiovisual subtitle translator paired with the benchmark called Mango-SubBench. Evaluations across all languages show it outperforms GPT, Gemini and DeepSeek with steady timeline compliance, consistent semantics and industrial scalability, see the technical report and GitHub for details. Our system holds four key advantages over prior work:

• Scenario-oriented & Multilingual Optimization: Professionally optimized for 11 different languages, perfectly adapting to complex industrial rules of video subtitle translation.
• Context-aware Translation Mechanism: Leverages global context modeling to stabilize plot logic and consistent character appellation translation.
• Structural Robustness: Rigidly retains original subtitle numbers and timestamps to ensure stable batch translation delivery.
• Industrial-grade evaluation benchmark : Unlike single-metric schemes, our benchmark integrates a full framework and high-quality dataset, evaluating subtitles against real delivery standards for reliable all-round quality assessment.

Installation

conda create -n mlt python==3.11
conda activate mlt
pip install -r requirements.txt

Usage

SGLang Server Usage

SGLang is a fast serving framework for large language models and vision language models. Please use the following command in a fresh environment:

uv pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=python&egg=sglang[all]'

See its documentation for more details.

The following will create API endpoints at http://localhost:8000/v1:

python -m sglang.launch_server --model-path model_dir --port 8000 --tp-size 1 --mem-fraction-static 0.8 --context-length 4096 --reasoning-parser qwen3

vLLM Server Usage

vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. Please use the following command in a fresh environment:

uv pip install vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly

See its documentation for more details.

For detailed usage guide, see the vLLM Qwen3.5 recipe.

The following will create API endpoints at http://localhost:8000/v1:

vllm serve model_dir --port 8000 --tensor-parallel-size 1 --max-model-len 4096 --reasoning-parser qwen3 --language-model-only

Performance

Mango-SubBench

Mango-SubBench is a novel benchmark for audiovisual subtitle translation. We evaluate our Mango-MT against three commercial large models (DeepSeek-v4-pro, Gemini-3-pro, GPT-5.4) across 11 languages with five core translation metrics: Semantic similarity, BLEU-2, COMET On translation benchmark, Mango-MT outperforms Gemini, DeepSeek and GPT across all 11 evaluated languages. This demonstrates that our model delivers powerful multilingual translation capabilities for film and drama content, and possesses significant commercial deployment value.

Evaluation Results on S(overall score) are:

Models	Malay	Thai	English	Indonesian	Vietnamese	Russian	French	Japanese	Korean	Spanish	Arabic
DeepSeek-V4-Pro	0.76	0.80	0.85	0.75	0.85	0.76	0.79	0.80	0.70	0.82	0.78
Gemini-3-Pro	0.78	0.88	0.86	0.79	0.87	0.79	0.82	0.82	0.73	0.83	0.78
GPT-5.4	0.75	0.83	0.83	0.76	0.84	0.77	0.79	0.81	0.71	0.81	0.73
Mango-MT	0.86	0.91	0.90	0.93	0.94	0.89	0.90	0.87	0.81	0.92	0.81

FLORES+

We evaluate the multilingual translation performance of our model on FLORES+ which is based on FLORES-200. This dataset was originally released by FAIR researchers at Meta under the name FLORES. The data is now being managed by OLDI, the Open Language Data Initiative. The + has been added to the name to disambiguate between the original datasets and this new actively developed version. For newer versions of this dataset, Please see FLORES+ HuggingFace repo . The data consists of translations primarily from English into over 200 language varieties. The original English sentences were sampled in equal amounts from Wikinews (an international news source), Wikijunior (a collection of age-appropriate non-fiction books), and Wikivoyage (a travel guide).

Models	Metric	Russian	Indonesian	Japanese	French	Thai	English	Spanish	Vietnamese	Arabic	Korean	Malay	Avg
DeepSeek-V4-Pro	BLEU	21.110	27.800	29.680	31.150	10.400	33.760	21.100	32.550	16.120	24.910	23.430	24.730
DeepSeek-V4-Pro	COMET	0.945	0.940	0.925	0.921	0.894	0.977	0.945	0.921	0.895	0.914	0.908	0.926
Gemini-3-Pro	BLEU	22.840	28.850	31.640	32.840	9.480	34.090	22.410	32.730	17.300	25.220	24.090	25.590
Gemini-3-Pro	COMET	0.948	0.943	0.930	0.924	0.903	0.977	0.948	0.930	0.903	0.919	0.914	0.931
GPT-5.4	BLEU	20.260	24.980	27.710	28.070	8.380	30.990	21.390	30.080	13.240	23.110	21.120	22.670
GPT-5.4	COMET	0.946	0.945	0.928	0.925	0.904	0.978	0.950	0.928	0.895	0.920	0.919	0.931
Mango-MT	BLEU	20.370	22.060	27.350	32.980	10.620	30.110	21.640	30.060	10.450	20.440	20.040	22.370
Mango-MT	COMET	0.944	0.932	0.923	0.932	0.905	0.978	0.946	0.925	0.870	0.908	0.911	0.925