laelhalawani/Seed-X-PPO-7B-EN-PL-AWQ-GGUF

Seed-X-PPO-7B — GGUF quantized variants

This is an AWQ GGUF conversion

This is an experimental conversion that uses AWQ to apply scaling and export in fp16, then pack it and quantize to GGUF. The AWQ calibration was done only using EN<-->PL pairs, 100 calibration samples each way, 200 total. It has not been tested nor evaluated yet. Depending on the outcomes:

• If AWQ turns helpful, I will try to add LoRA PEFT for the same languages to improve quality further, merge, then run AWQ, then release other language pairs/groups
• If AWQ doesn't provide improvements, or turns otherwise problematic I will replace AWQ with LoRA PEFT for the same languages, merge, then redo GGUF quants, and if that works I'll release for other language pairs/groups.

Important note on quality

This model, by the author's recommendation, should use beam search sampling, which isn't by default supported in llama.cpp and requires a custom sampling implementation. In compression to GGUF (8-2bit) model with knowledge already priorly condensed that much will really take a big hit in quality, especially if you use it with greedy sampling, so don't expect great results with it, without putting some effort into it. I'm releasing it for reasearch and further improvement purposes. Do expect giberish outputs at lower quants especially with greedy sampling. I will provide beam search implementation for llama-cpp-python once I had a chance to test there's even a point working with GGUF quants.

NOTE: If you'd like to quantize it yourself, you'll notice some necessary files (i.e. tokenizer_config.json) are missing from the official repo, you can still grab them from the official quants. ;)

Help out

If you want to help out with bringing this model in GGUF and proper performance to the community, you can help me get some more credits on RunPod.io All you have to do is sign up to RunPod.io using my link and deposit 10usd to your account. You'll get all your money usable for compute + a bonus, and I will receive a bonus of 5-500usd too. Runpod.io my ref link

Original model: ByteDance-Seed/Seed-X-PPO-7B

This upload contains Seed-X-PPO-7B (Mistral-based multilingual translation across 28 languages: Arabic (ar), Czech (cs), Danish (da), German (de), English (en), Spanish (es), Finnish (fi), French (fr), Croatian (hr), Hungarian (hu), Indonesian (id), Italian (it), Japanese (ja), Korean (ko), Malay (ms), Norwegian Bokmål (nb), Dutch (nl), Norwegian (no), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Swedish (sv), Thai (th), Turkish (tr), Ukrainian (uk), Vietnamese (vi), and Chinese (zh)), converted to GGUF.

Files & sizes

Made with llama.cpp.

File	Size (MB)
Seed-X-PPO-7B.Q2_K.gguf	2.88GB
Seed-X-PPO-7B.Q4_K_M.gguf	4.56G
Seed-X-PPO-7B.Q8_0.gguf	7.99G

Prompt format

Translate the following English sentence into Polish:
May the force be with you <pl>

Notice

• Add a target-language tag at the end of the prompt (e.g., , ). This is required because PPO training used language tags.
• No chat template: do not call tokenizer.apply_chat_template and avoid multi-turn chat formatting.
• The model is specialized for multilingual translation.

License

OpenMDW — see the upstream repository for details.

Citation

@misc{cheng2025seedxbuildingstrongmultilingual, title={Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters}, author={Shanbo Cheng and Yu Bao and Qian Cao and Luyang Huang and Liyan Kang and Zhicheng Liu and Yu Lu and Wenhao Zhu and Jingwen Chen and Zhichao Huang and Tao Li and Yifu Li and Huiying Lin and Sitong Liu and Ningxin Peng and Shuaijie She and Lu Xu and Nuo Xu and Sen Yang and Runsheng Yu and Yiming Yu and Liehao Zou and Hang Li and Lu Lu and Yuxuan Wang and Yonghui Wu}, year={2025}, eprint={2507.13618}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2507.13618} }