Rosetta-7B-Base

Introduction

Rosetta-7B-Base is a 7B-parameter bilingual (Korean-English) foundation model developed by PoSTMEDIA on the Rosetta dense decoder-only architecture. It was pretrained on trillions of tokens of curated bilingual text and further strengthened for Korean through a dedicated continual-pretraining stage on curated Korean corpora and in-house synthetic Korean data assets, with the vocabulary extended to 161K entries for efficient Korean tokenization.

Rosetta-7B-Base is the foundation of the Rosetta-7B family:

Model	Download	Note
Rosetta-7B-Base	HuggingFace	Foundation model (this model)
Rosetta-7B-Instruct	HuggingFace	Instruction following / chat
Rosetta-7B-Think	HuggingFace	Explicit reasoning (<think>)

Highlights

• Full-pipeline foundation — pretraining, staged mid-training, long-context extension, and Korean continual pretraining in a single lineage
• Korean-extended 161K vocabulary — substantially fewer tokens per Korean sentence, lowering serving cost
• 65,536-token context window with interleaved local-global attention
• Research-friendly — a clean base for SFT, RLHF/DPO, and domain adaptation experiments in Korean
• Apache-2.0 — unrestricted commercial use

Model Summary

Architecture	Rosetta dense decoder-only Transformer (RosettaForCausalLM)
Parameters	7B
Layers	32
Hidden size	4,096
Attention heads	32
Attention pattern	interleaved sliding-window (4,096) + global, 3:1, with QK-normalization
Context length	65,536
Vocabulary	161,425 (Korean-extended)
Type	base — no chat template, completion-style
License	Apache-2.0

Training Overview

1. Pretraining on trillions of tokens of curated bilingual web, code, and academic text
2. Staged mid-training enriching reasoning-dense data, followed by long-context extension up to 65K
3. Korean continual pretraining on curated Korean corpora plus in-house synthetic Korean data assets, with the vocabulary extended to 161K

No instruction tuning or preference optimization has been applied — this is a raw foundation model intended for completion-style use, fine-tuning, and research.

Quickstart

Transformers

Requires transformers>=5.13 and trust_remote_code=True.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "PoSTMEDIA/Rosetta-7B-Base"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, dtype="bfloat16", device_map="auto", trust_remote_code=True
)

ids = tokenizer("대한민국의 수도는", return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=64)
print(tokenizer.decode(out[0], skip_special_tokens=True))

vLLM

VLLM_USE_PRECOMPILED=1 pip install git+https://github.com/PoSTMEDIA-AI/vllm@rosetta-v0.26.0

vllm serve PoSTMEDIA/Rosetta-7B-Base --dtype bfloat16

Evaluation Results

Standardized base-suite results will be added in an upcoming update. For downstream capabilities, see the evaluation tables of Rosetta-7B-Instruct and Rosetta-7B-Think.

Limitations

• As a base model, it does not follow instructions or chat formats; use the Instruct/Think variants for assistant-style behavior.
• The model can generate factually incorrect or biased content reflecting web-scale training data.
• Optimized for Korean and English; other languages are not guaranteed.

License

Apache License 2.0 — see LICENSE. If you build something with Rosetta, we'd appreciate a "Built with Rosetta" attribution.

Citation

@misc{rosetta2026,
  title  = {Rosetta-7B: A Bilingual Korean-English Language Model Family},
  author = {{PoSTMEDIA AI Lab}},
  year   = {2026},
  url    = {https://huggingface.co/collections/PoSTMEDIA/rosetta-6a9db30fd1b4585b0c1845e9}
}

Contact

Questions and feedback — please open a discussion on the model page.