Opt.Gear-270M

This repository contains model weights and configuration files for the post-trained (instruction-tuned) model in the Hugging Face Transformers format.

These artifacts are compatible with Hugging Face Transformers, llama.cpp, ExecuTorch, and vLLM. Executable binaries optimized for NPU inference (Qualcomm Hexagon NPU, Apple ANE) are also provided.

Opt.Gear-270M targets lower-latency deployment on more constrained mobile and edge devices. For a stronger quality-efficiency trade-off at the same context length, see Opt.Gear-1B.

Opt.Gear is the first generation of OptAI Foundation Models, designed for efficient on-device deployment, real-time inference, and strong task capability. Rather than simply shrinking a server-scale LLM, Gear jointly optimizes downstream quality, device-side latency, memory efficiency, long-context capability, and practical deployability under hardware constraints such as limited SRAM, memory bandwidth, and power.

Opt.Gear-270M shares the same hybrid design and 64K context support as Opt.Gear-1B, while targeting lower-latency deployment on smaller mobile and edge environments.

Opt.Gear Highlights

• Data-efficient training: Trained on a curated 0.5T-token subset selected from a 2T-token candidate corpus, without knowledge distillation from a teacher model. Against the distilled Gemma3-270M (6T tokens), Opt.Gear-270M achieves competitive or superior scores with 12× fewer training tokens.

• Hybrid architecture with ConvKV-Gated Mixer: A small set of global GQA layers handles long-range routing, while ConvKV-Gated Mixer blocks replace many local attention layers with softmax-free, matrix-multiplication-free local mixing. The persistent local state scales with the convolution kernel (L_conv = 3) instead of the sliding window (W = 512), substantially reducing live decoding state and memory bandwidth.

• NPU-friendly by design: Dynamic QKV matrix multiplications and softmax normalization are replaced with static linear, convolution, and element-wise operations, making the model easy to map onto CPUs, GPUs, and NPUs.

• Korean-English bilingual: Built on the KORMo tokenizer (125,184 vocab) with Korean-English pretraining, leading same-scale baselines on Korean benchmarks (KMMLU, KoBEST, HAERAE).

• 64K context on device: Hybrid attention layout with separate global/local RoPE frequencies supports long-context modeling up to 65,536 tokens while keeping the local cache footprint small.

For more details, please refer to our tech report and blog post.

Model Overview

• Type: Causal Language Model (hybrid attention + convolutional mixer)
• Training Stage: Pre-training (0.5T tokens) → Long-context extension (4K → 32K → 64K) → Two-stage SFT (general instruction → reasoning-oriented)
• Architecture
  • Number of Parameters: 270M
  • Hidden Dimension: 640
  • Number of Layers: 18
  • Hidden Layout: hybrid of Global GQA, Local (sliding-window) Attention, and ConvKV-Gated Mixer
  • Grouped-Query Attention:
    • Number of Attention Heads: 4 for Q and 1 for KV
    • Head Dimension: 256
    • Sliding-Window Size (local attention): 512
    • QK-Normalization: QK-LN
  • ConvKV-Gated Mixer:
    • Causal depthwise 1D convolution on key/value streams
    • Convolution Kernel Size: 3 (fixed-size persistent state, independent of context length)
  • Feed-Forward Network:
    • Type: GeGLU (gated dense MLP)
    • Intermediate Dimension: 2,048
  • Rotary Position Embedding: global theta 1,000,000 / local theta 10,000
  • Tokenizer: KORMo (byte-level BPE), vocabulary 125,184 (shared with Opt.Gear-1B)
  • Word Embedding: untied (separate input embedding and LM head)
• Context Length: 65,536 natively

Base and Instruction models share the same tokenizer, but use different end-of-generation tokens: the Base model emits <EOS> while the Instruction model emits <EOT>. When fine-tuning, make sure training examples terminate with the same convention used by the target runtime — malformed termination wastes decoding steps and increases latency on device.

Benchmark Results

All results are obtained using the LM Evaluation Harness and may differ from scores reported elsewhere. Perplexity-based evaluation is adopted for HellaSwag, PIQA, WinoGrande, MMLU, GPQA, ARC, KMMLU, and KoBEST.

	Opt.Gear-270M	SmolLM2-135M	Gemma3-270M	LFM2.5-350M	Qwen3-0.6B
# Trained Tokens	0.5T	2T	6T	28T	36T
Distilled	✗	✗	✓	✓	✓
English
MMLU	25.9	25.3	26.5	41.0	47.3
Korean
KMMLU	30.0	29.1	28.0	31.4	35.0
KoBEST	51.9	48.7	50.0	51.1	54.4
CLIcK	27.2	22.5	27.3	24.0	36.7
HAERAE	21.5	18.9	20.4	24.8	37.0

* All results are obtained using LM Evaluation Harness and may differ from other reported scores.
* While baselines are trained on 2T–36T tokens and frequently leverage knowledge distillation, Opt.Gear-270M is trained on only 0.5T tokens without any distillation. Against the similarly-sized distilled Gemma3-270M, Opt.Gear-270M leads on reasoning (PIQA 67.3 vs 66.9, ARC-Easy 59.1 vs 56.1) and Korean benchmarks (KMMLU 30.0 vs 28.0, KoBEST 51.9 vs 50.0, HAERAE 21.5 vs 20.4).

On-Device Inference

Opt.Gear-270M shares the same NPU-friendly hybrid architecture as Opt.Gear-1B, which achieves up to ×4.9 faster prefill and decoding on NPUs compared to models of similar scale (measured with Qualcomm AI Runtime and CoreML — see the Opt.Gear-1B card and the tech report for detailed device benchmarks).

Quickstart

Hugging Face Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OptGear/Opt.Gear-270M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [
    {"role": "user", "content": "온디바이스 AI가 왜 중요한지 설명해줘."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

On-Device Deployment

Executable binaries optimized for NPU inference are provided for:

• Qualcomm Hexagon NPU — via Qualcomm AI Engine Direct (QAIRT); recommended for Snapdragon devices
• Apple ANE — via CoreML Runtime
• ExecuTorch — for mobile/edge PyTorch deployment

Best Practices

1. Termination tokens: The Instruction model terminates generation with <EOT>, while the Base model uses <EOS>. When fine-tuning either variant, ensure training examples terminate with the matching token.

2. Runtime selection: Snapdragon NPU → QAIRT; Apple devices → CoreML; CPU/GPU → llama.cpp. Avoid the llama.cpp NPU backend for models with sliding-window attention.

3. Languages: The model is trained primarily on English (92%) and Korean (6%) with mathematical text (~2%). It is best suited for Korean-English bilingual understanding, summarization, rewriting, and instruction following on latency-sensitive devices.

4. Not intended for code generation: The pre-training corpus deliberately excludes code data — models at the 270M–1B scale are not typically used for code generation tasks.

5. Long context: The model natively supports 65,536 tokens with a fixed-size local cache, making long-context decoding memory-efficient even at this scale.

Limitations

The constrained 0.5T-token training budget limits capabilities in complex reasoning and mathematical tasks that typically benefit from larger training corpora. At the 270M scale, knowledge-heavy benchmarks (e.g., MMLU) remain challenging; the model is best used for well-scoped generation and understanding tasks rather than open-domain knowledge QA. See the tech report for a detailed discussion.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{optgear2026,
    title  = {{Opt-Gear} Technical Report},
    author = {{Opt.Gear Team}},
    year   = {2026},
    url    = {https://huggingface.co/OptGear}
}

---

Correspondence: contact@opt-ai.kr · Hugging Face: huggingface.co/OptAI