OceanLabs/Ocean-3.7B-agentic

Ocean-Horizon-3.7B

OceanLabs / Ocean-Horizon-3.7B is a high-performance 3.7B-parameter dense decoder-only language model optimized for advanced reasoning, coding, agentic workflows, and ultra-long context understanding.

Built upon a rigorously engineered dense architecture with a native 524,288-token (512K) context window, Ocean-Horizon-3.7B delivers frontier-level capabilities in a compact, efficient package.

Key Capabilities

• Exceptional small-model performance. Strong results across agentic, coding, mathematical, and scientific reasoning benchmarks.
• Native 512K context. Full 524,288-token context window available from mid-training stages onward, enabling deep document analysis, long-horizon planning, and complex multi-turn interactions.
• Optimized reasoning depth. Supports configurable reasoning effort (high / medium / low) with dedicated thinking tokens for transparent chain-of-thought.
• Production-ready tool use. Native support for structured tool calling with multiple presentation formats (JSON, XML, Markdown).
• Fully open. Architecture, configuration, and evaluation resources are openly available for research and commercial use under Apache 2.0.

Architecture Summary

Parameter	Value
Parameters	3.7B
Architecture	Dense
Hidden size	2560
Intermediate size	10240
Layers	36
Attention heads	32
Key-value heads	8 (GQA)
Head dimension	128
Vocabulary size	250,624
Context length	524,288
Activation	SiLU
Precision	bfloat16
RoPE θ	10,000,000

Recommended Inference Settings

• Reasoning effort: always prefer "high" for maximum quality.
• Sampling: temperature=1.0, top_p=0.95.
• Maximum output tokens: at least 32,768 to avoid truncating reasoning traces.
• Serving backends: Validated with vLLM and SGLang (BF16, FlashAttention-3 recommended).

Example with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OceanLabs/Ocean-Horizon-3.7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype="bfloat16",
    low_cpu_mem_usage=True,
    trust_remote_code=True
)

inputs = tokenizer("Explain the advantages of a 512K context window for agentic systems.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example with OpenAI-compatible API (vLLM / SGLang)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="OceanLabs/Ocean-Horizon-3.7B",
    messages=[{"role": "user", "content": "Solve this step by step."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Training Pipeline Overview

Ocean-Horizon-3.7B follows a multi-stage curriculum:

1. Pretraining – large-scale general knowledge acquisition (22.9T tokens).
2. Midtraining – progressive context extension (32K → 128K → 512K) plus agentic and reasoning data.
3. Reinforcement Learning – specialized experts for mathematics, code, and STEM-code, followed by model merging.
4. Supervised Fine-Tuning – high-quality multi-domain instruction tuning with learning-rate decay.

This staged approach yields strong generalization while preserving long-context fidelity.

Best Practices

1. Always request high reasoning effort for evaluation and complex tasks.
2. Allocate sufficient output length (≥ 32k tokens) so that reasoning is never truncated.
3. Use the native k2_horizon reasoning and tool-call parsers when serving with vLLM or SGLang.
4. Pin a specific revision or commit hash for reproducible deployments.

License

Apache License 2.0

Citation

@misc{oceanhorizon2026,
  title  = {Ocean-Horizon-3.7B: Compact Dense Model with Frontier Reasoning and 512K Context},
  author = {{OceanLabs}},
  year   = {2026},
  url    = {https://huggingface.co/OceanLabs/Ocean-Horizon-3.7B},
}

---

OceanLabs – Advancing open, high-capability language models.