Refract-Labs/Orion-Flagship-Mini

🔺 Orion T2 Mini SFT

Orion T2 Mini SFT is the supervised-fine-tuned assistant variant of Project Prism's experimental Orion T2 architecture.

It starts from the completed Orion T2 Mini base checkpoint and was then trained with response-only SFT on 700,000 examples: 60% general/conversation, 20% math, and 20% code.

Chat format

Use the bundled Hugging Face chat template. The model was trained with literal text role markers <|system|>, <|user|>, and <|assistant|>. Those markers were not added as new tokenizer special tokens.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "Project-Prism/Orion-Flagship-Mini-SFT"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="auto"
)
messages = [
    {"role": "system", "content": "You are Orion, a helpful AI assistant."},
    {"role": "user", "content": "Explain why the sky is blue."},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=128, temperature=0.7, top_p=0.95, do_sample=True, use_cache=False)
new_tokens = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

Architecture

Orion T2 Mini has approximately 219M total parameters and combines a 12-stage Universal Cortex, shared sparse Procedure Banks, a sparse Knowledge Vault, explicit Working State, and recurrent late-stage deliberation.

The current Transformers compatibility layer uses use_cache=False, so generation recomputes the prefix each token.

Status

Experimental Project Prism research model. Architecture claims should be judged with matched evaluations rather than assumed from the design alone.