Noesis-0.6B-SFT

Noesis is a 0.6B text model fully fine-tuned from Qwen3-0.6B-Base on reasoning traces paired with final answers. It uses ChatML and can generate a <think>…</think> block before the answer. The weights are exported in BF16. Base model

Quick start

Install PyTorch for your device and run pip install "transformers>=4.57.1" accelerate.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Ma7ee7/Noesis-0.6B-SFT"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
).eval()

messages = [
    {"role": "user", "content": "A bat and ball cost $1.10 together. "
     "The bat costs $1 more than the ball. How much does the ball cost?"}
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

prompt_length = inputs["input_ids"].shape[-1]
remaining = 4096 - prompt_length
if remaining <= 0:
    raise ValueError("Shorten the prompt to leave room for generation.")

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=min(2048, remaining))

text = tokenizer.decode(output[0, prompt_length:], skip_special_tokens=False)
text = text.removesuffix("<|im_end|>").removesuffix("<|endoftext|>")
print(text)

The published generation config defaults to temperature 0.6, top-p 0.95, top-k 20, and up to 2,048 new tokens. It stops on <|im_end|> or <|endoftext|>. The chat template accepts text system, user, and assistant turns. It omits completed think blocks from earlier assistant turns when building the next prompt. Generation config · Chat template

Training data

Source	Eligible rows in the training pool
Claude Reasoning 4096	126,214
Polyglot Thoughts SFT Collection	16,000
Total	142,214

The Polyglot subset includes English, Spanish, German, French, Italian, and Japanese prompts. The sampler scanned 30,873 candidates and excluded 14,018 for length. Eligibility was checked after formatting and tokenization, with the prompt, reasoning, answer, and chat markers all counted toward the 4,096-token limit. Oversized rows were discarded whole. The training pool was shuffled, and the step budget covered a partial pass through it. Run manifest

Training setup

Setting	Value
Method	Full supervised fine-tuning
Hardware	NVIDIA L4, Google Colab
Precision	BF16 mixed precision with FP32 training weights
Training sequence limit	4,096 tokens
Optimizer step budget	2,000
Effective batch	16 examples (microbatch 1 × 16 accumulation steps)
Optimizer and schedule	AdamW, cosine decay
Learning rate and warmup	1e-5, 50 steps
Seed	42

The loss covers the final assistant response, including the think tags, reasoning text, answer, and turn terminator. Prompt and padding tokens are masked. The export uses the checkpoint selected by held-out validation loss. Run manifest

Evaluation

Checkpoint	Held-out assistant cross-entropy ↓
Qwen3-0.6B-Base before fine-tuning	1.8666
Noesis, best saved checkpoint	1.2589

Both scores use the same 256 held-out Claude Reasoning examples. The loss measures how well the model predicts the reference assistant text. Separate reasoning accuracy, multilingual performance, and longer-context behavior have yet to be measured here. Validation metrics

Generated reasoning can contain errors even when the final answer is correct. In one manual bat-and-ball check, Noesis answered 5¢ correctly but proposed the wrong subtraction in its think block.

Licenses and provenance

Qwen3-0.6B-Base is published under Apache 2.0. The Polyglot collection lists mixed source terms, and the selected subset includes 1,658 examples from the source labeled CC BY-NC 4.0 in its dataset card. See the upstream cards and the pinned source revisions for attribution and terms. Qwen base model · Claude Reasoning data · Polyglot source terms · Run manifest