CIRF-Qwen3-8B-Full

Full-result model. The manually trained Stage I checkpoint, retaining full explicit result supervision.

This is a standalone BF16 CIRF model derived from Qwen3-8B-Base. LoRA adapters have been merged into the dense model parameters. The trained input embeddings and output-head rows for all 514 new tokens are included: [sof], [eof], and [f_0] through [f_511]. Load it with standard Transformers; no PEFT adapter, original base checkpoint, CIRF checkout, teacher model, or VQ encoder is required for inference.

Checkpoint identity

• Source manual checkpoint: functional-qwen3-8b.

• Model parameters after merging: 8,192,758,784.

• Functional vocabulary: 512 codes.

• Original LoRA rank / alpha: 64 / 128.

• Stage I: 2 epochs; LoRA LR 0.0003; new-token LR 0.0001; batch size 32; gradient accumulation 1; maximum training length 768; seed 0; validation fraction .05.

• LoRA weight decay: .1; new-token embedding/head weight decay: 0.

• Exact training settings, checkpoint hashes and export checks are included in training_config.json, provenance.json, and export_verification.json.

These are the manual model family, not the hyperparameter-search checkpoints. The full model and the two compressed models are available at:

• Full
• Backward greedy 18
• Backward greedy 6

Input and output

Use the raw question followed immediately by [sof], without a chat template or an added BOS token. The generated trace contains functional codes and optional result values, then [eof] followed by the final answer. Stop generation at the normal tokenizer EOS token, not at [eof]: the answer comes after [eof]. Decode with skip_special_tokens=False to preserve the functional boundaries.

The 18/6-token budgets constrain explicit result values in training targets. They do not prune model weights, limit the number of functional codes, or impose an inference-time cap on total generated tokens. Backward greedy may prune additional nonpositive-loss result units after satisfying its token budget.

Quick start

pip install torch transformers accelerate safetensors

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "JhCIFT/CIRF-Qwen3-8B-Full"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id, dtype=torch.bfloat16, device_map="auto"
)
model.eval()
inputs = tokenizer(
    "What is 2 + 2?[sof]", add_special_tokens=False, return_tensors="pt"
).to(model.device)
with torch.inference_mode():
    output = model.generate(
        **inputs, max_new_tokens=384, do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )
ids = output[0, inputs.input_ids.shape[1]:].tolist()
if tokenizer.eos_token_id in ids:
    ids = ids[:ids.index(tokenizer.eos_token_id)]
trace = tokenizer.decode(ids, skip_special_tokens=False)
answer = trace.split("[eof]", 1)[1].strip() if "[eof]" in trace else None
print(trace)
print(answer)

The minimal example uses ordinary generation. To reproduce the CIRF evaluation limit of at most 32 generated functional codes, download the included inference.py and run:

python inference.py --model JhCIFT/CIRF-Qwen3-8B-Full --question "What is 2 + 2?"

That script masks further functional codes after the limit while allowing result values, [eof], and the answer. It requires only Torch and Transformers. Qwen3 requires Transformers 4.51 or later; this export and standalone reload were verified with Transformers 5.17.0.

Saved internal validation results

The following metrics are copied from the manual run's existing 1,068-example validation evaluation. They were not rerun during publication and are distinct from the search's 876-example screen and 2,138-example full validation set. They should not be interpreted as an independent official benchmark test.

Source	Examples	Answer accuracy
coinflip	251	100.00%
commonsenseqa	396	89.39%
gsm8k	367	80.38%
multiarith	20	100.00%
svamp	34	94.12%

• Equal-weight five-source macro accuracy: 92.78%.
• Example-weighted accuracy: 89.14%.
• Mean generated length in the saved evaluation: 28.92 tokens.

The training mixture contains coin-flip, CommonsenseQA, GSM8K, MultiArith and SVAMP reasoning data. These models are specialized for CIRF-formatted reasoning; longer action sequences and tasks outside that mixture require separate evaluation.

Export verification

Before publication, the export was safely merged, serialized as safetensors, and reloaded with AutoModelForCausalLM without adapters. All learned new-token input/output rows were preserved exactly, reloaded logits exactly matched the in-memory merged model on two probe traces, and a short greedy-generation check returned the correct answer to 2 + 2. No adapter tensors remain in the shards. These checks validate export integrity; they are not a new benchmark evaluation. checksums.json lists SHA-256 hashes for the published files.

License and attribution

The base model is Qwen/Qwen3-8B-Base, released by the Qwen team under Apache 2.0. This derivative retains that license; the license text is included in LICENSE. Changes comprise CIRF functional-token vocabulary expansion, learned embedding/output rows, supervised adaptation, and merging the learned LoRA updates into the base parameters.