JhCIFT/CIRF-Qwen3-4B-Full

CIRF Qwen3 4B: Full

Standalone BF16 parameters with LoRA merged and all 514 learned input/output token rows preserved. No PEFT adapter or original base model is needed for inference. Base revision: 906bfd4b4dc7f14ee4320094d8b41684abff8539. Parameters: 4,023,100,416.

Training

This release follows the published Qwen3-8B manual recipe on the same annotated coinflip, CommonsenseQA, GSM8K, MultiArith and SVAMP mixture and fixed 5% validation split (seed 0). Stage I uses 2 epochs, LoRA rank 64/alpha 128/dropout .05, LoRA LR 3e-4, new-token LR 1e-4, batch size 32, maximum length 768, warmup .03. Weight decay .1 applies only to LoRA; new-token rows use zero decay. Results are retained in full during Stage I. Plain completion prompts are used, without a chat template.

The same 512 code assignments are reused. Their decoded 4096-dimensional initialization vectors are mapped to hidden width 2560 through an SVD-span isometry with a seeded orthonormal basis. This preserves their Gram matrix within numerical tolerance; it does not assert alignment with the base model's neurons. Exact settings and input/code hashes appear in the accompanying JSON files. The inherited codebook was fitted to the original annotated mixture before the LM validation split, matching the 8B recipe; these are internal validation metrics.

This is the full-result Stage I model.

Budgets 18 and 6 constrain result values in training targets, not model weights, functional-token counts, or total inference length.

Inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "JhCIFT/CIRF-Qwen3-4B-Full"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto").eval()
x = tok("What is 2 + 2?[sof]", add_special_tokens=False, return_tensors="pt").to(model.device)
with torch.inference_mode():
    y = model.generate(**x, max_new_tokens=384, do_sample=False,
                       eos_token_id=tok.eos_token_id, pad_token_id=tok.pad_token_id)
trace = tok.decode(y[0, x.input_ids.shape[1]:], skip_special_tokens=False)
print(trace)

Keep [eof]: the final answer follows it. Stop at the tokenizer EOS. The included inference.py reproduces the evaluation cap of 32 generated functional codes. Transformers 5.17.0 was used for export verification.

Internal validation

Greedy question-only generation, batch size 32, up to 384 new tokens and 32 functional codes. These are the fixed internal validation split, not official test scores, and contain no uncertainty estimate over training seeds.

Source	Questions	Accuracy
coinflip	251	100.00%
commonsenseqa	396	87.37%
gsm8k	367	76.29%
multiarith	20	100.00%
svamp	34	94.12%
all	1068	86.99%

Coinflip training contains only two-action, initially heads-up questions querying heads; longer sequences and other query orientations require separate evaluation. The annotation granularity varies, so code count is not a guaranteed count of actions.

Integrity and license

LoRA was safely merged. A standalone reload exactly reproduced merged-model logits on two probe traces, and every learned input/output row was preserved exactly. The generation smoke check is recorded honestly in export_verification.json; export integrity is distinct from benchmark accuracy. File hashes are published. Apache 2.0, inherited from the Qwen base model; license text included.