Prience91/Climate-1000-Qwen-4b-Adapter

GIIS Mini Qwen3.5-4B LoRA adapter

Best-validation adapter from completed V17-Turbo climate SFT. This repository contains adapter weights and tokenizer/chat template, not merged base weights.

Provenance and selection

• Base: Qwen/Qwen3.5-4B, pinned revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.
• Training: unquantized frozen BF16 base, BF16 mixed precision, FP32 adapter storage.
• LoRA: all supported linear layers, rank 32, alpha 64, dropout 0.05, bias none.
• Dataset: 3,665 training and 406 validation examples, completion-only loss.
• Learning rate 1e-4; two epochs; microbatch 1; accumulation 16; seed 3407.
• Maximum sequence length 8,192; no packing; thinking disabled.
• Completed 460 optimizer steps on 22 October 2026. Selected checkpoint 230, epoch 1, validation loss 0.37746524810791016.
• Final step-460 validation loss: 0.38062456250190735.
• Training resumed from checkpoint 120 after a memory safety stop. Periodic CUDA cache clearing and memory telemetry enabled; optimization settings unchanged.
• The 100-paper benchmark remains provisional (98 authored/reviewed cases, 78 approved). Training validation does not establish downstream scientific accuracy.

Loading

Use transformers.AutoModelForCausalLM to load the text-generation base, followed by peft.PeftModel.from_pretrained. This matches the training architecture Qwen3_5ForCausalLM; adapter tensor names target the text model. Use compatible Transformers/PEFT versions (training used Transformers 5.15.1 and PEFT 0.19.1). Apply the included chat template with enable_thinking=False.

No merged-model inference evaluation is claimed. See checkpoint_manifest.json for pinned identities, data hashes and file checksums.