jetson-flat-cache-cpt-adapter

LoRA adapter from the Continual Pre-Training (CPT) stabilization pass in Phase 2 of the llm_distillery project. Trained on top of a Qwen2.5-0.5B model whose attention layers were surgically remapped to an alternating Sliding-Window Attention (SWA) / Gated Linear Attention (GLA) hybrid, to stabilize the newly-introduced GLA decay parameters and let the model adapt to its new attention mechanism.

• Base model (pre-surgery): mohith-das/jetson-assistant-0.5b
• 200 steps, LoRA r=8/alpha=16 on q_proj,k_proj,v_proj,o_proj, plus the GLA layers' log_decay parameters unfrozen
• Trained on Jetson Orin Nano Super (JetPack 7.2, CUDA 13.2)

See the merged full-precision model at mohith-das/jetson-hybrid-flat-cache-0.5b for the ready-to-use checkpoint with this adapter already folded in.