FrostiSteele/eden-4b-distilled-lora

Eden 4B Distilled LoRA

LoRA adapter distilled from Qwen3.5-9B-Instruct (20,000 steps, KL divergence, OpenOrca). Designed to be used with Bartowski's Qwen3.5-4B Q4_K_M base model.

Usage

# Download base
wget https://huggingface.co/bartowski/Qwen_Qwen3.5-4B-GGUF/resolve/main/Qwen_Qwen3.5-4B-Q4_K_M.gguf

# Run with eden.cpp
./eden-server -m Qwen_Qwen3.5-4B-Q4_K_M.gguf --lora eden-4b-distilled-lora.gguf --port 9094

Training

• Teacher: Qwen3.5-9B-Instruct (BitsAndBytes 4-bit)
• Student: Qwen3.5-4B
• Steps: 20,000
• Loss: KL divergence (T=2.0, α=0.8) + LM loss
• Engine: eden.cpp

Attribution

• Base: Qwen3.5-4B (Apache 2.0)
• LoRA weights: Project Glacie LLC (MIT)