shuff57/lfm2-24b-phase1-reasoning-GGUF

---

LFM2-24B-A2B Phase 1 Reasoning — GGUF Q4_K_M GGUF quantization of shuff57/lfm2-24b-phase1-reasoning (https://huggingface.co/shuff57/lfm2-24b-phase1-reasoning). Model Description This is the Phase 1 reasoning fine-tuned version of LiquidAI's LFM2-24B-A2B (24B MoE, 2.3B active parameters) model, trained on 13,201 synthetic reasoning examples generated for the O.G.R.E. (OllamaGradingRubricEvaluator) project. The base model (LiquidAI/LFM2-24B-A2B) was further fine-tuned using Unsloth's QLoRA approach with:

• LoRA rank: 16
• LoRA alpha: 16
• Target modules: ["q_proj", "k_proj", "v_proj", "out_proj", "in_proj", "w1", "w2", "w3"]
• Sequence length: 8192
• Packing: False (critical for LFM2 MoE routing stability)
• Training on responses only (using unsloth.chat_templates.train_on_responses_only)
• Optimizer: adamw_8bit
• Learning rate: 2e-5
• Epochs: 1 (effective steps: ~1,568)
• Precision: bfloat16 After training, the LoRA adapters were merged into the base model and saved in 16-bit format. GGUF Quantization The merged model was converted to GGUF format for efficient CPU usage with llama.cpp and Ollama. Conversion Process

1. Model Loading: Loaded with device_map="cpu" to avoid MoE-related VRAM overflow during transformation.
2. HF → GGUF Conversion: Used the llama.cpp conversion pipeline:
   • Cloned llama.cpp (commit post-Oct 2025, includes native lfm2_moe support)
   • Configured with cmake -B llama.cpp/build -S llama.cpp -DCMAKE_BUILD_TYPE=Release
   • Built all targets (cmake --build llama.cpp/build --config Release)
   • Located the llama-quantize binary via glob search
3. Quantization: Applied Q4_K_M quantization: ./llama.cpp/build/bin/llama-quantize lfm2-24b-phase1-F16.gguf lfm2-24b-phase1-Q4_K_M.gguf Q4_K_M
   4. Validation: The resulting GGUF file was tested with Ollama and llama.cpp for correct LFM2 MoE handling. Quantization Details

• Format: GGUF (GPT-Generated Unified Format)
• Quantization Method: Q4_K_M (4-bit with mixed precision, state-of-the-art balance)
• Estimated Size: ~14.42 GB
• VRAM Usage: Can be run on systems with <8GB VRAM when offloaded, or CPU-only
• Context Length: 8192 tokens (matches training) Usage with Ollama Create a Modelfile: FROM ./lfm2-24b-phase1-Q4_K_M.gguf SYSTEM "You are a careful analytical reasoner. Think step by step before answering." RENDERER lfm2 PARSER lfm2 PARAMETER temperature 0.1 PARAMETER top_k 50 PARAMETER top_p 0.95 PARAMETER repeat_penalty 1.05 PARAMETER num_ctx 8192 Then: ollama create lfm2-24b-phase1-reasoning -f Modelfile ollama run lfm2-24b-phase1-reasoning "Your prompt here" Usage with llama.cpp llama-cli -m lfm2-24b-phase1-Q4_K_M.gguf -p "You are a careful analytical reasoner." --temp 0.1 -n 512 Training Data The Phase 1 reasoning model was trained on a synthetic dataset of 13,201 examples designed to teach step-by-step reasoning, logical deduction, and mathematical problem-solving. The dataset is available at:
• shuff57/ogre-phase1-synth (https://huggingface.co/datasets/shuff57/ogre-phase1-synth) Related Models
• Base Model: LiquidAI/LFM2-24B-A2B (https://huggingface.co/LiquidAI/LFM2-24B-A2B)
• Phase 1 Reasoning (HF): shuff57/lfm2-24b-phase1-reasoning (https://huggingface.co/shuff57/lfm2-24b-phase1-reasoning) (16-bit merged weights)
• Phase 2 Stat Grader: shuff57/lfm2-24b-grader (https://huggingface.co/shuff57/lfm2-24b-grader) (in progress) Citation If you use this model, please cite the O.G.R.E. project: @misc{ogre2026, title={O.G.R.E.: OllamaGradingRubricEvaluator}, author={shuff57}, year={2026}, publisher={Hugging Face}, url={https://huggingface.co/shuff57} } License Apache 2.0 — same as the base LFM2-24B-A2B model.