LFM2.5-8B-A1B pi coding and defensive-security experiment
User-trained derivative of LiquidAI/LFM2.5-8B-A1B, pinned base revision dd83f5cb02aa2938b66bba9a9998d38d0517c1f3.
Experimental fine-tune: broader coding, security and retention gains have not been established.
Training
One complete pass: 126 optimizer updates, 656,800 input tokens, 210,697 supervised assistant tokens, 504 examples. BF16 frozen base and FP32 LoRA, rank16/alpha32/dropout0.05 on66 attention, convolution-projection and shallow dense-FFN modules. Experts, routers, tied embeddings/head and norms were frozen. No sequence packing. Sequence limit2048. Actual GPU: NVIDIA L4; A100 was unavailable.
The earlier2M-token plan was an upper budget. Licensing and execution checks reduced the accepted corpus; data was not repeated to fill that budget. Sources and revisions are in training_data_attribution.json. Tool demonstrations were driven by a deterministic fixture oracle and executed by real pi1.5.0; they are not target-model success claims.
Compact paired evaluation
Original: 0/8 protected pi tasks, 2/4 instruction checks. Trained BF16 adapter: 2/8 protected pi tasks, 4/4 instruction checks. These small clustered diagnostic cases do not establish a general improvement percentage. The broader proposed96-task final suite was not run. The diagnostic bridge permits file tools only on app.py and a small fixed command list. An out-of-scope call is rejected at the bridge and can terminate the case rather than returning a normal pi tool error; this limits conclusions about recovery and real repositories. Instruction probes use a 128-token output budget and include a task-directory marker in the system context. See evaluation_summary.json for aggregate results.
The token vocabulary is unchanged. The versioned corrected-v2 chat template omits empty tool-call blocks, escapes Pythonic argument values, normalizes tool wrappers and defaults to an explicit empty reasoning block. The identical template was used to prepare training data, serve the matched comparison and export GGUF. This is an experimental runtime change, so the paired scores above apply to that shared runtime rather than the unmodified template. LFM2.5 uses native Pythonic tool calls; pi requires a tested tool-format conversion and parser. GGUF load/generation checks do not establish an entire pi serving round trip.
License and attribution
See LICENSE, NOTICE and training_data_attribution.json. The base LFM Open License v1.0 includes commercial-use conditions and is not Apache-2.0. Modifications are the trained adapter, its merge, the corrected-v2 chat template and the named quantizations. Upstream source notices are retained. Raw private sessions, optimizer state and protected task content are not included.
Loading
Load the pinned base with Transformers5.22.0 in BF16, then PeftModel.from_pretrained(base, REPO_ID) using PEFT0.21.2. Load the tokenizer and chat_template.jinja from this repository so inference matches training. Set enable_thinking=True only as an unvalidated opt-in; default generation uses direct-answer mode, which may still generate reasoning. Explicit thinking control changes the prompt; it does not guarantee model compliance.
- Downloads last month
- -