experimentalmachines/LFM2.5-2.6B-heretic

Abliterated ("uncensored") version of LiquidAI/LFM2.5-2.6B, produced with heretic: directional ablation of the refusal direction in the residual stream, with a TPE search over per-layer ablation weights that minimises both the refusal rate and the KL divergence from the original model. Weights are merged, so this loads as a plain Lfm2ForCausalLM. Chat template, tool-calling tokens and everything else are unchanged from the base model.

heretic Pareto front for this run

study: ./checkpoints/LiquidAI--LFM2--5-2--6B.jsonl
completed trials: 200 / 200

idx trial refusals       KL  direction_index
  0   171     0.03   0.1522  None
  1   192     0.04   0.0806  None
  2   191     0.05   0.0801  None
  3   197     0.06   0.0648  None
  4     8     0.07   0.0343  None
  5   133     0.08   0.0238  17.379784537320543
  6    92     0.11   0.0225  None
  7    77     0.23   0.0210  None
  8   158     0.24   0.0103  17.028759195631267
  9    89     0.45   0.0051  17.485544563435877
 10    45     0.79   0.0046  17.737500883050494
 11   113     0.89   0.0035  16.83671987276088
 12   194     0.96   0.0020  17.497441205715695
 13   101     0.97   0.0016  18.074113894983057
 14    97     0.98   0.0009  17.811708539059808
 15   187     0.99   0.0008  11.863942700812128
selected pareto index: 4

ExecuTorch (CPU, KleidiAI on Arm)

file	max context (tokens)	size
executorch/lfm2_5_2_6b_heretic_8da4w_32k.pte	32,768	1.81 GB

Each is an ExecuTorch 1.4.1 program for the XNNPACK backend built with ExecuTorch's own LFM2 recipe: every linear layer is quantised to 8-bit dynamic per-token activations and 4-bit grouped weights with group size 32 (8da4w, the layout Arm's KleidiAI int4 kernels are written for), int8 embedding table, static KV cache, fused SDPA, 2048-token prefill chunk (longer prompts are prefilled in chunks; the context window is the figure in the table). XNNPACK dispatches these matmuls to KleidiAI on every Arm CPU (Apple Silicon, Snapdragon, Dimensity, Tensor, Graviton) in the stock ExecuTorch build; on x86 it falls back to AVX kernels. The program is the same file on every platform.

The KV cache for the full window is allocated when the program loads (fp32, about 0.8 GB for 1.2B and 1.1 GB for 2.6B); the runtime attends over filled positions only, so the window costs memory, not speed. Run it with llama_main from ExecuTorch (examples/models/llama) or the python runner, with this repo's tokenizer.json and the prompt format <|startoftext|><|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n.

Pipeline

Reproducible end to end from https://github.com/ (see the repository this was built with): make setup abliterate pareto, make save TRIAL=<idx>, make export, make publish.

Use responsibly; the safety training of the base model has been removed on purpose.