hypneum-lab/micro-kiki-v4-sota

micro-kiki V4-SOTA — domain-expert LoRA adapters for Qwen3.6-35B-A3B (34 published here)

The programme trained 35 per-domain LoRA adapters with mlx_lm (Apple MLX, BF16) on top of Qwen/Qwen3.6-35B-A3B (native MoE, 256 experts, ~3B active parameters per token). The shipped log-reasoning.txt ends with a Metal allocation error after iteration 150; whether a later run completed is not recorded. 34 adapters are published on this Hub repository. The missing one is reasoning: only config-reasoning.yaml and log-reasoning.txt are present and there is no reasoning/ adapter directory in the file list (checked 2026-10-31); the reasoning adapter is not on the Hub. This is the canonical micro-kiki adapter set (April 2026). It supersedes micro-kiki-v35b (rank-8 pilot) and micro-kiki-v3 (pre-pivot MoE-LoRA, dead weights).

Produced by Clément Saillant (L'Electron Rare) for the Hypneum Lab dreamOfkiki research program. Runtime, routing, cognitive layer and configs: https://github.com/hypneum-lab/micro-kiki. Training execution scripts and the mlx-lm fork: https://github.com/ailiance/ailiance-mac-tuner (formerly L-electron-Rare/KIKI-Mac_tunner).

Files and format

Hub revision f86cbc03630956e3d0f102a2c2682432179f4114 (last modified 2026-10-25T09:09:33Z), 181 files, 280 050 336 551 bytes (280 GB).

• <domain>/adapters.safetensors and <domain>/adapter_config.json for 34 domains (all domains in the list below except reasoning).
• <domain>/0000N00_adapters.safetensors: intermediate checkpoints for some domains.
• config-<domain>.yaml and log-<domain>.txt: training configs and logs for 35 domains (including reasoning).

Recipe

All values below come from the per-domain adapter_config.json, config-<domain>.yaml and log-<domain>.txt files shipped in this repository.

Parameter	Value
Base model	Qwen/Qwen3.6-35B-A3B (BF16, local copy models/Qwen3.6-35B-A3B)
Framework	mlx_lm.lora (mlx-lm 0.31.x), fine_tune_type: lora
LoRA rank	16
LoRA scale (effective)	20.0 (hard-coded in the training driver)
LoRA alpha	16.0 — recorded but unused: mlx-lm LoRA layers read only rank and scale, so alpha has no effect
Dropout	0.0
Target layers	32 (last 32 of 40) for 31 domains of the programme (30 on the Hub, reasoning not being published); 20 for components, electronics, llm-ops, security
Target modules	mlx-lm default LoRA surface for this architecture (linear attention in_proj_*/out_proj, self_attn q/k/v/o, MoE switch_mlp and shared-expert projections). See the GitHub README "Adapter surface".
Trainable parameters	1 025.8 M (2.96 % of 34 660.6 M) for 32-layer adapters (from logs)
Iterations	per domain: 1000 (chat-fr, python), 500 (e.g. electronics, kicad-pcb, sql), 300 (components, llm-ops, security), others per adapter_config.json — not a uniform 200. Counts over the 34 Hub adapter_config.json files (fetched 2026-10-31): 500 (13 adapters), 200 (11), 300 (5), 100 (3), 1000 (2)
Batch size	1, grad_accumulation_steps 1
Learning rate	1e-5, no schedule, optimizer adam
Max sequence length	1024 (15 adapters) or 512 (19 adapters, e.g. python, cpp, docker, rust, typescript), per adapter_config.json; longer samples were truncated
Gradient checkpointing	true
Seed	0
Checkpoints	every 100 iterations (0000NNN_adapters.safetensors); adapters.safetensors is the final one

Adapter file sizes: 4 103 288 730 bytes (32 layers) and 2 564 555 362 bytes (20 layers). The 4 smaller adapters are smaller only because they target 20 layers — not because of smaller datasets or a lower rank.

Training data

Per-domain JSONL under data/micro-kiki/<domain>/ built by the micro-kiki data pipeline (489 K examples across 35 domains; mix of Claude/Codex CLI session traces, curated public datasets and teacher distillation — see the GitHub README and the dataset card). Exact per-domain example counts for V4: not recorded in this repository. The dataset repository clemsail/micro-kiki-v3-dataset was not publicly resolvable on 2026-10-25 (HTTP 401 anonymous); provenance details live in the GitHub repo.

Domains (35 trained, 34 on the Hub)

chat-fr, components, cpp, devops, docker, dsp, electronics, embedded, emc, freecad, html-css, iot, kicad-dsl, kicad-pcb, llm-ops, llm-orch, lua-upy, math, ml-training, music-audio, platformio, power, python, reasoning, rust, security, shell, spice, spice-sim, sql, stm32, typescript, web-backend, web-frontend, yaml-json.

reasoning is in this list but its adapter is not on the Hub.

About the shipped logs

• log-chat-fr.txt is a complete run: val loss 2.417 (iter 1) → 0.849 (iter 1000), train loss 0.876, 0.429 it/s, peak memory 114.33 GB.
• log-components.txt and log-security.txt are failed first attempts (Metal allocation errors at iteration 1, with a different config; components: 500 iters). The adapters were then retrained successfully with the configs recorded in adapter_config.json; the logs of those successful runs are not in this repository.
• log-llm-ops.txt and log-ml-training.txt only contain a Metal initialisation crash; the corresponding adapters were produced by later runs whose logs are not recorded.

Training host & cost

Item	Value
Host	Mac Studio M3 Ultra, 512 GB unified memory, macOS, MLX (BF16, no quantization)
Peak memory	~107 GB (project README); 114.33 GB logged for chat-fr
Metal settings	mx.set_memory_limit(460 GB), mx.set_cache_limit(32 GB) (required to avoid GPU hangs)
Wall-clock	chat-fr: 1000 iterations at 0.429 it/s ≈ 39 min (derived from the log). Other domains: not recorded
Training window	2026-04-19 → 2026-04-26
Energy / monetary cost	not recorded
Reproducibility	the training host has been powered off since 2026-10; re-running requires a comparable Apple Silicon machine

Evaluation

Measured (from the GitHub README, results/ of hypneum-lab/micro-kiki):

Metric	Value
chat-fr val loss	0.849 (base 2.417)
reasoning val loss	0.638 (iteration 100 of a run that crashed at iteration 150)
HumanEval pilot (pass@1)	base 0.80 → cpp adapter 0.90
HumanEval+ n=164, chat-fr adapter, max_tokens 384	pass@1 0.543 (chat-fr-164-cross)
Forgetting (angle)	chat-fr ↔ reasoning mean 79.4°; typescript ↔ python 66.6°; only 5 of 41 adapter artefacts pairwise checked
Adapter health	README states all 35 V4 adapters pass the adapter-health validator; the committed sweep (results/adapter-health-sweep.json, 2026-04-19) covers the rank-8 v35b set, not this set. (all lora_B non-zero); reasoning is not on the Hub

Caveats

• The adapters have not been re-evaluated with a comparative benchmark since training finished (last bench-complete.json predates the end of V4 training).
• On the same base model, the sibling ailiance family of Qwen3.6-35B-A3B LoRA adapters (curriculum-trained, r32, 16 layers) was benchmarked in June 2026 with iact-bench: no adapter improved over the base model except html-css, and the base alone was routed in production afterwards. Those are different adapters from this set, but the result is the closest available evidence and it does not support a general "adapters beat the base" claim for this recipe. Treat these adapters as research artefacts, not as a validated improvement. (statement of the 2026-10-25 card; no source file in the repository, unverified on 2026-10-31)
• The project README notes that only 3 of 10 probed domains showed measurable improvement over the base model (README.md, section "Stacks vs base").
• The project follows an errata policy under which a number is reported only if a committed artefact reproduces it (docs/ERRATA.md).
• HumanEval+ pass@1 0.5427 (89/164) for chat-fr comes from evals/chat-fr-164-cross.json, a file hosted in hypneum-lab/micro-kiki-stack-chat-fr, not in this repository.
• No other evaluation is published on the Hub for this repository.

Limitations

• Research artefacts; no safety tuning beyond the base model.
• Sequences longer than 512 or 1024 tokens (per adapter) were truncated during training.
• Not designed for medical, legal or financial advice; outputs may be wrong or fabricated.
• Serving: Q4_K_M base recommended; at most 4 adapters active simultaneously.
• DoRA is not supported on this MoE architecture.

Usage (MLX)

from mlx_lm import load, generate

model, tokenizer = load("Qwen/Qwen3.6-35B-A3B", adapter_path="./math")
print(generate(model, tokenizer, prompt="Prove the Pythagorean theorem.", max_tokens=512))

Citation

@software{saillant_microkiki_2026,
  author    = {Saillant, Clément},
  title     = {micro-kiki: 35 Domain-Expert LoRA Adapters with Cognitive Layer on Qwen MoE},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19666759},
  url       = {https://doi.org/10.5281/zenodo.19666759}
}

Concept DOI 10.5281/zenodo.19666759 (resolves to the latest version; v0.3.0 is 10.5281/zenodo.19666760). Parent pre-registration: OSF 10.17605/OSF.IO/Q6JYN.

Provenance

• Code, configs, validators, eval results: https://github.com/hypneum-lab/micro-kiki
• Training execution (MLX scripts, status table): https://github.com/ailiance/ailiance-mac-tuner
• Base model: https://huggingface.co/Qwen/Qwen3.6-35B-A3B (Apache-2.0)
• Source repository: https://github.com/hypneum-lab/micro-kiki (Apache-2.0 LICENSE), checked at commit a3968d9e1f0b6a6b8cc4cb028d4fef3ced24770c.
• Zenodo DOI: 10.5281/zenodo.19666759, concept DOI of the source repository; no per-artefact DOI.
• Parent pre-registration: OSF 10.17605/OSF.IO/Q6JYN (CITATION.cff).
• Hub revision described by the file listing: f86cbc03630956e3d0f102a2c2682432179f4114.
• Card revision date: 2026-10-31.
• Related: hypneum-lab/micro-kiki-router-v3-minilm (domain router, called "Router V4" in the source repository), hypneum-lab/micro-kiki-v35b (rank-8 pilot), hypneum-lab/spikingkiki-v4-lif-metadata (metadata only, references this repository as base_model)

EU AI Act transparency (Regulation EU 2024/1689)

Field	Value
Provider	Clément Saillant / L'Electron Rare, for Hypneum Lab
Artefact type	LoRA adapter weights only; base model unchanged
Base model	Qwen/Qwen3.6-35B-A3B (upstream licence applies separately)
Training data origin	project corpus (CLI session traces, curated public datasets, teacher distillation) — see GitHub provenance
Intended use	technical assistance (electronics, embedded, code, FR/EN chat) in research settings
Out of scope	healthcare, legal, safety-critical autonomous decisions, malicious code
Risk classification	limited risk — Article 50 transparency obligations
Contact	clement@saillant.cc

You are using an AI model. Outputs may be inaccurate, biased or fabricated; verify independently.

AI assistance (transparency)

This repository is developed by a one-person lab with AI assistants used as tools, never as authors:

• Claude Code (Anthropic) assists with code, documentation, operations and literature search in interactive sessions driven by the PI.
• Codex CLI (OpenAI, gpt-6.1-sol / gpt-6-astra) runs independent review passes before a release, a pre-registration or a submission; the briefs, reports and verdict tables are published next to the reviewed material whenever a pass is run.

Every scientific claim, verdict and pre-registration decision is taken by the PI; results are reproduced by the test suite and CI, and review reports are evidence, not endorsements. No model is listed as a co-author of a publication or as a co-author of a commit.