Nullmega/Mellum2.1-CodeReasoning-Public-v1-LoRA

Mellum2.1-CodeReasoning-Public-v1 — adapter

Candidate fine-tune of JetBrains/Mellum2.1-12B-A2.5B-Thinking, revision 92ddae9fc7665e9f801d141d2e5a6b2caf2460c4. Run: mellum-coding-public-v1. Training mode: pilot / public_bootstrap. Completed optimizer steps: 963.

This is a candidate, not a demonstrated improvement over the original model. The public bootstrap trains coding/reasoning only. A full-plan run targets coding, frontend/backend, pi tool use, and authorized security work, according to the measured mix in run-manifest.json. Three diagnostic probes and validation NLL do not establish coding, reasoning or cybersecurity capability.

Training

BF16 frozen base; attention-only LoRA on q/k/v/o projections. Rank 16, alpha 32, dropout 0.05, LR 2e-05, sequence limit 4096, microbatch 1, gradient accumulation 16. No router/expert training and no RL. Loss is applied only to complete assistant targets. Related problems/projects share a split. Supervised tokens: 4,633,250; input tokens processed: 7,988,330.

Measured diagnostics

Base validation NLL: 0.665589; candidate NLL: 0.430868. Candidate probe passes: 2/3; base: 2/3. These are small diagnostics, not SWE-bench, Cybench or a security certification. See run-manifest.json and quantization-manifest.json where applicable.

Use and limitations

Use the native Mellum chat template and a runtime that supports Mellum and parses tool calls/reasoning. Native merged weights are BF16. GGUF variants are directly quantized from those merged weights. Start with Q4_K_M on a 12 GB RX 6700 XT, adjusting context/offload to measured memory use. Q3/Q2 variants are experimental; format and reasoning reliability may degrade. This notebook does not validate a 128K context or actual LM Studio/pi integration on your computer. MTP weights, if present in the source, were not adapted; GGUF export excludes MTP. Security evaluations belong in authorized, isolated environments. Verify generated code and findings.

Attribution

See ATTRIBUTIONS.md, the included base license, and the original dataset/source terms. Training data and credentials are not included in this repository.