1000luca/FlexDM-LoRA-v2.1

FlexDM-LoRA ver 2.1 · Gemma-4-31B

A LoRA adapter for google/gemma-4-31B-it @ 842da3794eaa0b77d5f08bae87a17459d91ff475 for structured decisions: a state (a document, a conversation, a device list, ...) and one or more multiple-choice or yes/no questions about it. The answer is a probability for every option, read from the option-label logits at Answer: (. Nothing is generated. It is served like FlexDM, with the same prompts and temperatures, with this adapter loaded in vLLM.

ver 2.1 keeps ver 2.0's recipe and data, adds about 65k questions aimed at formats ver 2.0 left weak (below), and averages two training runs. Earlier versions: ver 2.0 1000luca/FlexDM-LoRA-v2.0, ver 1.0 1000luca/FlexDM-LoRA-v1.0.

	ver 1.0	ver 2.0	ver 2.1
Training questions	90,007	115,249	180,101
Training runs averaged	1	1	2 (seeds 0 and 1)
Served rank / strength	64 / 0.5	64 / 0.5	128 / 0.7
Low-confidence re-read	no	yes	yes
Decision Index 0.3 public index	65.47	66.49	68.99

Files

Path	What
adapter_config.json, adapter_model.safetensors	The served adapter (load this one in vLLM). r 128, lora_alpha 89.6, all seven projections of the 60 text layers.
merge_info.json	How the served copy was made from the two trained adapters (the average, strength 0.7, the v_proj copies below), with their sha256.
peft/	The same adapter for transformers / PEFT.
code/jevlab/	The engine that ran the submission (sha256 as in the run's driver.json; the same files as ver 2.0).
code/requirements.txt	The exact Python 3.12 environment (vLLM 0.30.0, torch 2.13.0+cu130, transformers 5.16.1).

Why two copies: Gemma 4's full-attention layers have no v_proj (attention_k_eq_v: V is the k_proj output), so in transformers the k_proj LoRA changes both K and V. vLLM gives the V slice of its fused qkv_proj its own LoRA, so the served copy repeats the k_proj LoRA as v_proj on those 10 layers (5, 11, ..., 59). Without it, vLLM would serve V without the trained change. peft/ is the original for transformers.

How the average is stored: the two trained LoRAs (r 64 each) are stacked along the rank into one r 128 LoRA whose change is exactly the mean of the two (ΔW = ½ ΔW₀ + ½ ΔW₁). The stored lora_alpha 89.6 applies 0.7 of it: W + 0.7 · ΔW.

Serving (as submitted)

With the kit's own runner and code/ on PYTHONPATH (one GPU with about 75 GB free):

python -m decision_index run --engine jevlab.vllm_lora_reader:VllmLoraReader \
  --option lora=/path/to/this/repo --option layout=header --option read_twice=true --option twice_max_tokens=3000 \
  --option temperature_choice=10.9757 --option temperature_choice_log_slope=-1.7016 \
  --option temperature_noul=10.732 --option noul_bias=0 --option reverse_below=0.6 --compact --out runs/flexdm-lora

• vLLM 0.30.0 offline LLM, FP8 per-block quantization of the base at load (fp8_per_block), TRITON_ATTN, prefix caching, max_model_len 131,072, gpu_memory_utilization 0.70, enable_lora, max_lora_rank 128 (the engine reads it from the adapter), unmerged adapter. Set VLLM_USE_DEEP_GEMM=0 and put the venv's bin/ first on PATH (FlashInfer needs ninja).
• Read policy: the header layout (the request's question list, then the state, then the question and its options), Gemma's chat template with thinking off. Requests of at most 3,000 tokens have their user turn written twice, joined by Read the same input again before answering:. One rendering for every benchmark.
• A question whose most likely option has probability below 0.6 is read once more with its options in reverse order, and the two distributions are averaged (reverse_below=0.6, as in ver 2.0).
• Probabilities: softmax of the label log-probabilities at choice T = 10.9757 - 1.7016 ln(n options), noul T = 10.732.
• Limits: 131,071 tokens per row and 2-255 options per question. Longer rows are refused as unsupported; nothing is truncated.

Training

• Two runs with the same data and settings, seeds 0 and 1 (different data order and option shuffles). Each run: PEFT LoRA r 64, lora_alpha 128 during training, dropout 0, targets q/k/v/o/gate/up/down_proj of the 60 text layers, float32 LoRA on bf16 weights; one epoch over 180,101 questions (110,456 requests), 1,524 steps of about 62k tokens (93.9M tokens), AdamW, lr 1e-4, 3% warmup then cosine decay, gradient clipping 1.0; four RTX PRO 6000 Blackwell (data parallel), about 14 hours.

• Rows are rendered exactly as served: the header layout, the 3,000-token read-twice rule, the option order shuffled per question, and one forward pass per request with the request's questions side by side.

• Loss per question, on the served distribution (softmax of the soft-capped label logits at the serving temperatures): cross-entropy to the gold label (or the source's annotator distribution) + 1.0 · KL(teacher ‖ model) + 0.3 · Brier (+ RPS for rater scales). The teacher is the frozen base model itself:

  • for most questions, Gemma read by FlexDM (vLLM FP8), averaged over the original and the reversed option order;
  • for the reasoning-heavy questions marked below, Gemma's own answer distribution after thinking (thinking mode on, 2 samples, up to 3,072-4,096 thinking tokens).

  No other model is used as a teacher.

Data

ver 2.0's data, minus 287 rows, plus about 65k new questions. Only sources whose licences allow training. No non-commercial or research-only sources, no Jev outputs, no outputs of other entries. Rows carrying a benchmark canary are dropped.

Part	Questions	What
ver 2.0's data	114,962	As described for ver 2.0 (tasksource commercial-use rows; knowledge and reasoning train splits with thinking-mode labels; questions written by Gemma-4-31B-it itself; knowledge multiple choice; simulator-generated device decisions; POP909 chords from songs not in the suite; ver 1.0's rows; teacher-only rows). 287 rows with a negative gold answer were removed from five spam / toxicity sources (SMS Spam, hate_speech18, Wikipedia Detox, Jigsaw, Civil Comments), so their positive share matches the base model's instead of pulling every yes/no judgement toward "no".
Self-written product reviews	43,415	744 product reviews written by Gemma-4-31B-it itself, with aspect, category and opinion questions about each
Self-written emails	12,601	1,458 phishing and legitimate emails written by Gemma-4-31B-it itself from a written specification, with decision questions about each. Legitimate emails never carry shared-document links or come from personal webmail. 42 were dropped by decontamination.
Device decisions	4,838	300 more requests from the generator based on the kit's simulator logic (MIT), with a new seed. No state is equal to a public test row.
Clinical-trial statements	1,985	Fictional clinical-trial records generated by our code, with statements that are entailed or contradicted by a computed rule (50:50)
Causal questions	1,500	Random causal graphs generated by our code, with exactly computed answers (yes/no 50:50 per question type). The teacher is the base model's thinking-mode answer.
Next API call	800	Dialogues generated by our code over 53 API definitions taken from the API-Bank code (Apache-2.0), asking which API is called next
Official train splits of suite benchmarks	2,934 (1.6%)	Unchanged from ver 2.0: iSarcasmEval 1,500, GSM8K 1,039, Amazon ESCI 96, HoVer 95, BANKING77 56, WinoGrande 51, New Yorker (matching) 50, CLINC150 47. No test or evaluation split of any suite benchmark is used.

• Decontamination: every training row was checked against the 140,620 rows of the public 0.3 suite, the public JevBench items and jgeuter/OpenDecisions (exact strings, word 8-grams, long shared spans, a short-line fuzzy check, benchmark canaries). In the code-generated parts no record, causal graph or dialogue equals a public row; the device decisions share only the kit's fixed question and option wording.

Selection and disclosures

• The candidates were seed 0 alone and the two-seed average, each at strength 0.5 and 0.7. The rule (the largest estimated gain, with floors on every task; the averaged and lower-strength candidate when within a margin) was written before the results were seen. It was applied to public Decision Index items (a 2,000-row stratified sample and fixed subsamples of the suite benchmarks), so the public score is not blind to the choice.
• The new generated and self-written parts were aimed at task formats where ver 2.0 scored low on public items. They were written from task descriptions; no public test row was used to write them.
• jgeuter/OpenDecisions was used only as a held-out check (read, never trained on).
• The header layout follows wcole3/jev-brain (#193, MIT). Reading twice follows shisa-ai/shisa-de-2.

Results

Decision Index 0.3 public run: 1000luca/FlexDM-LoRA-v2.1-decision-index (runs/flexdm-l2-sub3).

License

The adapter is released under CC BY-SA 4.0, because some training sources are CC BY-SA. The base model is © Google, Apache 2.0. The code in code/ is MIT (FlexDM).