FlexDM-LoRA ver 1.0 · Gemma-4-31B

A LoRA adapter for google/gemma-4-31B-it @ 842da3794eaa0b77d5f08bae87a17459d91ff475 for structured decisions: a state (document, conversation, device list, ...) and one or more multiple-choice or yes/no questions about it. The answer is a probability for every option, read from the option-label logits at Answer: (; nothing is generated. It is served exactly like FlexDM (the same prompts, read policy and temperatures), with this adapter loaded in vLLM.

Files

Path	What
adapter_config.json, adapter_model.safetensors	The served adapter (load this in vLLM). r 64, lora_alpha 64, all seven projections of the text layers.
peft/	The same adapter for transformers / PEFT.
code/jevlab/	The engine that ran the submission: vllm_lora_reader.py (VllmLoraReader) on top of FlexDM v1.0's vllm_reader.py and reader.py (byte-identical, same sha256 as the run's driver.json).
code/requirements.txt	The exact Python 3.12 environment (vLLM 0.30.0, torch 2.13.0+cu130, transformers 5.16.1).

Why two copies: Gemma 4's full-attention layers have no v_proj (attention_k_eq_v: V is the k_proj output), so in transformers the k_proj LoRA changes both K and V. vLLM gives the V slice of its fused qkv_proj its own LoRA, so the served copy repeats the k_proj LoRA as v_proj on those 10 layers (5, 11, ..., 59). Without it vLLM would serve V without the trained change. peft/ is the original for transformers.

Serving (as submitted)

With the kit's own runner and the code of this repository on PYTHONPATH (one GPU with about 75 GB free):

python -m decision_index run --engine jevlab.vllm_lora_reader:VllmLoraReader \
  --option lora=/path/to/this/repo --option layout=header --option read_twice=true --option twice_max_tokens=3000 \
  --option temperature_choice=10.9757 --option temperature_choice_log_slope=-1.7016 \
  --option temperature_noul=10.732 --option noul_bias=0 --compact --out runs/flexdm-lora

• vLLM 0.30.0 offline LLM, FP8 per-block quantization of the base at load (fp8_per_block), TRITON_ATTN, prefix caching, max_model_len 131,072, gpu_memory_utilization 0.70, enable_lora, max_lora_rank 64, unmerged adapter. VLLM_USE_DEEP_GEMM=0 and the venv's bin/ first on PATH (FlashInfer needs ninja).
• Read policy (FlexDM v1.0): the header layout (the request's question list, then the state, then the question and its options), Gemma's chat template with thinking off; requests of at most 3,000 tokens have their user turn written twice, joined by Read the same input again before answering:. One rendering for every benchmark.
• Probabilities: softmax of the label log-probabilities at choice T = 10.9757 - 1.7016 ln(n options), noul T = 10.732.
• Limits: 131,071 tokens per row and 2-255 options per question; longer rows are refused as unsupported, nothing is truncated.

Training

• PEFT LoRA, r 64, lora_alpha 128 during training, dropout 0, targets q/k/v/o/gate/up/down_proj of the 60 text layers (0.49B trainable parameters), float32 LoRA on bf16 weights. Strength 0.5: the stored lora_alpha 64 applies half of the trained change (W + 0.5 · ΔW).
• One epoch over 90,007 questions (80,512 requests), 934 steps of about 64k tokens, AdamW, lr 1e-4, 3% warmup then cosine, gradient clipping 1.0; one RTX PRO 6000 Blackwell Max-Q, about 31 hours.
• Rows are rendered exactly as served: the header layout, the 3,000-token read-twice rule, the option order shuffled per question (a seeded hash), one forward per request with the request's questions side by side.
• Loss per question, on the served distribution (softmax of the soft-capped label logits at the serving temperatures): cross-entropy to the gold label (or the source's annotator distribution) + 1.0 · KL(base ‖ model) + 0.3 · Brier (+ RPS for rater scales). The base distribution is the frozen Gemma read by FlexDM (vLLM FP8), averaged over the original and the reversed option order: it keeps the adapter close to the base model.
• Data (licences that allow training; no non-commercial or research-only source; no model-written text; rows carrying a benchmark canary dropped):
  • 60% human-labelled public datasets: CommonsenseQA, QASC, OpenBookQA, AQuA-RAT, MATH (train), StrategyQA, QuaRTz, WiCE, FaithDial, IBM ArgQ-30k, PAWS, sarcasm headlines, NLU++, Taskmaster-2, oasst2 (human messages only), ColBERT humor;
  • 20% decisions generated by our own code with computed answers (arithmetic, calendars, logic puzzles, code execution, policy rules, table lookups, tool choice, long-document retrieval, probability and causal questions, chords);
  • 15% decisions from other domains: Civil Comments, Measuring Hate Speech, ETHICS, GoEmotions, financial topics, BIGPATENT, SciCite;
  • 5% (4,500 questions) from the official train splits of seven Decision Index benchmarks: HoVer, Amazon ESCI, New Yorker (matching), BANKING77, CLINC150, GSM8K, WinoGrande. No test or evaluation split of any suite benchmark is used.
• Decontamination: every training row was checked against the 140,620 rows of the public 0.3 suite and the public JevBench items (exact strings, word 8-grams, long shared spans, a short-line fuzzy check, benchmark canaries); 1,118 of 91,691 candidate rows were dropped.
• Selection: the strength (0.5 of the trained change, against 0.7 and 1.0) and the choice of this run over a shorter 10k-question run were made on public Decision Index items (a 2,000-row stratified sample, subsamples of five benchmarks and nine benchmarks), by a rule fixed before the last comparison. The public score is therefore not blind to these choices.

Results

Decision Index 0.3 public run: see the results dataset 1000luca/FlexDM-LoRA-decision-index.

License

The adapter is released under CC BY-SA 4.0 (some training sources are CC BY-SA). The base model is © Google, Apache 2.0. The code in code/ is MIT (FlexDM). Access is gated during the Decision Index evaluation period.