metask-jev-rain-31B-trained

Decision model behind the metask-jev-rain-31B-trained JevBench service. Base model: google/gemma-4-31B-it, used through Google's official quantization-aware-training (QAT) releases of it. The model has two parts that are loaded together and not merged:

Part	Location in this repository	What it is
Base	repository root	The files of google/gemma-4-31B-it-qat-w4a16-ct at revision 52f3f65bc7a02d555763bc923bd1d9094898219d, byte for byte: Google's official compressed-tensors W4A16 checkpoint (4-bit weights, group size 32, symmetric). Its own README and .gitattributes are replaced by this card and the repository default.
Adapter	adapter/	Our LoRA (PEFT format): rank 16, alpha 32, on the language model's q/k/v/o/gate/up/down projections, trained by us in BF16 on google/gemma-4-31B-it-qat-q4_0-unquantized at revision 1e4d8beecacb8b7590c1d8bedd7335f687bf311f.

adapter/adapter_model.safetensors contains the trained checkpoint tensors unchanged plus, for the 10 full-attention layers (5, 11, …, 59) that compute values with the key projection, a v_proj LoRA that is a byte copy of the trained k_proj LoRA (20 tensors), so that vLLM applies the update to keys and values as in training.

adapter/adapter_config.json names the training base above as base_model_name_or_path.

Serving

The service code (service/ in this repository) starts vLLM 0.31.0 on this repository's root with enable_lora, max_lora_rank=16, and the adapter/ folder as the single LoRA, applied to every request. Equivalent vLLM usage:

from vllm import LLM
from vllm.lora.request import LoRARequest
llm = LLM(model=MODEL_DIR, dtype="bfloat16", enable_lora=True, max_lora_rank=16)
adapter = LoRARequest("adapter", 1, MODEL_DIR + "/adapter")
# llm.generate(prompts, sampling_params, lora_request=adapter)

Loading only the root files gives Google's base checkpoint without our adapter.

Facts on data and the measurement conditions of cost and latency are in service/DISCLOSURE.md.

Contact: chenyuwang166@gmail.com