enochlev/KnowLine-31B-Gen1-NVFP4

KnowLine-31B-Gen1 NVFP4

KnowLine-31B-Gen1 as a LoRA for RedHatAI/gemma-4-31B-it-NVFP4.

Serve

vllm serve RedHatAI/gemma-4-31B-it-NVFP4 \
  --enable-lora --max-lora-rank 32 \
  --lora-modules knowline=enochlev/KnowLine-31B-Gen1-NVFP4

Send requests with "model": "knowline". Add --max-model-len 8192 on a 32 GB GPU.

Temperature

Greedy answers need no change. For probabilities, divide each label logprob by 1.44 before the softmax.

Use 1.0 instead for score questions.

Accuracy

The top answer matched FP8 KnowLine on 30 of 30 test questions.