Qwen2.5-Coder-1.5B Java code reviewer — Q4_K_M GGUF
QLoRA fine-tune of Qwen/Qwen2.5-Coder-1.5B-Instruct that reviews one Java method and returns a
single JSON object (severity, category, line, issue, suggested_fix). The LoRA adapter
was merged into the base model, converted to f16 GGUF, and quantized to Q4_K_M with llama.cpp
(4fea119). Size: 986 MB, 5.08 bits per weight.
Measured on a 40-record frozen holdout (CPU, greedy)
| Model | Schema-validity | Bug-catch |
|---|---|---|
| Fine-tuned adapter (HF, 4-bit) | 0.975 | 0.80 |
| This GGUF (Q4_K_M, llama.cpp) | 1.00 | 0.80 |
At n = 40, the one-record difference is within noise. Read it as "no measured loss from quantization", not as a gain.
Usage
Prompt it as a chat (the embedded chat template), not as raw completion. Raw completion skips the framing the model was fine-tuned on, and schema-validity drops to 0.00.
from llama_cpp import Llama
llm = Llama(model_path="model-Q4_K_M.gguf", n_ctx=4096)
out = llm.create_chat_completion(messages=[{"role": "user", "content": prompt}], temperature=0.0)
Source, the prompt and the eval harness: https://github.com/rahulyk09/3_LLM_from_scratch
- Downloads last month
- 112
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for rkg209/qwen2.5-coder-1.5b-java-review-gguf
Base model
Qwen/Qwen2.5-1.5B Finetuned
Qwen/Qwen2.5-Coder-1.5B Finetuned
Qwen/Qwen2.5-Coder-1.5B-Instruct