Novel (GGUF): small offline coding model, 1.5B

Novel is a small, local-first code generation model. It is the LoRA adapter Bedirhn77/novel-lora merged into Qwen/Qwen2.5-Coder-1.5B-Instruct and converted to GGUF with llama.cpp. It runs on CPU or a modest GPU with llama.cpp, Ollama, LM Studio, and other GGUF-compatible tools.

Quick facts

Base model	Qwen2.5-Coder-1.5B-Instruct
Method	LoRA fine-tune, merged, converted to GGUF
Size	2B parameters
Formats	Novel-Q4_K_M.gguf (smaller), Novel-Q8_0.gguf (higher quality)
Task	Code generation, code completion, coding Q&A
Runs on	llama.cpp, Ollama, LM Studio, any GGUF runtime
License	Apache-2.0 (inherited from base; verify training-data licenses)
Dataset	Bedirhn77/novel-dataset (Ardiuno, Esp32, Python, Java, C++, Opus4.6 Reasoning )

Benchmarks

Measured with greedy decoding and identical prompts for both models.

Benchmark	Qwen2.5-Coder-1.5B-Instruct (base)	Novel
HumanEval pass@1 (164 problems)	67.7% (111/164)	62.2% (102/164)
Novel-23 pass rate (23 problems)	43.5% (10/23)	39.1% (9/23)

[(Small diffrents come from small amount of questions in 1000 questions probably Novel can get %65-70)]

Honest reading of the results: Novel is currently slightly below the base model on both sets, and the differences are not statistically significant (especially on Novel-23, where one problem equals 4.3 points). Treat Novel as an experimental fine-tune, not an upgrade over the base model. A larger training set and a v2 are planned.

Illustrative chart (estimates, not measurements)

The chart below places Novel next to rough guesses for other models. Only the solid blue and orange columns are measured. Dashed columns are estimates, are not benchmark results, and their order is not a ranking.

Quickstart

llama.cpp

llama-cli -m Novel-Q4_K_M.gguf -p "Write a Python function that checks if a string is a palindrome."

Ollama

Put the Modelfile next to the GGUF file, then:

ollama create novel -f Modelfile
ollama run novel

The Modelfile uses the Qwen ChatML template. Adjust it if you swap in a non-Qwen base.

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="Novel-Q4_K_M.gguf", n_ctx=4096)
out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Write a Python function to merge two sorted lists."}]
)
print(out["choices"][0]["message"]["content"])

Which file should I download?

• Novel-Q4_K_M.gguf: smallest, fastest, good for low-RAM machines and laptops.
• Novel-Q8_0.gguf: closer to full precision, slightly larger and slower.

Intended use

• Local, offline code generation and small coding tasks
• Experimenting with LoRA fine-tuning of small code models
• Low-RAM environments where larger models do not fit

Limitations

• Fine-tuned on a small dataset. Expect limited gains, and see the benchmarks above: it does not beat the base model yet.
• Benchmarked only on HumanEval (164 problems) and a small custom set (Novel-23). Results may not reflect real-world performance.
• Small 1.5B models can produce incorrect or insecure code. Review and test all generated code.
• Check the license of the base model and of every training data source before using or redistributing.

Related

• Base model: Qwen/Qwen2.5-Coder-1.5B-Instruct
• LoRA adapter: Bedirhn77/novel-lora
• Runtime: llama.cpp, Ollama

FAQ

What is Novel? A LoRA fine-tune of Qwen2.5-Coder-1.5B-Instruct, packaged as GGUF for local use.

Can it run without internet or a GPU? Yes. The Q4_K_M file runs on CPU with llama.cpp or Ollama.

Is it better than the base model? Not yet. Measured scores are slightly lower, within noise.

What is Novel-23? A small custom set of 23 coding problems used alongside HumanEval.