Siddh07ETH/Atlas-Coder-0.5B-GGUF

⚡ Atlas-Coder-0.5B — GGUF

Quantized GGUF versions of Atlas-Coder-0.5B for CPU inference

---

About This Repository

This repository contains GGUF quantizations of Atlas-Coder-0.5B — a coding-specialized language model fine-tuned from Qwen2.5-Coder-0.5B base using QLoRA on 80K decontaminated code instructions.

These GGUF files are optimized for CPU inference on consumer hardware using llama.cpp, Ollama, and LM Studio. No GPU required.

For full training details, benchmark results, and the original FP16 model, see the Atlas-Coder-0.5B main model page.

---

Available Quantizations

File	Quantization	Size	Recommended For
Atlas-Coder-0.5B-Q8_0.gguf	Q8_0	531 MB	Highest quality — 8-bit, near-lossless. Use if RAM allows.
Atlas-Coder-0.5B-Q6_K.gguf	Q6_K	506 MB	Best quality/size balance — 6-bit K-quant. Recommended default.
Atlas-Coder-0.5B-Q5_K_M.gguf	Q5_K_M	420 MB	Great balance — 5-bit medium K-quant. Minimal quality loss.
Atlas-Coder-0.5B-Q4_K_M.gguf	Q4_K_M	398 MB	Most compressed — 4-bit medium K-quant. Best for low-RAM devices.

Which Quantization Should I Use?

• Q8_0 → Maximum fidelity, RAM not a concern, you want the closest to FP16 output
• Q6_K → Best all-rounder. This is the one to use if you are unsure
• Q5_K_M → Excellent quality at a smaller footprint. Barely distinguishable from Q6_K for most tasks
• Q4_K_M → Minimum RAM usage. Still surprisingly capable for a 400MB file

All four files run comfortably on any modern laptop with 4GB+ RAM.

---

Quick Start

Ollama

# Download the model file first, then create a Modelfile
cat > Modelfile << EOF
FROM ./Atlas-Coder-0.5B-Q5_K_M.gguf

SYSTEM "You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning."

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER num_predict 512
EOF

ollama create atlas-coder -f Modelfile
ollama run atlas-coder "Write a Python function to reverse a linked list"

LM Studio

1. Open LM Studio
2. Go to Search tab
3. Search for Siddh07ETH/Atlas-Coder-0.5B-GGUF
4. Download your preferred quantization
5. Load and chat

llama.cpp (CLI)

# Clone and build llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Run inference
./llama-cli \
  -m Atlas-Coder-0.5B-Q5_K_M.gguf \
  -p "<|im_start|>system\nYou are Atlas-Coder, an elite AI coding assistant.<|im_end|>\n<|im_start|>user\nWrite a Python binary search function.<|im_end|>\n<|im_start|>assistant\n" \
  -n 512 \
  --temp 0.3 \
  --top-p 0.9 \
  --repeat-penalty 1.1 \
  --no-display-prompt

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="Atlas-Coder-0.5B-Q5_K_M.gguf",
    n_ctx=1024,
    n_threads=4,
    verbose=False,
)

prompt = """<|im_start|>system
You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code.<|im_end|>
<|im_start|>user
Write a Python function to check if a number is prime.<|im_end|>
<|im_start|>assistant
"""

output = llm(
    prompt,
    max_tokens=512,
    temperature=0.3,
    top_p=0.9,
    repeat_penalty=1.1,
    stop=["<|im_end|>", "<|im_start|>"],
)

print(output["choices"][0]["text"])

---

Recommended Generation Settings

Setting	Value	Reason
temperature	0.2–0.4	Conservative — reduces hallucinations in code
top_p	0.9	Focused vocabulary sampling
repeat_penalty	1.1	Prevents repetitive patterns
max_tokens	256–512	Sufficient for most coding tasks
ctx_size	1024	Matches training sequence length

---

Chat Template

Atlas-Coder uses ChatML format. Always wrap prompts in this structure:

<|im_start|>system
You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning.<|im_end|>
<|im_start|>user
YOUR QUESTION HERE<|im_end|>
<|im_start|>assistant

---

Original Model

This GGUF repository is derived from the full FP16 model:

➡️ Siddh07ETH/Atlas-Coder-0.5B

The original model page contains:

• Full training methodology and pipeline details
• Dataset mixture and decontamination details
• Benchmark results (EvalPlus / HumanEval+ — publishing soon)
• FP16 and 4-bit inference code examples
• Research context and engineering decisions

---

About Atlas-Coder-0.5B

Atlas-Coder-0.5B is a coding-specialized LLM instruction-tuned from Qwen2.5-Coder-0.5B base (not instruct) using QLoRA on a Tesla T4. Key training highlights:

• 80K samples across 4 code datasets, including 50K execution-verified OSS-Instruct samples
• n-gram Jaccard decontamination against HumanEval — benchmark scores are honest
• Response-only loss masking — gradients only on assistant code output
• Trained entirely on free Kaggle T4 GPU — reproducible on consumer hardware
• 3-layer checkpoint recovery — survived Kaggle's 12-hour session limit

Part of the Pluto AI research project by Siddharth N.R.

---

Performance

Benchmarks (EvalPlus / HumanEval+) are currently being run on the original FP16 model and will be published on the main model page shortly.

GGUF quantization quality vs FP16 baseline (approximate):

Quantization	Quality Retention
Q8_0	~99.9%
Q6_K	~99.2%
Q5_K_M	~98.5%
Q4_K_M	~97.0%

---

Author

Siddharth N.R. Final-year B.Tech — AI & Data Science Pluto AI Research

---

Related Models

Model	Type	Description
Atlas-Coder-0.5B	FP16	Original full-precision model
Pluto-Genesis-0.6B	FP16	General reasoning, math, and code
Pluto-Genesis-0.6B-GGUF	GGUF	GGUF quantizations of Pluto-Genesis

---

Citation

@misc{atlascoder2026,
  author    = {Siddharth N.R.},
  title     = {Atlas-Coder-0.5B: A QLoRA-Trained Sub-1B Coding Model from Base},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Siddh07ETH/Atlas-Coder-0.5B}
}

---

License

Apache 2.0 — see LICENSE. Base model Qwen2.5-Coder-0.5B is also Apache 2.0.