Instructions to use Taimwe/qwen2.5-coder-7b-pro-merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Taimwe/qwen2.5-coder-7b-pro-merged with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Use Docker

docker model run hf.co/Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

LM Studio
Jan

vLLM

How to use Taimwe/qwen2.5-coder-7b-pro-merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Taimwe/qwen2.5-coder-7b-pro-merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Taimwe/qwen2.5-coder-7b-pro-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Ollama
How to use Taimwe/qwen2.5-coder-7b-pro-merged with Ollama:
```
ollama run hf.co/Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M
```
Unsloth Desktop

How to use Taimwe/qwen2.5-coder-7b-pro-merged with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Taimwe/qwen2.5-coder-7b-pro-merged with Docker Model Runner:
```
docker model run hf.co/Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M
```

Lemonade

How to use Taimwe/qwen2.5-coder-7b-pro-merged with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Run and chat with the model

lemonade run user.qwen2.5-coder-7b-pro-merged-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Taimwe/qwen2.5-coder-7b-pro-merged with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Taimwe/qwen2.5-coder-7b-pro-merged with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Non-commercial access request

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This model is licensed under the Taimwe Non-Commercial License 1.0 (https://huggingface.co/Taimwe/qwen2.5-coder-7b-pro-merged/blob/main/LICENSE). By requesting access you confirm that you will use it only for non-commercial purposes - personal projects, academic research, teaching or evaluation - and that your username and email address are shared with the model author. Commercial use requires a separate licence; see the LICENSE file.

Qwen2.5-Coder-7B-Pro (merged, GGUF)

GGUF export of the merged fine-tune Taimwe/qwen2.5-coder-7b-pro, built on Qwen2.5-Coder-7B-Instruct. The LoRA adapter Taimwe/qwen2.5-coder-7b-pro was merged into the base model and converted to GGUF with Unsloth / llama.cpp.

Architecture: Qwen2ForCausalLM (qwen2), 7.6 B parameters
Native context: 32768 tokens
Chat template: Qwen ChatML (<|im_start|> / <|im_end|>), with tool calling
Quantisation: see the table below

Files

File	Quant	Size	Notes
`qwen2.5-coder-7b-instruct.Q4_K_M.gguf`	Q4_K_M	4.7 GB	best size/quality trade-off; runs on 8 GB GPUs or CPU

More quantisations (Q5_K_M, Q6_K, Q8_0) are welcome via PRs - see Contributing quants.

Run it with llama.cpp

# one-shot prompt
llama cli -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M --jinja

# OpenAI-compatible server on http://localhost:8080
llama serve -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M \
  --host 0.0.0.0 --port 8080 -ngl 99 -c 8192 --jinja

curl localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"write a python fizzbuzz"}],
       "model":"local"}'

--jinja is recommended: it enables the tool-calling template stored in the GGUF, which agentic clients (Cline, Continue, Roo) rely on.

Legacy binaries (llama-cli, llama-server) accept the same -hf argument.

Run it with Ollama

ollama run hf.co/Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M

A ready-to-use Modelfile is included in this repository. Note that this downloads the weights to your machine (local inference).

Run it without owning a GPU

The model is self-contained GGUF, so it can be served on rented or free cloud compute with the llama.cpp container:

# Hugging Face Jobs - pulls the GGUF server-side, exposes an OpenAI-compatible URL
hf jobs run --detach --expose 8080 --flavor t4-small -s HF_TOKEN \
  ghcr.io/ggml-org/llama.cpp:server-cuda -- \
  /app/llama serve -hf Taimwe/qwen2.5-coder-7b-pro-merged:Q4_K_M \
  --host 0.0.0.0 --port 8080 -ngl 99 -c 8192 --jinja

Prompt format

ChatML, exactly as upstream Qwen2.5:

<|im_start|>system
You are a helpful coding assistant.<|im_end|>
<|im_start|>user
Write a binary search in Rust.<|im_end|>
<|im_start|>assistant

Tool calls use the <tools> / <tool_call> XML envelope emitted by the template in the GGUF metadata, so no extra prompt engineering is required.

Training and export pipeline

Reconstructed from the repository history - every step is verifiable in the commit log of this repository and of the adapter repository:

Step	Detail
Fine-tune	LoRA / 4-bit QLoRA, trained in Google Colab with Unsloth + TRL (PEFT 0.20.0)
Adapter	`Taimwe/qwen2.5-coder-7b-pro` - rank 16, alpha 16, dropout 0, all attention + MLP projections; pushed 2026-09-20 06:35 UTC
LoRA-in-GGUF (optional path)	`Taimwe/qwen2.5-coder-7b-pro-F16-GGUF`, 2026-09-20 07:38 UTC
Merge + GGUF export	Unsloth's GGUF export from the same notebook: commits "Trained with Unsloth", "- config", "- Ollama Modelfile" at 2026-09-20 09:20 UTC (this is also where the included `Modelfile` and `config.json` come from)
Quantisation	Q4_K_M -> `qwen2.5-coder-7b-instruct.Q4_K_M.gguf` (4.68 GB)

Known gaps

No fine-tuning data is published: the dataset, the number of steps/epochs, the learning rate, the training sequence length, the exact GPU and any evaluation numbers are absent from this account (it contains no datasets and the repositories contain no training logs). Until they are added, the effect of the fine-tune relative to Qwen/Qwen2.5-Coder-7B-Instruct is unverified.

Evaluation

No benchmark numbers have been published for this merge. Treat the deltas versus Qwen/Qwen2.5-Coder-7B-Instruct as unverified until measured (HumanEval, MBPP, or the Qwen2.5-Coder eval suite).

Limitations

Inherits all limitations of Qwen2.5-Coder-7B-Instruct, including weaker performance outside English and on very long reasoning chains.
A 7.6 B model cannot match ~30 B+ coders on complex repository-scale tasks.
Quantisation (Q4_K_M) costs a small amount of accuracy versus the F16 merge.

Contributing quants

# from the F16 GGUF of the merge
llama-quantize model-f16.gguf model-Q5_K_M.gguf Q5_K_M
# then open a PR against this repository

Licence and attribution

Released under Apache-2.0, following Qwen2.5-Coder-7B-Instruct (Apache-2.0). Please keep this licence and attribution if you redistribute.

Fine-tuned and exported with Unsloth, converted with llama.cpp.

Downloads last month: 231

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

4-bit

Model tree for Taimwe/qwen2.5-coder-7b-pro-merged

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-Coder-7B

Finetuned

Qwen/Qwen2.5-Coder-7B-Instruct

Quantized

(245)

this model