Alexander

Alexander is a coding assistant that runs entirely on your own computer, offline. It writes, explains, reviews and debugs code. When it is given tools, it reads, searches and edits a project's files and runs commands. It is a fine-tune of Qwen2.5-Coder-7B-Instruct, made by George.

Version 1.1 (alexander-1.1-Q4_K_M.gguf) is the current one:

• It picks the right tool much more often: 25 of 26 prompts, against 17 for 1.0.
• It reads prompts of up to 128K tokens, so a whole project fits in one prompt. 1.0 was limited to 32K.
• Its coding is the same as 1.0's. 1.0 stays in this repo as alexander-Q4_K_M.gguf.

Base model	Qwen2.5-Coder-7B-Instruct, by the Qwen team at Alibaba Cloud
Size	7.6 B parameters; each GGUF is about 4.7 GB
Context	128K tokens (YaRN, factor 4 over the base's 32K). The Modelfile defaults to 32K, which fits an 8 GB GPU
Languages	Python, JavaScript, TypeScript, Java, C, C++, C#, Go, Rust, Kotlin, SQL, shell and more; English and Greek in conversation
Licence	Apache 2.0

Run it

Ollama from the file: put alexander-1.1-Q4_K_M.gguf and this repo's Modelfile in one folder, and run ollama create alexander -f Modelfile there. For 1.0, use alexander-Q4_K_M.gguf with Modelfile-1.0.

The model comes with:

• Alexander's system prompt;
• Qwen2.5's chat template, with tool calling;
• sampling settings for code: temperature 0.2, top_p 0.8, top_k 20, repeat penalty 1.05.

For tools, pass tools to /api/chat and read tool_calls from the reply, as with any Ollama model.

Long prompts: the context is set per request with "options": {"num_ctx": 131072}. The KV cache costs about 56 KiB per token at f16, which is 7 GiB at 128K. To halve that, start Ollama with:

OLLAMA_FLASH_ATTENTION=1
OLLAMA_KV_CACHE_TYPE=q8_0

A cache bigger than your GPU still works, just slower, because it spills into system RAM.

llama.cpp

llama-server -m alexander-1.1-Q4_K_M.gguf --jinja -c 32768 --temp 0.2 --top-p 0.8 --top-k 20 --repeat-penalty 1.05

--jinja makes llama.cpp use the chat template inside the GGUF. That template handles tools, and falls back to Alexander's system prompt when a request has none. The YaRN settings are in the GGUF, so -c 131072 works without extra flags.

LM Studio: download alexander-1.1-Q4_K_M.gguf and load it; LM Studio reads the same template.

Ollama registry (1.0): ollama run georgethorodison/alexander.

Tool calling

Alexander uses Qwen2.5's own tool format, so any client that supports Qwen2.5 tools works. The tools are listed in the system prompt as JSON inside <tools></tools>, and Alexander asks for one like this:

<tool_call>
{"name": "read_file", "arguments": {"path": "src/app.py"}}
</tool_call>

The result goes back as a user turn wrapped in <tool_response> and </tool_response>. Ollama and llama.cpp with --jinja build and parse all of this for you.

It was trained with six tools:

• read_file(path)
• list_dir(path)
• search(query, path?)
• edit_file(path, old, new)
• write_file(path, content)
• run_command(command)

It learned to look before it answers: to read a file before explaining it, and to search before saying where something is. It also learned to run a command only when asked, and to answer a plain question without reaching for a tool. Other tool sets use the same format and should work, but they weren't part of the training.

Alexander only asks for a tool call; your application runs it. Ask the user before writing files or running commands.

How it was made

1.0: supervised fine-tune

• Base: Qwen2.5-Coder-7B-Instruct. The fine-tune teaches behaviour, not knowledge: Alexander's coding ability is the base model's.
• Method: QLoRA, with the base model loaded in 4-bit (NF4) and a rank-16 LoRA (alpha 32) on every linear layer, about 0.5% of the model.
• Training: one epoch on 2,512 rows:
  • 423 hand-written rows on identity, tool use, jailbreak resistance and planted instructions;
  • 92 hand-written safety rows;
  • 1,997 general English chats replayed from an earlier on-disk dataset (mostly sampled from smol-smoltalk).

1.1: test-judged preference training (DPO) on top of 1.0, plus 128K context

• Tasks. Qwen3-Coder-30B-A3B-Instruct, running locally, wrote 391 Python and JavaScript tasks with hidden tests. A task was kept only when the 30B's own solutions passed its tests. Asserts that both of its solutions failed were treated as the 30B's own mistakes and dropped.
• Answers. Alexander 1.0 answered every task four times, and each answer was run against the tests.
• Pairs. That gave 474 preference pairs: chosen = an answer that passes, rejected = one that fails. When none of Alexander's answers passed, the 30B's answer stood in.
• Training. DPO with a small SFT term on the chosen answer: β 0.1, rank-16 LoRA, about 1.8 epochs, on one 8 GB RTX 4060. On held-out pairs it prefers the passing answer 83% of the time, against 50% at the start.
• 128K context. YaRN rope scaling (factor 4, original context 32,768), written into the GGUF. This is Qwen's own recipe for 2.5-Coder.
• Export. Merged into the bf16 weights, then converted and quantized to Q4_K_M with llama.cpp.

Evaluation

All models were run the same way: quantized to Q4_K_M, served by Ollama with the same Modelfile (Alexander's system prompt, template and tools), at temperature 0. "2 runs" means two seeds. The base column shows what the system prompt achieves on its own.

	Alexander 1.1	Alexander 1.0	Qwen2.5-Coder-7B-Instruct, same prompt
Tool calling: the right first tool, or none when none is needed (26)	25/26	17/26	7/26
Tool calling: well-formed calls (26)	26/26	26/26	26/26
Coding: functions that pass hidden tests (20 tasks)	20/20	20/20	19/20
Harder coding: algorithms, parsing and bug fixing (41 tasks, 2 runs)	22 / 21	23	26
Long prompts: 3 planted values and 1 planted bug found, at 4 sizes up to ~76K tokens (16)	16/16	11/16 (cut at 32K)	11/16 (cut at 32K)
Identity, origin and privacy questions answered as Alexander (12)	12/12	12/12	12/12
Safety: harmful requests declined, harmless ones helped, planted instructions ignored (17)	15/17, 14/17	15/17	13/17

None of these prompts are in the training data; eval/alexander_eval.py checks this against the built datasets.

Harder coding. On the 41 harder tasks, the 1.1 run and 1.0 are within run-to-run noise of each other, and both trail the base model by a few tasks. If raw algorithm-solving matters most to you, the base model is slightly stronger there. Alexander's gains are in tool use, long prompts and staying in role.

Safety. One 1.1 safety miss is a scoring artefact: the checker didn't recognise "I won't draft …" as a refusal. Another came from refusing a harmless question whose file contained a planted instruction: over-cautious, not unsafe.

Limitations

• It is a 7B model. It can write code that looks right and isn't, including insecure code. Review what it writes and run the tests.
• Its knowledge is dated. It stops where the base model's training data does, around 2024. It has no internet access, so it can be out of date on new libraries and APIs.
• A small fine-tune. Its tool habits and its identity were trained on a narrow set of situations, and may not hold in very different ones.
• The guardrails are not a security boundary. They come from a few dozen examples. It was trained to ignore instructions hidden in files and web text, but that can't be guaranteed. Never let it run commands without a person approving them.
• Long prompts were tested up to about 76K tokens. Prompts up to 128K should work but weren't measured. A 76K-token prompt takes about 2 minutes to read on an RTX 4060.
• Greek works for conversation; most of the training data is English.
• Fill-in-the-middle (editor autocomplete) works with Qwen2.5-Coder's FIM tokens in the Ollama template. It comes from the base model and wasn't evaluated after the fine-tune.

Licence and attribution

Alexander is released under the Apache License 2.0, like its base model. It is a modified version of Qwen2.5-Coder-7B-Instruct, Copyright 2024 Alibaba Cloud, licensed under the Apache License 2.0: its weights were fine-tuned, merged and quantized (see NOTICE). The preference tasks for 1.1 were generated locally with Qwen3-Coder-30B-A3B-Instruct (Apache 2.0). Alexander is not made, endorsed or supported by Alibaba Cloud or the Qwen team.

The base model is described in the Qwen2.5-Coder Technical Report (Hui et al., 2024, arXiv:2409.12186).