angelgalvisc/agent-a1-alchemist-4bit

Agent-A1 (The Alchemist) — 4-bit

A 4-billion-parameter agentic model compressed from 8.41 GB to 2.4 GB without retraining. Runs on a MacBook Air. Text, tools, images and video.

Built by Datastrat by compressing Agents-A1-4B from Shanghai AI Laboratory, itself built on Alibaba's Qwen3.5-4B. See NOTICE for who did what.

On a phone, an NVIDIA card or a plain CPU, use the GGUF build instead: angelgalvisc/agent-a1-alchemist-gguf. Same weights, repacked for llama.cpp — 2.92 GB, text and tools, no vision. This repository is the one for Apple Silicon, and the only one with the image encoder.

The quick way

curl -fLO https://huggingface.co/angelgalvisc/agent-a1-alchemist-4bit/resolve/main/install.sh
bash install.sh

It makes a virtual environment, installs mlx-lm and mlx-vlm, downloads the weights into ~/alchemist, and leaves a command called alchemist behind. Nothing goes system-wide and no password is asked for; to undo it, delete that folder and ~/.local/bin/alchemist. Read the script first if you would rather see what it does — it is short and it is all in install.sh.

Then, from anywhere:

alchemist                              # a conversation
alchemist "how much is 17 x 23?"       # one question
alchemist photo.png "what is this?"    # about a picture

Python 3.11 or newer is needed. macOS ships 3.9, so if the installer says it cannot find one: brew install python@3.12.

Or by hand

On a Mac with Apple Silicon — this is where the model was built and measured.

python3 -m venv .venv
source .venv/bin/activate
pip install mlx-lm            # text and tools
pip install mlx-vlm           # add this for images

8 GB of RAM is enough, 16 GB is comfortable, and you need 2.4 GB of disk.

On Linux with an Nvidia GPU — same files, same command, MLX has a CUDA backend.

python3 -m venv .venv
source .venv/bin/activate
pip install "mlx[cuda13]" mlx-lm

Needs an Nvidia driver 580 or newer. Everything measured for this model was run on Apple Silicon, and MLX's quantized matmuls on CUDA are still being worked on, so run a question on both before trusting that the outputs match.

Python 3.11 or newer either way.

Run it

python run.py "A cafe sells 3 coffees for 4.50. How much do 7 coffees cost?"

That is all. run.py ships with the model and sets the two things that decide whether the experience is good or maddening.

python run.py --chat                        # interactive
python run.py --think "..."                 # let it reason out loud first
python run.py --image photo.png "..."       # ask about an image
python run.py --video clip.mov "..."        # ask about a video
python run.py --sampling "..."              # InternScience's recommended sampling
python run.py --system "You are..." "..."   # your own system prompt

Images and video

The vision tower ships in vision.safetensors, uncompressed, and handles both. The same folder serves every case: mlx-lm loads the text half and ignores the rest, mlx-vlm loads both.

from mlx_vlm import load, generate

model, processor = load(".")
tok = getattr(processor, "tokenizer", processor)
msgs = [{"role": "user", "content": [
    {"type": "video"},                       # or {"type": "image"}
    {"type": "text", "text": "What happens here?"}]}]
prompt = tok.apply_chat_template(msgs, tokenize=False,
                                 add_generation_prompt=True,
                                 enable_thinking=False)
print(generate(model, processor, prompt, video=["clip.mov"], max_tokens=500))

Build the prompt this way rather than with mlx_vlm.prompt_utils.apply_chat_template. For this architecture that helper emits no placeholder for video, and generation then fails with tokens: 0, features N — the tower processed the clip but the prompt had nowhere to put it. run.py already does it correctly.

No audio. This architecture has no audio tower and no token reserved for one. Of the 168 architectures mlx-vlm supports, eight have audio; this is not among them. Put a transcriber such as mlx-whisper in front if you need speech.

The vision half is not compressed and was not measured. Compression touched only the language module; the 297 vision tensors are the original weights, unchanged. It was verified to work — it read a scanned legal document, counted the plates in a colour-vision test, and described a 37-second clip — but none of the 60 agentic tasks involves images, so there is no measurement of how well it does, only evidence that it does.

The one thing to know: this model has two modes

With thinking on it writes a reasoning block inside <think>...</think> and answers after it. With thinking off it answers directly. Both modes stop on their own — neither runs away — but they cost very different amounts:

                                        direct      --think
  "What is the capital of France?"       8 tokens    207 tokens
  a 3-step arithmetic question         111          753
  "Explain 4-bit quantization"        1611         2781

run.py defaults to off, and that is the important default: every figure this model was measured on — 48 of 60 on the agentic bench, the recovery batch, the held-out set — came from thinking-off. --think is the model's original mode and it works, but nothing measured here applies to it.

The budgets are 4096 direct and 8192 with --think, deliberately generous. Generation stops when the model is done, so unused budget costs nothing.

Decoding

run.py decodes greedily, which is what every measurement used, so the same question gives the same answer every time.

InternScience recommends, for this model, temperature 0.85, top_p 0.95, top_k 20 and presence penalty 1.1. --sampling uses exactly those. Replies get more varied and less repetitive; they also stop being reproducible.

No generation_config.json ships with this model, nor with the original, nor with Qwen3.5-4B underneath it. Each runtime therefore picks its own starting point — mlx-lm and transformers begin greedy, vLLM begins at temperature 1.0 — so if the same question behaves differently somewhere else, this is why.

If you would rather use mlx-lm directly

from mlx_lm import load, generate

model, tok = load(".")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is the capital of France?"}],
    tokenize=False, add_generation_prompt=True,
    enable_thinking=False)          # <- omit this and it will reason first
print(generate(model, tok, prompt=prompt, max_tokens=512))

The mlx_lm command line does not expose enable_thinking, so it always reasons. If you use it, give it room:

python -m mlx_lm generate --model . --prompt "..." --max-tokens 2048

What to expect

load time	2–3 seconds
generation	roughly 100 tokens/s on an M-series chip
context window	262,144 tokens
resident memory	~2.5 GB plus whatever the context uses

It is a tool-using agent, so it is at its best when given tools and a multi-step job. On plain chat it works, but that is not what it was built for.

If you are giving it tools

Nothing below applies to plain chat. It matters when your own program gives the model tools and runs the back-and-forth.

Use the names it already knows

The model was trained and benchmarked with three specific tools. Give it those names and schemas rather than your own — with different names it calls them far less often.

tool	arguments
google_search	query — a search string
read_page	url, and goal, what you want extracted from that page
PythonInterpreter	code

The implementations are not included here, and there is no way to include them: they are network services. InternScience's own versions are in Agents-A1/evaluation/Search and use paid keys. Any backend works as long as the name and the shape of the reply stay the same. Note that read_page is not a page dump: it returns a summary of the page written against goal.

Tell it how to finish

For multi-step work, end your system prompt the way InternScience does:

When you have gathered sufficient information and are ready to provide the definitive response, you must enclose the entire final answer within <answer></answer> tags.

That tag is the signal the model was trained to close on; without it, it tends to keep calling tools instead of concluding. Stop on three conditions, not one:

• the turn contains <answer>...</answer> — take what is between the tags;
• the turn asks for no tool — it has answered, tags or not, and a loop waiting only for the tags will spin past a perfectly good reply;
• a ceiling on rounds, always. InternScience's own harness allows 300 calls and a time limit, and when the budget runs out it sends one last message demanding an answer in the format above. A loop with no ceiling is a bug that bills by the hour.

Make the first move a search

For questions of fact — who wrote this, when did that happen — do not leave the decision to search up to the model. Asked to identify a line of poetry it half remembered, it answered from memory and invented an author in six runs out of eight; forced to search first, it found the right one and said so. Requiring a google_search before any answer is accepted costs one call and removes the common failure.

Two more rules for the loop

Add this line to the system prompt:

The tools read and write the user's own notes, documents, calendar and drafts. If a question is about the user's own data, look it up with the tools instead of saying you do not have access.

Without it, asked "how much do I spend a month?", the model will say it does not have access — even with a search tool right there.

And limit your runner to one tool call per turn, or refuse to run the same call twice with identical arguments. When this model gets stuck it copies its previous turn and adds one more call, so a loop doubles on its own until the budget runs out. This is a property of the model family, not of the compression.

What is in this folder

file
model.safetensors	the language weights, 4-bit, 2.395 GB
vision.safetensors	the vision tower, bf16, untouched, 0.667 GB
config.json	architecture and quantization metadata
tokenizer.json, tokenizer_config.json	tokenizer
chat_template.jinja	chat and tool-call format, with the enable_thinking switch
run.py	runs it with the right defaults
install.sh	sets up the environment and the alchemist command
chat_template.original.jinja	the upstream template, before the identity was changed
NOTICE, LICENSE	attribution and terms

Verify the download:

shasum -a 256 model.safetensors
# b14868593d907bb3863c75ec2e8fe2fdf5947da1eed56ea2b4835afd344c0ad9

Two things to know before you judge it

The vision tower is not compressed. Everything measured in this work is text and tool use. Images work, but no figure here describes how well.

It still gets stuck. On one task in sixty it repeats a failed call up to 32 times. Handle it in your runner, as above.

What was done to it

Nothing was retrained. The weights were rewritten to take less space:

1. The 248 matrices of the language module are stored with 4 bits per number instead of 16, in groups of 128, each group with its own scale.

   "4-bit" is the usual shorthand. The real cost is higher, because each group of 128 also stores a scale and a zero point:

   	parameters	bits each
   248 projections	3569.1 M	4.25
   vocabulary table	635.7 M	6.25
   normalisations, untouched	1.0 M	16 (bf16)
   average over the model	4205.8 M	4.555

   2.395 GB, which is what the file actually weighs.

2. Before rounding, 216 of them are rescaled per channel. The weights that read the high-signal inputs are enlarged, and those inputs are divided by the same factor — folded into the neighbouring layer, so the model computes exactly the same thing and the file does not grow. Rounding error on the channels that matter ends up divided by that factor.

3. The other 32 are stored plain: they are the attention outputs, and there is no earlier linear layer to absorb the compensating division.

4. The vocabulary table, which serves as both input lookup and output head, goes to 6 bits — below that there is a cliff.

5. No range is clipped. Clipping helps the text metrics and makes the model repeat failed calls, so it was dropped.

On a 60-task bench where the model plans and calls real tools, this scores 48 against the 43 of the previously released 4-bit version, at the same file size.

The full account of how it was compressed and measured is a separate document; ask Angel for it.

License

Apache 2.0 for this packaging. The model is InternScience's, under its own terms.