Instructions to use lbernat/Starter-MAGMA-Qwen3-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lbernat/Starter-MAGMA-Qwen3-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lbernat/Starter-MAGMA-Qwen3-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lbernat/Starter-MAGMA-Qwen3-4B")
model = AutoModelForCausalLM.from_pretrained("lbernat/Starter-MAGMA-Qwen3-4B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lbernat/Starter-MAGMA-Qwen3-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lbernat/Starter-MAGMA-Qwen3-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lbernat/Starter-MAGMA-Qwen3-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lbernat/Starter-MAGMA-Qwen3-4B

SGLang

How to use lbernat/Starter-MAGMA-Qwen3-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lbernat/Starter-MAGMA-Qwen3-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lbernat/Starter-MAGMA-Qwen3-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lbernat/Starter-MAGMA-Qwen3-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lbernat/Starter-MAGMA-Qwen3-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lbernat/Starter-MAGMA-Qwen3-4B with Docker Model Runner:
```
docker model run hf.co/lbernat/Starter-MAGMA-Qwen3-4B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Starter MAGMA — Qwen3 4B

A small, experimental starter checkpoint for trying MAGMA when you do not have your own model yet.

This model was fine-tuned briefly with LoRA on MAGMA data, then merged into the weights of Qwen/Qwen3-4B. It is intended to make initial setup, integration checks, and early experiments easier. Training was limited; expect mistakes and inconsistent behavior. No benchmark results or performance improvements are claimed for this release.

The repository contains the complete merged model, ready to load with Hugging Face Transformers. You do not need PEFT, an adapter, or a separate download of the base model.

Model details

Property	Value
Repository	`lbernat/Starter-MAGMA-Qwen3-4B`
Base model	`Qwen/Qwen3-4B`
Architecture	`Qwen3ForCausalLM`, decoder-only text model
Parameters	4,021,784,576 (approximately 4B)
Training method	LoRA fine-tuning, merged into the base weights
Weight format	Safetensors, BF16, two shards
Weight download size	Approximately 8.04 GB (7.49 GiB)
Configured context limit	40,960 tokens, including prompt and generation
Prompt format	Included MAGMA robot/tool chat template
License	Apache 2.0; see LICENSE and NOTICE

The context limit comes from the model configuration. Long-context performance has not been established for this fine-tune. The tokenizer shipped here matches the merged checkpoint, including its 151,669-token vocabulary; always load it from this repository.

Quick start with Transformers

Qwen3 requires Transformers 4.51 or newer. Use at least 4.51.3 for the example below; installing a recent version is recommended.

python -m pip install --upgrade "transformers>=4.51.3" torch accelerate safetensors

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "lbernat/Starter-MAGMA-Qwen3-4B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)
model.eval()

messages = [{"role": "user", "content": "What can you help me with?"}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt")
inputs = inputs.to(model.get_input_embeddings().weight.device)

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=128, do_sample=True)

completion = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(completion, skip_special_tokens=True))

torch_dtype="auto" loads the checkpoint in its stored BF16 precision. In recent Transformers versions, the argument is named dtype; torch_dtype is used here for compatibility with Transformers 4.x. Sampling uses the included generation defaults (temperature=0.6, top_p=0.95, top_k=20). BF16 weights alone occupy about 7.49 GiB; allow additional memory for inference and the KV cache. CPU inference is possible but slower, and device support for BF16 varies.

MAGMA prompts and tools

Keep the included chat template when using this checkpoint. It supplies a fixed system instruction for robot control, then renders the history, task attributes, and current query. For the normal request flow, use string message contents and end the conversation with the current user or tool/environment message.

The template accepts these optional arguments to apply_chat_template:

Argument	Expected value
`tools`	A list of JSON-serializable tool definitions
`permanent_rules`	A list of rule strings
`task_attributes`	A string containing JSON-encoded task attributes

Send a request with a tool

Once model and tokenizer are loaded as shown in the quick start, run:

import json

messages = [{"role": "user", "content": "Report the status of robot1."}]
tools = [{
    "type": "function",
    "function": {
        "name": "get_robot_status",
        "description": "Report the current status of the target robot.",
        "parameters": {"type": "object", "properties": {}, "required": []},
    },
}]
prompt = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    permanent_rules=["Only call tools that are available."],
    task_attributes=json.dumps({"known_robots": ["robot1"]}),
    tokenize=False,
    add_generation_prompt=True,
)
print(prompt)

inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt")
inputs = inputs.to(model.get_input_embeddings().weight.device)
with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        temperature=None,
        top_p=None,
        top_k=None,
    )

# Keep only the generated tokens, excluding the input prompt.
completion = output[0, inputs["input_ids"].shape[1]:]
raw_completion = tokenizer.decode(completion, skip_special_tokens=False)
completion_text = tokenizer.decode(completion, skip_special_tokens=True)
print(raw_completion)

This example uses greedy decoding; the sampling parameters are cleared because sampling is disabled. The tool definition is illustrative: replace it and the task attributes with those from your application.

Exact rendered prompt

For the inputs above, print(prompt) produces:

<|im_start|>system
You control robots via tools.

# TOOL INSTRUCTIONS
When calling a tool, output no user-facing text.
A response must contain either a message or a tool call, never both.
If you want to call a tool use:
<tool_call>{"robot1":{"name": "tool_name", "arguments": {"arg1": "value1", ...}}}</tool_call>

Available tools:
{
    "type": "function",
    "function": {
        "name": "get_robot_status",
        "description": "Report the current status of the target robot.",
        "parameters": {
            "type": "object",
            "properties": {},
            "required": []
        }
    }
}

Permanent rules:
- Only call tools that are available.
<|im_end|>

# HISTORY

# TASK ATTRIBUTES
{"known_robots": ["robot1"]}

# QUERY
<|im_start|>user
Report the status of robot1.
<|im_end|><|im_start|>assistant

add_generation_prompt=True appends <|im_start|>assistant, so the model continues with an assistant reply. tokenize=False returns this rendered string; the next tokenizer call converts it into token IDs. If you prefer to obtain IDs directly, use tokenize=True with apply_chat_template.

Actual model response

The following completion was generated locally from this checkpoint with Transformers 5.1.0 on CPU, using the request and greedy settings above. It stopped on the end-of-message token after 20 generated tokens.

With skip_special_tokens=False, the raw completion is:

<tool_call>{"robot1": {"name": "get_robot_status", "arguments": {}}}</tool_call><|im_end|>

With skip_special_tokens=True, it becomes:

<tool_call>{"robot1": {"name": "get_robot_status", "arguments": {}}}</tool_call>

The JSON inside the tool-call markers identifies the target robot, function, and arguments:

{
  "robot1": {
    "name": "get_robot_status",
    "arguments": {}
  }
}

The delimiters have dedicated IDs in the tokenizer shipped with this checkpoint:

Token	ID	Meaning
`<	im_start	>`
`<	im_end	>`
`<tool_call>`	151657	Starts the MAGMA tool-call JSON
`</tool_call>`	151658	Ends the MAGMA tool-call JSON

The tool-call delimiters are registered as ordinary added tokens, so skip_special_tokens=True preserves them while removing <|im_end|>. The generated completion does not contain an opening assistant role marker: that marker is already at the end of the input prompt.

This is one observed response, not a guarantee that every request produces a valid tool call. MAGMA uses a JSON object keyed by robot name; it is your application's responsibility to parse and validate it, check the tool and its arguments, and execute the selected function. Generating this text alone does not execute a tool.

Feed a tool result back into the conversation

After your application executes the tool, append the assistant's tool call and the actual tool result to the history. For example, if the tool reports that robot1 is idle:

messages.append({"role": "assistant", "content": completion_text})
messages.append({
    "role": "tool",
    "content": json.dumps({"robot1": {"status": "idle"}}),
})
next_prompt = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    permanent_rules=["Only call tools that are available."],
    task_attributes=json.dumps({"known_robots": ["robot1"]}),
    tokenize=False,
    add_generation_prompt=True,
)

Here idle is an illustrative tool result, not a status measured by the model. Tokenize next_prompt and generate again as above to obtain the next decision. The template asks for either a user-facing message or a tool call in each reply. It does not implement Qwen's enable_thinking switch; use the shipped MAGMA template and avoid replacing it with the base model's chat template.

Download for local use

from huggingface_hub import snapshot_download

local_path = snapshot_download(
    repo_id="lbernat/Starter-MAGMA-Qwen3-4B",
    local_dir="./Starter-MAGMA-Qwen3-4B",
)

Pass local_path to both AutoTokenizer.from_pretrained and AutoModelForCausalLM.from_pretrained. Once downloaded, use local_files_only=True for offline loading. For reproducible experiments, pin revision to a published commit when downloading or loading from the Hub.

Training data and evaluation

The fine-tuning dataset has not been published yet. The author plans to publish it separately and add its link to this card once available. This release does not include the dataset, the LoRA adapter, or a full training run record. Dataset composition, size, LoRA hyperparameters, and exact training duration are not documented here.

No task-success scores, comparative benchmarks, or robustness evaluation are reported. Successful loading and text generation only establish that the checkpoint can run; they do not establish task competence.

Intended use and limitations

Use this checkpoint to get a MAGMA setup running, check the model/tool interface, or begin experimenting before training or selecting a stronger model. Treat it as an initial baseline with limited training.

It may produce malformed JSON, choose incorrect tools, invent arguments, or fail to follow instructions. Validate outputs in your application and evaluate it on your own tasks before relying on its decisions. Robot-control prompts do not establish suitability for autonomous physical robot operation. General chat, reasoning, multilingual behavior, and long-context quality have not been evaluated for this fine-tune. It can also retain errors and biases from its base model and fine-tuning data.

License and attribution

This checkpoint is distributed under Apache 2.0, consistent with the base Qwen3-4B release. The upstream license and copyright notice are preserved in LICENSE. NOTICE identifies the MAGMA fine-tuning and merge. The future dataset release will document its own license separately.