Lumma-0.6B-Tool

Based on Lumma-0.6B-Base, Lumma-0.6B-Tool is a lightweight, single-turn specialized model designed to accurately interpret user queries and generate precise tool calls in one step, enabling efficient and reliable function execution

The primary challenge was designing a non-thinking model that could match or exceed the tool-use performance of similarly sized thinking models, while delivering faster responses and greater efficiency.

Use cases:

• Mobile and edge devices: Enable low-latency API calls, database queries, and system integrations without relying on cloud-based reasoning.
• Real-time AI assistants: Power applications in vehicles, IoT devices, and customer support systems where fast response times are essential.
• Resource-constrained environments: Support efficient tool execution on embedded systems, battery-powered devices, and other hardware with limited compute and memory resources.

📄 Model details

List of Tools in System prompt: The system prompt must provide all the available tools, our chat template handles that.

Tool use: It consists of four main steps:

1. Function definition: Pass JSON tool definitions in the system prompt via tools_list= (inside <|im_start|><|system|>...<|endoftext|>).
2. Function call: Lumma returns a Pythonic call as assistant text: [FuncName(param=value)] between <|im_start|><|assistant|> and <|endoftext|>.
3. Function execution: Run the call and return the result as a tool message (usually JSON) between <|im_start|><|tool|> and <|endoftext|>.
4. Final answer: Lumma reads the tool output and replies in plain text as assistant.

🚀 Usage

import torch
import json
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "FrontiersMind/Lumma-0.6B-Tool"


tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_PATH, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="auto"
)
model.eval()


def generate(max_new_tokens,messages):
    prompt = tokenizer.apply_chat_template(
        messages,
        tools_list=tools_list,   # pass every turn
        tokenize=False,
        add_generation_prompt=True,
    )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(
            **inputs, max_new_tokens=max_new_tokens, do_sample=False,
            eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id,
        )
    text = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False)
    return text.split("<|endoftext|>")[0].strip()


tools = [
    {
        "name": "Quotes by Keywords",
        "description": "Returns a list of quotes containing the specified keyword.",
        "parameters": {
            "type": "dict",
            "properties": {"word": {"description": "The keyword to search for in quotes.", "type": "string"}},
            "required": ["word"],
        },
        "required": None,
    },
    {
        "name": "Get Zip Code Information",
        "description": "Retrieve information about a specific zip code in the United States.",
        "parameters": {
            "type": "dict",
            "properties": {
                "country": {"description": "The country code (default: 'us')", "type": "string"},
                "postal_code": {"description": "The zip code (default: '90210')", "type": "string"},
            },
            "required": ["country", "postal_code"],
        },
        "required": None,
    },
]

tools_list = json.dumps(tools)
messages = []

# --- Turn 1: user → tool call ---
messages.append({"role": "user", "content": 'Find quotes about "inspiration".'})
reply = generate(max_new_tokens=512,messages=messages)
messages.append({"role": "assistant", "content": reply})
print("Turn 1:", reply)

# --- Turn 2: tool result → answer ---
tool_call_response = """[{"name": "Quotes by Keywords", "results": {"quotes": [{"text": "Keep going.", "author": "Sam Levenson"}]}}]"""
messages.append({"role": "tool", "content": tool_call_response})
reply = generate(max_new_tokens=512,messages=messages)
messages.append({"role": "assistant", "content": reply})
print("Turn 2:", reply)

📬 Feedback & Suggestions

We’d love to hear your thoughts, feedback, and ideas!

• Discord: https://discord.gg/ZGdjCdRt
• Email: support@frontiersmind.ai
• Official Website https://www.frontiersmind.ai/
• LinkedIn: https://www.linkedin.com/company/frontiersmind/
• X (Twitter): https://x.com/FrontiersMind