antontuzovAI/MiniCPM5-2B-xLAM-Agent-LoRA

MiniCPM5-2B xLAM Agent Tool-Calling (QLoRA)

This is a highly optimized QLoRA adapter for openbmb/MiniCPM5-2B, specifically fine-tuned to act as a strict Function-Calling / Tool-Use agent.

It was trained via streaming on real-world data from the Salesforce/xlam-function-calling-60k dataset. The model is trained to output ONLY valid JSON arrays of tool calls, without any conversational filler, markdown, or <think> reasoning tags.

🚀 Key Features

• Strict JSON Output: Optimized to return [{"name": "tool", "arguments": {...}}] directly.
• Real-World APIs: Trained on 5,000 diverse, complex, real-world API schemas from the xLAM dataset.
• Lightweight: Uses 4-bit QLoRA (rank 16), making it extremely fast and cheap to deploy alongside the base MiniCPM5-2B model.
• T4 Friendly: Training and inference were optimized to run on standard 16GB VRAM GPUs (like the Tesla T4).

💻 How to Use

You need the base model and peft to load this adapter.

import torch
import json
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

model_id = "openbmb/MiniCPM5-2B"
adapter_id = "antontuzovAI/MiniCPM5-2B-xLAM-Agent-LoRA" # Replace with your actual repo ID

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token

base_model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True
)

# Load the Agent LoRA
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

# Define Tools
tools = [
    {
        "name": "get_weather",
        "description": "Get current weather.",
        "parameters": {"city": {"type": "string", "required": True}}
    }
]

prompt = f"""You are a strict tool-calling agent. Output ONLY JSON.
Tools: {json.dumps(tools)}
User: What is the weather in Tokyo?
Assistant:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
# Expected Output: [{"name": "get_weather", "arguments": {"city": "Tokyo"}}]