Thanush16/llama-3.2-1b-function-calling

Llama-3.2-1B-Instruct — Function Calling (QLoRA)

A LoRA adapter that teaches Llama-3.2-1B-Instruct to emit well-formed function/tool calls. The base model understands tool-use intent but produces structurally incorrect JSON; this adapter fixes the output format.

What it does

Before (base model):

{"type": "function", "function": "get_weather", "parameters": {"city": "Nagoya", "unit": "celsius"}}

After (with this adapter):

[{"name": "get_weather", "arguments": {"city": "Nagoya", "unit": "celsius"}}]

The base model used "parameters" (wrong key) and a flattened structure. The adapter corrects it to the standard name/arguments format, wrapped in a list.

How to use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "meta-llama/Llama-3.2-1B-Instruct"
adapter_id = "Thanush16/llama-3.2-1b-function-calling"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()
tokenizer = AutoTokenizer.from_pretrained(adapter_id)

Build prompts with tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True).

Training

• Base model: meta-llama/Llama-3.2-1B-Instruct
• Method: QLoRA (4-bit base, LoRA rank 16 on attention + MLP layers, ~0.9% of params trained)
• Dataset: Salesforce/xlam-function-calling-60k, 500-example subset
• Config: 2 epochs, effective batch size 16, learning rate 2e-4
• Hardware: single free Kaggle T4 GPU
• Final mean token accuracy: ~88%

Limitations

Trained on 500 examples to fix output format, not to be a broadly capable tool-use model. It reliably produces well-formed calls but wasn't evaluated for tool-selection accuracy on hard or ambiguous queries. Only the LoRA adapter is released — load it alongside the base model.