Instructions to use Sky-Kim/functiongemma-270m-finetune with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sky-Kim/functiongemma-270m-finetune with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Sky-Kim/functiongemma-270m-finetune")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Sky-Kim/functiongemma-270m-finetune", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Sky-Kim/functiongemma-270m-finetune with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sky-Kim/functiongemma-270m-finetune"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sky-Kim/functiongemma-270m-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Sky-Kim/functiongemma-270m-finetune

SGLang

How to use Sky-Kim/functiongemma-270m-finetune with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sky-Kim/functiongemma-270m-finetune" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sky-Kim/functiongemma-270m-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sky-Kim/functiongemma-270m-finetune" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sky-Kim/functiongemma-270m-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Sky-Kim/functiongemma-270m-finetune with Docker Model Runner:
```
docker model run hf.co/Sky-Kim/functiongemma-270m-finetune
```

FunctionGemma fine-tune + Sentis export (EN/KO)

A reproducible recipe that fine-tunes google/functiongemma-270m-it into a bilingual (Korean + English) smart-home function-calling model and exports it to a single ONNX graph that runs on-device in Unity Sentis 2.6 (com.unity.ai.inference). Ships everything end-to-end — the dataset, the merged model, the ONNX graph, and the scripts that produce them (the ~1 GB weights and graph are tracked with Git LFS). Swap the tools/utterances and the pipeline is generic.

Base: Gemma 3, 270M · 18 layers · MQA · head_dim 256 · vocab 262144 · sliding_window 512
Method: completion-only LoRA SFT (r=16), merged to a full model
Data: 4,091 rows (3,764 train / 327 val), EN+KO, single/multi/triple-call + refusals

Layout

scripts/   paths.py (folder layout) · schema.py (tools+prompt, single source of truth)
           1_gen_dataset · 2_finetune_lora · 3_merge_lora · 4_export_onnx_sentis
data/      train.jsonl · val.jsonl                          step 1 output
model/     merged model + config + chat_template + dev_block  step 2/3 output (weights via LFS)
onnx/      functiongemma_home.onnx                           step 4 output (~1 GB, via LFS)
base/      base model — download in step 0                   (not committed)
work/      LoRA adapter + train.log                          step 2 scratch (not committed)

Tools (16)

Rooms apply only to lights; TV / computer / vacuum / speaker are single devices with no room. General-knowledge questions go to web_search; chit-chat, unsupported devices, and relative changes with no number are refused in text (for a downstream router to escalate).

Tool	Args
`turn_on_light` / `turn_off_light`	`room` (req); `turn_on` also `brightness` 0–100
`set_light_color`	`room`, `color` enum (red/orange/yellow/green/blue/purple/pink/white/warm/cool)
`turn_on_tv` / `turn_off_tv` / `turn_on_computer` / `turn_off_computer` / `start_vacuum`	—
`play_music`	`genre` enum (rock/jazz/lofi), optional (default jazz)
`stop_music` / `get_volume` / `get_location`	—
`set_volume`	`volume` 0–100 (absolute)
`get_weather`	`city`
`get_time`	`city` (optional)
`web_search`	`query`

Output is FunctionGemma's native flat call, chained for multi-intent:

<start_function_call>call:turn_on_light{room:<escape>거실<escape>}<end_function_call><start_function_call>call:play_music{genre:<escape>jazz<escape>}<end_function_call>

Load the model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("model")
model = AutoModelForCausalLM.from_pretrained("model", dtype=torch.float32).eval()
CT = open("model/chat_template.jinja").read()

from scripts.schema import TOOLS, DEV   # or copy the two definitions
prompt = tok.apply_chat_template(
    [{"role": "developer", "content": DEV},
     {"role": "user", "content": "거실 불 30퍼센트로 켜줘"}],
    tools=TOOLS, chat_template=CT, add_generation_prompt=True, tokenize=False)
ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=[1, 50, 106])
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=False))
# -> <start_function_call>call:turn_on_light{brightness:30,room:<escape>거실<escape>}<end_function_call>

Reproduce

Run in order from the package root. Paths live in scripts/paths.py (override with FG_BASE/FG_DATA/FG_MODEL/FG_ONNX/FG_WORK); outputs land where the package ships them. Produced on Apple MPS, also runs on CUDA/CPU.

pip install -r requirements.txt

# 0. base model
huggingface-cli download google/functiongemma-270m-it --local-dir base

# 1. dataset  -> data/ + model/fg_dev_block_home.txt   (seeded; reproduces the shipped data)
python scripts/1_gen_dataset.py

# 2. LoRA SFT then merge  -> work/lora_best/ + model/   (BS=4 GRADCKPT=1 on a tight-memory Mac)
EPOCHS=5 BS=4 GRADCKPT=1 python scripts/2_finetune_lora.py

# 3. standalone merge (optional; step 2 already merges)
python scripts/3_merge_lora.py

# 4. export + ORT parity gate  -> onnx/functiongemma_home.onnx
python scripts/4_export_onnx_sentis.py
#    GATE_ONLY=1 re-gates an existing graph; FG_UNITY_ASSET=<path> also copies it into a Unity project.

Step 4 ends with an ORT parity gate: it decodes a held-out command set (BENCH in schema.py) through ONNXRuntime and checks each yields the expected call, confirming the exported graph still matches the model.

Sentis notes

The export handles the Sentis-specific surgery (weights inlined, Gelu→tanh ops, KV cache as plain graph I/O, stop tokens 1/50/106). The one trap worth knowing: Gemma 3 uses sliding-window attention (512) on most layers, so the graph takes an explicit mask_sliding and the runtime must build it too — feeding a plain causal mask silently collapses generation once the prefix exceeds 512 tokens. Put a schema/whitelist validator between the model output and execution; small SLMs still occasionally hallucinate a call.

License

Governed by the Gemma Terms of Use. Base model: Google FunctionGemma-270m-it.

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for Sky-Kim/functiongemma-270m-finetune

Base model

google/functiongemma-270m-it

Adapter

(30)

this model