Instructions to use elvin2929/functiongemma-270m-it-thinq-compat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use elvin2929/functiongemma-270m-it-thinq-compat with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M # Run inference directly in the terminal: llama cli -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M # Run inference directly in the terminal: llama cli -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
Use Docker
docker model run hf.co/elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use elvin2929/functiongemma-270m-it-thinq-compat with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "elvin2929/functiongemma-270m-it-thinq-compat" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "elvin2929/functiongemma-270m-it-thinq-compat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
- Ollama
How to use elvin2929/functiongemma-270m-it-thinq-compat with Ollama:
ollama run hf.co/elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use elvin2929/functiongemma-270m-it-thinq-compat with Docker Model Runner:
docker model run hf.co/elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
- Lemonade
How to use elvin2929/functiongemma-270m-it-thinq-compat with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull elvin2929/functiongemma-270m-it-thinq-compat:Q4_K_M
Run and chat with the model
lemonade run user.functiongemma-270m-it-thinq-compat-Q4_K_M
List all available models
lemonade list
- Atomic Chat
FunctionGemma 270M IT โ preformatted-prompt metadata compatibility experiment
Modified GGUF. Experimental, metadata-only derivative; not a fine-tuned model.
This repository contains one Q4_K_M GGUF derived from Google's
FunctionGemma 270M IT.
The immediate input was the Q4_K_M artifact distributed as version 1 of
palm/functiongemma-270m-it on Melange.
The original public model and its weights have not been modified in place.
This is an independent compatibility experiment, not an official Google or LG model.
Intended use and limitations
The target application supplies a complete, already serialized FunctionGemma prompt as the last message's content, including the function declarations, developer and user turns, and open model turn. The passthrough template avoids serializing those turns a second time. It is not a normal conversational chat template: do not pass an ordinary multi-message conversation to this model and expect it to format the conversation or tools for you. Earlier messages are ignored by this template. BOS insertion remains tokenizer-owned.
The added EOT metadata marks <start_function_response> as an intended
generation boundary after a function call. Whether a particular SDK/runtime
honors this metadata remains to be tested. This does not add native function
calling support to an SDK.
The target ThinQ demo executes tools through a mock adapter. This repository does not contain appliance profiles, app source, credentials, private prompts, device logs, or any real-appliance connection. It does not expand the model's training, supported appliance operations, or demonstrated accuracy.
Status at publication: structural integrity checks and host template rendering checks passed. Android/iOS SDK generation, stop behavior, function calling accuracy, Melange delivery, and Device Farm validation have not yet been established for this derivative. No runtime success or safety claim is made.
Exact metadata changes
Only these two stored GGUF metadata entries differ from the immediate input:
| Key | New value | GGUF type |
|---|---|---|
tokenizer.chat_template |
{{ messages[-1]['content'] }} |
STRING |
tokenizer.ggml.eot_token_id |
50 (<start_function_response>) |
UINT32 |
The template has no trailing newline and emits no textual BOS. Existing
add_bos_token=true, BOS ID 2, EOS ID 1, and add_eos_token=false are unchanged.
All other stored metadata, token vocabulary, tensor types, shapes, and weights
are preserved. The GGUF header's metadata-entry count necessarily increases by
one when adding the previously absent EOT key.
Artifact integrity
File: functiongemma-270m-it-thinq-compat-Q4_K_M.gguf
- Format: GGUF v3; 236 tensors.
- File size: 253,113,408 bytes.
- File SHA-256:
35373153ddbb6ad6ede046d45efdfba49c1ab918a75f7adac83a6beb7250d19d. - Immediate input size: 253,127,136 bytes.
- Immediate input SHA-256:
51c8c84d40dcf313482970bf8b3db34d11e61e9bdb3450314be5b3d647dc92ab. - Ordered raw tensor payload SHA-256 (identical before/after):
79ff0e9b47749b89f38d7bd303c6c0248cadd56746d073641ca205308e785a59.
Verification compared every tensor payload byte and all non-target stored metadata. A separate raw GGUF parser independently checked the same invariants. This is evidence of preserved model data, not runtime equivalence or accuracy. Modification date: 2026-10-21.
License and required use restrictions
The model and this derivative are governed by the Gemma Terms of Use, reproduced in LICENSE, including Section 3.2 and the incorporated Gemma Prohibited Use Policy. Use and distribution of this derivative are subject to those restrictions; no additional grant overrides them. See also NOTICE. By using, modifying, or distributing the model, you are subject to the agreement as specified in that license. Google retains the rights in the upstream model.
- Downloads last month
- -
4-bit
Model tree for elvin2929/functiongemma-270m-it-thinq-compat
Base model
google/functiongemma-270m-it