tarif2108/gemma-3-270m-json-extractor

🚀 Gemma 3 270M — Structured JSON & Function Calling Extractor

gemma-3-270m-json-extractor is a fine-tuned version of google/gemma-3-270m-it optimized for strict, raw JSON output and structured function-calling extraction without conversational filler.

Despite its tiny footprint (270 million parameters), this fine-tuned model delivers a 6x reduction in latency and a **4x increase in valid JSON formatting accuracy** compared to the base model.

---

📊 Key Benchmark & Evaluation Results

Evaluated on a held-out test split of 100 complex structured JSON extraction prompts:

Metric	Base gemma-3-270m-it	Fine-Tuned (gemma-3-270m-json-extractor)	Improvement
JSON Validity Rate	23.0%	90.0%	+291%
Exact Schema Match	0.0%	21.0%	+21.0%
Key Coverage Rate	0.0%	59.0%	+59.0%
ROUGE-L Score	26.58	67.79	+155%
Avg Inference Latency	11.54s	1.92s	6x Faster

Key takeaway: The base model frequently rambled with conversational preamble ("Sure, here is your JSON..."), leading to long generation times and broken JSON syntax. The fine-tuned model immediately triggers JSON generation and cleanly emits the <eos> token upon completion.

---

💻 How to Use

Basic Inference with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tarif2108/gemma-3-270m-json-extractor"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

system_prompt = "You are a JSON generator. Reply ONLY with a single valid JSON object and nothing else."
user_prompt = "Extract user info into JSON with keys: name, age, city. Input: 'Hi, I'm Ada Lovelace, 28 years old, living in London.'"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id
)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)
# Expected Output: {"name":"Ada Lovelace","age":28,"city":"London"}

🛡️ Guaranteeing 100% Valid JSON (Constrained Decoding) While this fine-tuned model achieves 90% valid JSON natively, you can achieve 100% guaranteed schema enforcement during inference by combining this model with lm-format-enforcer:

pip install lm-formatenforcer pydantic

from pydantic import BaseModel
from lmformatenforcer import JsonSchemaParser
from lmformatenforcer.integrations.transformers import build_transformers_prefix_allowed_tokens_fn

class UserSchema(BaseModel):
    name: str
    age: int
    city: str

parser = JsonSchemaParser(UserSchema.model_json_schema())
prefix_fn = build_transformers_prefix_allowed_tokens_fn(tokenizer, parser)

# Pass prefix_allowed_tokens_fn to model.generate
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    prefix_allowed_tokens_fn=prefix_fn
)

---

⚙️ Training Details

• Base Model: google/gemma-3-270m-it
• Dataset: NousResearch/hermes-function-calling-v1 (~11,500 clean JSON rows across all subsets)
• Fine-Tuning Technique: QLoRA (4-bit NF4 quantization)
• LoRA Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• LoRA Parameters: r=16, alpha=32, dropout=0.05
• Hardware: NVIDIA RTX 3050 (4GB VRAM)
• Optimizer: paged_adamw_8bit
• Precision: bfloat16 compute
• Sequence Length: 512