gemma4-e4b-agent-lime-v1-fft100

Experimental Gemma4 E4B full-finetune snapshot for coding-agent behavior, tool-call style formatting, repository/debug workflows, structured reasoning, failure recovery, and a small amount of Korean-first Lime-style response tone.

This is a research artifact. It is meant to document and test a specific data mixture, not to claim production readiness.

Model Summary

• Base checkpoint: local gemma4-e4b-it export at training time.
• Architecture: Gemma4ForConditionalGeneration.
• Published format: BF16 model.safetensors.
• Hugging Face safetensors metadata reports about 7.94B BF16 parameters.
• Training method: full fine-tune, not LoRA.
• Primary language coverage: English plus Korean.
• Primary use: coding-agent prompts, tool-call shaped outputs, repo/debug task summaries, structured reasoning, and concise Korean responses.
• Not intended for: autonomous real-world tool execution without a controller, deployment without validation, hidden chain-of-thought training, vision/audio tasks, or safety-critical decisions.

The repository name uses fft100 to identify the published checkpoint-style snapshot. The local training config was set up as a 300-step run with saves every 100 steps, so this should be treated as an intermediate experimental snapshot unless a trainer_state.json is published later.

Intended Behavior

The data was built around this Agent-Lime system prompt:

You are Lime, a pragmatic Gemma4 E4B coding and tool-use agent. Read the current task carefully, use structured visible reasoning when useful, make tool calls only when the prompt provides tools, recover from errors, prefer small verifiable steps, and keep a concise Korean-first assistant voice when Korean is used.

The target behavior is practical agent output:

• read the task before acting;
• make small, verifiable coding/debugging steps;
• emit tool-call shaped JSON only when tools are provided in the prompt;
• recover from failed commands or bad assumptions;
• keep Korean responses concise and task-focused when the user writes Korean.

Training Recipe

Setting	Value
Method	full_finetune_agent_lime_e4b
Base model path	gemma4 e4b
Precision	BF16
Gradient checkpointing	enabled
Input field	pre-rendered Gemma4 turn text
Max sequence length	2048
Learning rate	1.5e-6
Scheduler	cosine
Warmup ratio	0.03
Weight decay	0.01
Optimizer	adamw_torch_fused
Micro batch size	1
Gradient accumulation	16
Configured max steps	300
Eval interval	50 steps
Save interval	100 steps
Seed	20260709

Data Mix

The dataset was rebuilt from raw/local source folders under data/; previous Gemma4-derived SFT mixes were not used as inputs.

Source family	Rows
qwen_toolcalling	11,356
open_perfectblend	10,000
OpenThoughts-114k	9,092
glaive_function_calling	8,734
hermes_function_calling	6,204
open_swe_traces	5,160
korean_style	1,200
deepseek_ko_reasoning	796
synthetic_agent_lime	800
Total	53,342

Split:

Split	Rows
Train	50,842
Eval	2,500

Data Policy

• Raw inputs were read from source folders, not previous Gemma4 derived SFT mixes.
• Tool-calling rows preserved tool schemas as user-visible context.
• Function calls were normalized toward JSON tool_call objects.
• Tool role outputs were converted to user-visible Tool result messages so the Gemma4 text-only turn format stayed valid.
• Open-SWE traces were condensed into coding-agent patch-summary rows instead of raw full trajectories.
• OpenThoughts and DeepSeek reasoning rows had hidden-thought tags stripped before writing.
• Korean writing style was kept small to avoid overpowering coding and tool-use behavior.
• Synthetic Agent-Lime rows taught repo/debug/tool-routing/failure-recovery reporting habits.
• System prompts were replaced with the Agent-Lime Gemma4 E4B prompt.
• Rows were de-duplicated by normalized message content.
• Text was rendered with the local Gemma4 turn template: <bos><|turn>role\n...<turn|>.

Denied or stripped markers included:

• hidden-thought markers such as <think>, <|think|>, and related tags;
• Qwen ChatML markers such as <|im_start|> and <|im_end|>;
• raw tool-call/tool-response wrapper variants that did not match the target Gemma4 format;
• image rows.

Usage

Use a Transformers build that supports Gemma4ForConditionalGeneration and the included chat_template.jinja.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "naksyu/gemma4-e4b-agent-lime-v1-fft100"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": "You are Lime, a pragmatic Gemma4 E4B coding and tool-use agent. Read the current task carefully, use structured visible reasoning when useful, make tool calls only when the prompt provides tools, recover from errors, prefer small verifiable steps, and keep a concise Korean-first assistant voice when Korean is used.",
    },
    {
        "role": "user",
        "content": "A unit test fails after a config change. Give me a short debugging plan.",
    },
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
inputs = {key: value.to(model.device) for key, value in inputs.items()}

outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(processor.decode(outputs[0], skip_special_tokens=True))

Tool-Call Caveat

This model learned tool-call shaped text from prompt examples. It does not execute tools by itself. A runtime/controller must parse, validate, authorize, and execute any tool call. Do not connect it directly to destructive, private, or paid tools without an explicit safety layer.

Limitations

• This model has not been released with a full benchmark report.
• Tool-call JSON may still be malformed or mismatched to a runtime schema.
• Coding/debug suggestions still need real tests and code review.
• Korean style data is intentionally a small slice, so tone may be inconsistent.
• It was tuned as text-only even though the base architecture may expose multimodal tokens.
• Safety behavior was not the main target of this tune.

License and Terms

This model is a derivative experimental tune of the base Gemma4 E4B checkpoint. Use is subject to the base model terms and to the licenses or terms of the upstream data sources used in the mix. This model card does not grant separate rights to redistribute any underlying datasets.

Provenance Files

Local build metadata used for this card:

• train_gemma4_e4b_agent_lime_v1_fft.yaml
• gemma4_e4b_agent_lime_v1_manifest.json
• gemma4_e4b_agent_lime_v1_used.md