Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-121/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-242/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-363/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,181 @@
+---
+library_name: peft
+license: gemma
+base_model: google/gemma-4-26b-a4b-it
+tags:
+  - security
+  - secure-code
+  - cybersecurity
+  - qlora
+  - gemma4
+  - code-generation
+  - owasp
+  - ai-security
+datasets:
+  - scthornton/securecode
+  - scthornton/securecode-web
+pipeline_tag: text-generation
+model-index:
+  - name: gemma4-26b-securecode
+    results: []
+---
+
+# Gemma 4 26B-A4B SecureCode
+
+**Security-specialized code generation model** fine-tuned on the [SecureCode](https://huggingface.co/datasets/scthornton/securecode) and [SecureCode Web](https://huggingface.co/datasets/scthornton/securecode-web) datasets.
+
+Part of the [SecureCode model collection](https://huggingface.co/collections/scthornton/securecode) by [perfecXion.ai](https://perfecxion.ai).
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| **Base Model** | [google/gemma-4-26b-a4b-it](https://huggingface.co/google/gemma-4-26b-a4b-it) |
+| **Architecture** | Gemma 4 Mixture-of-Experts (26B total, 4B active per token) |
+| **Method** | QLoRA (4-bit NormalFloat quantization) |
+| **Parameters Trained** | ~1-2% via LoRA adapters |
+| **Tier** | Tier 3: Large Security Specialist |
+
+## Training Configuration
+
+### QLoRA Settings
+
+| Parameter | Value |
+|-----------|-------|
+| Quantization | 4-bit NormalFloat (NF4) |
+| Compute Dtype | bfloat16 |
+| Double Quantization | Enabled |
+| LoRA Rank | 16 |
+| LoRA Alpha | 32 |
+| LoRA Dropout | 0.05 |
+| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+
+### Training Hyperparameters
+
+| Parameter | Value |
+|-----------|-------|
+| Learning Rate | 2e-4 |
+| LR Scheduler | Cosine with 100-step warmup |
+| Epochs | 3 |
+| Per-device Batch Size | 2 |
+| Gradient Accumulation | 8x |
+| Effective Batch Size | 16 |
+| Max Sequence Length | 4,096 tokens |
+| Optimizer | paged_adamw_8bit |
+| Precision | bf16 |
+
+### Hardware
+
+| Component | Specification |
+|-----------|--------------|
+| System | NVIDIA DGX Spark |
+| GPU | NVIDIA GB10 |
+| Memory | 128 GB Unified (CPU/GPU) |
+
+## Training Data
+
+Combined and deduplicated from two datasets:
+
+| Dataset | Examples | Focus |
+|---------|----------|-------|
+| [scthornton/securecode](https://huggingface.co/datasets/scthornton/securecode) | 2,185 | Web + AI/ML security (OWASP Top 10 2021 + LLM Top 10 2025) |
+| [scthornton/securecode-web](https://huggingface.co/datasets/scthornton/securecode-web) | 1,378 | Web security with framework-specific patterns |
+
+### Coverage
+
+**Vulnerability Standards:**
+- OWASP Top 10 2021 (Web/Application Security)
+- OWASP LLM Top 10 2025 (AI/ML Security)
+- 92+ CWEs mapped
+
+**Programming Languages:** Python, JavaScript, Java, Go, PHP, TypeScript, C#, Ruby, Rust, Kotlin, YAML, HCL
+
+**Frameworks:** 49+ including LangChain, OpenAI, Anthropic, HuggingFace, Django, Express.js, Spring Boot, FastAPI, and more
+
+**Training Format:** 4-turn conversational examples:
+1. Developer asks about implementing a feature
+2. Assistant provides vulnerable + secure implementations with attack demonstrations
+3. Developer asks about testing and edge cases
+4. Assistant delivers defense-in-depth operational guidance
+
+Every example is grounded in real CVEs and published security incidents.
+
+## Usage
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+import torch
+
+# Load with 4-bit quantization (matches training)
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+)
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    "google/gemma-4-26b-a4b-it",
+    quantization_config=bnb_config,
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained("scthornton/gemma4-26b-securecode")
+model = PeftModel.from_pretrained(base_model, "scthornton/gemma4-26b-securecode")
+
+messages = [
+    {"role": "user", "content": "How do I implement JWT authentication with refresh tokens in Python?"}
+]
+
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
+outputs = model.generate(inputs, max_new_tokens=2048, temperature=0.7)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## What Makes This Different
+
+Standard code models generate functional but often **insecure** code. SecureCode-trained models:
+
+- Generate **secure implementations by default** with proper input validation, parameterized queries, and cryptographic best practices
+- Provide **vulnerable AND secure** code side-by-side so developers understand the risk
+- Include **defense-in-depth guidance**: logging, monitoring, SIEM integration, and infrastructure hardening
+- Cover **AI/ML-specific vulnerabilities**: prompt injection defenses, RAG security, model supply chain protection
+
+## SecureCode Model Collection
+
+| Model | Parameters | Base |
+|-------|-----------|------|
+| [llama-3.2-3b-securecode](https://huggingface.co/scthornton/llama-3.2-3b-securecode) | 3B | Llama 3.2 3B |
+| [codegemma-7b-securecode](https://huggingface.co/scthornton/codegemma-7b-securecode) | 7B | CodeGemma 7B IT |
+| [deepseek-coder-6.7b-securecode](https://huggingface.co/scthornton/deepseek-coder-6.7b-securecode) | 6.7B | DeepSeek Coder |
+| [qwen-coder-7b-securecode](https://huggingface.co/scthornton/qwen-coder-7b-securecode) | 7B | Qwen Coder 7B |
+| [codellama-13b-securecode](https://huggingface.co/scthornton/codellama-13b-securecode) | 13B | Code Llama 13B |
+| [qwen2.5-coder-14b-securecode](https://huggingface.co/scthornton/qwen2.5-coder-14b-securecode) | 14B | Qwen 2.5 Coder 14B |
+| [starcoder2-15b-securecode](https://huggingface.co/scthornton/starcoder2-15b-securecode) | 15B | StarCoder2 15B |
+| [granite-20b-code-securecode](https://huggingface.co/scthornton/granite-20b-code-securecode) | 20B | Granite 20B Code |
+| **gemma4-26b-securecode** | **26B (4B active)** | **Gemma 4 26B-A4B IT** |
+
+## Limitations
+
+- Training data focuses on defensive security patterns; not designed for offensive security tooling
+- 4-turn conversation format may not generalize to all coding interaction patterns
+- MoE architecture means only 4B parameters are active per token despite 26B total
+- Security guidance reflects best practices as of early 2026; new vulnerabilities may not be covered
+
+## License
+
+- **Model:** Gemma license (inherited from base model)
+- **Dataset:** CC BY-NC-SA 4.0
+- **Adapters:** CC BY-NC-SA 4.0
+
+## Citation
+
+```bibtex
+@misc{thornton2026securecode,
+  title={SecureCode: A Production-Grade Multi-Turn Dataset for Training Security-Aware Code Generation Models},
+  author={Thornton, Scott},
+  year={2026},
+  publisher={perfecXion.ai},
+  url={https://huggingface.co/datasets/scthornton/securecode},
+  note={arXiv:2512.18542}
+}
+```

adapter_config.json

@@ -0,0 +1,246 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-4-26b-a4b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.2.dev0@e7355a3b2233820f6f30e558ce133ed22673a087",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.4.self_attn.k_proj",
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.3.mlp.up_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.8.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.k_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.6.mlp.up_proj",
+    "model.language_model.layers.24.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.q_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.up_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.26.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.down_proj",
+    "model.language_model.layers.11.mlp.up_proj",
+    "model.language_model.layers.2.self_attn.o_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.k_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.v_proj",
+    "model.language_model.layers.10.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.25.self_attn.q_proj",
+    "model.language_model.layers.5.self_attn.o_proj",
+    "model.language_model.layers.2.mlp.gate_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.23.self_attn.o_proj",
+    "model.language_model.layers.27.mlp.gate_proj",
+    "model.language_model.layers.0.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.4.self_attn.o_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.11.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.o_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.up_proj",
+    "model.language_model.layers.2.mlp.up_proj",
+    "model.language_model.layers.0.mlp.gate_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.10.mlp.down_proj",
+    "model.language_model.layers.10.mlp.gate_proj",
+    "model.language_model.layers.0.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.1.self_attn.v_proj",
+    "model.language_model.layers.13.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.3.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.o_proj",
+    "model.language_model.layers.7.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.v_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.k_proj",
+    "model.language_model.layers.5.self_attn.k_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.1.mlp.gate_proj",
+    "model.language_model.layers.8.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.q_proj",
+    "model.language_model.layers.23.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.k_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.10.mlp.up_proj",
+    "model.language_model.layers.11.mlp.gate_proj",
+    "model.language_model.layers.1.mlp.up_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.gate_proj",
+    "model.language_model.layers.7.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.up_proj",
+    "model.language_model.layers.5.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.down_proj",
+    "model.language_model.layers.22.mlp.gate_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.gate_proj",
+    "model.language_model.layers.26.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.q_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.28.self_attn.k_proj",
+    "model.language_model.layers.24.self_attn.o_proj",
+    "model.language_model.layers.26.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.25.mlp.up_proj",
+    "model.language_model.layers.27.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.0.mlp.down_proj",
+    "model.language_model.layers.6.self_attn.v_proj",
+    "model.language_model.layers.4.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.0.self_attn.q_proj",
+    "model.language_model.layers.27.mlp.up_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.12.mlp.up_proj",
+    "model.language_model.layers.6.mlp.down_proj",
+    "model.language_model.layers.2.mlp.down_proj",
+    "model.language_model.layers.6.mlp.gate_proj",
+    "model.language_model.layers.24.self_attn.v_proj",
+    "model.language_model.layers.4.mlp.up_proj",
+    "model.language_model.layers.9.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.gate_proj",
+    "model.language_model.layers.5.mlp.down_proj",
+    "model.language_model.layers.13.self_attn.o_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.24.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.26.mlp.up_proj",
+    "model.language_model.layers.26.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.o_proj",
+    "model.language_model.layers.8.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.o_proj",
+    "model.language_model.layers.6.self_attn.k_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.12.self_attn.k_proj",
+    "model.language_model.layers.13.mlp.down_proj",
+    "model.language_model.layers.1.mlp.down_proj",
+    "model.language_model.layers.3.mlp.gate_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.o_proj",
+    "model.language_model.layers.0.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.8.self_attn.k_proj",
+    "model.language_model.layers.12.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.13.mlp.up_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.25.self_attn.k_proj",
+    "model.language_model.layers.8.self_attn.q_proj",
+    "model.language_model.layers.4.self_attn.v_proj",
+    "model.language_model.layers.23.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.o_proj",
+    "model.language_model.layers.5.mlp.up_proj",
+    "model.language_model.layers.13.self_attn.k_proj",
+    "model.language_model.layers.7.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.o_proj",
+    "model.language_model.layers.22.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.24.self_attn.q_proj",
+    "model.language_model.layers.12.self_attn.q_proj",
+    "model.language_model.layers.2.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.v_proj",
+    "model.language_model.layers.13.mlp.gate_proj",
+    "model.language_model.layers.12.mlp.down_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.gate_proj",
+    "model.language_model.layers.13.self_attn.v_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.5.mlp.gate_proj",
+    "model.language_model.layers.1.self_attn.q_proj",
+    "model.language_model.layers.11.mlp.down_proj",
+    "model.language_model.layers.0.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.7.mlp.up_proj",
+    "model.language_model.layers.22.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.o_proj",
+    "model.language_model.layers.23.mlp.down_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd22f3389135f43d99a3e2a496b4df257c053e7777182b566f577e496b885a88
+size 74403016

chat_template.jinja

@@ -0,0 +1,266 @@
+{%- macro format_parameters(properties, required) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'OBJECT' -%}
+                ,properties:{
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                {%- elif value is mapping -%}
+                    {{- format_parameters(value, value['required'] | default([])) -}}
+                {%- endif -%}
+                }
+                {%- if value['required'] -%}
+                    ,required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    ,items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(prev_message_type=None) -%}
+{%- set loop_messages = messages -%}
+{{ bos_token }}
+{#- Handle System/Tool Definitions Block -#}
+{%- if (enable_thinking is defined and enable_thinking) or tools or messages[0]['role'] in ['system', 'developer'] -%}
+    {{- '<|turn>system\n' -}}
+
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking is defined and enable_thinking -%}
+        {{- '<|think|>' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{- messages[0]['content'] | trim -}}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+        {{- '<|turn>' + role + '\n' }}
+
+            {%- if message['tool_calls'] -%}
+                {%- for tool_call in message['tool_calls'] -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is string -%}
+                        {{- function['arguments'] -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- if message['tool_responses'] -%}
+                {#- Tool Response handling -#}
+                {%- for tool_response in message['tool_responses'] -%}
+                    {{- '<|tool_response>' -}}
+                    {%- if tool_response['response'] is mapping -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{' -}}
+                        {%- for key, value in tool_response['response'] | dictsort -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                            {%- if not loop.last %},{% endif -%}
+                        {%- endfor -%}
+                        {{- '}' -}}
+                    {%- else -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{value:' + format_argument(tool_response['response'], escape_keys=False) + '}' -}}
+                    {%- endif -%}
+                    {{- '<tool_response|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_response' -%}
+            {%- endif -%}
+
+            {%- if message['content'] is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message['content'] is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item['type'] == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item['type'] == 'image' -%}
+                        {{- '\n\n<|image|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'image' -%}
+                    {%- elif item['type'] == 'audio' -%}
+                        {{- '<|audio|>' -}}
+                        {%- set ns.prev_message_type = 'audio' -%}
+                    {%- elif item['type'] == 'video' -%}
+                        {{- '\n\n<|video|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'video' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+        {%- if not (message['tool_responses'] and not message['content']) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' -%}
+        {{- '<|turn>model\n' -}}
+    {%- endif -%}
+    {%- if not enable_thinking | default(false) -%}
+        {{- '<|channel>thought\n<channel|>' -}}
+    {%- endif -%}
+{%- endif -%}

checkpoint-121/README.md

@@ -0,0 +1,209 @@
+---
+base_model: google/gemma-4-26b-a4b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:google/gemma-4-26b-a4b-it
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.2.dev0

checkpoint-121/adapter_config.json

@@ -0,0 +1,246 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-4-26b-a4b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.2.dev0@e7355a3b2233820f6f30e558ce133ed22673a087",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.4.self_attn.k_proj",
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.3.mlp.up_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.8.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.k_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.6.mlp.up_proj",
+    "model.language_model.layers.24.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.q_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.up_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.26.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.down_proj",
+    "model.language_model.layers.11.mlp.up_proj",
+    "model.language_model.layers.2.self_attn.o_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.k_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.v_proj",
+    "model.language_model.layers.10.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.25.self_attn.q_proj",
+    "model.language_model.layers.5.self_attn.o_proj",
+    "model.language_model.layers.2.mlp.gate_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.23.self_attn.o_proj",
+    "model.language_model.layers.27.mlp.gate_proj",
+    "model.language_model.layers.0.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.4.self_attn.o_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.11.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.o_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.up_proj",
+    "model.language_model.layers.2.mlp.up_proj",
+    "model.language_model.layers.0.mlp.gate_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.10.mlp.down_proj",
+    "model.language_model.layers.10.mlp.gate_proj",
+    "model.language_model.layers.0.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.1.self_attn.v_proj",
+    "model.language_model.layers.13.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.3.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.o_proj",
+    "model.language_model.layers.7.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.v_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.k_proj",
+    "model.language_model.layers.5.self_attn.k_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.1.mlp.gate_proj",
+    "model.language_model.layers.8.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.q_proj",
+    "model.language_model.layers.23.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.k_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.10.mlp.up_proj",
+    "model.language_model.layers.11.mlp.gate_proj",
+    "model.language_model.layers.1.mlp.up_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.gate_proj",
+    "model.language_model.layers.7.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.up_proj",
+    "model.language_model.layers.5.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.down_proj",
+    "model.language_model.layers.22.mlp.gate_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.gate_proj",
+    "model.language_model.layers.26.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.q_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.28.self_attn.k_proj",
+    "model.language_model.layers.24.self_attn.o_proj",
+    "model.language_model.layers.26.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.25.mlp.up_proj",
+    "model.language_model.layers.27.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.0.mlp.down_proj",
+    "model.language_model.layers.6.self_attn.v_proj",
+    "model.language_model.layers.4.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.0.self_attn.q_proj",
+    "model.language_model.layers.27.mlp.up_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.12.mlp.up_proj",
+    "model.language_model.layers.6.mlp.down_proj",
+    "model.language_model.layers.2.mlp.down_proj",
+    "model.language_model.layers.6.mlp.gate_proj",
+    "model.language_model.layers.24.self_attn.v_proj",
+    "model.language_model.layers.4.mlp.up_proj",
+    "model.language_model.layers.9.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.gate_proj",
+    "model.language_model.layers.5.mlp.down_proj",
+    "model.language_model.layers.13.self_attn.o_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.24.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.26.mlp.up_proj",
+    "model.language_model.layers.26.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.o_proj",
+    "model.language_model.layers.8.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.o_proj",
+    "model.language_model.layers.6.self_attn.k_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.12.self_attn.k_proj",
+    "model.language_model.layers.13.mlp.down_proj",
+    "model.language_model.layers.1.mlp.down_proj",
+    "model.language_model.layers.3.mlp.gate_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.o_proj",
+    "model.language_model.layers.0.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.8.self_attn.k_proj",
+    "model.language_model.layers.12.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.13.mlp.up_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.25.self_attn.k_proj",
+    "model.language_model.layers.8.self_attn.q_proj",
+    "model.language_model.layers.4.self_attn.v_proj",
+    "model.language_model.layers.23.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.o_proj",
+    "model.language_model.layers.5.mlp.up_proj",
+    "model.language_model.layers.13.self_attn.k_proj",
+    "model.language_model.layers.7.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.o_proj",
+    "model.language_model.layers.22.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.24.self_attn.q_proj",
+    "model.language_model.layers.12.self_attn.q_proj",
+    "model.language_model.layers.2.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.v_proj",
+    "model.language_model.layers.13.mlp.gate_proj",
+    "model.language_model.layers.12.mlp.down_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.gate_proj",
+    "model.language_model.layers.13.self_attn.v_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.5.mlp.gate_proj",
+    "model.language_model.layers.1.self_attn.q_proj",
+    "model.language_model.layers.11.mlp.down_proj",
+    "model.language_model.layers.0.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.7.mlp.up_proj",
+    "model.language_model.layers.22.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.o_proj",
+    "model.language_model.layers.23.mlp.down_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-121/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75c05a4dbd6f957da19a4fcd5fa768f553c1cf5fa86a54f286530b8f94bd4e89
+size 37232104

checkpoint-121/chat_template.jinja

@@ -0,0 +1,266 @@
+{%- macro format_parameters(properties, required) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'OBJECT' -%}
+                ,properties:{
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                {%- elif value is mapping -%}
+                    {{- format_parameters(value, value['required'] | default([])) -}}
+                {%- endif -%}
+                }
+                {%- if value['required'] -%}
+                    ,required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    ,items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(prev_message_type=None) -%}
+{%- set loop_messages = messages -%}
+{{ bos_token }}
+{#- Handle System/Tool Definitions Block -#}
+{%- if (enable_thinking is defined and enable_thinking) or tools or messages[0]['role'] in ['system', 'developer'] -%}
+    {{- '<|turn>system\n' -}}
+
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking is defined and enable_thinking -%}
+        {{- '<|think|>' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{- messages[0]['content'] | trim -}}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+        {{- '<|turn>' + role + '\n' }}
+
+            {%- if message['tool_calls'] -%}
+                {%- for tool_call in message['tool_calls'] -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is string -%}
+                        {{- function['arguments'] -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- if message['tool_responses'] -%}
+                {#- Tool Response handling -#}
+                {%- for tool_response in message['tool_responses'] -%}
+                    {{- '<|tool_response>' -}}
+                    {%- if tool_response['response'] is mapping -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{' -}}
+                        {%- for key, value in tool_response['response'] | dictsort -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                            {%- if not loop.last %},{% endif -%}
+                        {%- endfor -%}
+                        {{- '}' -}}
+                    {%- else -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{value:' + format_argument(tool_response['response'], escape_keys=False) + '}' -}}
+                    {%- endif -%}
+                    {{- '<tool_response|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_response' -%}
+            {%- endif -%}
+
+            {%- if message['content'] is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message['content'] is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item['type'] == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item['type'] == 'image' -%}
+                        {{- '\n\n<|image|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'image' -%}
+                    {%- elif item['type'] == 'audio' -%}
+                        {{- '<|audio|>' -}}
+                        {%- set ns.prev_message_type = 'audio' -%}
+                    {%- elif item['type'] == 'video' -%}
+                        {{- '\n\n<|video|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'video' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+        {%- if not (message['tool_responses'] and not message['content']) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' -%}
+        {{- '<|turn>model\n' -}}
+    {%- endif -%}
+    {%- if not enable_thinking | default(false) -%}
+        {{- '<|channel>thought\n<channel|>' -}}
+    {%- endif -%}
+{%- endif -%}

checkpoint-121/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93915687bf2358b3909212d14e3049eb99e10df082b649515b609996d0f43a0d
+size 38229709

checkpoint-121/processor_config.json

@@ -0,0 +1,75 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "dither": 0.0,
+    "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+    "feature_size": 128,
+    "fft_length": 512,
+    "fft_overdrive": false,
+    "frame_length": 320,
+    "hop_length": 160,
+    "input_scale_factor": 1.0,
+    "max_frequency": 8000.0,
+    "mel_floor": 0.001,
+    "min_frequency": 0.0,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "per_bin_mean": null,
+    "per_bin_stddev": null,
+    "preemphasis": 0.0,
+    "preemphasis_htk_flavor": true,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4ImageProcessor",
+    "image_seq_length": 280,
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4Processor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4VideoProcessor"
+  }
+}

checkpoint-121/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:959e8e9e42ca24ad2d2375468311a443a85029f21f69e8aeecbbf05f12d75103
+size 14645

checkpoint-121/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6be25e5f07aa9f30e24f822ea2c4935aeae0eb0c636c45fdbb3d908d7c804c2b
+size 1465

checkpoint-121/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2619fe11b50dbed06ac443c51d757b354d0b62d64baa514404d4e84e6713519
+size 32169780

checkpoint-121/tokenizer_config.json

@@ -0,0 +1,95 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<content>(?:(?!\\<\\|tool_call\\>)(?!\\<turn\\|\\>).)+)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?:\\<turn\\|\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>"
+}

checkpoint-121/trainer_state.json

@@ -0,0 +1,165 @@
+{
+  "best_global_step": 121,
+  "best_metric": 0.6695265769958496,
+  "best_model_checkpoint": "/home/plucky/ml-workspace/models/gemma4-26b-securecode/checkpoint-121",
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 121,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.1113492242991925,
+      "epoch": 0.0827300930713547,
+      "grad_norm": 10.3125,
+      "learning_rate": 1.8e-05,
+      "loss": 93.48836059570313,
+      "mean_token_accuracy": 0.4107020549476147,
+      "num_tokens": 81920.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.8875315530225635,
+      "epoch": 0.1654601861427094,
+      "grad_norm": 6.15625,
+      "learning_rate": 3.8e-05,
+      "loss": 67.76697998046875,
+      "mean_token_accuracy": 0.5182974558323622,
+      "num_tokens": 163840.0,
+      "step": 20
+    },
+    {
+      "entropy": 0.673606987670064,
+      "epoch": 0.2481902792140641,
+      "grad_norm": 2.421875,
+      "learning_rate": 5.8e-05,
+      "loss": 37.221334838867186,
+      "mean_token_accuracy": 0.6476027386263012,
+      "num_tokens": 245760.0,
+      "step": 30
+    },
+    {
+      "entropy": 1.0845661748200655,
+      "epoch": 0.3309203722854188,
+      "grad_norm": 1.3671875,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 22.017848205566406,
+      "mean_token_accuracy": 0.7083170266821981,
+      "num_tokens": 327680.0,
+      "step": 40
+    },
+    {
+      "entropy": 1.1636322166770696,
+      "epoch": 0.4136504653567735,
+      "grad_norm": 0.703125,
+      "learning_rate": 9.8e-05,
+      "loss": 17.47879638671875,
+      "mean_token_accuracy": 0.7332558700814843,
+      "num_tokens": 409600.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9551631901413202,
+      "epoch": 0.4963805584281282,
+      "grad_norm": 0.40625,
+      "learning_rate": 0.000118,
+      "loss": 15.09481201171875,
+      "mean_token_accuracy": 0.7555772982537746,
+      "num_tokens": 491520.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.8048430571332574,
+      "epoch": 0.5791106514994829,
+      "grad_norm": 0.375,
+      "learning_rate": 0.000138,
+      "loss": 13.297686767578124,
+      "mean_token_accuracy": 0.7774828754365444,
+      "num_tokens": 573440.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.8100443260744215,
+      "epoch": 0.6618407445708376,
+      "grad_norm": 0.4609375,
+      "learning_rate": 0.00015800000000000002,
+      "loss": 12.752572631835937,
+      "mean_token_accuracy": 0.7837084107100963,
+      "num_tokens": 655360.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.7172152267768979,
+      "epoch": 0.7445708376421923,
+      "grad_norm": 2.1875,
+      "learning_rate": 0.00017800000000000002,
+      "loss": 11.629959106445312,
+      "mean_token_accuracy": 0.799449609220028,
+      "num_tokens": 737280.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.7284062243998051,
+      "epoch": 0.827300930713547,
+      "grad_norm": 0.40625,
+      "learning_rate": 0.00019800000000000002,
+      "loss": 11.506278991699219,
+      "mean_token_accuracy": 0.8022871781140566,
+      "num_tokens": 819200.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.6922262106090784,
+      "epoch": 0.9100310237849017,
+      "grad_norm": 0.341796875,
+      "learning_rate": 0.00019942266891397815,
+      "loss": 11.149666595458985,
+      "mean_token_accuracy": 0.8068982377648354,
+      "num_tokens": 901120.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.6608987387269736,
+      "epoch": 0.9927611168562565,
+      "grad_norm": 0.373046875,
+      "learning_rate": 0.00019743551343638324,
+      "loss": 10.666960906982421,
+      "mean_token_accuracy": 0.8124388422816992,
+      "num_tokens": 983040.0,
+      "step": 120
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.6862195637336997,
+      "eval_loss": 0.6695265769958496,
+      "eval_mean_token_accuracy": 0.8135074851124786,
+      "eval_num_tokens": 990208.0,
+      "eval_runtime": 255.0413,
+      "eval_samples_per_second": 0.843,
+      "eval_steps_per_second": 0.843,
+      "step": 121
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 363,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.4904406021973606e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-121/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2eaf4e1eba101412810b250e27914b2df87f93b0a9c62028451f50813e692b8e
+size 5713

checkpoint-242/README.md

@@ -0,0 +1,209 @@
+---
+base_model: google/gemma-4-26b-a4b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:google/gemma-4-26b-a4b-it
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.2.dev0

checkpoint-242/adapter_config.json

@@ -0,0 +1,246 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-4-26b-a4b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.2.dev0@e7355a3b2233820f6f30e558ce133ed22673a087",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.4.self_attn.k_proj",
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.3.mlp.up_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.8.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.k_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.6.mlp.up_proj",
+    "model.language_model.layers.24.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.q_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.up_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.26.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.down_proj",
+    "model.language_model.layers.11.mlp.up_proj",
+    "model.language_model.layers.2.self_attn.o_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.k_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.v_proj",
+    "model.language_model.layers.10.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.25.self_attn.q_proj",
+    "model.language_model.layers.5.self_attn.o_proj",
+    "model.language_model.layers.2.mlp.gate_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.23.self_attn.o_proj",
+    "model.language_model.layers.27.mlp.gate_proj",
+    "model.language_model.layers.0.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.4.self_attn.o_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.11.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.o_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.up_proj",
+    "model.language_model.layers.2.mlp.up_proj",
+    "model.language_model.layers.0.mlp.gate_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.10.mlp.down_proj",
+    "model.language_model.layers.10.mlp.gate_proj",
+    "model.language_model.layers.0.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.1.self_attn.v_proj",
+    "model.language_model.layers.13.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.3.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.o_proj",
+    "model.language_model.layers.7.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.v_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.k_proj",
+    "model.language_model.layers.5.self_attn.k_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.1.mlp.gate_proj",
+    "model.language_model.layers.8.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.q_proj",
+    "model.language_model.layers.23.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.k_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.10.mlp.up_proj",
+    "model.language_model.layers.11.mlp.gate_proj",
+    "model.language_model.layers.1.mlp.up_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.gate_proj",
+    "model.language_model.layers.7.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.up_proj",
+    "model.language_model.layers.5.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.down_proj",
+    "model.language_model.layers.22.mlp.gate_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.gate_proj",
+    "model.language_model.layers.26.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.q_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.28.self_attn.k_proj",
+    "model.language_model.layers.24.self_attn.o_proj",
+    "model.language_model.layers.26.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.25.mlp.up_proj",
+    "model.language_model.layers.27.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.0.mlp.down_proj",
+    "model.language_model.layers.6.self_attn.v_proj",
+    "model.language_model.layers.4.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.0.self_attn.q_proj",
+    "model.language_model.layers.27.mlp.up_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.12.mlp.up_proj",
+    "model.language_model.layers.6.mlp.down_proj",
+    "model.language_model.layers.2.mlp.down_proj",
+    "model.language_model.layers.6.mlp.gate_proj",
+    "model.language_model.layers.24.self_attn.v_proj",
+    "model.language_model.layers.4.mlp.up_proj",
+    "model.language_model.layers.9.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.gate_proj",
+    "model.language_model.layers.5.mlp.down_proj",
+    "model.language_model.layers.13.self_attn.o_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.24.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.26.mlp.up_proj",
+    "model.language_model.layers.26.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.o_proj",
+    "model.language_model.layers.8.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.o_proj",
+    "model.language_model.layers.6.self_attn.k_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.12.self_attn.k_proj",
+    "model.language_model.layers.13.mlp.down_proj",
+    "model.language_model.layers.1.mlp.down_proj",
+    "model.language_model.layers.3.mlp.gate_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.o_proj",
+    "model.language_model.layers.0.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.8.self_attn.k_proj",
+    "model.language_model.layers.12.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.13.mlp.up_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.25.self_attn.k_proj",
+    "model.language_model.layers.8.self_attn.q_proj",
+    "model.language_model.layers.4.self_attn.v_proj",
+    "model.language_model.layers.23.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.o_proj",
+    "model.language_model.layers.5.mlp.up_proj",
+    "model.language_model.layers.13.self_attn.k_proj",
+    "model.language_model.layers.7.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.o_proj",
+    "model.language_model.layers.22.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.24.self_attn.q_proj",
+    "model.language_model.layers.12.self_attn.q_proj",
+    "model.language_model.layers.2.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.v_proj",
+    "model.language_model.layers.13.mlp.gate_proj",
+    "model.language_model.layers.12.mlp.down_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.gate_proj",
+    "model.language_model.layers.13.self_attn.v_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.5.mlp.gate_proj",
+    "model.language_model.layers.1.self_attn.q_proj",
+    "model.language_model.layers.11.mlp.down_proj",
+    "model.language_model.layers.0.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.7.mlp.up_proj",
+    "model.language_model.layers.22.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.o_proj",
+    "model.language_model.layers.23.mlp.down_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-242/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6c31eb96af91247162f1fdd882f14bf908d1c3c4b7925203d58312809b5007e
+size 37232104

checkpoint-242/chat_template.jinja

@@ -0,0 +1,266 @@
+{%- macro format_parameters(properties, required) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'OBJECT' -%}
+                ,properties:{
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                {%- elif value is mapping -%}
+                    {{- format_parameters(value, value['required'] | default([])) -}}
+                {%- endif -%}
+                }
+                {%- if value['required'] -%}
+                    ,required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    ,items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(prev_message_type=None) -%}
+{%- set loop_messages = messages -%}
+{{ bos_token }}
+{#- Handle System/Tool Definitions Block -#}
+{%- if (enable_thinking is defined and enable_thinking) or tools or messages[0]['role'] in ['system', 'developer'] -%}
+    {{- '<|turn>system\n' -}}
+
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking is defined and enable_thinking -%}
+        {{- '<|think|>' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{- messages[0]['content'] | trim -}}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+        {{- '<|turn>' + role + '\n' }}
+
+            {%- if message['tool_calls'] -%}
+                {%- for tool_call in message['tool_calls'] -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is string -%}
+                        {{- function['arguments'] -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- if message['tool_responses'] -%}
+                {#- Tool Response handling -#}
+                {%- for tool_response in message['tool_responses'] -%}
+                    {{- '<|tool_response>' -}}
+                    {%- if tool_response['response'] is mapping -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{' -}}
+                        {%- for key, value in tool_response['response'] | dictsort -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                            {%- if not loop.last %},{% endif -%}
+                        {%- endfor -%}
+                        {{- '}' -}}
+                    {%- else -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{value:' + format_argument(tool_response['response'], escape_keys=False) + '}' -}}
+                    {%- endif -%}
+                    {{- '<tool_response|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_response' -%}
+            {%- endif -%}
+
+            {%- if message['content'] is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message['content'] is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item['type'] == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item['type'] == 'image' -%}
+                        {{- '\n\n<|image|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'image' -%}
+                    {%- elif item['type'] == 'audio' -%}
+                        {{- '<|audio|>' -}}
+                        {%- set ns.prev_message_type = 'audio' -%}
+                    {%- elif item['type'] == 'video' -%}
+                        {{- '\n\n<|video|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'video' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+        {%- if not (message['tool_responses'] and not message['content']) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' -%}
+        {{- '<|turn>model\n' -}}
+    {%- endif -%}
+    {%- if not enable_thinking | default(false) -%}
+        {{- '<|channel>thought\n<channel|>' -}}
+    {%- endif -%}
+{%- endif -%}

checkpoint-242/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01cc0e44a43e58c8b45161465df8b4fadbd469fd35fab1d5c423759ee5ba8a68
+size 38229709

checkpoint-242/processor_config.json

@@ -0,0 +1,75 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "dither": 0.0,
+    "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+    "feature_size": 128,
+    "fft_length": 512,
+    "fft_overdrive": false,
+    "frame_length": 320,
+    "hop_length": 160,
+    "input_scale_factor": 1.0,
+    "max_frequency": 8000.0,
+    "mel_floor": 0.001,
+    "min_frequency": 0.0,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "per_bin_mean": null,
+    "per_bin_stddev": null,
+    "preemphasis": 0.0,
+    "preemphasis_htk_flavor": true,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4ImageProcessor",
+    "image_seq_length": 280,
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4Processor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4VideoProcessor"
+  }
+}

checkpoint-242/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67a6c6b866e89a5917944b74173a0b8536ce4695e579297378c983b24e5a507b
+size 14645

checkpoint-242/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7702a97a98d09da954174b71008538d77d6adeaeaeb17c9732ebeab8932e0e3e
+size 1465

checkpoint-242/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2619fe11b50dbed06ac443c51d757b354d0b62d64baa514404d4e84e6713519
+size 32169780

checkpoint-242/tokenizer_config.json

@@ -0,0 +1,95 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<content>(?:(?!\\<\\|tool_call\\>)(?!\\<turn\\|\\>).)+)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?:\\<turn\\|\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>"
+}

checkpoint-242/trainer_state.json

@@ -0,0 +1,296 @@
+{
+  "best_global_step": 242,
+  "best_metric": 0.6102388501167297,
+  "best_model_checkpoint": "/home/plucky/ml-workspace/models/gemma4-26b-securecode/checkpoint-242",
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 242,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.1113492242991925,
+      "epoch": 0.0827300930713547,
+      "grad_norm": 10.3125,
+      "learning_rate": 1.8e-05,
+      "loss": 93.48836059570313,
+      "mean_token_accuracy": 0.4107020549476147,
+      "num_tokens": 81920.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.8875315530225635,
+      "epoch": 0.1654601861427094,
+      "grad_norm": 6.15625,
+      "learning_rate": 3.8e-05,
+      "loss": 67.76697998046875,
+      "mean_token_accuracy": 0.5182974558323622,
+      "num_tokens": 163840.0,
+      "step": 20
+    },
+    {
+      "entropy": 0.673606987670064,
+      "epoch": 0.2481902792140641,
+      "grad_norm": 2.421875,
+      "learning_rate": 5.8e-05,
+      "loss": 37.221334838867186,
+      "mean_token_accuracy": 0.6476027386263012,
+      "num_tokens": 245760.0,
+      "step": 30
+    },
+    {
+      "entropy": 1.0845661748200655,
+      "epoch": 0.3309203722854188,
+      "grad_norm": 1.3671875,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 22.017848205566406,
+      "mean_token_accuracy": 0.7083170266821981,
+      "num_tokens": 327680.0,
+      "step": 40
+    },
+    {
+      "entropy": 1.1636322166770696,
+      "epoch": 0.4136504653567735,
+      "grad_norm": 0.703125,
+      "learning_rate": 9.8e-05,
+      "loss": 17.47879638671875,
+      "mean_token_accuracy": 0.7332558700814843,
+      "num_tokens": 409600.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9551631901413202,
+      "epoch": 0.4963805584281282,
+      "grad_norm": 0.40625,
+      "learning_rate": 0.000118,
+      "loss": 15.09481201171875,
+      "mean_token_accuracy": 0.7555772982537746,
+      "num_tokens": 491520.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.8048430571332574,
+      "epoch": 0.5791106514994829,
+      "grad_norm": 0.375,
+      "learning_rate": 0.000138,
+      "loss": 13.297686767578124,
+      "mean_token_accuracy": 0.7774828754365444,
+      "num_tokens": 573440.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.8100443260744215,
+      "epoch": 0.6618407445708376,
+      "grad_norm": 0.4609375,
+      "learning_rate": 0.00015800000000000002,
+      "loss": 12.752572631835937,
+      "mean_token_accuracy": 0.7837084107100963,
+      "num_tokens": 655360.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.7172152267768979,
+      "epoch": 0.7445708376421923,
+      "grad_norm": 2.1875,
+      "learning_rate": 0.00017800000000000002,
+      "loss": 11.629959106445312,
+      "mean_token_accuracy": 0.799449609220028,
+      "num_tokens": 737280.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.7284062243998051,
+      "epoch": 0.827300930713547,
+      "grad_norm": 0.40625,
+      "learning_rate": 0.00019800000000000002,
+      "loss": 11.506278991699219,
+      "mean_token_accuracy": 0.8022871781140566,
+      "num_tokens": 819200.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.6922262106090784,
+      "epoch": 0.9100310237849017,
+      "grad_norm": 0.341796875,
+      "learning_rate": 0.00019942266891397815,
+      "loss": 11.149666595458985,
+      "mean_token_accuracy": 0.8068982377648354,
+      "num_tokens": 901120.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.6608987387269736,
+      "epoch": 0.9927611168562565,
+      "grad_norm": 0.373046875,
+      "learning_rate": 0.00019743551343638324,
+      "loss": 10.666960906982421,
+      "mean_token_accuracy": 0.8124388422816992,
+      "num_tokens": 983040.0,
+      "step": 120
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.6862195637336997,
+      "eval_loss": 0.6695265769958496,
+      "eval_mean_token_accuracy": 0.8135074851124786,
+      "eval_num_tokens": 990208.0,
+      "eval_runtime": 255.0413,
+      "eval_samples_per_second": 0.843,
+      "eval_steps_per_second": 0.843,
+      "step": 121
+    },
+    {
+      "entropy": 0.6788679953617386,
+      "epoch": 1.0744570837642193,
+      "grad_norm": 0.3984375,
+      "learning_rate": 0.00019405971991583108,
+      "loss": 10.533837127685548,
+      "mean_token_accuracy": 0.8129133717923225,
+      "num_tokens": 1063936.0,
+      "step": 130
+    },
+    {
+      "entropy": 0.5800832805223763,
+      "epoch": 1.157187176835574,
+      "grad_norm": 0.333984375,
+      "learning_rate": 0.00018934339971482674,
+      "loss": 9.498150634765626,
+      "mean_token_accuracy": 0.8281555753201246,
+      "num_tokens": 1145856.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.6344770405441522,
+      "epoch": 1.2399172699069285,
+      "grad_norm": 0.388671875,
+      "learning_rate": 0.00018335376920472097,
+      "loss": 10.217367553710938,
+      "mean_token_accuracy": 0.8195327781140804,
+      "num_tokens": 1227776.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.6310219537466765,
+      "epoch": 1.3226473629782833,
+      "grad_norm": 0.380859375,
+      "learning_rate": 0.00017617619180688085,
+      "loss": 10.081737518310547,
+      "mean_token_accuracy": 0.8219178050756455,
+      "num_tokens": 1309696.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.5863334746100008,
+      "epoch": 1.4053774560496382,
+      "grad_norm": 0.341796875,
+      "learning_rate": 0.00016791296140450545,
+      "loss": 9.392319488525391,
+      "mean_token_accuracy": 0.8319227002561093,
+      "num_tokens": 1391616.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.6232900662347675,
+      "epoch": 1.4881075491209927,
+      "grad_norm": 0.44921875,
+      "learning_rate": 0.0001586818444637402,
+      "loss": 10.051438140869141,
+      "mean_token_accuracy": 0.8215264175087214,
+      "num_tokens": 1473536.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.6163463215343654,
+      "epoch": 1.5708376421923473,
+      "grad_norm": 0.384765625,
+      "learning_rate": 0.0001486144016415862,
+      "loss": 9.878226470947265,
+      "mean_token_accuracy": 0.8220768082886935,
+      "num_tokens": 1555456.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.588023800123483,
+      "epoch": 1.6535677352637022,
+      "grad_norm": 0.3515625,
+      "learning_rate": 0.00013785411280082746,
+      "loss": 9.45407943725586,
+      "mean_token_accuracy": 0.8305283710360527,
+      "num_tokens": 1637376.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.599842881783843,
+      "epoch": 1.736297828335057,
+      "grad_norm": 0.37890625,
+      "learning_rate": 0.00012655433215401438,
+      "loss": 9.548422241210938,
+      "mean_token_accuracy": 0.8284735765308142,
+      "num_tokens": 1719296.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.6552030782215297,
+      "epoch": 1.8190279214064116,
+      "grad_norm": 0.361328125,
+      "learning_rate": 0.00011487610267952142,
+      "loss": 10.46890640258789,
+      "mean_token_accuracy": 0.8134295467287302,
+      "num_tokens": 1801216.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.5984975789207965,
+      "epoch": 1.9017580144777662,
+      "grad_norm": 0.353515625,
+      "learning_rate": 0.00010298586095833151,
+      "loss": 9.603475952148438,
+      "mean_token_accuracy": 0.827079250663519,
+      "num_tokens": 1883136.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.5947112645488233,
+      "epoch": 1.984488107549121,
+      "grad_norm": 0.64453125,
+      "learning_rate": 9.10530651419099e-05,
+      "loss": 9.561953735351562,
+      "mean_token_accuracy": 0.8265655554831028,
+      "num_tokens": 1965056.0,
+      "step": 240
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6100467269503793,
+      "eval_loss": 0.6102388501167297,
+      "eval_mean_token_accuracy": 0.8254676164582718,
+      "eval_num_tokens": 1980416.0,
+      "eval_runtime": 254.828,
+      "eval_samples_per_second": 0.844,
+      "eval_steps_per_second": 0.844,
+      "step": 242
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 363,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.980881204394721e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-242/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2eaf4e1eba101412810b250e27914b2df87f93b0a9c62028451f50813e692b8e
+size 5713

checkpoint-363/README.md

@@ -0,0 +1,209 @@
+---
+base_model: google/gemma-4-26b-a4b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:google/gemma-4-26b-a4b-it
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.2.dev0

checkpoint-363/adapter_config.json

@@ -0,0 +1,246 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-4-26b-a4b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.2.dev0@e7355a3b2233820f6f30e558ce133ed22673a087",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.4.self_attn.k_proj",
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.3.mlp.up_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.8.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.k_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.6.mlp.up_proj",
+    "model.language_model.layers.24.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.q_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.up_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.26.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.down_proj",
+    "model.language_model.layers.11.mlp.up_proj",
+    "model.language_model.layers.2.self_attn.o_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.k_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.v_proj",
+    "model.language_model.layers.10.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.25.self_attn.q_proj",
+    "model.language_model.layers.5.self_attn.o_proj",
+    "model.language_model.layers.2.mlp.gate_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.23.self_attn.o_proj",
+    "model.language_model.layers.27.mlp.gate_proj",
+    "model.language_model.layers.0.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.4.self_attn.o_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.11.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.o_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.up_proj",
+    "model.language_model.layers.2.mlp.up_proj",
+    "model.language_model.layers.0.mlp.gate_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.10.mlp.down_proj",
+    "model.language_model.layers.10.mlp.gate_proj",
+    "model.language_model.layers.0.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.1.self_attn.v_proj",
+    "model.language_model.layers.13.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.3.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.o_proj",
+    "model.language_model.layers.7.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.v_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.k_proj",
+    "model.language_model.layers.5.self_attn.k_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.1.mlp.gate_proj",
+    "model.language_model.layers.8.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.q_proj",
+    "model.language_model.layers.23.self_attn.k_proj",
+    "model.language_model.layers.3.self_attn.k_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.10.mlp.up_proj",
+    "model.language_model.layers.11.mlp.gate_proj",
+    "model.language_model.layers.1.mlp.up_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.gate_proj",
+    "model.language_model.layers.7.mlp.gate_proj",
+    "model.language_model.layers.8.mlp.up_proj",
+    "model.language_model.layers.5.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.down_proj",
+    "model.language_model.layers.22.mlp.gate_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.o_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.k_proj",
+    "model.language_model.layers.24.mlp.gate_proj",
+    "model.language_model.layers.26.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.q_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.28.self_attn.k_proj",
+    "model.language_model.layers.24.self_attn.o_proj",
+    "model.language_model.layers.26.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.25.mlp.up_proj",
+    "model.language_model.layers.27.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.0.mlp.down_proj",
+    "model.language_model.layers.6.self_attn.v_proj",
+    "model.language_model.layers.4.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.0.self_attn.q_proj",
+    "model.language_model.layers.27.mlp.up_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.12.mlp.up_proj",
+    "model.language_model.layers.6.mlp.down_proj",
+    "model.language_model.layers.2.mlp.down_proj",
+    "model.language_model.layers.6.mlp.gate_proj",
+    "model.language_model.layers.24.self_attn.v_proj",
+    "model.language_model.layers.4.mlp.up_proj",
+    "model.language_model.layers.9.self_attn.o_proj",
+    "model.language_model.layers.22.self_attn.v_proj",
+    "model.language_model.layers.23.mlp.gate_proj",
+    "model.language_model.layers.5.mlp.down_proj",
+    "model.language_model.layers.13.self_attn.o_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.24.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.26.mlp.up_proj",
+    "model.language_model.layers.26.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.o_proj",
+    "model.language_model.layers.8.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.o_proj",
+    "model.language_model.layers.6.self_attn.k_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.12.self_attn.k_proj",
+    "model.language_model.layers.13.mlp.down_proj",
+    "model.language_model.layers.1.mlp.down_proj",
+    "model.language_model.layers.3.mlp.gate_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.o_proj",
+    "model.language_model.layers.0.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.8.self_attn.k_proj",
+    "model.language_model.layers.12.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.13.mlp.up_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.25.self_attn.k_proj",
+    "model.language_model.layers.8.self_attn.q_proj",
+    "model.language_model.layers.4.self_attn.v_proj",
+    "model.language_model.layers.23.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.o_proj",
+    "model.language_model.layers.5.mlp.up_proj",
+    "model.language_model.layers.13.self_attn.k_proj",
+    "model.language_model.layers.7.self_attn.k_proj",
+    "model.language_model.layers.22.self_attn.o_proj",
+    "model.language_model.layers.22.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.24.self_attn.q_proj",
+    "model.language_model.layers.12.self_attn.q_proj",
+    "model.language_model.layers.2.self_attn.v_proj",
+    "model.language_model.layers.12.self_attn.v_proj",
+    "model.language_model.layers.13.mlp.gate_proj",
+    "model.language_model.layers.12.mlp.down_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.26.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.7.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.gate_proj",
+    "model.language_model.layers.13.self_attn.v_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.5.mlp.gate_proj",
+    "model.language_model.layers.1.self_attn.q_proj",
+    "model.language_model.layers.11.mlp.down_proj",
+    "model.language_model.layers.0.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.7.mlp.up_proj",
+    "model.language_model.layers.22.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.3.self_attn.o_proj",
+    "model.language_model.layers.23.mlp.down_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-363/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fc9c87e6f1fd7f94d16281ee425e6587460ef9e8104aac93d25e0de6b7b31a9
+size 37232104

checkpoint-363/chat_template.jinja

@@ -0,0 +1,266 @@
+{%- macro format_parameters(properties, required) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'OBJECT' -%}
+                ,properties:{
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                {%- elif value is mapping -%}
+                    {{- format_parameters(value, value['required'] | default([])) -}}
+                {%- endif -%}
+                }
+                {%- if value['required'] -%}
+                    ,required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    ,items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(prev_message_type=None) -%}
+{%- set loop_messages = messages -%}
+{{ bos_token }}
+{#- Handle System/Tool Definitions Block -#}
+{%- if (enable_thinking is defined and enable_thinking) or tools or messages[0]['role'] in ['system', 'developer'] -%}
+    {{- '<|turn>system\n' -}}
+
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking is defined and enable_thinking -%}
+        {{- '<|think|>' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{- messages[0]['content'] | trim -}}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+        {{- '<|turn>' + role + '\n' }}
+
+            {%- if message['tool_calls'] -%}
+                {%- for tool_call in message['tool_calls'] -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is string -%}
+                        {{- function['arguments'] -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- if message['tool_responses'] -%}
+                {#- Tool Response handling -#}
+                {%- for tool_response in message['tool_responses'] -%}
+                    {{- '<|tool_response>' -}}
+                    {%- if tool_response['response'] is mapping -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{' -}}
+                        {%- for key, value in tool_response['response'] | dictsort -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                            {%- if not loop.last %},{% endif -%}
+                        {%- endfor -%}
+                        {{- '}' -}}
+                    {%- else -%}
+                        {{- 'response:' + tool_response['name'] | default('unknown') + '{value:' + format_argument(tool_response['response'], escape_keys=False) + '}' -}}
+                    {%- endif -%}
+                    {{- '<tool_response|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_response' -%}
+            {%- endif -%}
+
+            {%- if message['content'] is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message['content'] is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item['type'] == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item['type'] == 'image' -%}
+                        {{- '\n\n<|image|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'image' -%}
+                    {%- elif item['type'] == 'audio' -%}
+                        {{- '<|audio|>' -}}
+                        {%- set ns.prev_message_type = 'audio' -%}
+                    {%- elif item['type'] == 'video' -%}
+                        {{- '\n\n<|video|>\n\n' -}}
+                        {%- set ns.prev_message_type = 'video' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+        {%- if not (message['tool_responses'] and not message['content']) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' -%}
+        {{- '<|turn>model\n' -}}
+    {%- endif -%}
+    {%- if not enable_thinking | default(false) -%}
+        {{- '<|channel>thought\n<channel|>' -}}
+    {%- endif -%}
+{%- endif -%}

checkpoint-363/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:704bd56fcfb881393baa83113d34a7cf3d7745744dc252c023ac9d7dee1ed1e8
+size 38230093

checkpoint-363/processor_config.json

@@ -0,0 +1,75 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "dither": 0.0,
+    "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+    "feature_size": 128,
+    "fft_length": 512,
+    "fft_overdrive": false,
+    "frame_length": 320,
+    "hop_length": 160,
+    "input_scale_factor": 1.0,
+    "max_frequency": 8000.0,
+    "mel_floor": 0.001,
+    "min_frequency": 0.0,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "per_bin_mean": null,
+    "per_bin_stddev": null,
+    "preemphasis": 0.0,
+    "preemphasis_htk_flavor": true,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4ImageProcessor",
+    "image_seq_length": 280,
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4Processor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4VideoProcessor"
+  }
+}

checkpoint-363/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b277407ef7cf34b19b7f76b4063ef02c942ff37191ea6603533d3e5bb877696d
+size 14645

checkpoint-363/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a27b1dfda3b0692906cbec10c46258cb4f99d28ec188b1d9b066e584a6708792
+size 1465

checkpoint-363/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2619fe11b50dbed06ac443c51d757b354d0b62d64baa514404d4e84e6713519
+size 32169780

checkpoint-363/tokenizer_config.json

@@ -0,0 +1,95 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<content>(?:(?!\\<\\|tool_call\\>)(?!\\<turn\\|\\>).)+)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?:\\<turn\\|\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>"
+}

checkpoint-363/trainer_state.json

@@ -0,0 +1,427 @@
+{
+  "best_global_step": 363,
+  "best_metric": 0.6060348153114319,
+  "best_model_checkpoint": "/home/plucky/ml-workspace/models/gemma4-26b-securecode/checkpoint-363",
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 363,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.1113492242991925,
+      "epoch": 0.0827300930713547,
+      "grad_norm": 10.3125,
+      "learning_rate": 1.8e-05,
+      "loss": 93.48836059570313,
+      "mean_token_accuracy": 0.4107020549476147,
+      "num_tokens": 81920.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.8875315530225635,
+      "epoch": 0.1654601861427094,
+      "grad_norm": 6.15625,
+      "learning_rate": 3.8e-05,
+      "loss": 67.76697998046875,
+      "mean_token_accuracy": 0.5182974558323622,
+      "num_tokens": 163840.0,
+      "step": 20
+    },
+    {
+      "entropy": 0.673606987670064,
+      "epoch": 0.2481902792140641,
+      "grad_norm": 2.421875,
+      "learning_rate": 5.8e-05,
+      "loss": 37.221334838867186,
+      "mean_token_accuracy": 0.6476027386263012,
+      "num_tokens": 245760.0,
+      "step": 30
+    },
+    {
+      "entropy": 1.0845661748200655,
+      "epoch": 0.3309203722854188,
+      "grad_norm": 1.3671875,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 22.017848205566406,
+      "mean_token_accuracy": 0.7083170266821981,
+      "num_tokens": 327680.0,
+      "step": 40
+    },
+    {
+      "entropy": 1.1636322166770696,
+      "epoch": 0.4136504653567735,
+      "grad_norm": 0.703125,
+      "learning_rate": 9.8e-05,
+      "loss": 17.47879638671875,
+      "mean_token_accuracy": 0.7332558700814843,
+      "num_tokens": 409600.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9551631901413202,
+      "epoch": 0.4963805584281282,
+      "grad_norm": 0.40625,
+      "learning_rate": 0.000118,
+      "loss": 15.09481201171875,
+      "mean_token_accuracy": 0.7555772982537746,
+      "num_tokens": 491520.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.8048430571332574,
+      "epoch": 0.5791106514994829,
+      "grad_norm": 0.375,
+      "learning_rate": 0.000138,
+      "loss": 13.297686767578124,
+      "mean_token_accuracy": 0.7774828754365444,
+      "num_tokens": 573440.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.8100443260744215,
+      "epoch": 0.6618407445708376,
+      "grad_norm": 0.4609375,
+      "learning_rate": 0.00015800000000000002,
+      "loss": 12.752572631835937,
+      "mean_token_accuracy": 0.7837084107100963,
+      "num_tokens": 655360.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.7172152267768979,
+      "epoch": 0.7445708376421923,
+      "grad_norm": 2.1875,
+      "learning_rate": 0.00017800000000000002,
+      "loss": 11.629959106445312,
+      "mean_token_accuracy": 0.799449609220028,
+      "num_tokens": 737280.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.7284062243998051,
+      "epoch": 0.827300930713547,
+      "grad_norm": 0.40625,
+      "learning_rate": 0.00019800000000000002,
+      "loss": 11.506278991699219,
+      "mean_token_accuracy": 0.8022871781140566,
+      "num_tokens": 819200.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.6922262106090784,
+      "epoch": 0.9100310237849017,
+      "grad_norm": 0.341796875,
+      "learning_rate": 0.00019942266891397815,
+      "loss": 11.149666595458985,
+      "mean_token_accuracy": 0.8068982377648354,
+      "num_tokens": 901120.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.6608987387269736,
+      "epoch": 0.9927611168562565,
+      "grad_norm": 0.373046875,
+      "learning_rate": 0.00019743551343638324,
+      "loss": 10.666960906982421,
+      "mean_token_accuracy": 0.8124388422816992,
+      "num_tokens": 983040.0,
+      "step": 120
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.6862195637336997,
+      "eval_loss": 0.6695265769958496,
+      "eval_mean_token_accuracy": 0.8135074851124786,
+      "eval_num_tokens": 990208.0,
+      "eval_runtime": 255.0413,
+      "eval_samples_per_second": 0.843,
+      "eval_steps_per_second": 0.843,
+      "step": 121
+    },
+    {
+      "entropy": 0.6788679953617386,
+      "epoch": 1.0744570837642193,
+      "grad_norm": 0.3984375,
+      "learning_rate": 0.00019405971991583108,
+      "loss": 10.533837127685548,
+      "mean_token_accuracy": 0.8129133717923225,
+      "num_tokens": 1063936.0,
+      "step": 130
+    },
+    {
+      "entropy": 0.5800832805223763,
+      "epoch": 1.157187176835574,
+      "grad_norm": 0.333984375,
+      "learning_rate": 0.00018934339971482674,
+      "loss": 9.498150634765626,
+      "mean_token_accuracy": 0.8281555753201246,
+      "num_tokens": 1145856.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.6344770405441522,
+      "epoch": 1.2399172699069285,
+      "grad_norm": 0.388671875,
+      "learning_rate": 0.00018335376920472097,
+      "loss": 10.217367553710938,
+      "mean_token_accuracy": 0.8195327781140804,
+      "num_tokens": 1227776.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.6310219537466765,
+      "epoch": 1.3226473629782833,
+      "grad_norm": 0.380859375,
+      "learning_rate": 0.00017617619180688085,
+      "loss": 10.081737518310547,
+      "mean_token_accuracy": 0.8219178050756455,
+      "num_tokens": 1309696.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.5863334746100008,
+      "epoch": 1.4053774560496382,
+      "grad_norm": 0.341796875,
+      "learning_rate": 0.00016791296140450545,
+      "loss": 9.392319488525391,
+      "mean_token_accuracy": 0.8319227002561093,
+      "num_tokens": 1391616.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.6232900662347675,
+      "epoch": 1.4881075491209927,
+      "grad_norm": 0.44921875,
+      "learning_rate": 0.0001586818444637402,
+      "loss": 10.051438140869141,
+      "mean_token_accuracy": 0.8215264175087214,
+      "num_tokens": 1473536.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.6163463215343654,
+      "epoch": 1.5708376421923473,
+      "grad_norm": 0.384765625,
+      "learning_rate": 0.0001486144016415862,
+      "loss": 9.878226470947265,
+      "mean_token_accuracy": 0.8220768082886935,
+      "num_tokens": 1555456.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.588023800123483,
+      "epoch": 1.6535677352637022,
+      "grad_norm": 0.3515625,
+      "learning_rate": 0.00013785411280082746,
+      "loss": 9.45407943725586,
+      "mean_token_accuracy": 0.8305283710360527,
+      "num_tokens": 1637376.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.599842881783843,
+      "epoch": 1.736297828335057,
+      "grad_norm": 0.37890625,
+      "learning_rate": 0.00012655433215401438,
+      "loss": 9.548422241210938,
+      "mean_token_accuracy": 0.8284735765308142,
+      "num_tokens": 1719296.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.6552030782215297,
+      "epoch": 1.8190279214064116,
+      "grad_norm": 0.361328125,
+      "learning_rate": 0.00011487610267952142,
+      "loss": 10.46890640258789,
+      "mean_token_accuracy": 0.8134295467287302,
+      "num_tokens": 1801216.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.5984975789207965,
+      "epoch": 1.9017580144777662,
+      "grad_norm": 0.353515625,
+      "learning_rate": 0.00010298586095833151,
+      "loss": 9.603475952148438,
+      "mean_token_accuracy": 0.827079250663519,
+      "num_tokens": 1883136.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.5947112645488233,
+      "epoch": 1.984488107549121,
+      "grad_norm": 0.64453125,
+      "learning_rate": 9.10530651419099e-05,
+      "loss": 9.561953735351562,
+      "mean_token_accuracy": 0.8265655554831028,
+      "num_tokens": 1965056.0,
+      "step": 240
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6100467269503793,
+      "eval_loss": 0.6102388501167297,
+      "eval_mean_token_accuracy": 0.8254676164582718,
+      "eval_num_tokens": 1980416.0,
+      "eval_runtime": 254.828,
+      "eval_samples_per_second": 0.844,
+      "eval_steps_per_second": 0.844,
+      "step": 242
+    },
+    {
+      "entropy": 0.5080371947511088,
+      "epoch": 2.066184074457084,
+      "grad_norm": 0.453125,
+      "learning_rate": 7.924777985705556e-05,
+      "loss": 8.056553649902344,
+      "mean_token_accuracy": 0.8497857213774814,
+      "num_tokens": 2045952.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.5341692148707807,
+      "epoch": 2.1489141675284387,
+      "grad_norm": 0.384765625,
+      "learning_rate": 6.773825246734622e-05,
+      "loss": 8.356841278076171,
+      "mean_token_accuracy": 0.8431262206286192,
+      "num_tokens": 2127872.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.5629857819527387,
+      "epoch": 2.231644260599793,
+      "grad_norm": 0.328125,
+      "learning_rate": 5.668851523397829e-05,
+      "loss": 9.067486572265626,
+      "mean_token_accuracy": 0.8315435405820608,
+      "num_tokens": 2209792.0,
+      "step": 270
+    },
+    {
+      "entropy": 0.5280973493587225,
+      "epoch": 2.314374353671148,
+      "grad_norm": 0.361328125,
+      "learning_rate": 4.625604754968839e-05,
+      "loss": 8.390058135986328,
+      "mean_token_accuracy": 0.8423923663794994,
+      "num_tokens": 2291712.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.5421305931173265,
+      "epoch": 2.3971044467425027,
+      "grad_norm": 0.353515625,
+      "learning_rate": 3.658953156328857e-05,
+      "loss": 8.713886260986328,
+      "mean_token_accuracy": 0.8375489212572574,
+      "num_tokens": 2373632.0,
+      "step": 290
+    },
+    {
+      "entropy": 0.5257686520460993,
+      "epoch": 2.479834539813857,
+      "grad_norm": 0.373046875,
+      "learning_rate": 2.7826733181357932e-05,
+      "loss": 8.388682556152343,
+      "mean_token_accuracy": 0.8447284691035748,
+      "num_tokens": 2455552.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.5735760541632772,
+      "epoch": 2.562564632885212,
+      "grad_norm": 0.421875,
+      "learning_rate": 2.0092538646774072e-05,
+      "loss": 9.259294891357422,
+      "mean_token_accuracy": 0.8287671197205781,
+      "num_tokens": 2537472.0,
+      "step": 310
+    },
+    {
+      "entropy": 0.5352369678206742,
+      "epoch": 2.6452947259565667,
+      "grad_norm": 0.369140625,
+      "learning_rate": 1.3497174676506674e-05,
+      "loss": 8.547685241699218,
+      "mean_token_accuracy": 0.8413160435855389,
+      "num_tokens": 2619392.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.540962244477123,
+      "epoch": 2.7280248190279215,
+      "grad_norm": 0.365234375,
+      "learning_rate": 8.134637525034839e-06,
+      "loss": 8.591437530517577,
+      "mean_token_accuracy": 0.838882090896368,
+      "num_tokens": 2701312.0,
+      "step": 330
+    },
+    {
+      "entropy": 0.5567054254934192,
+      "epoch": 2.8107549120992763,
+      "grad_norm": 0.353515625,
+      "learning_rate": 4.081353362167406e-06,
+      "loss": 8.788534545898438,
+      "mean_token_accuracy": 0.8374510746449232,
+      "num_tokens": 2783232.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.5575114467181266,
+      "epoch": 2.8934850051706307,
+      "grad_norm": 0.35546875,
+      "learning_rate": 1.3950890573852126e-06,
+      "loss": 8.935771179199218,
+      "mean_token_accuracy": 0.8345768079161644,
+      "num_tokens": 2865152.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.5235911178402602,
+      "epoch": 2.9762150982419855,
+      "grad_norm": 0.36328125,
+      "learning_rate": 1.1412889406192673e-07,
+      "loss": 8.273484039306641,
+      "mean_token_accuracy": 0.8450831711292267,
+      "num_tokens": 2947072.0,
+      "step": 360
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5448623623265777,
+      "eval_loss": 0.6060348153114319,
+      "eval_mean_token_accuracy": 0.8278159535208414,
+      "eval_num_tokens": 2970624.0,
+      "eval_runtime": 254.9017,
+      "eval_samples_per_second": 0.843,
+      "eval_steps_per_second": 0.843,
+      "step": 363
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 363,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.471321806592082e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-363/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2eaf4e1eba101412810b250e27914b2df87f93b0a9c62028451f50813e692b8e
+size 5713

processor_config.json

@@ -0,0 +1,75 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "dither": 0.0,
+    "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+    "feature_size": 128,
+    "fft_length": 512,
+    "fft_overdrive": false,
+    "frame_length": 320,
+    "hop_length": 160,
+    "input_scale_factor": 1.0,
+    "max_frequency": 8000.0,
+    "mel_floor": 0.001,
+    "min_frequency": 0.0,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "per_bin_mean": null,
+    "per_bin_stddev": null,
+    "preemphasis": 0.0,
+    "preemphasis_htk_flavor": true,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4ImageProcessor",
+    "image_seq_length": 280,
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4Processor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4VideoProcessor"
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

tokenizer_config.json

@@ -0,0 +1,95 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<content>(?:(?!\\<\\|tool_call\\>)(?!\\<turn\\|\\>).)+)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?:\\<turn\\|\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2eaf4e1eba101412810b250e27914b2df87f93b0a9c62028451f50813e692b8e
+size 5713