i-KenTanaka/merged_exp01

merged_exp01

This repository provides a fine-tuned model developed for the LLM Engineering Main Competition.

The model is trained to improve structured output generation for tasks requiring machine-readable formats such as JSON.

Model Overview

Model name

merged_exp01

Base model

unsloth/Qwen3-4B-Instruct-2507

Training approach

Supervised Fine-Tuning (SFT)

Objective

Improve accuracy of structured outputs such as:

JSON

YAML

CSV

XML

The training objective focuses on generating valid structured outputs suitable for automated evaluation pipelines.

Training Data

The model was trained using datasets provided for the competition.

Examples include:

structured_data_with_cot_dataset_512_v2

structured_data_with_cot_dataset_512_v4

structured_data_with_cot_dataset_512_v5

structured_data_with_cot_dataset

structured-3k-mix-sft

structured-5k-mix-sft

structured-hard-sft-4k

These datasets are designed for structured output tasks with reasoning support (CoT).

Note:

Use of these datasets does not guarantee higher scores and participants may combine or customize them.

メインコンペ_使用データまとめ.txt

Training Configuration

Typical training configuration used in this experiment:

Base Model unsloth/Qwen3-4B-Instruct-2507

Method QLoRA (4-bit)

Max sequence length 512

Epochs 1

Learning rate 1e-4

LoRA parameters

r = 64 alpha = 128

Usage

Example code for loading the model:

from transformers import AutoModelForCausalLM, AutoTokenizer import torch

model_id = "i-KenTanaka/merged_exp01"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.float16, device_map="auto" )

prompt = "Generate structured JSON output for the given task."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)

print(tokenizer.decode(outputs[0], skip_special_tokens=True)) Competition Submission

This model is submitted as part of the Main Competition submission requirements.

Submission components:

Inference JSON generated using the official inference code

Public Hugging Face model repository (this repository)

Limitations

The model is optimized for structured output tasks.

Performance may degrade for general conversational tasks.

Structured output validity depends on prompt design.

License and Compliance

Users must comply with:

The license of the base model

The license of the training datasets

Attribution requirements must be respected when using the datasets.

Author

Kenji Tanaka https://huggingface.co/i-KenTanaka

Acknowledgements

This model was developed using:

Hugging Face Transformers

Unsloth

QLoRA training method

Competition datasets provided by the organizers