youngseok12/AX-3.1-Light-sft_v3_0

A.X-3.1-Light-sft_v3_0

skt/A.X-3.1-Light를 기반으로 AI Hub 한국어 데이터로 수행한 첫 번째 clean SFT baseline입니다. LoRA adapter를 base model에 병합한 standalone BF16 모델이므로 별도의 adapter나 custom Python code 없이 Transformers 및 표준 vLLM에서 로드할 수 있습니다.

이 모델은 연구 및 2026 K-DATA SCIENCE 해커톤의 로컬 검증을 위한 실험 산출물입니다. K-AI 리더보드의 공식 점수를 포함하지 않으며, 공개 benchmark 문항은 학습 데이터에 포함하지 않았습니다.

Model Details

• Base model: skt/A.X-3.1-Light
• Architecture: LlamaForCausalLM (unchanged from the base model)
• Parameters: approximately 7B
• Weight format: BF16 safetensors
• Chat template: the official A.X chat template
• Custom code: none
• Merge method: PEFT LoRA merge_and_unload
• Base revision used for training and merge: 9b41bb2406472634d8812c0b8931fa40fa9a6c3a

Training Data

The SFT view contained 36,000 training examples and 3,000 validation examples. Training and validation were kept separate at the source level. The following AI Hub data domains were used:

• Civil law LLM pretraining and instruction-tuning data
• Criminal law LLM pretraining and instruction-tuning data
• Administrative law LLM pretraining and instruction-tuning data
• Corporate accounting standards data
• Essential medical knowledge data
• News article machine-reading data
• CoT-Fabric technology valuation data

Public evaluation benchmark data, including KMMLU-Pro, CLIcK, HLE, SNU Ko-MuSR, Com2-main, and Original MuSR, was not used for SFT.

Training Procedure

• Objective: standard assistant-token causal language-model cross entropy
• Epochs: 1
• Learning rate: 5e-5
• Scheduler: cosine
• Warmup ratio: 0.03
• Weight decay: 0.01
• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• LoRA target modules: q_proj, k_proj, v_proj, o_proj
• Effective batch size: 32
• Maximum sequence length: 2048
• Precision: BF16
• Packing: disabled
• Random seed: 20260825

The final training view used 800 reasoning examples. Their output tokens accounted for approximately 24.37% of the total assistant target tokens. The training format validator passed all 36,000 training and 3,000 validation examples.

Usage with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "youngseok12/AX-3.1-Light-sft_v3_0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "대한민국의 수도는 어디인가요?"}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=64, do_sample=False)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Usage with vLLM

vllm serve youngseok12/AX-3.1-Light-sft_v3_0 \
  --dtype bfloat16 \
  --max-model-len 32768

The repository is intended to be publicly accessible and directly loadable by a standard vLLM OpenAI-compatible server. The model does not require trust_remote_code=True.

Intended Use and Limitations

This model is intended for Korean-language research, controlled evaluation, and experimentation with instruction-following and domain-oriented SFT. It is not a substitute for professional legal, accounting, or medical advice. Responses may contain factual errors, unsupported reasoning, or format deviations. The reported training configuration does not imply an official K-AI leaderboard result.

The AI Hub source data remains subject to its original access terms and licenses. This repository contains model weights and documentation, not the source training datasets.

License

The base model is distributed under the Apache License 2.0. Please review the base model license and the terms applicable to the source training data before using or redistributing this derivative model.