haninmb/train_qewn3_final

Qwen3-14B fine-tuned for STR DNA family analysis

• Developed by: haninmb
• License: apache-2.0
• Finetuned from model: unsloth/Qwen3-14B-unsloth-bnb-4bit (Qwen3-14B)
• Weights in this repo: LoRA merged into the base model, 16-bit

The model reads an STR profile (father, mother and child alleles per marker) and returns an analysis of Mendelian inheritance: which markers mismatch, the mutation type and direction, the likely source parent, and a biological / not-biological classification. The answer is written as a short Analysis: section followed by a Final Response: JSON object.

This qwen3 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Training summary

Method	QLoRA supervised fine-tuning (4-bit base, LoRA adapters), 1 epoch
Total training time	118,618 s (about 32 h 57 min) wall-clock
Peak VRAM (reserved)	about 13 GB on a single GPU
Hardware	1 GPU with 32 GB VRAM
Throughput	0.78 samples/s, 0.196 steps/s (about 5.1 s per optimizer step)
Optimizer steps	23,203 (effective batch size 4)
Training examples	92,811
Held-out examples	938 (1% split, seed 42). Not used for evaluation during training.
Final training loss	about 0.068 (running mean over the whole run: 0.072)
Checkpoints	every 400 steps, last 3 kept

Loss fell from about 1.27 at step 10 to about 0.07 by step 2,000 and stayed between 0.066 and 0.070 for the rest of the run. The loss is computed over the whole sequence, including the long system prompt, not only the assistant answer, so it is not comparable with response-only losses.

Training data

Dataset: haninmb/all_merged_families (updates_prompts_all_merged_families_10_4v.jsonl), about 93.7k examples. Each example is a system prompt describing the inheritance rules and output format, a user message with the family STR profile, and an assistant answer. The original analysis and final assistant turns were merged into one answer (Analysis: followed by Final Response:), and the developer role was mapped to system.

Training procedure

Hyperparameter	Value
Base model	Qwen3-14B, loaded in 4-bit
LoRA rank / alpha / dropout	32 / 32 / 0
LoRA target modules	q, k, v, o, gate, up, down projections
Max sequence length	8,048 tokens
Per-device batch size	1
Gradient accumulation	4
Learning rate	2e-4, linear decay
Warmup steps	500
Weight decay	0.001
Optimizer	AdamW 8-bit
Precision	bf16
Gradient checkpointing	Unsloth
Seed	3407

Software: transformers 4.57.1, TRL 0.22.2, Unsloth 2026.10.11, PyTorch 2.11 (CUDA 12.8).

The training code is in training/ and the TensorBoard logs are in tensorboard/. The run can be resumed from its latest checkpoint after an interruption.

Usage

The 16-bit weights are about 30 GB. On a GPU with less than 40 GB of VRAM, load the model in 4-bit (for example with bitsandbytes or Unsloth, which needs about 10 GB for weights).

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "haninmb/train_qewn3_final", max_seq_length=16384, load_in_4bit=True
)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "<the STR analysis system prompt used in training>"},
    {"role": "user", "content": "Input family DNA profile:\nCSF1PO_Father: 10, 12 | Mother: 13, 14 | Child: 11, 13\n..."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to("cuda")
out = model.generate(**inputs, max_new_tokens=4048)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Use the system role for the instructions, as in training. Inference on a single consumer GPU in 4-bit runs at roughly 60 to 80 tokens per second, and a typical answer is 300 to 900 tokens.

Repository contents

• Model weights and tokenizer (merged, 16-bit)
• training/: training script and launcher
• tensorboard/: TensorBoard event logs for the full run