Aeon-Q4 : GGUF

Aeon Q4 is a Turkish-focused fine-tune of meta-llama/Llama-3.1-8B-Instruct, trained with Unsloth and QLoRA on a Turkish SFT (Supervised Fine-Tuning) dataset, then converted to GGUF format for fast local inference.

• What it is: an 8B parameter Turkish instruction-tuned chat model
• How it was trained: QLoRA fine-tuning with Unsloth + TRL SFTTrainer
• Why GGUF: quantized for efficient CPU/GPU inference via llama.cpp and Ollama
• Where to use it: llama.cpp, llama-cli, llama-mtmd-cli, or Ollama

⚠️ Alpha / Research Preview — This is an early research checkpoint (Stage 1 SFT, ~end of epoch 1/3), released for community testing and feedback rather than production use. Several known weaknesses (hallucination, inconsistent JSON schema output, multi-character dialogue degradation) have not yet been addressed — see Known Limitations below.

Model Details

Property	Value
Base model	Llama 3.1 8B Instruct
Training method	QLoRA (Unsloth + TRL SFTTrainer)
LoRA rank (r)	16
LoRA alpha	32
Max sequence length	128k
Effective batch size	16
Optimizer	AdamW 8-bit
Scheduler	Cosine
Total steps	~5292 (3 epochs)
Hardware	T4 (Google Colab)
Quantization	Q4_K_M (GGUF)

Training Summary

Aeon Q4 was fine-tuned on a Turkish identity/instruction-following SFT dataset using QLoRA to keep memory usage low enough for a single T4 GPU. Training used an 8-bit AdamW optimizer with a cosine learning-rate schedule and gradient checkpointing (via Unsloth) to enable a large effective context window. This release corresponds to a checkpoint taken near the end of the first epoch of a planned three-epoch Stage 1 run; evaluation was performed every 150 steps (eval_steps=150).

Usage

Use case	Command
Text only LLMs	llama-cli -hf yusuf3416/Aeon-Q4 --jinja
Multimodal models	llama-mtmd-cli -hf yusuf3416/Aeon-Q4 --jinja

Available Model Files

File	Quantization
meta-llama-3.1-8b-instruct.Q4_K_M.gguf	Q4_K_M

Additional quantization variants (Q4_0, Q5_K_M, Q8_0, etc.) may be added later.

Ollama

An Ollama Modelfile is included for easy deployment:

ollama create aeon-q4 -f Modelfile
ollama run aeon-q4

Training Metrics

The charts below track key metrics throughout Stage 1 SFT training (eval_steps=150).

Eval Loss

Evaluation loss showed a consistent downward trend across checkpoints, decreasing from roughly 1.36 at step 150 to roughly 1.05 at step 1950, without visible signs of overfitting at this stage of training.

Train Loss

Gradient Norm

Learning Rate

Evaluation loss showed a consistent downward trend across checkpoints, from ~1.36 (step 150) to ~1.05 (step 1950).

Current Strengths

• Turkish fluency: More natural, idiomatic Turkish phrasing compared to the base model's out-of-the-box Turkish output.
• Instruction following: Improved adherence to Turkish-language prompts and formatting requests.
• Identity consistency: The "Aeon Q4" persona remained stable and resisted adversarial jailbreak attempts even across extended contexts.
• Stable optimization: Smooth, monotonic loss decrease with no divergence or instability observed during Stage 1.

Known Limitations

Comprehensive inference testing surfaced the following weaknesses, which are targeted for Stage 2 training:

• Hallucination: Tendency to fabricate information when asked about nonexistent entities (confirmed as a greedy-decoding issue).
• Strategic math: Failure on math problems requiring correct method selection.
• JSON schema inconsistency: Output doesn't always conform to the requested schema.
• Multi-character dialogue: Loss of coherence in longer/complex multi-character dialogue.

Planned Improvements (Roadmap)

• Targeted data and training adjustments to reduce hallucination
• Reasoning-focused SFT to improve strategic math and multi-step problem solving
• Stricter JSON-schema adherence training
• Extended multi-turn / multi-character dialogue training
• Turkish-language benchmark evaluation before the next public checkpoint

Evaluation

No formal benchmark results are available yet for this alpha checkpoint. Planned evaluations for upcoming releases include:

• MMLU
• TruthfulQA
• GSM8K
• IFEval
• MT-Bench (Turkish)

This section will be updated as benchmark results become available.

License

This model is released under the Llama 3.1 Community License, inherited from the base model meta-llama/Llama-3.1-8B-Instruct. Use of this model is subject to Meta's Llama 3.1 license terms and acceptable use policy.

Acknowledgements

Thanks to the teams and projects that made this work possible:

• Meta for the Llama 3.1 base model
• Unsloth for fast, memory-efficient fine-tuning
• Hugging Face TRL for the SFTTrainer
• Hugging Face for model hosting and tooling
• llama.cpp and Ollama for inference support

Training Details

This model was trained 2x faster using Unsloth.

Citation

If you use Aeon-Q4 in your research, applications, or derivative work, please cite:

@software{aeonq4,
  author    = {Yusuf},
  title     = {Aeon-Q4: Turkish QLoRA Fine-Tuning of Llama 3.1 8B Instruct},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/yusuf3416/Aeon-Q4}
}