astrevallion/Qwen3-14B-FT

Uploaded model

• Developed by: astrevallion
• License: apache-2.0
• Finetuned from model : unsloth/qwen3-14b-unsloth-bnb-4bit

Qwen3-14B-FT: A Dungeons & Dragons SRD Specialist

This repository contains a fine-tuned version of Qwen3-14B-Instruct, specialized for question-answering and content generation related to the Dungeons & Dragons (D&D) 5.1 System Reference Document (SRD).

This model was fine-tuned as part of a Master's thesis research project comparing the effectiveness of Fine-Tuning (FT) and Retrieval-Augmented Generation (RAG) for domain-specific applications on consumer-grade hardware.

Full Project:

GitHub Repository: https://github.com/ErcagCan/ft_vs_rag_dnd

Scientific Article: Fine-Tuning versus Retrieval-Augmented Generation in Large Language Models: A Comparative Study on Dungeons & Dragons

Model Description

This model was fine-tuned using QLoRA with the Unsloth library on a dataset of 1,779 question-answer pairs derived and augmented from the D&D 5.1 SRD. The fine-tuning process aimed to imbue the base model with the specific terminology, structure, and "voice" of the D&D ruleset.

The key finding of the research was that while this FT model showed improvements in stylistic alignment, the most significant gains in factual accuracy came from using Retrieval-Augmented Generation (RAG). Intended Uses & Limitations

This model is intended for:

Answering questions about Dungeons & Dragons rules, lore, and mechanics as covered by the SRD 5.1.

Serving as a creative assistant for Dungeon Masters to generate narrative hooks, item descriptions, and encounter ideas based on SRD content.

Academic research into the effects of fine-tuning on domain-specific knowledge and stylistic adaptation in LLMs.

Limitations:

The model's knowledge is limited to the D&D 5.1 SRD. It does not contain information from other sourcebooks or editions.

While fine-tuning improved stylistic alignment, the standalone model may still produce factual inaccuracies (hallucinations). For the highest factual reliability, it is strongly recommended to use this model within a Retrieval-Augmented Generation (RAG) pipeline.

Training Data

The model was fine-tuned on a dataset created through a two-stage process:

Preprocessing: The D&D 5.1 SRD, sourced from an open-license JSON dump, was parsed into 1,779 distinct chat pairs.

Chain-of-Thought (CoT) Augmentation: For each pair, a base LLM was used to generate a more natural-sounding user question and a <think> block emulating the reasoning process to arrive at the answer.

This augmented dataset encouraged the model to not only learn the SRD content but also to adopt a more structured, reasoning-forward approach to its answers.

This qwen3 model was trained 2x faster with Unsloth and Huggingface's TRL library.