ti-analyst-9b (GGUF Q4_K_M)

GitHub Repository: tacit-knowledge-lora

🛑 Disclaimer & Implementation Details

Status: Academic prototype; not production-tested. GGUF versions may exhibit unexpected variance. Issue: The model may occasionally predict the start of a new user turn (e.g., <|im_start|>user) instead of stopping. Solution: Strongly recommend explicitly defining stop sequences in your inference pipeline. Use <|im_start|> and <|endoftext|> to prevent the model from generating beyond the current turn. These are included in the repository's configuration files by default.

Model Description

LoRA adapters encoding talent intelligence analytical reasoning into the Qwen3.5 model family. The 9B adapter was trained on 350 expert-curated behavioral examples.

Architecture: Dense hybrid attention (Gated DeltaNet + full softmax, 3:1 ratio). Training method: bf16 LoRA (not QLoRA. The Qwen3.5 hybrid attention layers produce NaN loss under 4-bit NF4 quantization). Intended use: Thought partner for talent intelligence analysis. The model should assist analytical reasoning, not replace it.

Training Data

• 350 examples covering 7 knowledge subcategories across talent intelligence
• Single-analyst training data: encodes one practitioner's analytical priorities and reasoning patterns
• Constructed through AI-assisted generation directed by domain expertise, with manual review and iterative evaluation-driven refinement
• Single normalized system prompt across all examples

Evaluation Results

Cross-Scale Summary (350 examples)

Metric	9B	4B	2B	0.8B
Judge score (1-5)	3.46	3.18	2.45	Below threshold
Signal density (FT/Base)	1.7x	2.4x	3.2x	1.6x
General knowledge preserved	0.88	0.88	0.71	0.71

Strongest and Weakest Subcategories

Strongest: Compensation & Benefits. Highest cross-scale scores, most consistent ablation performance, highest token agreement (5.9%) in divergence analysis. Structured frameworks with defensible answers produce the most robust encoding.

Weakest: Competitive Intelligence. Lowest scores at 2B and 0.8B, with "competitor" token demoted 10.3 rank positions in divergence analysis. The model reframes competitive analysis as benchmarking. Correctable with targeted training examples.

Known Biases and Limitations

Single-Analyst Bias

The adapter encodes one practitioner's analytical priorities. Vocabulary shift analysis quantified the emphasis distribution:

• Compensation terms: +1.87x amplification
• Labor market terms: +1.54x amplification
• Data quality terms: 0.45x suppression

Behavioral Limitations

• Diagnostic questioning transferred successfully at 9B (judge score 4.0/5) but did not consistently generalize across smaller scales
• Actionability was the weakest qualitative dimension at 9B (2.82/5). The model learned to reason like a senior analyst at the cost of less immediately actionable output
• System prompt switching instability was observed at 9B: when the inference-time system prompt conflicted with the training prompt, the model sometimes reproduced the training persona.
• Note on Sequence Termination During inference, the model may exhibit "over-generation" by predicting subsequent conversational turns. This is a result of the training distribution where <|endoftext|> is often followed by a new user prompt. Technical Mitigation: Developers should implement a stopping criterion on the following tokens to ensure clean outputs: <|endoftext|>, <|im_start|>, <|im_end|> (if applicable)

What This Model Is Not

• Not an oracle. It encodes one analyst's judgment, not ground truth.
• Not tested on real workforce data. Evaluation used synthetic stakeholder scenarios, not operational data with messy real-world inputs.
• Not a replacement for expertise. It is a thought partner that can structure analytical reasoning, not a substitute for domain knowledge.

Responsible Use

This model should be used as a reasoning reason aid.

• Validate against additional sources.
• Be aware of emphasis biases.
• Understand the maturity level.
• Context matters.

Technical Specifications

Parameter	9B
Training examples	350
LoRA rank	64
LoRA alpha	128
Epochs	3
Precision	bf16
Target modules	q/k/v/o_proj, gate/up/down_proj

References

Culshaw, T. (2022). Talent intelligence: Use business and people data to drive organizational performance. Kogan Page.

---

Quantization & Usage Details

This is a Q4_K_M GGUF quantization of the fine-tuned model.

• Source precision: bf16 (merged fine-tune)
• Quantization: Q4_K_M via llama.cpp
• Architecture: Qwen3.5 (4096h, 32L)

Usage with Ollama

# Download the GGUF and Modelfile, then:
ollama create ti-analyst-9b -f Modelfile
ollama run ti-analyst-9b

Usage with llama.cpp

llama-cli -m ti-analyst-9b-Q4_K_M.gguf --jinja --color -ngl 99 -fa -c 4096