ti-analyst-4b-150 (GGUF Q4_K_M)

GitHub Repository: tacit-knowledge-lora

🛑 Disclaimer & Implementation Details

Status: Academic prototype; not production-tested. GGUF versions may exhibit unexpected variance. Issue: The model may occasionally predict the start of a new user turn (e.g., <|im_start|>user) instead of stopping. Solution: Strongly recommend explicitly defining stop sequences in your inference pipeline. Use <|im_start|> and <|endoftext|> to prevent the model from generating beyond the current turn. These are included in the repository's configuration files by default.

Model Description

LoRA adapters encoding talent intelligence analytical reasoning into the Qwen3.5 model family. This specific adapter represents the "empirical sweet spot" of the project: trained on 150 expert-curated behavioral examples.

Architecture: Dense hybrid attention (Gated DeltaNet + full softmax, 3:1 ratio). Training method: bf16 LoRA. Intended use: Thought partner for talent intelligence analysis. Optimized for high-density domain reasoning on consumer hardware.

The "150-Example Sweet Spot"

A central finding of this project's ablation study was that 100–150 well-curated examples are sufficient for meaningful domain transfer while preserving full general knowledge. Beyond 250 examples, general knowledge began to erode. The 4B-150 model represents the optimal balance between domain expertise and generalist capability.

Hardware Accessibility

This model is designed for broad accessibility. It provides Level 3 talent intelligence reasoning (per Culshaw 2022) and can run comfortably on any laptop with 8 GB of RAM, offline and without an API.

Training Data

• 150 examples covering 7 knowledge subcategories across talent intelligence
• Single-analyst training data: encodes one practitioner's analytical priorities and reasoning patterns
• Constructed through AI-assisted generation directed by domain expertise, with manual review and iterative evaluation-driven refinement

Evaluation Results

Cross-Scale Summary (150 examples focus)

Metric	4B	2B	0.8B
Judge score (1-5)	3.18	2.45	Below threshold
Signal density (FT/Base)	2.4x	3.2x	1.6x
General knowledge preserved	0.88	0.71	0.71

Known Biases and Limitations

Single-Analyst Bias

The adapter encodes one practitioner's analytical priorities. Vocabulary shift analysis quantified the emphasis distribution:

• Compensation terms: +1.87x amplification
• Labor market terms: +1.54x amplification
• Data quality terms: 0.45x suppression

Behavioral Limitations & Hallucinations

• Stop-Sequence Hallucinations: Like many conversational fine-tunes, this model may occasionally predict the start of a new dialogue turn (e.g., generating <|im_start|>user).
• Mitigation: Users should implement a stopping criterion on the following tokens: <|endoftext|>, <|im_start|>, and <|im_end|>. The provided Modelfile handles this automatically for Ollama users.
• Diagnostic questioning: Transferred successfully at larger scales, but is less consistent at the 4B scale compared to the 9B.

What This Model Is Not

• Not an oracle. It encodes one analyst's judgment, not ground truth.
• Not a replacement for expertise. It is a thought partner that can structure analytical reasoning, not a substitute for domain knowledge.

Responsible Use

This model should be used as a reasoning aid.

• Validate against additional sources.
• Be aware of emphasis biases.
• Understand the maturity level.

Technical Specifications

Parameter	4B
Training examples	150
LoRA rank	64
LoRA alpha	128
Epochs	3
Precision	bf16

References

Culshaw, T. (2022). Talent intelligence: Use business and people data to drive organizational performance. Kogan Page.

---

Quantization & Usage Details

This is a Q4_K_M GGUF quantization of the fine-tuned model.

• Source precision: bf16 (merged fine-tune)
• Quantization: Q4_K_M via llama.cpp
• Architecture: Qwen3.5 (4096h, 32L)

Usage with Ollama

# Download the GGUF and Modelfile, then:
ollama create ti-analyst-4b-150 -f Modelfile
ollama run ti-analyst-4b-150

Usage with llama.cpp

llama-cli -m ti-analyst-4b-150-Q4_K_M.gguf --jinja --color -ngl 99 -fa -c 4096