Title: DHKR: Dynamic Hierarchical Knowledge Routing for Efficient Low‑Resource Alignment
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Knowledge Alignment, Domain adaptation, Domain Shift Tuning, Lottery Hedge Fund Hypothesis, PEFT, MoE

Abstract: Domain adaptation for large language models (LLMs) remains a significant challenge, often requiring extensive fine-tuning or inefficient architectural modifications. 
Knowledge-Aligned Domain Shift Tuning (KADA) is a parameter-efficient fine-tuning (PEFT) method that leverages the Lottery Hedge Fund Hypothesis (LHFH) to identify and reuse latent knowledge fragments. 
However, KADA is limited by its reliance on a static, predefined set of subnetworks, which restricts flexible adaptation to unseen domains.

Inspired by the routing principles of Mixture-of-Experts (MoE) and portfolio balancing strategies, 
we propose Dynamic Hierarchical Knowledge Routing (DHKR). 
DHKR extends KADA by introducing \textbf{two-level subnetwork growth}, a hierarchical routing mechanism that dynamically expands both high-level subnetworks ($K$) and their second-level components ($L$) on demand, thereby overcoming the static-size constraints of prior methods.
To ensure stability, DHKR employs a composite growth trigger—based on training stagnation, entropy reduction, usage imbalance, and routing stability—together with a multi-level Loss-Free Balancing (LFB) strategy. 
This transforms KADA’s static mechanism into a self-adaptive, robust knowledge routing system that adjusts capacity and structure to new domains while maintaining stable performance.

Experiments on instruction based benchmarks show that DHKR not only preserves the strengths of KADA but also addresses its key limitations.
Our work provides a principled framework for dynamic, knowledge-aligned adaptation of LLMs.

TL;DR: Dynamic Hierarchical Knowledge Routing is knowledge alignment tuning framework