## 5 Experiment
### Reliability  
The reliability of KGQA systems directly impacts their trustworthiness in real-world applications (e.g., medical diagnosis and financial decision making). Reliability here encompasses output stability--the model's ability to consistently return correct answers across multiple executions [43; 44]. For LLMs, three critical parameters govern output reliability: temperature, top-k, and top-p.  
Temperature: The temperature parameter fundamentally reshapes the token probability distribution through entropy scaling. Given the original logits \(z_{w}\) for token \(w\), the temperature-scaled probability is computed as follows:  
\[P(w)=\frac{\exp(z_{w}/T)}{\sum_{w^{\prime}}\exp(z_{w^{\prime}}/T)} \tag{14}\]  
where \(T\) controls the steepness of the distribution: Low temperatures (\(T\to 0\)) maximize the probability gap between the top token and others, effectively implementing greedy decoding. High temperatures (\(T>1\)) flatten the distribution, increasing the generation diversity at the cost of reliability.  
Top-k Sampling constrains decoding to the top k tokens by probability at each step:  
\[V_{top-k}=\{w_{i}|P(w_{i})\geq P(w(k))\} \tag{15}\]  
A smaller k suppresses low-probability noise but risks ignoring domain-specific rare tokens.  
Top-p dynamically constructs a minimal candidate set covering cumulative probability p:  
\[\sum_{w\in V_{top-p}}P(w)\geq p \tag{16}\]  
Compared to fixed-k sampling, top-p adapts to the distribution shape, balancing reliability and flexibility.  
To quantify the parameter impacts, we evaluated six parameter combinations on the MetaQA and PathQuestion datasets, chosen for their comprehensive KG coverage to minimize noise from incomplete knowledge. During testing, 100 inference runs were conducted for each parameter combination using the same validation dataset. The average correct rate (ACR) and average variance (AV) of the inferred results were calculated, with the formula defined as follows:  
\[\text{Average Correct Rate}=\frac{1}{N}\sum_{i=1}^{N}\frac{\text{Correct Predictions}_{i}}{\text{Total Predictions}_{i}} \tag{17}\]  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**MetaQA(3-hop)**} & \multicolumn{2}{c}{**PQ(3-hop)**} & \multicolumn{2}{c}{**WebQSP**} & \multicolumn{2}{c}{**CWQ**} \\ \hline  & hit@1 & F1 & hit@1 & F1 & hit@1 & F1 & hit@1 & F1 \\ ES1 & 70.9 & 76.22 & 33.4 & 33.4 & 53.27 & 73.59 & 13.5 & 15.92 \\ ES2 & 28.63 & 73.91 & 41.6 & 41.6 & 44.49 & 52.03 & 28.51 & 23.21 \\ DRKG & 100 & 100 & 89.3 & 89.3 & 78.16 & 74.05 & 66.99 & 58.04 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Ablation experiment results.  
\[\text{Average Variance}=\frac{1}{N}\sum_{i=1}^{N}(\text{Accuracy}_{i}-\text{ Mean Accuracy})^{2} \tag{18}\]  
As shown in Table 10, the experimental results demonstrate that temperature significantly impacts model accuracy. Low-temperature settings (\(T\leq 0.1\)) achieve perfect accuracy, while high-temperature settings (\(T\geq 0.8\)) cause substantial performance degradation. This occurs because low temperatures enforce greedy decoding, which suppresses stochasticity by consistently selecting the highest-probability entities. Notably, this pattern holds across both the MetaQA and PathQuestion datasets, yielding optimal performance under low-temperature conditions. In contrast, top-k and top-p exhibit minimal regulatory effects except in extreme parameter ranges (e.g., \(k\leq 5\) or \(p\leq 0.3\)). These characteristics necessitate a stability-first optimization principle. Low-temperature configurations maximize reliability by eliminating randomness, while top-k/top-p parameters show negligible influence due to DRKG's inherent structural constraints. For applications demanding precision without generative diversity (e.g., mission-critical systems), we recommend prioritizing low-temperature settings (\(T\leq 0.1\)) and minimal top-p values (\(p\leq 0.1\)) to ensure deterministic, high-fidelity outputs.