## 5 Experiment
### Ablation Study
#### 5.3.1 Natural Language Question Types  
As previously stated, the LC-QuAD 2.0 dataset, with its considerable size and complexity, encompasses a broad range of question types. Table 4 shows all of the question types in this dataset, which include single or multi-fact, multi-intentions, boolean, count, ranking, string operation, and temporal aspect, among others. This section presents experimental evaluations assessing the performance of our model across these question types.  
Figure 3 provides a detailed visual representation of the QM scores for each question type. The QM scores, depicted through horizontal bars, represent the performance of two models: TSET and T5. Question types are listed on the y-axis, with corresponding QM scores on the x-axis. The figure clearly demonstrates that the TSET model outperforms the T5 model for most problem types. Specifically, TSET has a higher QM score in 8 out of the 13 problem types, indicating its superior performance. Moreover, TSET matches T5's performance in 2 out of 13 problem types, showcasing its robustness and versatility. The significant improvement of TSET over T5, denoted by the red line in the figure, is evident across several question types. In several cases, the improvement exceeds 5%, highlighting the substantial enhancement offered by the TSET model. Despite these improvements, it is worth noting that the T5 model performed better in three problem types. However, the overall superiority of TSET across most question types underscores  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Ratio**} & **Model** & \multicolumn{2}{c}{**Small**} & \multicolumn{2}{c}{**Base**} \\ \cline{3-6}  & **/Impr.** & **QM** & **F1** & **QM** & **F1** \\ \hline \multirow{4}{*}{1\%} & T5 & 49.34 & 54.01 & 55.39 & 61.11 \\  & TSET & 70.53 & 74.57 & 75.22 & 79.58 \\ \cline{2-6}  & Impr. & +21.19 & +20.56 & +19.83 & +18.47 \\ \hline \multirow{4}{*}{5\%} & T5 & 78.58 & 81.45 & 80.23 & 83.09 \\  & TSET & 83.18 & 86.50 & 85.81 & 89.04 \\ \cline{2-6}  & Impr. & +4.60 & +5.05 & +5.58 & +5.95 \\ \hline \multirow{4}{*}{10\%} & T5 & 82.47 & 85.25 & 84.35 & 87.25 \\  & TSET & 86.09 & 89.61 & 88.32 & 91.14 \\ \cline{2-6}  & Impr. & +3.62 & +4.36 & +3.97 & +3.89 \\ \hline \multirow{4}{*}{20\%} & T5 & 85.49 & 87.51 & 86.52 & 88.61 \\  & TSET & 88.31 & 90.96 & 90.39 & 92.48 \\ \cline{1-1} \cline{2-6}  & Impr. & +2.82 & +3.45 & +3.87 & +3.87 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The QM scores for low resource evaluation. For training, we selected 1%, 5%, 10%, and 20% of the data, respectively. “Impr.” is short for “Improvement”.  
the effectiveness of our proposed model and its potential for tackling a wide array of problem types.  
Figure 3: Comparison of TSET and T5 model performance across various problem types. The problem types are displayed on the y-axis, while the performance scores of the models are shown on the x-axis. The blue and light blue bars represent the scores of the T5 and TSET models, respectively, for each problem type. The red line with circular markers, which corresponds to the secondary x-axis at the top, shows the improvement of the TSET model over the T5 model. A higher score or a more positive improvement value indicates better performance. The problem types are ordered by the improvement of TSET over T5, with the problem type having the most significant improvement at the bottom.