## 5 Discussion  
It is worth noting that the modified BertGCN model with longer input length achieved a performance comparable to the highest validation accuracy achieved by the original  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline  & \(\mathbf{\lambda}\) = 0.0 & \(\mathbf{\lambda}\) = 0.1 & \(\mathbf{\lambda}\) = 0.2 & \(\mathbf{\lambda}\) = 0.3 & \(\mathbf{\lambda}\) = 0.4 & \(\mathbf{\lambda}\) = 0.5 & \(\mathbf{\lambda}\) = 0.6 & \(\mathbf{\lambda}\) = 0.7 & \(\mathbf{\lambda}\) = 0.8 & \(\mathbf{\lambda}\) = 0.9 & \(\mathbf{\lambda}\) = 1.0 \\ \hline Clued & 0.8949 & 0.8874 & 0.8879 & 0.8839 & 0.8879 & 0.8897 & 0.8950 & 0.8847 & 0.8963 & 0.8913 & 0.8816 \\ Un-clued & 0.8959 & 0.8892 & 0.8931 & 0.8979 & 0.8885 & 0.8922 & 0.8885 & 0.8870 & 0.8917 & 0.8929 & 0.8867 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Validation accuracy by \(\lambda\) (length = 280 tokens).  
Figure 2: Document classification performance of the three modes. This shows the document classification performances of the three modes, including the modified BertGCN, the original BertGCN, and the citation-based GCN models.  
BertGCN. This finding indicates that it can consistently achieve high validation accuracy without relying on the composition parameter, simply by connecting the last layers of the BERT and GCN models. This idea deserves consideration when aiming to surpass existing BERT document classification performance using GCN, to implement a practical system.  
However, it was found that the citation graph exhibited a lower performance than the statistical GCN, achieving a validation accuracy of about 10% lower. The smaller size of the nodes and edges constituting the citation graph may explain this gap, but further research is necessary to determine the exact reason.  
As for various factors influencing the original BertGCN, The length of input data fed to the model is a primary factor for classification performance in BertGCN. The validation performance disparity of up to approximately 4% between the two length conditions of the input sequence (128, 280 tokens) is rooted in the richer BERT embeddings (Figure 2). This is because as the input sequence length increases, BERT can capture more accurate and informative feature embeddings with the help of the multi-headed self-attention mechanism. The longer sentence condition is slightly less sensitive to the \(\lambda\) variation compared to the shorter condition. While the shorter case shows a gradual validation accuracy increase from \(\lambda\) of 0.0 to 0.8, the longer case exhibits a bumpy pattern that requires more trials to establish a stable pattern. The shorter-sequence model shows a deviation of approximately 2%, whereas there is no significant difference under the long-sentence condition except for the sudden drop near \(\lambda\) of 1.0. Therefore, it implies that the importance of the composition parameter \(\lambda\) increases as the length of input sequences decreases.  
In the BertGCN model, the final output is a combination of the logit values from the BERT and GCN subnetworks. The observed trend of increasing validation accuracy with higher values of \(\lambda\) for shorter input sequences suggests that the GCN model has a greater impact on improving classification accuracy than the BERT model as the value of \(\lambda\) increases.  
The experiments revealed no noticeable performance gap between cloud and uncluded conditions. The observed improvement in classification performance without critical keywords can also be attributed to the ability of Transformer-related models to capture contextual information during training using multi-headed attention mechanisms, enabling accurate classification even in the absence of explicit keyword cues, such as \"in vivo\" or \"in vitro\".