# Integrating PubMed Label Hierarchy Knowledge into a Complex Hierarchical Deep Neural Network +
## 5 Conclusions  
This paper presented an innovative deep learning network architecture based on a hierarchical structure, named _Hierarchical Deep Neural Network_ (HDNN), specifically devoted to the hierarchical XMTC problem. The topology of the HDNN reproduces the same hierarchical structure of the labels. This general concept can be applied in all multilabel classification problems where the labels are organized in a hierarchical structure. After an introduction to the problem and a related works section, we described the details of the HDNN and a methodology for the automatic creation of this DNN, \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{2}{*}{**Test**} & \multicolumn{8}{c}{**FLAT Measures**} \\ \cline{2-7}  & **Acc** & **EBP** & **EBR** & **EBF** & **MiP** & **MiR** & **MiF** \\ \hline
**CNN WE** & 0.1353 & 0.7528 & 0.1398 & 0.2324 & 0.7522 & 0.1336 & 0.2268 \\ \hline
**CNN BioBERT** & 0.2344 & 0.7828 & 0.1619 & 0.2684 & 0.7810 & 0.1592 & 0.2644 \\ \hline
**HDNN WE** & **0.1719** & **0.7578** & **0.1828** & **0.2894** & **0.7578** & **0.1739** & **0.2829** \\ \hline
**HDNN BioBERT** & **0.3015** & **0.8213** & **0.2187** & **0.3453** & **0.8191** & **0.2041** & **0.3270** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performances of HDNN and CNN with static and dynamic embeddings in terms of flat measures. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Test**} & \multicolumn{8}{c}{**Hierarchical Measures**} \\ \cline{2-7}  & **HiP** & **HiR** & **HiF** & **LCA-P** & **LCA-R** & **LCA-F** \\ \hline
**CNN WE** & 0.7839 & 0.1635 & 0.2665 & 0.5530 & 0.0738 & 0.1273 \\ \hline
**CNN BioBERT** & 0.8117 & 0.1775 & 0.2899 & 0.5824 & 0.1113 & 0.1819 \\ \hline
**HDNN WE** & 0.7820 & **0.2052** & **0.3194** & **0.6350** & **0.1239** & **0.2020** \\ \hline
**HDNN BioBERT** & **0.8229** & **0.2573** & **0.3941** & **0.7177** & **0.1944** & **0.3005** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performances of HDNN and CNN with static and dynamic embeddings in terms of hierarchical measures. starting from the label hierarchy structure, based on a recursive algorithm that iterates basic DNN blocks. We also defined a method for the regularization of the training set, named Hierarchical Label Set Expansion (HLSE), which adds to each document label set the corresponding ancestors if they are not present, obtaining a training set consistent with the predefined hierarchy. The experimental assessment focused on the automatic labeling of the biomedical scientific literature indexed in PubMed, which adopts the MeSHs (Medical Subjects Headings) hierarchical label set. The obtained results can be useful for the improvement of the large biomedical indexing task, integrating it in more complex methodologies, as proposed by the most recent methodologies in the BioASQ distributed challenges [13; 14; 15; 16], which usually integrate more methodologies to obtain state-of-the-art performances. A limit of the proposed methodology is its high computation time, mainly required to update the network weights, caused by the need to wait for the complete update of the upper layers' weights before the calculation of the corresponding children nodes' new neural network weights. This issue can be addressed by adopting more powerful GPU architecture, such as the newer Nvidia H100 architecture, as well as implementing a parallel version of the HDNN training routine, capable of distributing the computational complexity among different nodes and GPUs. We envisage the development of a parallel version of the HDNN code as a future work. A possible strategy for the parallel implementation includes the development of a multi-GPU code, which runs each internal node of the same level of the HDNN on separate GPUs in parallel. Another possibility is to adapt some GNN parallel approaches, such as the ones proposed in [41; 42; 43], allowing better exploitation of the features of modern multi-GPU hardware architectures.