# Integrating PubMed Label Hierarchy Knowledge into a Complex Hierarchical Deep Neural Network +
## 1 Introduction  
The most recent studies in the field of artificial intelligence, and in particular in the Natural Language Processing (NLP) area, have proposed innovative deep learning (DL) network architectures, which were able to sensibly improve the previous state-of-the-art systems. Examples of these topologies are the Attention and Transformer-based Deep Neural Network architectures [1; 2], which have provided a breakthrough in several application domains of medicine and healthcare, such as medical imaging or biomedical information extraction tasks [3; 4; 5; 6]. On the other hand, among the various open issues related to DNNs, there is a need to incorporate prior and external knowledge directly into the neural architectures, allowing exploitation of predefined rules or knowledge by the neural architecture independently from its training [7]. Recently, some contributions to address this issue have been provided by Graph Neural Networks (GNNs) [8; 9]. Moreover, it has been demonstrated that incorporating innate structures into artificial intelligence (AI) systems could provide substantial benefits [10]. Following these results, this paper presents a solution for incorporating an external innate structure into AI systems to integrate this kind of external knowledge within the DNN architecture. We applied the proposed approach to a specific difficult task: theeXtreme Multilabel Text Classification_ (XMTC) [11; 12]. This task consists of the labeling of textual documents with a variable number of labels belonging to a very large set, formed by thousands or more elements. In many real-world applications, the labels lie in a hierarchical tree-like structure, and in this case, the task can be called _hierarchical multilabel text classification_. An example of a hierarchical XMTC task is the automatic assignment of the MeSH (Medical Subject Headings) terms to the biomedical scientific articles indexed by PubMed ([[https://www.ncbi.nlm.nih.gov/pubmed/](https://www.ncbi.nlm.nih.gov/pubmed/)]([https://www.ncbi.nlm.nih.gov/pubmed/](https://www.ncbi.nlm.nih.gov/pubmed/)) (accessed on 5 December 2023)), the official indexing and search engine for the biomedical scientific literature provided and maintained by the US National Library of Medicine (NLM). Domain experts manually tag each paper in PubMed for indexing and searching purposes. The classes used in PubMed belong to a large set called MeSH, where each label lies in a hierarchical tree structure. The manual annotation task is very long and difficult, and several methods have been proposed in the literature to support this process and implement a hierarchical extreme multilabel text classifier, like the ones described in the more recent BioASQ distributed tasks results [13; 14; 15; 16]. The large-scale MeSH indexing task has previously been faced with hybrid DNNs and classic machine learning techniques [17], BERT-based solutions [18; 19; 20], or by adopting GNN-based approaches [21], improving the obtained performances over the years and often combining the integration of more approaches to exploit their respective advantages. Anyway, the complexity of this task and the need for more innovative and performing architectures is supported by the recent works presented in the literature, as well as by the distributed task proposed every year by BioASQ. In this context, this paper proposes an innovative approach for the XMTC problem based on a Hierarchical Deep Neural Network (HDNN). This network topology internally reproduces the hierarchy of the labels within its structure. In this way, each node is specialized for the classification of its children nodes labels, considering both the results of the parent nodes and a representation of the input text. The network topology is dynamically and automatically obtained by an algorithm, which implements the network topology by iterating basic DNN blocks, following the hierarchical graph of the labels.