## 3 Materials and Methods
Word-piece embedding involves the embedding of a word that is an actual input, and is used to divide a sentence into token units. It enables more effective discrimination of tokens compared to simply separating tokens using spaces. For example, the word \"playing\" can be considered to be a token comprising the components, \"play\" and \"ing\". This enables the clear comprehension of each token by the deep learning model; thus, even when a misspelt or new word is used as input, the deep learning model performs satisfactorily as the words are divided into components that may have been trained previously. Segment embedding informs the deep learning model of the existence of two different sentences. When two sentences are input, they are distinguished by assigning a distinct number to each sentence. Position embedding encodes the relative location information of tokens. The deep learning model uses position embedding to determine the order of tokens. In general, position embedding uses the sine and cosine functions to create a matrix whose elements comprise word vectors depending on their positions. However, in BERT, positional information is obtained based on training, rather than the use of sine and cosine functions. **BioBERT Fine-tuning:** BERT normally uses pretrained word embeddings in text mining tasks, but BioBERT creates a new entity by training word-piece embedding by itself during the fine-tuning and pretraining processes. However, if the word included in the existing text does not exist in the vocabulary of the embedding, providing alternative expressions for the corresponding word is more difficult that when it exists in the vocabulary of the embedding. To overcome this, BioBERT employs word-piece embedding used in BERT to represent each word by dividing it into lower-level words. Since it can elicit more meaning, it is easy to deduce the meaning of a term from low-level words, even if there are unseen words or misspellings. To utilize this advantage properly, the pretrained weights are adjusted using word-piece embedding while fine-tuning BioBERT. \[p(y_{i}=k|T_{i})=softmax(T_{i}W^{\intercal}+b)_{k^{\prime}},\ \ k=0,1,\ldots,6 \tag{3}\]  
Figure 6: Input embedding of BERT (cited from [30]). Here, \(T_{i}\) denotes the last hidden size corresponding to the token, \(i\); \(p\) denotes the label probability; \(b\) denotes bias; \(W\) denotes the classification layer; and \(k\) = 7. The equation for classification loss, \(L\), is as follows:  
\[L(\theta)=-\frac{1}{N}\sum_{i=1}^{N}\log(p(y_{i}|T_{i};\theta)) \tag{4}\]  
where \(N\) denotes the sequence length and \(\theta\) denotes the trainable parameter. BioBERT NER is a text mining task that extracts domain-specific entities via directly trained word-piece embedding. Token classification, which assigns labels to word tokens, is performed using a single output layer. In the BERT embedding process, the meanings of even low-level words are extracted to facilitate the inference of meanings of input words that are not included in the embedding vocabulary. RE classifies the relation between entities in a biomedical corpus. Entity identification in the NER model must be performed before processing RE. The RE task, which follows the NER task, identifies all possible entity pairs in the sentence and classifies the relation. For instance, consider a sentence in which the words disease category A, B, and gene/protein category C are extracted from the NER task. In such a case, the pair of relationships between the two categories will be (A, C), (B, C). The two relation pairs are substituted with a specific mask to generate two relation sentences that serve as input. The outcomes of NER and RE fine-tuning are visually given to the user. NetworkX was utilized to convert RE data to graphs, and the Cytoscape module was utilized to visualize graph data for interaction. A more detailed description is offered in the result section. **NER Dataset Generation for Retraining:** The existing web service visually displays the NER results and RE graph corresponding to the desired PMID.