# Document Retrieval System for Biomedical Question Answering
## 4 Discussion  
Commonly used ranking algorithms cannot measure words' semantic similarity or their similarity/closeness to other words. These algorithms work on a dictionary basis. Generally, the similarity score between the query and the documents is calculated based on the number of common words. Although this gives fast and good results in searches based on word similarity, it is insufficient for queries based on comprehension. In our study, we tried to eliminate the disadvantages of ranking algorithms by using information retrieval techniques and textual and semantic similarity. Therefore, we used UMLS services to provide semantically similar concepts to enrich the query. At the same time, we added important words (nouns) to the query and thus increased the effect of these words in similarity calculations, since nouns are more significant and distinctive for similarity calculations. These applied techniques provided a general increase in performance, although it remains below the desired level. The performance did not reach a certain level because \begin{table}
\begin{tabular}{l l l l} \hline  & **Precision** & **Recall** & **F1-Score** \\ \hline First Method (5 feature F1 weight) & 0.3609410430839 & 0.268139543582234 & 0.256271218796207 \\ Second Method (3 feature F1 weight) & 0.313103254769921 & 0.272229307505606 & 0.260814282105723 \\ Third Method (NER Filter with 0.2 weight) & 0.303397311305475 & 0.260304412863095 & 0.249995189380743 \\ \hline \end{tabular}
\end{table} 
Table 18: Top 10 evaluation score. \begin{table}
\begin{tabular}{l l l l} \hline  & **Precision** & **Recall** & **F1-Score** \\ \hline First Method (5 feature F1 weight) & 0.265168962321771 & 0.424878821734178 & 0.297632486789945 \\ Second Method (3 feature F1 weight) & 0.260647496638209 & 0.413577298763878 & 0.290564749074941 \\ Third Method (NER Filter with 0.2 weight) & 0.263240786175486 & 0.401791106023006 & 0.289514758013646 \\ \hline \end{tabular}
\end{table} 
Table 19: Top 20 evaluation score. \begin{table}
\begin{tabular}{l l l l} \hline  & **Precision** & **Recall** & **F1-Score** \\ \hline One-by-One (NER Filter, 0.2 weight) top10 & 0.387081128747795 & 0.277551033286955 & 0.310881366606519 \\ One-by-One (5 feature F1 weight) top10 & 0.391542408209075 & 0.279592440704706 & 0.313518331658926 \\ One-by-One (NER Filter, 0.2 weight) top20 & 0.388856258638562 & 0.35390159282724 & 0.366408330602924 \\ One-by-One (5 feature F1 weight) top20 & 0.3944740846232 & 0.358815270716613 & 0.371674766230661 \\ One-by-One (NER Filter, 0.2 weight) & 0.39280291978339 & 0.385307940268726 & 0.388352093573967 \\ One-by-One (5 feature F1 weight) & 0.396920212051186 & 0.389355326112682 & 0.392433244564683 \\ \hline \end{tabular}
\end{table} 
Table 20: One-by-One evaluation score. the similarity score calculation was carried out using the dictionary-based bag-of-words method. To overcome this problem, it is necessary to use embedding/transformer-based methods that capture the meaning of words in the relevant context, calculate the semantic similarity between words, and can be trained according to the specific domains. As for the answer extraction module, we obtained five sentences created with different features of the same sentence. One is the original sentence, whereas the other four consist of an NER sentence, CUI sentence, Semantic Type sentence, and Semantic Groups sentence. We have employed two distinct score calculation methodologies. In the initial model, the similarity score is determined based on the textual resemblance between the question and the sentence. In the second model, the similarity score is computed using various factors, including textual similarity, named entity similarity, CUI similarity, semantic type similarity, and semantic group similarity scores. We achieved approximately a 25% performance increase for the top 20 results using the second model compared to the first. To obtain even better results, we need deep learning algorithms that can understand the question and sentences in their context.