# A Purely Entity-Based Semantic Search Approach for Document Retrieval
## 3 Materials and Methods
### Retrieval and Ranking Method
#### 3.3.3 Algorithm
Consequently, the absence of highly annotated queries (with an annotation avg \(\geq\) 0.75) annotated by the REL tool and the absence of corresponding qrels for the highly annotated queries annotated by the DBpedia Spotlight tool (with an annotation avg = 1.0) are the two reasons why we did not provide the combination of both PESS4 IR and Long (Longformer) on the MSMARCO collection. ## 5 Discussion

In the discussion section, we explain and discuss the added value achieved by PESS4 IR when it is tested with queries annotated by the REL and DBpedia Spotlight entity linking methods. Moreover, we discuss our experiments presenting the strengths and limitations of our approach (PESS4 IR). ### Pess4 IR Tested by REL

As a purely entity-based method is appropriate only for completely annotated queries, the results are partial when only completely annotated queries are considered. Among them, the ones with higher scores perform better than the rest. Thus, the purely entity-based approach is recommended for highly represented queries (whose entities have high annotation scores). Furthermore, this session shows how our approach achieves the maximum nDCG@5 score. The following experiment shows how our approach offers added value. In the experiment, we use the REL entity linking method for query annotation (see Appendix A.2). Table 10 contains highly represented queries whose entities have an average annotation score greater than or equal to 0.75. The table also contains the query text which is the original text (query title). For each query, in REL annotation column, it shows the detected mentions with the corresponding entities and their annotation scores. In addition, to analyze the performance achieved by our purely entity-based approach, we present the results of the PESS4 IR together with the results of the LongP (Longformer) and the Galago (Dirichlet) models. Moreover, Figure 4 shows the results of our approach in comparison with those of the two models. In the experiment, as illustrated in Table 11 and Figure 4, the nDCG@k scores are for the top five ranked documents, and the results of the LongP (Longformer) model are provided. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**qID** & **Query Text** & **REL Annotations (\(\boldsymbol{\text{``Mention''}}\rightarrow\text{Entity}\rightarrow\text{Score}\))** & **Annotation Avg Scores** \\ \hline
365 & El Nino &
\begin{tabular}{c} \textasci{``El Nino''} \rightarrow\text{El\_Nino \rightarrow 0.76}\end{tabular} & 0.76 \\
423 & [PERSON], [PERSON] &
\begin{tabular}{c} \textasci{``[PERSON]''} \rightarrow\textasci{[PERSON]} \rightarrow\textasci{0.99}\end{tabular} & 0.98 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: REL annotations whose average scores are higher than or equal to 0.75. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**nDCG@20**} \\ \cline{2-7}
**Method** & \multicolumn{3}{c}{**TREC DL 2019**} & \multicolumn{3}{c}{**TREC DL 2020**} \\ \cline{2-7}  & \begin{tabular}{c} **Avg \(\geq\) Min** \\ **(24 Queries)** \\ \end{tabular} & \begin{tabular}{c} **Avg \(\geq\) 0.95** \\ **(16 Queries)** \\ \end{tabular} & \begin{tabular}{c} **Avg = 1.0** \\ **(1 Query)** \\ \end{tabular} & \begin{tabular}{c} **Avg \(\geq\) Min** \\ **(24 Queries)** \\ \end{tabular} & \
\begin{tabular}{c} **Avg \(\geq\) 0.95** \\ **(10 Queries)** \\ \end{tabular} &
\begin{tabular}{c} **Avg = 1.0** \\ **(1 Query)** \\ \end{tabular} \\ \hline LongP (Longformer) & **0.7179** & **0.7464** & None & **0.6850** & **0.6605** & None \\ PESS4 IR & 0.3734 & 0.3970 & None & 0.2970 & 0.3055 & None \\ \hline \hline \end{tabular}
\end{table} 
Table 9: nDCG@20 scores for PESS4 IR and LongP (Longformer) on MSMARCO. With these results, our approach achieves the maximum nDCG@5 score of 1.000 for the queries with the highest representation (annotated by the REL entity linking method). This score is an added value for any document retrieval method which does not reach that score.