## 3 Results
### SME Use Cases  
Based on the MeSH terms extracted from the product description, an initial search in the local PubMed database took place, and the SMEs rated the hundred most frequently occurring MeSH terms of the retrieved publications. The numbers of rated MeSH terms of the SMEs are shown in Table 2. It can be observed for SME A that the majority of presented MeSH terms in the questionnaire should be excluded from the search, whereas for SME B the majority of terms were rated as highly relevant, subsequently leading to a narrower search for SME A and a wider search for SME B. In total SME A rated 39 and SME B rated 76 of the respective hundred MeSH terms as relevant. When comparing the MeSH terms derived from the product descriptions and rated by the SMEs as relevant (Highly Relevant, Relevant) and the terms rated as irrelevant (Irrelevant, Exclude) with the annotated MeSH terms of the provided seed publications, a disconnect becomes evident as visualized in Figure 4: Only a small part of the MeSH terms rated as relevant by either SMEs are also present in the MeSH terms from the initial \begin{table}
\begin{tabular}{c c c} \hline \hline
**Relevance** & **SME A** & **SME B** \\ \hline Highly Relevant & 23 & 63 \\ Relevant & 16 & 13 \\ Irrelevant & 0 & 20 \\ Exclude & 61 & 4 \\ \hline Total & 100 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Overview of the MeSH term relevance screening results of both SMEs for the hundred most frequently occurring MeSH terms from the MeSH term expansion step. seeds. Moreover, in the case of SME A (a), six MeSH terms rated as irrelevant are present in the provided initial seeds. For SME B (b) no overlap between MeSH terms of the initial seeds and terms rated as irrelevant can be observed. For both SMEs, less than a third of the MeSH terms present in the initial seeds overlap with the MeSH terms rated as relevant by the SMEs. Based on the hundred MeSH terms rated by the SMEs, an initial MeSH-based search was conducted alongside a BERT-based search based on the product description without any preprocessing. After discussing the results with the SMEs, a second search was conducted and expanded on a larger time frame as specified by the SME requirements. For a third search, clinical trials without a specified timeframe were searched for as well. After each search, the top 10 results were screened by SMEs for relevance. The average precision for each search for publications and clinical trials by the MeSH- and BERT-based methods for both SMEs are visualized in Figure 5.  
For both search methods, an increased precision from the first to the second search for publications can be observed. For the MeSH-based search, the precision increased considerably from 25% to 65% in the second search. For the BERT-based search, the precision was already relatively high in the first stage at 65% and increased to a smaller extent to 80%. During the search for relevant clinical trials, the MeSH-based method achieved only a precision of 10% while the BERT-based achieved a precision of 75%. Figure 4: Venn diagram indicating the overlap of MeSH terms from the initial seed publications and the hundred MeSH terms rated by the SMEs in the MeSH term expansion step. Figure 5: Average precision at top 10 search results for the two MeSH- and BERT-based methods, applied to publications (PubMed) and clinical trials (ClinicalTrials.gov) in the different searches for the two SME use cases. After the two screening stages, a last search, which incorporated all relevant publications, was conducted to produce the final result set for both SMEs. The goodness of the ranking was evaluated via the MAP score as visualized in Figure 6.  
It is noticeable that the ranking of the BERT-based method is better as indicated by a higher MAP score of 0.29 overall, i.e., at rank 50, than the ranking of the MeSH-based method with a score of 0.19. The MAP score of the MeSH-based method starts at 1 and then continues to decline.