## 2 Materials and Methods
### Workflow  
For resource-efficient collaboration with SMEs during the development and evaluation of the search methods, we conceptualized a workflow revolving around the SMEs with multiple short 30-min relevance screening sessions online as visualized in Figure 3. In accordance with the SME representatives, we agreed on 30-min sessions for the relevance screening and feedback discussion regarding the process, to keep the required SME resources low. During the online sessions in the form of video calls, the SME representatives were able to provide feedback and ask for clarifications if needed.  
As the first data collection step, SMEs provide a product description as well as relevant documents specific to their medical device by answering the questionnaire.  
For the second step, MeSH terms from the product description are extracted using the Medical Term Indexer (MTI). Since not every product description might result in multiple terms due to varying length or content, an additional term expansion is performed. For the term expansion all direct MeSH terms as subnodes of the extracted MeSH terms were added to the MeSH term list. Afterward, in a first MeSH search, all publications with at least one matching MeSH term were retrieved from PubMed and ordered by descending  
Figure 3: Illustration of the workflow to guide the automated searches tailored to the SME use cases. The process comprises five distinct steps with respective output: (1) data about the medical device in question are collected via a short questionnaire; (2) MeSH terms extracted from the product description are expanded; (3) a first search for publications with a filter for the last 10 years is conducted; (4) a second search without filters is conducted; and (5) a third search for clinical trials without filters is conducted. Extracted MeSH terms and the top 10 results of each search are relevance screened by the SMEs.  
matches. SMEs then rated the relevance of the 100 most frequent MeSH terms out of all publications from the search results via a four-point weighing scheme analogous to a Likert scale (highly relevant: 2, relevant: 1, irrelevant: 0, exclude: \(-1\)). After initial exchanges with the SMEs, we decided to label an answer option \"exclude\" instead of \"highly irrelevant\", to allow SMEs to specify terms to exclude during the search.  
In the third to fifth steps, three separate searches are performed by each search method. The BERT-based search uses the product description without any preprocessing for each of the three performed searches. The top 10 results of each search method are presented to the SMEs for binary relevance screening. Decision fatigue during the screening is prevented by alternating results from the top 10 lists of the search methods. To avoid redundancies in the search results, previously identified relevant documents, such as seed documents or already screened documents, are removed and replaced with the next most relevant documents in case those documents are retrieved through a search.  
For the search in the third step, publications of the last ten years from 1 January 2012 to 31 August 2022 based on the SME requirements are searched for. The MeSH-based search uses the hundred SME-rated MeSH terms.  
For the search in the fourth step, the search is expanded to cover all documents in PubMed without a specified time frame in accordance with SMEs. Documents from the previous screening serve as seed documents for the MeSH-based search. Based on their relevance rating, documents are used as positive and negative seeds, e.g., an irrelevant document is used as a negative seed.  
For the search in the concluding fifth step, the screened publications are incorporated in the MeSH-based search and a last search for clinical trials is carried out.