Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: This paper talks about extracting paths from SNOMED CT and ranking them by scoring against electronic medical health records. The authors include a couple of examples where they generate an embedding h of the input text and rank the paths from SNOMED CT based on the aggregated cosine similarity between the path and the embedding h.
strengths_and_weaknesses: Strengths: 
1. The paper is well-written, and the setup is clearly explained.
2. The premise of the paper is interesting, and it has the potential to make a significant impact with a defined problem statement.

Weaknesses:
1. More rigorous and systematic experimentation is needed in order to conclude whether scoring the paths based on the aggregated cosine similarity works efficiently.
2. Only one type of concept (Findings) and relationship (hierarchial) is considered. It would be interesting to see the analysis of multiple concepts and relationships.
suggested_changes: 1. A stopping criteria can be explored in order to avoid unwanted further nodes in the path (some sort of thresholding in the cosine similarities), such that the model avoids going to the leaf nodes, that is, towards a more specific concept than required from the input text, as discussed in the second "Swollen Fingers" example.
2. More ranking algorithms, other than aggregating cosine similarity, can be explored and compared with some baselines.
3. and 4. Same as weaknesses

Minor editing: In equation (2), in c_1, 1 should be subscript
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: The paper studies an important area of healthcare delivery by automating the mapping process of concepts in healthcare records to a standardized healthcare taxonomy.
strengths_and_weaknesses: STRENGTHS
1. The paper provides a unique method for extracting and ranking complete pathways from the SNOMED CT graph against EHRs which is a significant contribution to the healthcare field.
2. The author details the limitation of the paper and proposes grounds for improvement
3. The methodology of using sentence encoding models to generate embeddings and compute cosine similarity scores is a common practice in NLP, which adds to the reliability and reproducibility of the results.
4. The paper also provides a custom dataset of FSNs and synonyms, along with paraphrased sentences, which enhances the quantity and reliability of the dataset.

WEAKNESSES
1. The author only focus on generating paths for "Findings" concepts for hierarchical relationships, which limits the applicability of the proposed approach. Within the healthcare space, much more will be needed to provide robust and accurate matching result
suggested_changes: 1. The author should consider expanding their methodology to other types of concepts and relationships in SNOMED CT, to improve the applicability of the proposed approach. SNOMED CT is robust, and even we the author do not explore all, certainly, just one really limits the usage of the approach, considering the medical sector.
review_title: null

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: The paper proposes to extract a SNOMET CT path for medical records using embedding similarity and path ranking.
strengths_and_weaknesses: Strengths:
1. The paper proposes to rank entire paths as opposed to finding relevant concepts from SNOMET.

Weaknesses: 
1. It is suggested that the results are positively received by experts in the field but no details regarding the same are provided.
2. No baseline is compared (term-similarity) and it is unclear why adding scores to rank a path is a reasonable approach.
suggested_changes: It would have been useful to provide a visualization that scores multiple paths for an input text using the proposed approach and potentially contrast against simple term-matching baselines.
review_title: null

Meta-review: metareview: The paper proposes a method to extract paths from SNOMED CT and rank them based on cosine similarity with embeddings generated from electronic medical health records. The paper is well-written and addresses an important problem in healthcare. The paper uses familiar concepts and conjoins them into a new application for medical diagnosis. The paper's custom dataset of FSNs and synonyms, along with paraphrased sentences, also enhances the quantity and reliability of the dataset.

Apart from the strengths, there are some weaknesses that should be addressed for the paper to be in an archivable state: There is a need for some additional experimentation with methodology to make a case for some choices made, and comparison against relevant baselines.
An exemplar query with the complete flow of the technique would have been very beneficial to the reader's understanding. Moreover, the claims about the impression from medical experts could be more concrete.

Overall, we recommend an invitation to archive after some revisions.
confidence: 3: The area chair is somewhat confident
Decision: Invite to archive