Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, the authors proposed a causal model, DrugRec, for drug recommendation. The model utilizes different historic and clinical information to target three specific problems: (1) elimination of drug recommendation bias (2) better utilisation of historic patient data (3) use of the clinical information to reduce harmful drug-drug interaction (DDI). There are two variants of DrugRec which are compared against multiple baseline models and they outperformed all the existing baselines over the publicly available MIMIC dataset. 
strengths_and_weaknesses: This is a strong paper but have some (fundamental) weaknesses that I would like the authors to address during the rebuttal period.

Strengths:
1. The paper is neatly written with very few grammatical errors and is easy to follow. 
2. DrugRec is compared against all the appropriate baselines for drug recommendation. 
3. I appreciate that the authors have tried to justify their decisions at all steps. (I have a few questions regarding some of them below)
4. The ablation study is important and shows that not only the historic patient data is important but also the way in which they are modelled makes a significant difference in the performance. 

Weakness/Questions: 
1. The Related Works section can be improved a lot, the authors should discuss how their model is different from the existing work cited in the section. 1-2 liner discussions in each section of Related Works would also be enough. This helps the reader in identifying the clear advantage of the proposed model.
2. Reference Figure 1: Why is M not connected to S? M^j_t is the multi-hot encoded medication set of the jth patient right? Because D and P would not cover what medications or drugs were previously given to the patients in earlier visits. 
3. Ref DrugRec-k: Why did author choose k visits, rather than a specific time period before the visit? This provides different context windows for different patients. One patient can have a lookback period of 5 years whereas other can have of 5 months.
4. (i) Reference L249-251: If you did not want to use ICD coding, a better way would have been to use an entity normalizer to identify which 
        ICD codes belong to the same set of families. ICD codes also have a hierarchy provided by UMLS or other clinical meta-thesaurus.
    (ii) Secondly, how was negation handled here? Such as in scenarios like "The pt. did not have XYZ ......" 
5. The limitations of the proposed model are not discussed and there is no specific error analysis to discuss the short-comings of the model or future work.
questions: Copied from weaknesses: 
2. Reference Figure 1: Why is M not connected to S? M^j_t is the multi-hot encoded medication set of the jth patient right? Because D and P would not cover what medications or drugs were previously given to the patients in earlier visits. 

3. Ref DrugRec-k: Why did the authors choose k visits, rather than a specific time period before the visit? This provides different context windows for different patients. One patient can have a lookback period of 5 years whereas other can have of 5 months.

4. (i) Reference L249-251: If you did not want to use ICD coding, a better way would have been to use an entity normalizer to identify which 
        ICD codes belong to the same set of families. ICD codes also have a hierarchy provided by UMLS or other clinical meta-thesaurus.
    (ii) Secondly, how was negation handled here? Such as in scenarios like "The pt. did not have XYZ ......" 

5. The limitations of the proposed model are not discussed and there is no specific error analysis to discuss the short-comings of the model or future work. Could authors elaborate a bit as to what are the potential limitations of their proposed model?
limitations: The limitations are not discussed by the authors.
1. The Related work section can be improved by clearly identifying the difference between the proposed model and the class of related works. Each para that identifies a particular direction of work can have a line or two to identify the difference.
2. Limitation and error analysis is not discussed in the paper.
3. It would be great if authors could add the algo of their proposed model in the paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper address the task of drug recommendation from EHR data. The authors propose a causal inference graphical model with front-door adjustment to address the issue of hidden confounders, which leads to bias. The proposed model handles data with multiple visits (DrugRec-a: averaging visits, DrugRec-k: taking the last k visits) by extending the causal graph to link consecutive visits. The authors propose a method to tackle drug-drug interaction (DDI) in the recommender system by solving a propositional satisfiability (SAT) problem. Experiments on MIMIC datasets show the effectiveness of the model based on various metrics.
strengths_and_weaknesses: Strengths:
- The paper addresses both hidden confounders and DDI in a single model
- The paper includes a causal modeling of the sequential aspect of EHR  
Weakness:
- The novelty is rather incremental and a combination of known methods
-  A simulation example could be added to better assess the hidden confounders

questions: What is the distribution of number of visits in the datasets? 
The role of k could be better highlighted by stratifying the datasets based on different number of visits.

Typos:
L299: addtion -> addition
L319: methrics -> metrics


limitations: The experiments are based on two MIMIC datasets which are likely overlapping in data
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This is an important area of research for clinical/medical innovation. The authors propose DrugRec, a causal inference-based drug recommendation model. The causal graphical model can identify and deconfound the recommendation bias with front-door adjustment. In addition, the authors model the drug-drug interactions (DDIs) as the propositional satisfiability (SAT) problem and solving the SAT problem can help better coordinate the recommendation. Experiments are performed using real-world clinical data. 

strengths_and_weaknesses: Strengths: Use of real-world data, even though MIMIC has certain characteristics and does not represent typical EHR data for general patients. 
Weaknesses: A number of assumptions are not discussed or addressed. See sections "Questions"
questions: some important assumptions are not discussed and are worth mentioning and further expanding upon:
1.	A key point that is not discussed in the assumptions is the time intervals between the visits, especially when historical data are used. For instance, having 5 visits over the past 5 years is very different from having 5 visits during the past month. 
2.	Frequent refills (indication of safety and chronic conditions), and dosage adjustment are important elements in the recommendations that are also not discussed.
3.	Medication corrections – a modification that is only a few days apart can easily be corrected and provide better quality data in the pre-processing.
4.	Medication recommendations for in-patients vs. out-patients are very different; which one is being modeled here? The modeling assumption and data pre-processing need a different level of attention depending on the target population. 
5.	Regarding the use of ICD codes. Often the ICD codes are used with a rule of 2, especially for complex conditions that require further investigation to confirm the correct diagnosis. 
6.	Natural language processing: it is not clear how symptoms were extracted from clinical notes. Also, there are different types of notes, and it is not clear what type of notes were used at what time during the encounter. 
limitations: A number of assumptions and clarifications are lacking, limiting the scope and utility of the study.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a causal inference-based drug recommendation model by considering three key factors in EHRs. The proposed two modeling schemes (DrugRec-a and DrugRec-k) handle the multi-visit patients to better model a patient’s historical health condition. And controllable DDI with 2-SAT is proposed to coordinate the recommendation and improve the recommendation safety. It is well-organized. The extensive experiments demonstrate the proposed model's effectiveness.
strengths_and_weaknesses: Strengths:
1. The paper proposes a causal inference-based model to address the drug recommendation task and provides causal analysis.
2. DrugRec-a and DrugRec-k handle the multi-visit patients to better model a patient’s historical health condition.
3. Extensive experiments on real-world datasets MIMIC-III and MIMIC-IV evaluate the recommendation quality and the DDI rate.

Weaknesses:
1. It is claimed that the symptoms are extracted from the clinical notes. As the clinical notes are generated at the discharge stage, it means medications have been given to patients at the current visit. So it is confusing to use the symptoms as inputs at the current visit.
2. It would be better to provide more information about how previous research addressed the limitations without the three key factors.
3. It seems MIMIC-IV does not have clinical notes. How do the experiments conduct on MIMIC-IV-related symptoms?
questions: 1. How do the experiments conduct on MIMIC-IV-related symptoms?
2. How do previous methods consider three essential factors in a drug recommendation model?
limitations: 1. It is claimed that the symptoms are extracted from the clinical notes. As the clinical notes are generated at the discharge stage, it means medications have been given to patients at the current visit. So it is confusing to use the symptoms as inputs at the current visit.
2. It would be better to provide more information about how previous research addressed the limitations without the three key factors.
3. It seems MIMIC-IV does not have clinical notes. How do the experiments conduct on MIMIC-IV-related symptoms?
4. The proposed model is complicated, it would be better to give a complexity analysis.
review_title: null

Meta-review: metareview: This paper proposes a drug recommendation model, called DrugRec, based on causal inference. The two proposed modeling schemes are capable of handling multiple patient visits to better model the patient's past health status. The authors performed large-scale experiments and showed the effectiveness of the proposed model. The model formulates bias as a confounder through causal effect. The authors clearly answered the questions from the reviewers, and so the revised paper would be an achievement worthy of being accepted as a new NeurIPS paper.  
confidence: Certain
Decision: Accept