Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper moves beyond the traditional way of improving the prediction accuracy of the patient’s pathology. It enables the system to optimize with the differential diagnosis with the exploration-confirmation approach while prioritizing severe pathologies. The proposed methodology performed well on benchmark datasets. 
strengths_and_weaknesses: ### Strength
-------------------------
1. Reproducibility: I liked how the authors provided and documented the code well. Some code snippets are $a \ little$ hard to follow, but I am confident that the authors would clean them at the open source time. 
2. Figures and tables: Starting from the motivational diagram to the supl. material - the authors provided good figures that helped me understand the paper's central theme and the texts. 
3. Formulation: the equations are easy to follow and supported with adequate description - making them easy to read and criticize.


### Weaknesses
-------------------------
1. Presentation style of the paper is relatively not very impressive. A few suggestions could be; (1) merging the Intro and the Medical history taking, and (2) the preliminaries could be merged with the method.
2. A few mathematical formulations are used without proper justification; pls see Sec. Questions.
questions: 1. Eqn 1, it is unclear why another divergence or distance was not used. Might not we consider KL or the earth-mover's in place of JSD. A small discussion behind the justification of taking JSD would help the paper to understand more. 
2. Eqn 2, the weight is learned, but what if the dataset has some bias, such as gender, race, belief, etc.? Also, if we say the end decision would be taken by humans, aren't we encouraging human bias in the systems too!
3. Eqn 5. what are the values of $\gamma$ for this study? Do we have any ablation study on the same - or am I missing something here. 
4. It seems like the work is close to the work [1]

## Ref:
--------------
[1] Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices. Vincent S. Chen et al. https://vincentsc.com/papers/chen2019slice.pdf

limitations: Justified clearly
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: An interesting paper which focuses on imitating doctor's reasoning in automated clinical diagnosis tasks. As per the authors, the paper proposes a novel reinforcement learning (RL) agent to this end, and utilises it to model the evidence acquisition and automatic
diagnosis tasks. The proposed RL agent outperforms multiple baseline models based on experiments with benchmark datasets.
strengths_and_weaknesses: The paper has a real-world application and the approach can be quite impactful in safety critical applications like clinical diagnosis
However, there are few things which reduce the overall merit of the paper:
- The results and experiments provide an evaluation of the model based on metrics like F1 scores etc. (e.g. in Figure 3) which is useful. However, it is still not entirely clear for an average. reader in the AI community (who may not be specialised in the domain of DRL) that how exactly is the doctor's reason emulated by the model? It would have been useful if the authors provide some example(s) to describe the medical diagnosis reasoning imitation by the proposed DRL framework in comparison to a medical doctor. Can it be done with e.g. a visual depiction of the model's outputs/predictions? Figure 1 shows the proposed framework and the example of the synthetic interaction with the proposed model is useful - but it does not resonate clearly with the results. It is suggested that a better representation of the final model outputs/predictions later on in the paper's results section (besides the quantitative evaluation would be useful).

questions: - Did the authors perform a qualitative evaluation of the model's performance by discussing the predictions/outputs with domain-experts (real-world doctors) etc.? Besides the comparison on the benchmark datasets - how trustworthy/transparent/explainable is the proposed DRL framework for real-world utility - please clarify.

UPDATE: Thanks for addressing this reviewer's concerns. I have increased the score from 6 to 7.
limitations: Limitations have been properly addressed in the paper.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper focuses on the automatic diagnosis task and the authors reformulate the task of evidence acquisition and automated diagnosis by introducing doctors’ trust as a desideratum. The authors propose a new RL model (i.e., CASANDE) with DQN as the backbone and mimic human doctors’ reasoning by incorporating the exploration-confirmation and ruling out severe pathologies with new reward functions. The experiments on a synthetic dataset show that the proposed model outperforms the baselines.
strengths_and_weaknesses: Strengths:
1. The authors propose a new RL model, CASANDE, to interact with patients and produce differential diagnoses automatically. The task is significant in the clinical setting and could reduce human doctors’ workload.
2. The authors mimic human doctors’ reasoning, namely using differential diagnosis with the exploration-confirmation approach while prioritizing severe pathologies, by designing new reward functions, which is helpful for convincing doctors and real-world deployment.
3. The authors conduct experiments on a synthetic dataset and the results show that the proposed model outperforms state-of-the-art methods.

Weaknesses:
1. The proposed model is similar to an RL baseline AARLC. The main difference is that the proposed model has four additional reward components. Table 2 shows that after removing the four components, the performance is still much better than AARLC in Table 1. It is worth more discussion and explanation.
2. The paper’s main contribution is the four reward components in Table 2. However, the performance difference among the proposed model's various versions is insignificant (i.e., about 1% improvement on DDF1, 0.5% improvement on DSF1 and PER, and worse performance on IL). 
3. The expense of misdiagnosis is much more expensive than the cost of interaction with patients in clinical settings. It is unclear how the authors balance the two kinds of expenses. It would be better if the authors conducted experiments to show the automatic diagnosis performance over different lengths of interaction with patients.
4. The patients in the dataset are simulated (including the demographics, symptoms, and diagnosis). The experimental results on a totally synthetic dataset are not convincing enough.  
5. It would be better if the authors displayed the standard deviation in Table 2. 
6. It would be better if the authors gave some case studies to show how the proposed model is beneficial from designed reward functions.

questions: The authors will at least need to address questions 1, 2, 3, and 5 in the Weaknesses section.
limitations: Yes. The section "Limitations and potential negative social impact" is sufficient.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a novel RL architecture which is specially designed to help medical doctor’s diagnosis. The system utilizes differentiable diagnosis by incorporating a classification network, and considers three essential aspects of doctors’ reasoning, namely exploration, confirmation and severe pathologies. The system is shown to empirically work by experiments. 


strengths_and_weaknesses: Strength:
1.The paper considers a system which physically models the diagnosis process of the doctor.
2.Experimental results are promising
3.The presentation is clear
Weakness:
1.Theoretical contribution seems limited.

questions: 1.DDXPlus dataset along is used. Although evaluated thoroughly, is there any special reason that you do not consider other dataset?
2.What’s the impact when one adjust the architecture of the classifier network?
3.The system depends essentially on the data quality. However, some patients may be tricking, so that the data quality of the patient is not good. Do you consider this case since I think this often happens? Like using an adversarial agent to deal with this?

limitations: Not obvious.
review_title: null

Meta-review: metareview: **Technical Review and Decision**: This paper proposes four sets of rewards such that an RL maximizing the sum of those rewards (combined with the environment reward) to mimic doctors' behavior and increase the trust in the automatic differential diagnosis systems. The paper is well-written and the exchange between the reviewers and the authors have been constructive. There are multiple questions and the reviewers are convinced by the response. The authors should include the clarifications in the camera-ready version of the paper. While the methodological contributions are limited to a reward design, this paper qualifies as a good application paper. 

**Ethics Review**: The ethical reviewers have identified that the authors need to elaborate more on the doctor consultation process and make it more transparent. I strongly suggest including a discussion of ethical concerns, as discussed in the ethical reviews.
confidence: Less certain
Decision: Accept