Medical Dialogue Generation via Dual Flow Modeling
Year: 2023

Abstract:
Medical dialogue systems (MDS) aim to provide patients with medical services, such as diagnosis and prescription. Since most patients cannot precisely describe their symptoms, dialogue understanding is challenging for MDS. Previous studies mainly addressed this by extracting the mentioned medical entities as critical dialogue history information. In this work, we argue that it is also essential to capture the transitions of the medical entities and the doctor’s dialogue acts in each turn, as they help the understanding of how the dialogue flows and enhance the prediction of the entities and dialogue acts to be adopted in the following turn. Correspondingly, we propose a Dual Flow enhanced Medical (DFMED) dialogue generation framework. It extracts the medical entities and dialogue acts used in the dialogue history and models their transitions with an entity-centric graph flow and a sequential act flow, respectively. We employ two sequential models to encode them and devise an interweaving component to enhance their interactions. Experiments on two datasets demonstrate that our method exceeds baselines in both automatic and manual evaluations.

heading: 1 Introduction
text: Medical dialogue systems (MDS) have drawn considerable research attention with the increasing demand for telemedicine, especially after the outbreak of the COVID-19 pandemic (Zeng et al., 2020; Liu et al., 2020; Zhou et al., 2021; He et al., 2022; Xia et al., 2020; Yan et al., 2022), as they can provide much more people with in-time and affordable access to medical services such as health consultation, diagnosis, and prescription.
∗Equal Contributions.
For MDS, an efficient understanding of the dialogue history is challenging, as patients usually cannot describe their symptoms precisely and tend to convey lots of redundant information unnecessary for diagnosis (Liu et al., 2020; Mengel et al., 2002). To extract the critical information in the lengthy dialogue, previous research focused on identifying the important medical entities mentioned in the context, such as diseases, medicine, and symptoms (Liu et al., 2020; Lin et al., 2021; Liu et al., 2021).
In our work, we argue that capturing the transitions of the medical entities and the doctor’s dialogue acts in each turn (as depicted in Figure
6771
1) is also essential for the construction of MDS, which was largely overlooked by previous studies. In medical dialogues, the flows of medical entities and dialogue acts both follow particular transition patterns. For the medical entity flow, entities to appear in the following utterance are usually closely related to the ones mentioned recently. As in Figure 1, the entities mentioned in adjacent dialogue rounds are logically related, being neighboring nodes in the medical knowledge. For the dialogue act flow, though variations are allowed to some extent, it usually needs to follow the medical consultation framework suggested in Silverman et al. (2016). Modeling these two types of transitions would be helpful in dialogue understanding as they effectively capture how the dialogue history flows. Moreover, learning their transition patterns would also enhance the prediction of the dialogue acts and the medical entities to be adopted in the future turn.
Based on the above intuition, we propose a Dual Flow enhanced Medical (DFMED) dialogue generation framework. At each dialogue turn, it extracts the medical entities and the dialogue acts used in the dialogue history, and models their transitions with an entity-centric graph flow and a sequential act flow, respectively. Two sequential models are constructed to encode their transitions, with an interweaving component to enhance their interactions. The output representations are then used to predict the entities and the acts to be adopted in the following turn, which are employed to guide the response generation through gate control. Our main contributions are summarized as follows: • We propose a novel MDS framework, DFMED,
which models the transitions of medical entities and dialogue acts via step-by-step interweaving.
• We summarize the dialogue acts in the medical consultation scenario grounded on the medical documentation standards, SOAP notes (Cameron and Turtle-Song, 2002), including make a diagnosis, prescribe medications, etc.
• Experimental results show the superiority of DFMED over the previous frameworks and demonstrate the effectiveness of introducing the medical entity and dialogue act flows.

heading: 2 Related Work
text: Medical dialogue systems aim to provide medical services for patients. Early studies focus on automatic diagnosis in the form of a task-oriented
dialogue system, and the purpose is to collect hidden symptoms in minimal turns and make a diagnosis at the end (Liao et al., 2020; Lin et al., 2019; Chen et al., 2021; Liu et al., 2022). Wei et al. (2018) creates a dataset annotated with symptom phrases and constructs a reinforcement learning based MDS. Xu et al. (2019) improves the topic transition in MDS by introducing a medical knowledge graph. With the release of large-scale medical dialogue datasets (e.g., MedDialog (Zeng et al., 2020), MedDG (Liu et al., 2020), and KaMed (Li et al., 2021)), dialogue response generation attracts increasing attention. Liu et al. (2020) frames medical dialogue generation as entity prediction and entity-aware response generation. Furthermore, Liu et al. (2021) unifies the dialogue context understanding and entity reasoning through a heterogeneous graph. Li et al. (2021) considers medical entities in patient and doctor utterances as states and actions and presents semi-supervised variation reasoning with a patient state tracker and a physician action network. The proposed model, VRBot, achieves comparable performance without entity supervision. Lin et al. (2021) analyses a low-resource challenge in medical dialogue generation and develops an entity-involved meta-learning framework to enhance diagnostic experience sharing between different diseases.
Although many studies have tried to improve medical dialogue generation by incorporating predicted medical entities, they simply treat entities in different turns as nodes in one entity graph with no entity transition modeling. Besides, few works focus on sequential entity-guided dialogue act prediction and sequential act-involved entity selection. Our framework exploits the transition and interaction of entity and act flows to strengthen dialogue understanding and guide response generation.

heading: 3 Preliminary
text: Problem Formulation. We define a medical dialogue as U={(Pk, Dk)}Tk=1, where P and D represent utterances from patients and doctors. Each utterance contains several medical entities E={ei}, and each doctor utterance is annotated with multiple dialogue acts A={aj}. Given the dialogue history Ut={P1, D1, ..., Pt}, the system is supposed to generate the t-th doctor utterance Dt.
Dialogue Acts. We summarize several common dialogue acts implied in medical dialogues. One type is medical-related dialogue acts. We design
acts that represent a function in a medical documentation standard, the SOAP note (Cameron and Turtle-Song, 2002), and occur throughout the dialogue. For example, State a required medical test and Prescribe medications in the “Plan” function of the SOAP note are included in our designed acts. The other type is general open-domain dialogue acts. We choose some acts introduced by Zhao et al. (2022) and further refine them, such as merging acts that behave as social obligation management into Chitchat. Eventually, we obtain 7 dialogue acts for flow modeling, and later experiments demonstrate the effectiveness of guidance for response generation. Detailed description of each dialogue act can be found in Appendix A.2.

heading: 4 Method
text: The proposed method learns two flows (i.e., a medical entity flow and a dialogue act flow) to model the propagation of medical entities and medicalrelated interactions and guide response generation with the corresponding hints. As shown in Figure 2, the overall framework contains two modules. The Dual Flow Modeling module learns the entity and act transitions and infers probable entities and acts. The Response Generation module outputs a response under the guidance of the selected entities and predicted acts.

heading: 4.1 Dual Flow Modeling
text: Figure 3 left displays the architecture of the Dual Flow Modeling module. To extract flow transitions, we encode the medical entities and the dialogue acts in a sequential way. Besides, at each turn, entity and act embeddings are mutually integrated through an interweaving component, named Interweaver. The context states Sc produced by a context encoder also play a role in the integration. After encoding the whole dialogue history, we obtain the entity state Set and act state S a t . The entity states are adopted to select the relevant entities. The act states are sent to a multi-label act predictor to estimate the next acts.
We apply BERT (Devlin et al., 2019) as the context encoder to encode the dialogue history. All utterances are concatenated with a unique role token (i.e., “[P]” or “[D]”) to separate the patient and doctor utterances. We compute the context states with different history ranges Uk={P1, D1, ..., Pk}(k ∈ [1, t]) for supporting the interweaving at each turn. The mean embedding of tokens in history Uk is ap-
plied as the k-th turn context state Sck ∈ Rd, where d is the dimension of the state.

heading: 4.1.1 Entity Flow
text: To learn the sequential transition of medical entities, we create the Entity Flow that encodes an entity graph transition and selects the most relevant entities. Specifically, the flow consists of an entity graph for each turn. Medical entities of each dialogue turn and entities from an external knowledge graph with a one-hop connection to the entities in dialogue are included in the graph of the k-th turn, denoted as Gk. Since we assume entities hop along the links on the graph, the ones in the sub-graph partially provide future transition hints.
We define graphs that include entities until the t-th turn as G≤t. For all entities in graph G≤t, we use the same encoder as above to get entity embeddings. Instead of randomly initializing an embedding, BERT-based encoding keeps token-level semantics. Then, the average token embedding of each entity is employed as the raw embedding, denoted as he0 ∈ Rd. Finally, Graph Attention Network (GAT) (Velickovic et al., 2018) is implemented to merge neighboring information for each entity:
αkij = exp
( σ1 ( aT[W khe0i ∥W khe0j ] ))
∑ µ∈Ni exp (σ1 (a T[W khe0i ∥W khe0µ ])) ,
(1)
hei =  σ2  ∑
j∈Ni αkijW khe0j
    K
k=1
, (2)
where hei ∈ Rd is the updated entity embedding, K is the number of heads, a ∈ R2d is a trainable weight, W k ∈ Rdh×d is the linear transformation matrix for the k-th head, σ1 and σ2 are the activation function, and Ni represents neighboring entities that connect to entity i with one hop. The updated embedding is used to compute each turn’s overall graph embedding via mean pooling:
h̄ek = 1 |Gk| ∑
i∈Gk hei , k ∈ [1, t], (3)
where h̄ek ∈ Rd is the entity graph embedding, t represents the turn of the target response. Following Tu et al. (2022), we employ a GRU to model the entity graph transition throughout the dialogue. The GRU takes all the previous graph embeddings {h̄e1, h̄e2, ..., h̄et} as input and produces the entity state as follows:
Set = GRUEntity(S e t−1, h̄ e t ), (4)
where Set ∈ Rd is the final hidden state of the GRU. We denote Set as the entity state that implies clues for the entity transition in the previous context.
Then, we apply the entity state Set to calculate relevant scores for candidate entities. These entities are from the sub-graph of entities in a historical dialogue context. The score is defined as follows:
Score = ⟨Set , hei ⟩ , i ∈ G1≤t, (5) where ⟨, ⟩ represents a similarity function, hei is the entity embedding, G1≤t is the one-hop sub-graphs for entities until the t-th turn. We select top-k relevant entities Êt to guide response generation.

heading: 4.1.2 Act Flow
text: To learn the medical-related interactions of the doctor, we design the Act Flow that encodes act sequences and predicts the next acts. Specifically, the flow is composed of the act sequence of each turn. We first randomly initialize the trainable embedding of each dialogue act, denoted as ha ∈ Rd. The act sequence of the k-th turn can be defined as Ak = {ha1, ha2, ..., hanka}, where n k a represents the number of dialogue acts for each turn. Then, we compute the act sequence embedding h̄ak ∈ Rd, k ∈ [1, t] through mean pooling.
Similar to the entity flow, we employ a GRU to model dialogue act transition. With all sequence embeddings as input, the final hidden state of the GRU is calculated and denoted as the act state:
Sat = GRUAct(S a t−1, h̄ a t ), (6)
where Sat ∈ Rd is the act state of the t-th turn. Then, the multi-act probability of the t-th turn is computed based on the act state Sat with a sigmoid and linear transformation layer:
Prob = sigmoid(WaSat + ba), (7)
where Wa ∈ Rna×d and ba ∈ Rna are model parameters, and na denotes the number of candidate dialogue acts. The predicted dialogue acts Ât are obtained through an appropriate threshold.

heading: 4.1.3 Flow Interweaving
text: To achieve the integration of these two flows, we present an Interweaver to extend the entity graph embedding and act sequence embedding, as shown in Figure 3 right. This component incorporates the dialogue context into entity/act states and integrates entity/act sequential information from each other.
For the entity flow, we first fuse the historical context into the entity graph embedding. The context-aware graph embedding of each turn is computed via cross-attention:
αki = softmax( Qck TKei√ d ), (8) h̄e c k = ∑
i∈Gk αkiV
e i , k ∈ [1, t], (9)
where h̄e c k ∈ Rd is the context-aware graph embedding at the k-th turn, Kei and V e i are linear projected vectors based on the entity embedding hei , and Q c k is based on the context state S c k. We denote this operation on Sck and [h e i ]i∈Gk as CA(Sck, [h e i ]i∈Gk). Then, the act transition is fused into the graph embedding via CA as follows:
h̄e a k = CA(h̄ e k, [h a j ]j∈A≤k), k ∈ [1, t], (10)
where h̄e a k ∈ Rd is the act-aware graph embedding, and A≤k represents act sequences until the k-th turn. The final entity graph embedding is defined as the concatenation of three embeddings:
h̄e ′ = [h̄e; h̄e c ; h̄e a ], (11)
where h̄e ′ k ∈ R3d is the extended graph embedding that incorporates historical dialogue context and act transition pattern.
For the act flow, we apply the CA operation to compute the context-aware and entity-aware act sequence embedding following the same way. The context-aware sequence embedding h̄a c
k is based on Sck and [h a j ]j∈Ak , and the entity-aware sequence embedding h̄a e
k is based on h̄ a k and [h e i ]i∈G≤k .
These two embeddings incorporate the historical dialogue context and entity transition pattern. Then, the final sequence embedding h̄a
′ ∈ R3d is concatenated as follows:
h̄a ′ = [h̄a; h̄a c ; h̄a e ]. (12)
We send the extended embeddings h̄e ′ and h̄a ′
instead of the pure embeddings to the above GRUs, allowing two flow models to acquire context information and lead each other.

heading: 4.1.4 Training Objective
text: The training of the dual flow modeling module includes two tasks, medical entity ranking and multi-act classification. The first one follows a contrastive learning (Gao et al., 2021) way with a negative log likelihood loss:
Le = − T∑
t
∑
t+
log e
〈 Set ,h e t+ 〉
∑ i−∈G1≤t e
〈 Set ,h e i− 〉 , (13)
where het+ is the embedding of a target entity mentioned in the t-th response, ⟨, ⟩ is the dot product operation that calculates relevant scores (see Eq. 5), and G1≤t denotes the one-hop sub-graphs for
the dialogue history until the t-th turn. We randomly select several entities instead of the whole sub-graph as negative entities. The second one is defined as a multi-label classification with a binary cross-entropy loss:
La = T∑
t
na∑
j
BCE(ŷat,j , y a t,j), (14)
where ŷat,j is the probability of the dialogue act j for the t-th response (see Eq. 7), and yat,j is the groundtruth act label. The overall training objective of the dual flow modeling module can be calculated as:
LF = λeLe + λaLa, (15)
where λe and λa are weights for each task.

heading: 4.2 Flow-guided Response Generation
text: After training the dual flow modeling module, we exploit the top-k relevant entities Êt (see Sec. 4.1.1) and predicted dialogue acts Ât (see Sec. 4.1.2) to guide response generation. We first encode the entity/act and dialogue history separately, allowing a relatively complete dialogue context. Then, these two types of information is merged into the decoder via a fusion component.
Act-Entity Fusion As shown in Figure 4, the dialogue acts and entities are concatenated into one token sequence. We assign a unique token to each dialogue act. Given the entity/act sequence and dialogue history sequence as input, the corresponding encoder produces final hidden states, Hea and Hc. We fuse two types of information through a gate mechanism in the decoder:
hl,eaw = CA(h l w, H ea), (16)
hl,cw = CA(h l w, H c), (17)
glw = sigmoid(W lhl,cw ), (18) hl ′ w = FFN(g l wh l,c w + (1− glw)hl,eaw ), (19)
where hl ′ w is the final hidden state at the w-th token after fusion, hlw denotes the output of the l-th selfattention layer, FFN is the feed forward network, and W l is the trainable parameter. We adopt the final hidden state hL ′ w of the decoder to compute the probability distribution of the next token:
p(Dt,w+1) = softmax(WdhL ′ w + bd), (20)
where Wd and bd are linear mapping parameters.
Training and Inference When training the response generation module, we use the selected topk entities from the dual flow modeling module and ground truth dialogue acts as encoder input. The training objective is defined as follows:
LG = − T∑
t
w∑
w=1
log p(Dt,w+1), (21)
where p(Dt,w+1) denotes the probability of the next token in the t-th response. Then in inference, we apply the top-k entities and predicted dialogue acts to generate the next response.

heading: 5 Experiments

heading: 5.1 Dataset
text: Our experiments are conducted on two medical dialogue datasets, MedDG (Liu et al., 2020) and KaMed (Li et al., 2021). The MedDG dataset contains 17K dialogues, focusing on 12 diseases in the gastroenterology department. The medical entities mentioned in the dialogues are annotated in MedDG. We split the dataset into 14862/1999/999 dialogues as the training/validation/test sets, following its original division. The KaMed dataset contains over 63K dialogues, covering diverse diseases in about 100 hospital departments. We filter out some dialogues with privacy concerns in KaMed (see Appendix A.3) and obtain 29,159/1,532/1,539 dialogues as the training/validation/test sets. The final data accounts for around 51% of the total.

heading: 5.2 Baseline models
text: We compare DFMED with five baseline models. (1) Seq2Seq (Sutskever et al., 2014) is an RNNbased sequence to sequence model with an attention mechanism. (2) HRED (Serban et al., 2016) is a hierarchical RNN that models a dialogue as a token sequence and an utterance sequence. (3) GPT-2 (Radford et al., 2019) is a transformer decoder-based language model. (4) BART
(Lewis et al., 2020) is a transformer-based encoderdecoder model. (5) VRBot (Li et al., 2021) is a medical dialogue generation model with patient entity tracking and doctor entity learning.
For the experiments on the MedDG dataset, we extend Seq2Seq, HRED, GPT-2, and BART with entity hints, following Liu et al. (2020). Specifically, the extracted medical entities are appended at the end of the input sequence. In the following, these models with entity modeling are displayed with the -Entity suffix (e.g., Seq2seq-Entity).

heading: 5.3 Evaluation Metrics
text: Automatic Evaluation. We evaluate modules in the proposed framework separately. For the dual flow modeling module, the top-20 recall rate (R@20) of target entities and the weighted F1 score (Weighted-F1) of different dialogue acts considering the act imbalance are adopted. For the response generation module, BLEU (Papineni et al., 2002) and ROUGE (Lin, 2004) scores in different n-grams (i.e., B-1, B-2, B-4, R-1, and R-2) are adopted to assess the response quality. Besides, we also measure the precision, recall, and F1 of entities (i.e., E-P, E-R, and E-F1) in responses to demonstrate the reliability following Liu et al. (2020).
Human Evaluation. We chose 100 cases at random and invited three annotators to evaluate them manually. Results of the proposed framework are compared with different baseline models. We use three metrics to evaluate all of the generated responses based on past research (Liu et al., 2020; Li et al., 2021): sentence fluency (FLU), knowledge accuracy (KC), and entire quality (EQ). On a 5- point Likert scale, from 1 (worst) to 5 (best), three annotators are asked to score these responses.

heading: 5.4 Implementation Details
text: For all baselines, we implement the open-source algorithm following Liu et al. (2020) and Li et al. (2021). We use the MedBERT1 pretrained in the medical domain as the backbone of the dual flow modeling module. To extract the medical entities in the dialogue history, we refer to the medical knowledge graph CMeKG2 and extract the text spans that match the string of nodes in the graph. Entities with a one-hop connection to the historical entities are target ones for entity flow modeling. Besides, following Yan et al. (2022), to identify
1https://github.com/trueto/medbert 2http://cmekg.pcl.ac.cn/
the dialogue acts, we apply an open-source pseudolabeling algorithm3 to automatically label each utterance. We train the dual flow modeling module with the AdamW (Loshchilov and Hutter, 2019) optimizer. The learning rate and batch size are 4e-5 and 12 with 1000 warmup steps. The best loss weights λe and λa are 1 and 0.05 through grid searching. After training ten epochs, checkpoints with the highest average F1 for act prediction and the highest recall rate for top-20 entity selection on the validation set are selected. We select the threshold for each dialogue act, which achieves the best F1 score for the corresponding act on all validation samples. Then, we use Chinese pre-trained BARTbase4 model with a six-layer encoder and a six-layer decoder for the response generation module. The entity/act encoder and context encoder share the same encoder. We adopt the AdamW optimizer and set the learning rate to 3e-5 with 2000 warmup steps. The model is trained for ten
3https://github.com/yanguojun123/Medical-Dialogue 4https://huggingface.co/fnlp/bart-base-chinese
epochs with a batch size of 4. We implement all experiments on a single RTX 3090 GPU.

heading: 6 Results and Analysis

heading: 6.1 Automatic Evaluation
text: The overall comparison of DFMED and other baseline models on the MedDG dataset is illustrated in Table 1, and the KaMed dataset is in Table 2. The observations from the comparison are as follows:
(1) Our proposed framework DFMED outperforms these baseline models in most metrics. Specifically, on the MedDG dataset, it is 8.42%, 6.15%, 4.11%, and 0.79% higher than the best baseline model, BART-Entity, on B-1, B-2, B-4, and R-1. On the KaMed dataset, DFMED outperforms BART by 6.58%, 4.54%, 3.12%, and 0.37% on B1, B-2, B-4, and R-1, indicating better similarity to the content of ground truth responses. Besides, on the MedDG dataset, DFMED exceeds BART-Entity by 3.00% on E-F1, meaning that it can generate responses with more accurate entity mentions. The above increases are because DFMED learns the transition of a medical entity flow and a dialogue act flow and predicts the entities and acts to guide response generation. Moreover, interweaving two flows enhances the prediction of entities and acts.
(2) DFMED effectively fuses the guidance from
medical entities and dialogue acts. Compared to GPT-2 and BART, which have a performance drop on BLEU with the incorporation of medical entities (i.e., -Entity), DFMED shows increases in all these metrics. The main reason is that the force incorporation of entities may reduce response fluency. In DFMED, the dialogue act guidance influences attention to entities in the encoder and implicitly prevents the enforcement. The gate mechanism in the decoder controls the proportion of information from entities and acts. Besides, dialogue acts also provide essential content for responses without entity hints. This improvement is significant as about half of the responses in datasets do not match an entity in CMeKG via automatic string matching.

heading: 6.2 Human Evaluation
text: We select methods with high accuracy on the MedDG dataset to conduct a human evaluation, as shown in Table 3. Our framework displays an overall better response quality. Especially on the EQ, DFMED performs significantly better than baselines due to the incorporation of dialogue acts. The Cohen’s Kappa coefficient equals 0.53 and indicates a moderate agreement (Cohen, 1968).

heading: 6.3 Analysis of Dual Flow Modeling
text: To further explore the effectiveness of our method, we investigate the following variants of DFMED: (1) w/o Flow Modeling, where we use a context state produced by mean pooling of all hidden states of dialogue context tokens to rank entities and predict dialogue acts. (2) w/o Interweaving, which removes the interweaving between entities and acts. (3) w/o Entity attends to Act, which removes the interweaving from act sequences. (4) w/o Act attends to Entity, which removes the interweaving from sequential entity graphs.
Table 4 shows the ablation study results. We observe drops in all metrics with the ablation variants, indicating the effectiveness of our proposed module. Specifically, the result of w/o Flow Mod-
eling significantly drops on the entity recall rate and slightly drops on the overall act prediction F1 compared to the full model. It demonstrates that flow modeling can be necessary for learning the transition of entities and acts in dialogues. Besides, comparison among variants of the interweaver illustrates that incorporating sequential entity graphs and act sequences assists in the transition. Notably, the act sequence is more conducive to the transition of the entity and act flows. It may be because the entity graph introduce noise entities from patient utterances. These entities are more variable and can be deviated from the main content.

heading: 6.4 Case Study
text: A case generated by the above methods is illustrated in Table 5. Compared to baseline models, DFMED can produce responses more consistent
with the dialogue context reflected by medical entities and dialogue acts. We can observe that the dual flow modeling module correctly predicts all acts and selects several medical entities, although two medications (e.g., “mosapride”) are different from ground truth mentions. Then, the beneficial guidance from medical entities and acts is employed to generate the next response. The response generation module fuses the entities and acts and produces the response containing these two hints.

heading: 7 Conclusion
text: In this paper, we propose a dual flow enhanced medical dialogue generation framework, DFMED, that models a medical entity flow and a dialogue act flow to improve the relevant entity selection and dialogue act prediction. Besides, we design an interweaver to strengthen interactions of two flows. The selected entities and predicted acts are applied to guide response generation. Experiments validate the effectiveness of our DFMED on two datasets.
Limitations
Although our proposed framework beats several baseline methods for medical dialogue generation, there is still room for progress. We exploit an entity flow and a dialogue act flow to improve dialogue understanding and guide response generation. However, our summarized dialogue acts are limited in the types and granularity of functions they denote. We can manually annotate more medical-related dialogue acts in our future research following the SOAP notes. Besides, more medical knowledge with different formats, such as medical articles and medical examination reports, can be incorporated. Finally, it is crucial to recognize the potential risks associated with system utilization and the possibility of patient privacy leakage. A collaborative approach involving both dialogue systems and medical professionals should be considered. This will ensure that responses are endorsed by physicians and stringently overseen by reliable authorities.
Ethics Statement
Our proposed system aims to provide medical services, such as diagnosis and prescription, for patients who suffer from certain diseases. All datasets have been anonymized when released in dataset papers. However, since we train the model with limited and incomplete samples in two datasets,
the generated responses may involve misleading information about diagnosis, treatment, and precautions. We recommend that users adopt the system as an auxiliary tool and go to the hospital for help if necessary. Besides, when interacting with the system, there is a risk of sensitive information leak (e.g., gender as reported by users). It can be addressed by adopting anonymous technology in the future. Thus, we strongly advise users to consider the ethical implications of the generated responses carefully. Furthermore, the scientific artifacts that we used are freely available for research, including NLTK, ROUGE, Transformers, and other GitHub codes. And this paper’s use of these artifacts is consistent with their intended use.

heading: Acknowledgment
text: This work was supported by the Research Grants Council of Hong Kong (15207920, 15207821, 15207122) and National Natural Science Foundation of China (62076212).

heading: A Appendix
text: A.1 Details of Packages
We use the NLTK package in version 3.4.1 for calculating BLEU scores, the Pyrouge package in version 0.1.3 for calculating ROUGE scores, and the Transformers package in version 4.21.3.
A.2 Details of Dialogue Acts
In this section, we will describe the detail of our summarized dialogue acts. The seven dialogue acts can be divided into two types: (i) medical-related functions and (ii) general open-domain dialogue acts. The specific meaning of each act is interpreted as follows:
Medical-related functions. (1) Inquire. The doctor asks questions about the history of the present illness (e.g., the location and duration of one symptom), previous surgery and medical conditions, current medications, allergies, etc. It corresponds to the “Subjective” function of the SOAP note (Cameron and Turtle-Song, 2002). (2) Make a diagnosis. The doctor makes a differential diagnosis based on historical dialogue context. It corresponds to the “Assessment” function of the SOAP note. (3) Prescribe medications. The doctor provide medication names and instructions. (4) State a required medical test. The doctor explains which tests are required and why each one was chosen to resolve diagnostic ambiguities; besides, what the next step would be if the results are positive or negative. (5) Provide daily precautions. The doctor explains the things that need to be paid attention to every day. Acts (3), (4), and (5) correspond to the “Plan” function of the SOAP note.
General open-domain dialogue acts. (6) Inform. The doctor tells the patient some information that he assumes to be correct. (7) Chitchat. The doctor expresses welcome, goodbye, apology and thanks to the patient.
The proportion of each act in the MedDG and KaMed datasets is shown in the Table 6. Examples for each dialogue act are listed as follows:
1. Inquire: “Hello, do you usually have diarrhea?”, “Have you taken any medicine before?”;
2. Make a diagnosis: “You may have gastroenteritis”, “You may have allergic rhinitis, which is easy to get sick this season.”;
3. Prescribe medications: “Please take Amoxicillin capsule 1.0g 2 times a day and Clarithromycin tablet 0.5g 2 times a day. Both are taken after meals.”;
4. State a required medical test: “If you often feel sick to your stomach, you can do a gastroscopy.”, “If your condition does not improve, I suggest you do a gastroscopy and Helicobacter pylori detection.”;
5. Provide daily precautions: “Please drink plenty of water, eat more fruits and vegetables. And try to have a morning poop.”;
6. Inform: “Migraine is a primary headache whose etiology and pathogenesis are not fully understood.”, “It can be effective in five days, and individual differences are relatively large.”;
7. Chitchat: “You’re welcome, and I wish you a speedy recovery!”, “Thank you so much”, “Hello!”;
A.3 Extra Process of the KaMed dataset
We filter out dialogues that include texts such as “The image is not available for privacy concerns”
and “The voice is not available for privacy concerns” since these dialogues are incomplete and hard to understand. We have obtained approximately 51% samples for our experiments. The reason is as follows.
The KaMed dataset is collected from an online medical consultation platform. The raw dialogues contain multi-modal information such as pictures (e.g., medical examination reports and photos of body parts) and voice messages. These messages are crucial for dialogue development since the doctor will respond to the picture or voice. For example, they will discuss the result of an examination report, or patients will directly use voice messages instead of texts to express their condition. However, this information is replaced by meaningless text such as "The image is not available" for privacy concerns when collecting the dataset. Thus, the dialogue context is incomplete and difficult to understand. We argue that filtering dialogues that contain these texts can help us build a more robust model for dialogue understanding.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: 8 Limitations.

heading: 3 A2. Did you discuss any potential risks of your work?
text: 9 Ethics Statement.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: 0 Abstract; 1 Introduction.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: 5.1 Dataset; 5.2 Baseline models; 5.3 Evaluation Metrics.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: 5.1 Dataset; 5.2 Baseline models; 5.3 Evaluation Metrics.

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: 9 Ethics Statement.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 9 Ethics Statement.

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? 9 Ethics Statement.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? 5.1 Dataset.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 5.1 Dataset; Appendix A.2 Details of Dialogue Acts.
C 3 Did you run computational experiments?

heading: 5 Experiments.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 5.4 Implementation Details.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 5.4 Implementation Details.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 6 Results and Analysis.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix A.1 Details of Package.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We can manually annotate more medical-related dialogue acts in our future research following the SOAP notes. Besides, more medical knowledge with different formats, such as medical articles and medical examination reports, can be incorporated. Finally, it is crucial to recognize the potential risks associated with system utilization and the possibility of patient privacy leakage. A collaborative approach involving both dialogue systems and medical professionals should be considered. This will ensure that responses are endorsed by physicians and stringently overseen by reliable authorities