prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What features are used?
Context 1: Introduction
Cancer is one of the leading causes of death in the world, with over 80,000 deaths registered in Canada in 2017 (Canadian Cancer Statistics 2017)  A computer-aided system for cancer diagnosis usually involves a pathologist rendering a descriptive report after examining the tissue glass slides obtained from the biopsy of a patient  A pathology report contains specific analysis of cells and tissues, and other histopathological indicators that are crucial for diagnosing malignancies  An average sized laboratory may produces a large quantity of pathology reports annually (e g , in excess of 50,000), but these reports are written in mostly unstructured text and with no direct link to the tissue sample  Furthermore, the report for each patient is a personalized document and offers very high variability in terminology due to lack of standards and may even include misspellings and missing punctuation, clinical diagnoses interspersed with complex explanations, different terminology to label the same malignancy, and information about multiple carcinoma appearances included in a single report BIBREF0  
In Canada, each Provincial and Territorial Cancer Registry (PTCR) is responsible for collecting the data about cancer diseases and reporting them to Statistics Canada (StatCan)  Every year, Canadian Cancer Registry (CCR) uses the information sources of StatCan to compile an annual report on cancer and tumor diseases  Many countries have their own cancer registry programs  These programs rely on the acquisition of diagnostic, treatment, and outcome information through manual processing and interpretation from various unstructured sources (e g , pathology reports, autopsy/laboratory reports, medical billing summaries)  The manual classification of cancer pathology reports is a challenging, time-consuming task and requires extensive training BIBREF0  
With the continued growth in the number of cancer patients, and the increase in treatment complexity, cancer registries face a significant challenge in manually reviewing the large quantity of reports BIBREF1 , BIBREF0   In this situation, Natural Language Processing (NLP) systems can offer a unique opportunity to automatically encode the unstructured reports into structured data  Since, the registries already have access to the large quantity of historically labeled and encoded reports, a supervised machine learning approach of feature extraction and classification is a compelling direction for making their workflow more effective and streamlined  If successful, such a solution would enable processing reports in much lesser time allowing trained personnel to focus on their research and analysis  However, developing an automated solution with high accuracy and consistency across wide variety of reports is a challenging problem 
For cancer registries, an important piece of information in a pathology report is the associated ICD-O code which describes the patient's histological diagnosis, as described by the World Health Organization's (WHO) International Classification of Diseases for Oncology BIBREF2   Prediction of the primary diagnosis from a pathology report provides a valuable starting point for exploration of machine learning techniques for automated cancer surveillance  A major application for this purpose would be “auto-reporting” based on analysis of whole slide images, the digitization of the biopsy glass slides  Structured, summarized and categorized reports can be associated with the image content when searching in large archives  Such as system would be able to drastically increase the efficiency of diagnostic processes for the majority of cases where in spite of obvious primary diagnosis, still time and effort is required from the pathologists to write a descriptive report 
The primary objective of our study is to analyze the efficacy of existing machine learning approaches for the automated classification of pathology reports into different diagnosis categories  We demonstrate that TF-IDF feature vectors combined with linear SVM or XGBoost classifier can be an effective method for classification of the reports, achieving up to 83% accuracy  We also show that TF-IDF features are capable of identifying important keywords within a pathology report  Furthermore, we have created a new dataset consisting of 1,949 pathology reports across 37 primary diagnoses  Taken together, our exploratory experiments with a newly introduced dataset on pathology reports opens many new opportunities for researchers to develop a scalable and automatic information extraction from unstructured pathology reports 
Background
NLP approaches for information extraction within the biomedical research areas range from rule-based systems BIBREF3 , to domain-specific systems using feature-based classification BIBREF1 , to the recent deep networks for end-to-end feature extraction and classification BIBREF0   NLP has had varied degree of success with free-text pathology reports BIBREF4   Various studies have acknowledge the success of NLP in interpreting pathology reports, especially for classification tasks or extracting a single attribute from a report BIBREF4 , BIBREF5  
The Cancer Text Information Extraction System (caTIES) BIBREF6 is a framework developed in a caBIG project focuses on information extraction from pathology reports  Specifically, caTIES extracts information from surgical pathology reports (SPR) with good precision as well as recall 
Another system known as Open Registry BIBREF7 is capable of filtering the reports with disease codes containing cancer  In BIBREF8 , an approach called Automated Retrieval Console (ARC) is proposed which uses machine learning models to predict the degree of association of a given pathology or radiology with the cancer  The performance ranges from an F-measure of 0 75 for lung cancer to 0 94 for colorectal cancer  However, ARC uses domain-specific rules which hiders with the generalization of the approach to variety of pathology reports 
This research work is inspired by themes emerging in many of the above studies  Specifically, we are evaluating the task of predicting the primary diagnosis from the pathology report  Unlike previous approaches, the system does not rely on custom rule-based knowledge, domain specific features, balanced dataset with fewer number of classes 
Materials and Methods
We assembled a dataset of 1,949 cleaned pathology reports  Each report is associated with one of the 37 different primary diagnoses based on IDC-O codes  The reports are collected from four different body parts or primary sites from multiple patients  The distribution of reports across different primary diagnoses and primary sites is reported in tab:report-distribution  The dataset was developed in three steps as follows 
Collecting pathology reports: The total of 11,112 pathology reports were downloaded from NCI's Genomic Data Commons (GDC) dataset in PDF format BIBREF9   Out of all PDF files, 1,949 reports were selected across multiple patients from four specific primary sites—thymus, testis, lung, and kidney  The selection was primarily made based on the quality of PDF files 
Cleaning reports: The next step was to extract the text content from these reports  Due to the significant time expense of manually re-typing all the pathology reports, we developed a new strategy to prepare our dataset  We applied an Optical Character Recognition (OCR) software to convert the PDF reports to text files  Then, we manually inspected all generated text files to fix any grammar/spelling issues and irrelevant characters as an artefact produced by the OCR system 
Splitting into training-testing data: We split the cleaned reports into 70% and 30% for training and testing, respectively  This split resulted in 1,364 training, and 585 testing reports 
Pre-Processing of Reports
We pre-processed the reports by setting their text content to lowercase and filtering out any non-alphanumeric characters  We used NLTK library to remove stopping words, e g , `the', `an', `was', `if' and so on BIBREF10   We then analyzed the reports to find common bigrams, such as “lung parenchyma”, “microscopic examination”, “lymph node” etc  We joined the biagrams with a hyphen, converting them into a single word  We further removed the words that occur less than 2% in each of the diagnostic category  As well, we removed the words that occur more than 90% across all the categories  We stored each pre-processed report in a separate text file 
TF-IDF features
TF-IDF stands for Term Frequency-Inverse Document Frequency, and it is a useful weighting scheme in information retrieval and text mining  TF-IDF signifies the importance of a term in a document within a corpus  It is important to note that a document here refers to a pathology report, a corpus refers to the collection of reports, and a term refers to a single word in a report  The TF-IDF weight for a term INLINEFORM0 in a document INLINEFORM1 is given by DISPLAYFORM0
We performed the following steps to transform a pathology report into a feature vector:
Create a set of vocabulary containing all unique words from all the pre-processed training reports 
Create a zero vector INLINEFORM0 of the same length as the vocabulary 
For each word INLINEFORM0 in a report INLINEFORM1 , set the corresponding index in INLINEFORM2 to INLINEFORM3  
The resultant INLINEFORM0 is a feature vector for the report INLINEFORM1 and it is a highly sparse vector 
Keyword extraction and topic modelling
The keyword extraction involves identifying important words within reports that summarizes its content  In contrast, the topic modelling allows grouping these keywords using an intelligent scheme, enabling users to further focus on certain aspects of a document  All the words in a pathology report are sorted according to their TF-IDF weights  The top INLINEFORM0 sorted words constitute the top INLINEFORM1 keywords for the report  The INLINEFORM2 is empirically set to 50 within this research  The extracted keywords are further grouped into different topics by using latent Dirichlet allocation (LDA) BIBREF11   The keywords in a report are highlighted using the color theme based on their topics 
Evaluation metrics
Each model is evaluated using two standard NLP metrics—micro and macro averaged F-scores, the harmonic mean of related metrics precision and recall  For each diagnostic category INLINEFORM0 from a set of 37 different classes INLINEFORM1 , the number of true positives INLINEFORM2 , false positives INLINEFORM3 , and false negatives INLINEFORM4 , the micro F-score is defined as DISPLAYFORM0
whereas macro F-score is given by DISPLAYFORM0
In summary, micro-averaged metrics have class representation roughly proportional to their test set representation (same as accuracy for classification problem with a single label per data point), whereas macro-averaged metrics are averaged by class without weighting by class prevalence BIBREF12  
Experimental setting
In this study, we performed two different series of experiments: i) evaluating the performance of TF-IDF features and various machine learning classifiers on the task of predicting primary diagnosis from the text content of a given report, and ii) using TF-IDF and LDA techniques to highlight the important keywords within a report  For the first experiment series, training reports are pre-processed, then their TF-IDF features are extracted  The TF-IDF features and the training labels are used to train different classification models  These different classification models and their hyper-parameters are reported in tab:classifier  The performance of classifiers is measured quantitatively on the test dataset using the evaluation metrics discussed in the previous section  For the second experiment series, a random report is selected and its top 50 keywords are extracted using TF-IDF weights  These 50 keywords are highlighted using different colors based on their associated topic, which are extracted through LDA  A non-expert based qualitative inspection is performed on the extracted keywords and their corresponding topics 
Experiment Series 1
A classification model is trained to predict the primary diagnosis given the content of the cancer pathology report  The performance results on this task are reported in tab:results  We can observe that the XGBoost classifier outperformed all other models for both the micro F-score metric, with a score of 0 92, and the macro F-score metric, with a score of 0 31  This was an improvement of 7% for the micro F-score over the next best model, SVM-L, and a marginal improvement of 5% for macro F-score  It is interesting to note that SVM with linear kernels performs much better than SVM with RBF kernel, scoring 9% on the macro F-score and 12% more on the micro F-score  It is suspected that since words used in primary diagnosis itself occur in some reports, thus enabling the linear models to outperform complex models 
Experiment Series 2
fig:keywords shows the top 50 keywords highlighted using TF-IDF and LDA  The proposed approach has performed well in highlighting the important regions, for example the topic highlighted with a red color containing “presence range tumor necrosis” provides useful biomarker information to readers 
Conclusions
We proposed a simple yet efficient TF-IDF method to extract and corroborate useful keywords from pathology cancer reports  Encoding a pathology report for cancer and tumor surveillance is a laborious task, and sometimes it is subjected to human errors and variability in the interpretation  One of the most important aspects of encoding a pathology report involves extracting the primary diagnosis  This may be very useful for content-based image retrieval to combine with visual information  We used existing classification model and TF-IDF features to predict the primary diagnosis  We achieved up to 92% accuracy using XGBoost classifier  The prediction accuracy empowers the adoption of machine learning methods for automated information extraction from pathology reports 
Context 2: Problem Statement
Spoken conversations still remain the most natural and effortless means of human communication  Thus a lot of valuable information is conveyed and exchanged in such an unstructured form  In telehealth settings, nurses might call discharged patients who have returned home to continue to monitor their health status  Human language technology that can efficiently and effectively extract key information from such conversations is clinically useful, as it can help streamline workflow processes and digitally document patient medical information to increase staff productivity  In this work, we design and prototype a dialogue comprehension system in the question-answering manner, which is able to comprehend spoken conversations between nurses and patients to extract clinical information 
Motivation of Approach
Machine comprehension of written passages has made tremendous progress recently  Large quantities of supervised training data for reading comprehension (e g  SQuAD BIBREF0 ), the wide adoption and intense experimentation of neural modeling BIBREF1 , BIBREF2 , and the advancements in vector representations of word embeddings BIBREF3 , BIBREF4 all contribute significantly to the achievements obtained so far  The first factor, the availability of large scale datasets, empowers the latter two factors  To date, there is still very limited well-annotated large-scale data suitable for modeling human-human spoken dialogues  Therefore, it is not straightforward to directly port over the recent endeavors in reading comprehension to dialogue comprehension tasks 
In healthcare, conversation data is even scarcer due to privacy issues  Crowd-sourcing is an efficient way to annotate large quantities of data, but less suitable for healthcare scenarios, where domain knowledge is required to guarantee data quality  To demonstrate the feasibility of a dialogue comprehension system used for extracting key clinical information from symptom monitoring conversations, we developed a framework to construct a simulated human-human dialogue dataset to bootstrap such a prototype  Similar efforts have been conducted for human-machine dialogues for restaurant or movie reservations BIBREF5   To the best of our knowledge, no one to date has done so for human-human conversations in healthcare 
Human-human Spoken Conversations
Human-human spoken conversations are a dynamic and interactive flow of information exchange  While developing technology to comprehend such spoken conversations presents similar technical challenges as machine comprehension of written passages BIBREF6 , the challenges are further complicated by the interactive nature of human-human spoken conversations:
(1) Zero anaphora is more common: Co-reference resolution of spoken utterances from multiple speakers is needed  For example, in Figure FIGREF5 (a) headaches, the pain, it, head bulging all refer to the patient's headache symptom, but they were uttered by different speakers and across multiple utterances and turns  In addition, anaphors are more likely to be omitted (see Figure FIGREF5 (a) A4) as this does not affect the human listener’s understanding, but it might be challenging for computational models 
(2) Thinking aloud more commonly occurs: Since it is more effortless to speak than to type, one is more likely to reveal her running thoughts when talking  In addition, one cannot retract what has been uttered, while in text communications, one is more likely to confirm the accuracy of the information in a written response and revise if necessary before sending it out  Thinking aloud can lead to self-contradiction, requiring more context to fully understand the dialogue; e g , in A6 in Figure FIGREF5 (a), the patient at first says he has none of the symptoms asked, but later revises his response saying that he does get dizzy after running 
(3) Topic drift is more common and harder to detect in spoken conversations: An example is shown in Figure FIGREF5 (a) in A3, where No is actually referring to cough in the previous question, and then the topic is shifted to headache  In spoken conversations, utterances are often incomplete sentences so traditional linguistic features used in written passages such as punctuation marks indicating syntactic boundaries or conjunction words suggesting discourse relations might no longer exist 
Dialogue Comprehension Task
Figure FIGREF5 (b) illustrates the proposed dialogue comprehension task using a question answering (QA) model  The input are a multi-turn symptom checking dialogue INLINEFORM0 and a query INLINEFORM1 specifying a symptom with one of its attributes; the output is the extracted answer INLINEFORM2 from the given dialogue  A training or test sample is defined as INLINEFORM3   Five attributes, specifying certain details of clinical significance, are defined to characterize the answer types of INLINEFORM4 : (1) the time the patient has been experiencing the symptom, (2) activities that trigger the symptom (to occur or worsen), (3) the extent of seriousness, (4) the frequency occurrence of the symptom, and (5) the location of symptom  For each symptom/attribute, it can take on different linguistic expressions, defined as entities  Note that if the queried symptom or attribute is not mentioned in the dialogue, the groundtruth output is “No Answer”, as in BIBREF6  
Reading Comprehension
Large-scale reading comprehension tasks like SQuAD BIBREF0 and MARCO BIBREF7 provide question-answer pairs from a vast range of written passages, covering different kinds of factual answers involving entities such as location and numerical values  Furthermore, HotpotQA BIBREF8 requires multi-step inference and provides numerous answer types  CoQA BIBREF9 and QuAC BIBREF10 are designed to mimic multi-turn information-seeking discussions of the given material  In these tasks, contextual reasoning like coreference resolution is necessary to grasp rich linguistic patterns, encouraging semantic modeling beyond naive lexical matching  Neural networks contribute to impressive progress in semantic modeling: distributional semantic word embeddings BIBREF3 , contextual sequence encoding BIBREF11 , BIBREF12 and the attention mechanism BIBREF13 , BIBREF14 are widely adopted in state-of-the-art comprehension models BIBREF1 , BIBREF2 , BIBREF4  
While language understanding tasks in dialogue such as domain identification BIBREF15 , slot filling BIBREF16 and user intent detection BIBREF17 have attracted much research interest, work in dialogue comprehension is still limited, if any  It is labor-intensive and time-consuming to obtain a critical mass of annotated conversation data for computational modeling  Some propose to collect text data from human-machine or machine-machine dialogues BIBREF18 , BIBREF5   In such cases, as human speakers are aware of current limitations of dialogue systems or due to pre-defined assumptions of user simulators, there are fewer cases of zero anaphora, thinking aloud, and topic drift, which occur more often in human-human spoken interactions 
NLP for Healthcare
There is emerging interest in research and development activities at the intersection of machine learning and healthcare , of which much of the NLP related work are centered around social media or online forums (e g , BIBREF19 , BIBREF20 ), partially due to the world wide web as a readily available source of information  Other work in this area uses public data sources such as MIMIC in electronic health records: text classification approaches have been applied to analyze unstructured clinical notes for ICD code assignment BIBREF21 and automatic intensive emergency prediction BIBREF22   Sequence-to-sequence textual generation has been used for readable notes based on medical and demographic recordings BIBREF23   For mental health, there has been more focus on analyzing dialogues  For example, sequential modeling of audio and text have helped detect depression from human-machine interviews BIBREF24   However, few studies have examined human-human spoken conversations in healthcare settings 
Data Preparation
We used recordings of nurse-initiated telephone conversations for congestive heart failure patients undergoing telemonitoring, post-discharge from the hospital  The clinical data was acquired by the Health Management Unit at Changi General Hospital  This research study was approved by the SingHealth Centralised Institutional Review Board (Protocol 1556561515)  The patients were recruited during 2014-2016 as part of their routine care delivery, and enrolled into the telemonitoring health management program with consent for use of anonymized versions of their data for research 
The dataset comprises a total of 353 conversations from 40 speakers (11 nurses, 16 patients, and 13 caregivers) with consent to the use of anonymized data for research  The speakers are 38 to 88 years old, equally distributed across gender, and comprise a range of ethnic groups (55% Chinese, 17% Malay, 14% Indian, 3% Eurasian, and 11% unspecified)  The conversations cover 11 topics (e g , medication compliance, symptom checking, education, greeting) and 9 symptoms (e g , chest pain, cough) and amount to 41 hours 
Data preprocessing and anonymization were performed by a data preparation team, separate from the data analysis team to maintain data confidentiality  The data preparation team followed standard speech recognition transcription guidelines, where words are transcribed verbatim to include false starts, disfluencies, mispronunciations, and private self-talk  Confidential information were marked and clipped off from the audio and transcribed with predefined tags in the annotation  Conversation topics and clinical symptoms were also annotated and clinically validated by certified telehealth nurses 
Linguistic Characterization on Seed Data
To analyze the linguistic structure of the inquiry-response pairs in the entire 41-hour dataset, we randomly sampled a seed dataset consisting of 1,200 turns and manually categorized them to different types, which are summarized in Table TABREF14 along with the corresponding occurrence frequency statistics  Note that each given utterance could be categorized to more than one type  We elaborate on each utterance type below 
Open-ended Inquiry: Inquiries about general well-being or a particular symptom; e g , “How are you feeling?” and “Do you cough?”
Detailed Inquiry: Inquiries with specific details that prompt yes/no answers or clarifications; e g , “Do you cough at night?”
Multi-Intent Inquiry: Inquiring more than one symptom in a question; e g , “Any cough, chest pain, or headache?”
Reconfirmation Inquiry: The nurse reconfirms particular details; e g , “Really? At night?” and “Serious or mild?”  This case is usually related to explicit or implicit coreferencing 
Inquiry with Transitional Clauses: During spoken conversations, one might repeat what the other party said, but it is unrelated to the main clause of the question  This is usually due to private self-talk while thinking aloud, and such utterances form a transitional clause before the speaker starts a new topic; e g , “Chest pain    no chest pain, I see    any cough?” 
Yes/No Response: Yes/No responses seem straightforward, but sometimes lead to misunderstanding if one does not interpret the context appropriately  One case is tag questions: A:“You don't cough at night, do you?” B:`Yes, yes” A:“cough at night?” B:“No, no cough”  Usually when the answer is unclear, clarifying inquiries will be asked for reconfirmation purposes 
Detailed Response: Responses that contain specific information of one symptom, like “I felt tightness in my chest” 
Response with Revision: Revision is infrequent but can affect comprehension significantly  One cause is thinking aloud so a later response overrules the previous one; e

ground_truth: Unanswerable

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What labels do they create on their dataset?
Context 2: g , “No dizziness, oh wait    last week I felt a bit dizzy when biking” 
Response with Topic Drift: When a symptom/topic like headache is inquired, the response might be: “Only some chest pain at night”, not referring to the original symptom (headache) at all 
Response with Transitional Clauses: Repeating some of the previous content, but often unrelated to critical clinical information and usually followed by topic drift  For example, “Swelling    swelling    I don't cough at night” 
Simulating Symptom Monitoring Dataset for Training
We divide the construction of data simulation into two stages  In Section SECREF16 , we build templates and expression pools using linguistic analysis followed by manual verification  In Section SECREF20 , we present our proposed framework for generating simulated training data  The templates and framework are verified for logical correctness and clinical soundness 
Template Construction
Each utterance in the seed data is categorized according to Table TABREF14 and then abstracted into templates by replacing entity phrases like cough and often with respective placeholders “#symptom#” and “#frequency#”  The templates are refined through verifying logical correctness and injecting expression diversity by linguistically trained researchers  As these replacements do not alter the syntactic structure, we interchange such placeholders with various verbal expressions to enlarge the simulated training set in Section SECREF20   Clinical validation was also conducted by certified telehealth nurses 
For the 9 symptoms (e g  chest pain, cough) and 5 attributes (e g , extent, frequency), we collect various expressions from the seed data, and expand them through synonym replacement  Some attributes are unique to a particular symptom; e g , “left leg” in #location# is only suitable to describe the symptom swelling, but not the symptom headache  Therefore, we only reuse general expressions like “slight” in #extent# across different symptoms to diversify linguistic expressions 
Two linguistically trained researchers constructed expression pools for each symptom and each attribute to account for different types of paraphrasing and descriptions  These expression pools are used in Section SECREF20 (c) 
Simulated Data Generation Framework
Figure FIGREF15 shows the five steps we use to generate multi-turn symptom monitoring dialogue samples 
(a) Topic Selection: While nurses might prefer to inquire the symptoms in different orders depending on the patient's history, our preliminary analysis shows that modeling results do not differ noticeably if topics are of equal prior probabilities  Thus we adopt this assumption for simplicity 
(b) Template Selection: For each selected topic, one inquiry template and one response template are randomly chosen to compose a turn  To minimize adverse effects of underfitting, we redistributed the frequency distribution in Table TABREF14 : For utterance types that are below 15%, we boosted them to 15%, and the overall relative distribution ranking is balanced and consistent with Table TABREF14  
(c) Enriching Linguistic Expressions: The placeholders in the selected templates are substituted with diverse expressions from the expression pools in Section UID19 to characterize the symptoms and their corresponding attributes 
(d) Multi-Turn Dialogue State Tracking: A greedy algorithm is applied to complete conversations  A “completed symptoms” list and a “to-do symptoms” list are used for symptom topic tracking  We also track the “completed attributes" and “to-do attributes"  For each symptom, all related attributes are iterated  A dialogue ends only when all possible entities are exhausted, generating a multi-turn dialogue sample, which encourages the model to learn from the entire discussion flow rather than a single turn to comprehend contextual dependency  The average length of a simulated dialogue is 184 words, which happens to be twice as long as an average dialogue from the real-world evaluation set  Moreover, to model the roles of the respondents, we set the ratio between patients and caregivers to 2:1; this statistic is inspired by the real scenarios in the seed dataset  For both the caregivers and patients, we assume equal probability of both genders  The corresponding pronouns in the conversations are thus determined by the role and gender of these settings 
(e) Multi-Turn Sample Annotation: For each multi-turn dialogue, a query is specified by a symptom and an attribute  The groundtruth output of the QA system is automatically labeled based on the template generation rules, but also manually verified to ensure annotation quality  Moreover, we adopt the unanswerable design in BIBREF6 : when the patient does not mention a particular symptom, the answer is defined as “No Answer”  This process is repeated until all logical permutations of symptoms and attributes are exhausted 
Experiments
Model Design
We implemented an established model in reading comprehension, a bi-directional attention pointer network BIBREF1 , and equipped it with an answerable classifier, as depicted in Figure FIGREF21   First, tokens in the given dialogue INLINEFORM0 and query INLINEFORM1 are converted into embedding vectors  Then the dialogue embeddings are fed to a bi-directional LSTM encoding layer, generating a sequence of contextual hidden states  Next, the hidden states and query embeddings are processed by a bi-directional attention layer, fusing attention information in both context-to-query and query-to-context directions  The following two bi-directional LSTM modeling layers read the contextual sequence with attention  Finally, two respective linear layers with softmax functions are used to estimate token INLINEFORM2 's INLINEFORM3 and INLINEFORM4 probability of the answer span INLINEFORM5  
In addition, we add a special tag “[SEQ]” at the head of INLINEFORM0 to account for the case of “No answer” BIBREF4 and adopt an answerable classifier as in BIBREF25   More specifically, when the queried symptom or attribute is not mentioned in the dialogue, the answer span should point to the tag “[SEQ]” and answerable probability should be predicted as 0 
Implementation Details
The model was trained via gradient backpropagation with the cross-entropy loss function of answer span prediction and answerable classification, optimized by Adam algorithm BIBREF26 with initial learning rate of INLINEFORM0   Pre-trained GloVe BIBREF3 embeddings (size INLINEFORM1 ) were used  We re-shuffled training samples at each epoch (batch size INLINEFORM2 )  Out-of-vocabulary words ( INLINEFORM3 ) were replaced with a fixed random vector  L2 regularization and dropout (rate INLINEFORM4 ) were used to alleviate overfitting BIBREF27  
Evaluation Setup
To evaluate the effectiveness of our linguistically-inspired simulation approach, the model is trained on the simulated data (see Section SECREF20 )  We designed 3 evaluation sets: (1) Base Set (1,264 samples) held out from the simulated data  (2) Augmented Set (1,280 samples) built by adding two out-of-distribution symptoms, with corresponding dialogue contents and queries, to the Base Set (“bleeding” and “cold”, which never appeared in training data)  (3) Real-World Set (944 samples) manually delineated from the the symptom checking portions (approximately 4 hours) of real-world dialogues, and annotated as evaluation samples 
Results
Evaluation results are in Table TABREF25 with exact match (EM) and F1 score in BIBREF0 metrics  To distinguish the correct answer span from the plausible ones which contain the same words, we measure the scores on the position indices of tokens  Our results show that both EM and F1 score increase with training sample size growing and the optimal size in our setting is 100k  The best-trained model performs well on both the Base Set and the Augmented Set, indicating that out-of-distribution symptoms do not affect the comprehension of existing symptoms and outputs reasonable answers for both in- and out-of-distribution symptoms  On the Real-World Set, we obtained 78 23 EM score and 80 18 F1 score respectively 
Error analysis suggests the performance drop from the simulated test sets is due to the following: 1) sparsity issues resulting from the expression pools excluding various valid but sporadic expressions  2) nurses and patients occasionally chit-chat in the Real-World Set, which is not simulated in the training set  At times, these chit-chats make the conversations overly lengthy, causing the information density to be lower  These issues could potentially distract and confuse the comprehension model  3) an interesting type of infrequent error source, caused by patients elaborating on possible causal relations of two symptoms  For example, a patient might say “My giddiness may be due to all this cough”  We are currently investigating how to close this performance gap efficiently 
Ablation Analysis
To assess the effectiveness of bi-directional attention, we bypassed the bi-attention layer by directly feeding the contextual hidden states and query embeddings to the modeling layer  To evaluate the pre-trained GloVe embeddings, we randomly initialized and trained the embeddings from scratch  These two procedures lead to 10% and 18% performance degradation on the Augmented Set and Real-World Set, respectively (see Table TABREF27 ) 
Conclusion
We formulated a dialogue comprehension task motivated by the need in telehealth settings to extract key clinical information from spoken conversations between nurses and patients  We analyzed linguistic characteristics of real-world human-human symptom checking dialogues, constructed a simulated dataset based on linguistically inspired and clinically validated templates, and prototyped a QA system  The model works effectively on a simulated test set using symptoms excluded during training and on real-world conversations between nurses and patients  We are currently improving the model's dialogue comprehension capability in complex reasoning and context understanding and also applying the QA model to summarization and virtual nurse applications 
Acknowledgements
Research efforts were supported by funding for Digital Health and Deep Learning I2R (DL2 SSF Project No: A1718g0045) and the Science and Engineering Research Council (SERC Project No: A1818g0044), A*STAR, Singapore  In addition, this work was conducted using resources and infrastructure provided by the Human Language Technology unit at I2R  The telehealth data acquisition was funded by the Economic Development Board (EDB), Singapore Living Lab Fund and Philips Electronics – Hospital to Home Pilot Project (EDB grant reference number: S14-1035-RF-LLF H and W) 
We acknowledge valuable support and assistance from Yulong Lin, Yuanxin Xiang, and Ai Ti Aw at the Institute for Infocomm Research (I2R); Weiliang Huang at the Changi General Hospital (CGH) Department of Cardiology, Hong Choon Oh at CGH Health Services Research, and Edris Atikah Bte Ahmad, Chiu Yan Ang, and Mei Foon Yap of the CGH Integrated Care Department 
We also thank Eduard Hovy and Bonnie Webber for insightful discussions and the anonymous reviewers for their precious feedback to help improve and extend this piece of work 
Context 3: Introduction
The irony is a kind of figurative language, which is widely used on social media BIBREF0   The irony is defined as a clash between the intended meaning of a sentence and its literal meaning BIBREF1   As an important aspect of language, irony plays an essential role in sentiment analysis BIBREF2 , BIBREF0 and opinion mining BIBREF3 , BIBREF4  
Although some previous studies focus on irony detection, little attention is paid to irony generation  As ironies can strengthen sentiments and express stronger emotions, we mainly focus on generating ironic sentences  Given a non-ironic sentence, we implement a neural network to transfer it to an ironic sentence and constrain the sentiment polarity of the two sentences to be the same  For example, the input is “I hate it when my plans get ruined" which is negative in sentiment polarity and the output should be ironic and negative in sentiment as well, such as “I like it when my plans get ruined"  The speaker uses “like" to be ironic and express his or her negative sentiment  At the same time, our model can preserve contents which are irrelevant to sentiment polarity and irony  According to the categories mentioned in BIBREF5 , irony can be classified into 3 classes: verbal irony by means of a polarity contrast, the sentences containing expression whose polarity is inverted between the intended and the literal evaluation; other types of verbal irony, the sentences that show no polarity contrast between the literal and intended meaning but are still ironic; and situational irony, the sentences that describe situations that fail to meet some expectations  As ironies in the latter two categories are obscure and hard to understand, we decide to only focus on ironies in the first category in this work  For example, our work can be specifically described as: given a sentence “I hate to be ignored", we train our model to generate an ironic sentence such as “I love to be ignored"  Although there is “love" in the generated sentence, the speaker still expresses his or her negative sentiment by irony  We also make some explorations in the transformation from ironic sentences to non-ironic sentences at the end of our work  Because of the lack of previous work and baselines on irony generation, we implement our model based on style transfer  Our work will not only provide the first large-scale irony dataset but also make our model as a benchmark for the irony generation 
Recently, unsupervised style transfer becomes a very popular topic  Many state-of-the-art studies try to solve the task with sequence-to-sequence (seq2seq) framework  There are three main ways to build up models  The first is to learn a latent style-independent content representation and generate sentences with the content representation and another style BIBREF6 , BIBREF7   The second is to directly transfer sentences from one style to another under the control of classifiers and reinforcement learning BIBREF8   The third is to remove style attribute words from the input sentence and combine the remaining content with new style attribute words BIBREF9 , BIBREF10   The first method usually obtains better performances via adversarial training with discriminators  The style-independent content representation, nevertheless, is not easily obtained BIBREF11 , which results in poor performances  The second method is suitable for complex styles which are difficult to model and describe  The model can learn the deep semantic features by itself but sometimes the model is sensitive to parameters and hard to train  The third method succeeds to preserve content but cannot work for some complex styles such as democratic and republican  Sentences with those styles usually do not have specific style attribute words  Unfortunately, due to the lack of large irony dataset and difficulties of modeling ironies, there has been little work trying to generate ironies based on seq2seq framework as far as we know  Inspired by methods for style transfer, we decide to implement a specifically designed model based on unsupervised style transfer to explore irony generation 
In this paper, in order to address the lack of irony data, we first crawl over 2M tweets from twitter to build a dataset with 262,755 ironic and 112,330 non-ironic tweets  Then, due to the lack of parallel data, we propose a novel model to transfer non-ironic sentences to ironic sentences in an unsupervised way  As ironic style is hard to model and describe, we implement our model with the control of classifiers and reinforcement learning  Different from other studies in style transfer, the transformation from non-ironic to ironic sentences has to preserve sentiment polarity as mentioned above  Therefore, we not only design an irony reward to control the irony accuracy and implement denoising auto-encoder and back-translation to control content preservation but also design a sentiment reward to control sentiment preservation 
Experimental results demonstrate that our model achieves a high irony accuracy with well-preserved sentiment and content  The contributions of our work are as follows:
Related Work
Style Transfer: As irony is a complicated style and hard to model with some specific style attribute words, we mainly focus on studies without editing style attribute words 
Some studies are trying to disentangle style representation from content representation  In BIBREF12 , authors leverage adversarial networks to learn separate content representations and style representations  In BIBREF13 and BIBREF6 , researchers combine variational auto-encoders (VAEs) with style discriminators 
However, some recent studies BIBREF11 reveal that the disentanglement of content and style representations may not be achieved in practice  Therefore, some other research studies BIBREF9 , BIBREF10 strive to separate content and style by removing stylistic words  Nonetheless, many non-ironic sentences do not have specific stylistic words and as a result, we find it difficult to transfer non-ironic sentences to ironic sentences through this way in practice 
Besides, some other research studies do not disentangle style from content but directly learn representations of sentences  In BIBREF8 , authors propose a dual reinforcement learning framework without separating content and style representations  In BIBREF7 , researchers utilize a machine translation model to learn a sentence representation preserving the meaning of the sentence but reducing stylistic properties  In this method, the quality of generated sentences relies on the performance of classifiers to a large extent  Meanwhile, such models are usually sensitive to parameters and difficult to train  In contrast, we combine a pre-training process with reinforcement learning to build up a stable language model and design special rewards for our task 
Irony Detection: With the development of social media, irony detection becomes a more important task  Methods for irony detection can be mainly divided into two categories: methods based on feature engineering and methods based on neural networks 
As for methods based on feature engineering, In BIBREF1 , authors investigate pragmatic phenomena and various irony markers  In BIBREF14 , researchers leverage a combination of sentiment, distributional semantic and text surface features  Those models rely on hand-crafted features and are hard to implement 
When it comes to methods based on neural networks, long short-term memory (LSTM) BIBREF15 network is widely used and is very efficient for irony detection  In BIBREF16 , a tweet is divided into two segments and a subtract layer is implemented to calculate the difference between two segments in order to determine whether the tweet is ironic  In BIBREF17 , authors utilize a recurrent neural network with Bi-LSTM and self-attention without hand-crafted features  In BIBREF18 , researchers propose a system based on a densely connected LSTM network 
Our Dataset
In this section, we describe how we build our dataset with tweets  First, we crawl over 2M tweets from twitter using GetOldTweets-python  We crawl English tweets from 04/09/2012 to /12/18/2018  We first remove all re-tweets and use langdetect to remove all non-English sentences  Then, we remove hashtags attached at the end of the tweets because they are usually not parts of sentences and will confuse our language model  After that, we utilize Ekphrasis to process tweets  We remove URLs and restore remaining hashtags, elongated words, repeated words, and all-capitalized words  To simplify our dataset, We replace all “ INLINEFORM0 money INLINEFORM1 " and “ INLINEFORM2 time INLINEFORM3 " tokens with “ INLINEFORM4 number INLINEFORM5 " token when using Ekphrasis  And we delete sentences whose lengths are less than 10 or greater than 40  In order to restore abbreviations, we download an abbreviation dictionary from webopedia and restore abbreviations to normal words or phrases according to the dictionary  Finally, we remove sentences which have more than two rare words (appearing less than three times) in order to constrain the size of vocabulary  Finally, we get 662,530 sentences after pre-processing 
As neural networks are proved effective in irony detection, we decide to implement a neural classifier in order to classify the sentences into ironic and non-ironic sentences  However, the only high-quality irony dataset we can obtain is the dataset of Semeval-2018 Task 3 and the dataset is pretty small, which will cause overfitting to complex models  Therefore, we just implement a simple one-layer RNN with LSTM cell to classify pre-processed sentences into ironic sentences and non-ironic sentences because LSTM networks are widely used in irony detection  We train the model with the dataset of Semeval-2018 Task 3  After classification, we get 262,755 ironic sentences and 399,775 non-ironic sentences  According to our observation, not all non-ironic sentences are suitable to be transferred into ironic sentences  For example, “just hanging out   watching   is it monday yet" is hard to transfer because it does not have an explicit sentiment polarity  So we remove all interrogative sentences from the non-ironic sentences and only obtain the sentences which have words expressing strong sentiments  We evaluate the sentiment polarity of each word with TextBlob and we view those words with sentiment scores greater than 0 5 or less than -0 5 as words expressing strong sentiments  Finally, we build our irony dataset with 262,755 ironic sentences and 102,330 non-ironic sentences 
[t] Irony Generation Algorithm
INLINEFORM0 pre-train with auto-encoder Pre-train INLINEFORM1 , INLINEFORM2 with INLINEFORM3 using MLE based on Eq  EQREF16 Pre-train INLINEFORM4 , INLINEFORM5 with INLINEFORM6 using MLE based on Eq  EQREF17 INLINEFORM7 pre-train with back-translation Pre-train INLINEFORM8 , INLINEFORM9 , INLINEFORM10 , INLINEFORM11 with INLINEFORM12 using MLE based on Eq  EQREF19 Pre-train INLINEFORM13 , INLINEFORM14 , INLINEFORM15 , INLINEFORM16 with INLINEFORM17 using MLE based on Eq  EQREF20
INLINEFORM0 train with RL each epoch e = 1, 2,    , INLINEFORM1 INLINEFORM2 train non-irony2irony with RL INLINEFORM3 in N INLINEFORM4 update INLINEFORM5 , INLINEFORM6 , using INLINEFORM7 based on Eq  EQREF29 INLINEFORM8 back-translation INLINEFORM9 INLINEFORM10 INLINEFORM11 update INLINEFORM12 , INLINEFORM13 , INLINEFORM14 , INLINEFORM15 using MLE based on Eq  EQREF19 INLINEFORM16 train irony2non-irony with RL INLINEFORM17 in I INLINEFORM18 update INLINEFORM19 , INLINEFORM20 , using INLINEFORM21 similar to Eq  EQREF29 INLINEFORM22 back-translation INLINEFORM23 INLINEFORM24 INLINEFORM25 update INLINEFORM26 , INLINEFORM27 , INLINEFORM28 , INLINEFORM29 using MLE based on Eq  EQREF20
Our Method
Given two non-parallel corpora: non-ironic corpus N={ INLINEFORM0 , INLINEFORM1 ,    , INLINEFORM2 } and ironic corpus I={ INLINEFORM3 , INLINEFORM4 ,    , INLINEFORM5 }, the goal of our irony generation model is to generate an ironic sentence from a non-ironic sentence while preserving the content and sentiment polarity of the source input sentence  We implement an encoder-decoder framework where two encoders are utilized to encode ironic sentences and non-ironic sentences respectively and two decoders are utilized to decode ironic sentences and non-ironic sentences from latent representations respectively  In order to enforce a shared latent space, we share two layers on both the encoder side and the decoder side  Our model architecture is illustrated in Figure FIGREF13   We denote irony encoder as INLINEFORM6 , irony decoder as INLINEFORM7 and non-irony encoder as INLINEFORM8 , non-irony decoder as INLINEFORM9   Their parameters are INLINEFORM10 , INLINEFORM11 , INLINEFORM12 and INLINEFORM13  
Our irony generation algorithm is shown in Algorithm SECREF3   We first pre-train our model using denoising auto-encoder and back-translation to build up language models for both styles (section SECREF14 )  Then we implement reinforcement learning to train the model to transfer sentences from one style to another (section SECREF21 )  Meanwhile, to achieve content preservation, we utilize back-translation for one time in every INLINEFORM0 time steps 
Pretraining
In order to build up our language model and preserve the content, we apply the auto-encoder model  To prevent the model from simply copying the input sentence, we randomly add some noises in the input sentence  Specifically, for every word in the input sentence, there is 10% chance that we delete it, 10 % chance that we duplicate it, 10% chance that we swap it with the next word, or it remains unchanged  We first encode the input sentence INLINEFORM0 or INLINEFORM1 with respective encoder INLINEFORM2 or INLINEFORM3 to obtain its latent representation INLINEFORM4 or INLINEFORM5 and reconstruct the input sentence with the latent representation and respective decoder  So we can get the reconstruction loss for auto-encoder INLINEFORM6 : DISPLAYFORM0 DISPLAYFORM1
In addition to denoising auto-encoder, we implement back-translation BIBREF19 to generate a pseudo-parallel corpus  Suppose our model takes non-ironic sentence INLINEFORM0 as input  We first encode INLINEFORM1 with INLINEFORM2 to obtain its latent representation INLINEFORM3 and decode the latent representation with INLINEFORM4 to get a transferred sentence INLINEFORM5   Then we encode INLINEFORM6 with INLINEFORM7 and decode its latent representation with INLINEFORM8 to reconstruct the original input sentence INLINEFORM9   Therefore, our reconstruction loss for back-translation INLINEFORM10 : DISPLAYFORM0
And if our model takes ironic sentence INLINEFORM0 as input, we can get the reconstruction loss for back-translation as: DISPLAYFORM0
Reinforcement Learning
Since the gold transferred result of input is unavailable, we cannot evaluate the quality of the generated sentence directly  Therefore, we implement reinforcement learning and elaborately design two rewards to describe the irony accuracy and sentiment preservation, respectively 
A pre-trained binary irony classifier based on CNN BIBREF20 is used to evaluate how ironic a sentence is  We denote the parameter of the classifier as INLINEFORM0 and it is fixed during the training process 
In order to facilitate the transformation, we design the irony reward as the difference between the irony score of the input sentence and that of the output sentence  Formally, when we input a non-ironic sentence INLINEFORM0 and transfer it to an ironic sentence INLINEFORM1 , our irony reward is defined as: DISPLAYFORM0
where INLINEFORM0 denotes ironic style and INLINEFORM1 is the probability of that a sentence INLINEFORM2 is ironic 
To preserve the sentiment polarity of the input sentence, we also need to use classifiers to evaluate the sentiment polarity of the sentences  However, the sentiment analysis of ironic sentences and non-ironic sentences are different  In the case of figurative languages such as irony, sarcasm or metaphor, the sentiment polarity of the literal meaning may differ significantly from that of the intended figurative meaning BIBREF0   As we aim to train our model to transfer sentences from non-ironic to ironic, using only one classifier is not enough  As a result, we implement two pre-trained sentiment classifiers for non-ironic sentences and ironic sentences respectively  We denote the parameter of the sentiment classifier for ironic sentences as INLINEFORM0 and that of the sentiment classifier for non-ironic sentences as INLINEFORM1  
A challenge, when we implement two classifiers to evaluate the sentiment polarity, is that the two classifiers trained with different datasets may have different distributions of scores  That means we cannot directly calculate the sentiment reward with scores applied by two classifiers  To alleviate this problem and standardize the prediction results of two classifiers, we set a threshold for each classifier and subtract the respective threshold from scores applied by the classifier to obtain the comparative sentiment polarity score  We get the optimal threshold by maximizing the ability of the classifier according to the distribution of our training data 
We denote the threshold of ironic sentiment classifier as INLINEFORM0 and the threshold of non-ironic sentiment classifier as INLINEFORM1   The standardized sentiment score is defined as INLINEFORM2 and INLINEFORM3 where INLINEFORM4 denotes the positive sentiment polarity and INLINEFORM5 is the probability of that a sentence is positive in sentiment polarity 
As mentioned above, the input sentence and the generated sentence should express the same sentiment  For example, if we input a non-ironic sentence “I hate to be ignored" which is negative in sentiment polarity, the generated ironic sentence should be also negative, such as “I love to be ignored"  To achieve sentiment preservation, we design the sentiment reward as that one minus the absolute value of the difference between the standardized sentiment score of the input sentence and that of the generated sentence  Formally, when we input a non-ironic sentence INLINEFORM0 and transfer it to an ironic sentence INLINEFORM1 , our sentiment reward is defined as: DISPLAYFORM0
To encourage our model to focus on both the irony accuracy and the sentiment preservation, we apply the harmonic mean of irony reward and sentiment reward: DISPLAYFORM0
Policy Gradient
The policy gradient algorithm BIBREF21 is a simple but widely-used algorithm in reinforcement learning  It is used to maximize the expected reward INLINEFORM0   The objective function to minimize is defined as: DISPLAYFORM0
where INLINEFORM0 , INLINEFORM1 is the reward of INLINEFORM2 and INLINEFORM3 is the input size 
Training Details
INLINEFORM0 , INLINEFORM1 , INLINEFORM2 and INLINEFORM3 in our model are Transformers BIBREF22 with 4 layers and 2 shared layers  The word embeddings of 128 dimensions are learned during the training process  Our maximum sentence length is set as 40  The optimizer is Adam BIBREF23 and the learning rate is INLINEFORM4   The batch size is 32 and harmonic weight INLINEFORM5 in Eq 9 is 0 5  We set the interval INLINEFORM6 as 200  The model is pre-trained for 6 epochs and trained for 15 epochs for reinforcement learning 
Irony Classifier: We implement a CNN classifier trained with our irony dataset  All the CNN classifiers we utilize in this paper use the same parameters as BIBREF20  
Sentiment Classifier for Irony: We first implement a one-layer LSTM network to classify ironic sentences in our dataset into positive and negative ironies  The LSTM network is trained with the dataset of Semeval 2015 Task 11 BIBREF0 which is used for the sentiment analysis of figurative language in twitter  Then, we use the positive ironies and negative ironies to train the CNN sentiment classifier for irony 
Sentiment Classifier for Non-irony: Similar to the training process of the sentiment classifier for irony, we first implement a one-layer LSTM network trained with the dataset for the sentiment analysis of common twitters to classify the non-ironies into positive and negative non-ironies  Then we use the positive and negative non-ironies to train the sentiment classifier for non-irony 
Baselines
We compare our model with the following state-of-art generative models:
BackTrans BIBREF7 : In BIBREF7 , authors propose a model using machine translation in order to preserve the meaning of the sentence while reducing stylistic properties 
Unpaired BIBREF10 : In BIBREF10 , researchers implement a method to remove emotional words and add desired sentiment controlled by reinforcement learning 
CrossAlign BIBREF6 : In BIBREF6 , authors leverage refined alignment of latent representations to perform style transfer and a cross-aligned auto-encoder is implemented 
CPTG BIBREF24 : An interpolated reconstruction loss is introduced in BIBREF24 and a discriminator is implemented to control attributes in this work 
DualRL BIBREF8 : In BIBREF8 , researchers use two reinforcement rewards simultaneously to control style accuracy and content preservation 
Evaluation Metrics
In order to evaluate sentiment preservation, we use the absolute value of the difference between the standardized sentiment score of the input sentence and that of the generated sentence  We call the value as sentiment delta (senti delta)  Besides, we report the sentiment accuracy (Senti ACC) which measures whether the output sentence has the same sentiment polarity as the input sentence based on our standardized sentiment classifiers  The BLEU score BIBREF25 between the input sentences and the output sentences is calculated to evaluate the content preservation performance  In order to evaluate the overall performance of different models, we also report the geometric mean (G2) and harmonic mean (H2) of the sentiment accuracy and the BLEU score  As for the irony accuracy, we only report it in human evaluation results because it is more accurate for the human to evaluate the quality of irony as it is very complicated 
We first sample 50 non-ironic input sentences and their corresponding output sentences of different models  Then, we ask four annotators who are proficient in English to evaluate the qualities of the generated sentences of different models  They are required to rank the output sentences of our model and baselines from the best to the worst in terms of irony accuracy (Irony), Sentiment preservation (Senti) and content preservation (Content)  The best output is ranked with 1 and the worst output is ranked with 6  That means that the smaller our human evaluation value is, the better the corresponding model is 
Results and Discussions
Table TABREF35 shows the automatic evaluation results of the models in the transformation from non-ironic sentences to ironic sentences  From the results, our model obtains the best result in sentiment delta  The DualRL model achieves the highest result in other metrics, but most of its outputs are the almost same as the input sentences  So it is reasonable that DualRL system outperforms ours in these metrics but it actually does not transfer the non-ironic sentences to ironic sentences at all  From this perspective, we cannot view DualRL as an effective model for irony generation  In contrast, our model gets results close to those of DualRL and obtains a balance between irony accuracy, sentiment preservation, and content preservation if we also consider the irony accuracy discussed below 
And from human evaluation results shown in Table TABREF36 , our model gets the best average rank in irony accuracy  And as mentioned above, the DualRL model usually does not change the input sentence and outputs the same sentence  Therefore, it is reasonable that it obtains the best rank in sentiment and content preservation and ours is the second  However, it still demonstrates that our model, instead of changing nothing, transfers the style of the input sentence with content and sentiment preservation at the same time 
Case Study
In the section, we present some example outputs of different models  Table TABREF37 shows the results of the transformation from non-ironic sentences to ironic sentences  We can observe that: (1) The BackTrans system, the Unpaired system, the CrossAlign system and the CPTG system tends to generate sentences which are towards irony but do not preserve content  (2) The DualRL system preserves content and sentiment very well but even does not change the input sentence  (3) Our model considers both aspects and achieves a better balance among irony accuracy, sentiment and content preservation 
Error Analysis
Although our model outperforms other style transfer baselines according to automatic and human evaluation results, there are still some failure cases because irony generation is still a very challenging task  We would like to share the issues we meet during our experiments and our solutions to some of them in this section 
No Change: As mentioned above, many style transfer models, such as DualRL, tend to make few changes to the input sentence and output the same sentence  Actually, this is a common issue for unsupervised style transfer systems and we also meet it during our experiments  The main reason for the issue is that rewards for content preservation are too prominent and rewards for style accuracy cannot work well  In contrast, in order to guarantee the readability and fluency of the output sentence, we also cannot emphasize too much on rewards for style accuracy because it may cause some other issues such as word repetition mentioned below  A method to solve the problem is tuning hyperparameters and this is also the method we implement in this work  As for content preservation, maybe MLE methods such as back-translation are not enough because they tend to force models to generate specific words  In the future, we should further design some more suitable methods to control content preservation for models without disentangling style and content representations, such as DualRL and ours 
Word Repetition: During our experiments, we observe that some of the outputs prefer to repeat the same word as shown in Table TABREF38   This is because reinforcement learning rewards encourage the model to generate words which can get high scores from classifiers and even back-translation cannot stop it  Our solution is that we can lower the probability of decoding a word in decoders if the word has been generated in the previous time steps during testing  We also try to implement this method during training time but obtain worse performances because it may limit the effects of training  Some previous studies utilize language models to control the fluency of the output sentence and we also try this method  Nonetheless, pre-training a language model with tweets and using it to generate rewards is difficult because tweets are more casual and have more noise  Rewards from that kind of language model are usually not accurate and may confuse the model  In the future, we should come up with better methods to model language fluency with the consideration of irony accuracy, sentiment and content preservation, especially for tweets 
Improper Words: As ironic style is hard for our model to learn, it may generate some improper words which make the sentence strange  As the example shown in the Table TABREF38 , the sentiment word in the input sentence is “wonderful" and the model should change it into a negative word such as “sad" to make the output sentence ironic  However, the model changes “friday" and “fifa" which are not related to ironic styles  We have not found a very effective method to address this issue and maybe we should further explore stronger models to learn ironic styles better 
Additional Experiments
In this section, we describe some additional experiments on the transformation from ironic sentences to non-ironic sentences  Sometimes ironies are hard to understand and may cause misunderstanding, for which our task also explores the transformation from ironic sentences to non-ironic sentences 
As shown in Table TABREF46 , we also conduct automatic evaluations and the conclusions are similar to those of the transformation from non-ironic sentences to ironic sentences  As for human evaluation results in Table TABREF47 , our model still can achieve the second-best results in sentiment and content preservation  Nevertheless, DualRL system and ours get poor performances in irony accuracy  The reason may be that the other four baselines tend to generate common and even not fluent sentences which are irrelevant to the input sentences and are hard to be identified as ironies  So annotators usually mark these output sentences as non-ironic sentences, which causes these models to obtain better performances than DualRL and ours but much poorer results in sentiment and content preservation  Some examples are shown in Table TABREF52  
Conclusion and Future Work
In this paper, we first systematically define irony generation based on style transfer  Because of the lack of irony data, we make use of twitter and build a large-scale dataset  In order to control irony accuracy, sentiment preservation and content preservation at the same time, we also design a combination of rewards for reinforcement learning and incorporate reinforcement learning with a pre-training process  Experimental results demonstrate that our model outperforms other generative models and our rewards are effective  Although our model design is effective, there are still many errors and we systematically analyze them  In the future, we are interested in exploring these directions and our work may extend to other kinds of ironies which are more difficult to model 

ground_truth: (1) the time the patient has been experiencing the symptom, (2) activities that trigger the symptom (to occur or worsen), (3) the extent of seriousness, (4) the frequency occurrence of the symptom, and (5) the location of symptom, No Answer

the time the patient has been experiencing the symptom, activities that trigger the symptom, the extent of seriousness, the frequency occurrence of the symptom, the location of symptom, 9 symptoms

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What experiments are conducted?
Context 4: 015 
For the pun detection task, if the predicted tag sequence contains at least one INLINEFORM0 tag, we regard the output (i e , the prediction of our pun detection model) for this task as true, otherwise false  For the pun location task, a predicted pun is regarded as correct if and only if it is labeled as the gold pun in the dataset  As to pun location, to make fair comparisons with prior studies, we only consider the instances that are labeled as the ones containing a pun  We report precision, recall and INLINEFORM1 score in Table TABREF11   A list of prior works that did not employ joint learning are also shown in the first block of Table TABREF11  
Results
We also implemented a baseline model based on conditional random fields (CRF), where features like POS tags produced by the Stanford POS tagger BIBREF27 , n-grams, label transitions, word suffixes and relative position to the end of the text are considered  We can see that our model with the INLINEFORM0 tagging scheme yields new state-of-the-art INLINEFORM1 scores on pun detection and competitive results on pun location, compared to baselines that do not adopt joint learning in the first block  For location on heterographic puns, our model's performance is slightly lower than the system of BIBREF25 , which is a rule-based locator  Compared to CRF, we can see that our model, either with the INLINEFORM2 or the INLINEFORM3 scheme, yields significantly higher recall on both detection and location tasks, while the precisions are relatively close  This demonstrates the effectiveness of BiLSTM, which learns the contextual features of given texts – such information appears to be helpful in recalling more puns 
Compared to the INLINEFORM0 scheme, the INLINEFORM1 tagging scheme is able to yield better performance on these two tasks  After studying outputs from these two approaches, we found that one leading source of error for the INLINEFORM2 approach is that there exist more than one words in a single instance that are assigned with the INLINEFORM3 tag  However, according to the description of pun in BIBREF9 , each context contains a maximum of one pun  Thus, such a useful structural constraint is not well captured by the simple approach based on the INLINEFORM4 tagging scheme  On the other hand, by applying the INLINEFORM5 tagging scheme, such a constraint is properly captured in the model  As a result, the results for such a approach are significantly better than the approach based on the INLINEFORM6 tagging scheme, as we can observe from the table  Under the same experimental setup, we also attempted to exclude word position features  Results are given by INLINEFORM7 - INLINEFORM8   It is expected that the performance of pun location drops, since such position features are able to capture the interesting property that a pun tends to appear in the second half of a sentence  While such knowledge is helpful for the location task, interestingly, a model without position knowledge yields improved performance on the pun detection task  One possible reason is that detecting whether a sentence contains a pun is not concerned with such word position information 
Additionally, we conduct experiments over sentences containing a pun only, namely 1,607 and 1,271 instances from homographic and heterographic pun corpora separately  It can be regarded as a “pipeline” method where the classifier for pun detection is regarded as perfect  Following the prior work of BIBREF4 , we apply 10-fold cross validation  Since we are given that all input sentences contain a pun, we only report accumulated results on pun location, denoted as Pipeline in Table TABREF11   Compared with our approaches, the performance of such an approach drops significantly  On the other hand, such a fact demonstrates that the two task, detection and location of puns, can reinforce each other  These figures demonstrate the effectiveness of our sequence labeling method to detect and locate English puns in a joint manner 
Error Analysis
We studied the outputs from our system and make some error analysis  We found the errors can be broadly categorized into several types, and we elaborate them here  1) Low word coverage: since the corpora are relatively small, there exist many unseen words in the test set  Learning the representations of such unseen words is challenging, which affects the model's performance  Such errors contribute around 40% of the total errors made by our system  2) Detection errors: we found many errors are due to the model's inability to make correct pun detection  Such inability harms both pun detection and pun location  Although our approach based on the INLINEFORM0 tagging scheme yields relatively higher scores on the detection task, we still found that 40% of the incorrectly predicted instances fall into this group  3) Short sentences: we found it was challenging for our model to make correct predictions when the given text is short  Consider the example “Superglue! Tom rejoined," here the word rejoined is the corresponding pun  However, it would be challenging to figure out the pun with such limited contextual information 
Related Work
Most existing systems address pun detection and location separately  BIBREF22 applied word sense knowledge to conduct pun detection  BIBREF24 trained a bidirectional RNN classifier for detecting homographic puns  Next, a knowledge-based approach is adopted to find the exact pun  Such a system is not applicable to heterographic puns  BIBREF28 applied Google n-gram and word2vec to make decisions  The phonetic distance via the CMU Pronouncing Dictionary is computed to detect heterographic puns  BIBREF10 used the hidden Markov model and a cyclic dependency network with rich features to detect and locate puns  BIBREF23 used a supervised approach to pun detection and a weakly supervised approach to pun location based on the position within the context and part of speech features  BIBREF25 proposed a rule-based system for pun location that scores candidate words according to eleven simple heuristics  Two systems are developed to conduct detection and location separately in the system known as UWAV BIBREF3   The pun detector combines predictions from three classifiers  The pun locator considers word2vec similarity between every pair of words in the context and position to pinpoint the pun  The state-of-the-art system for homographic pun location is a neural method BIBREF4 , where the word senses are incorporated into a bidirectional LSTM model  This method only supports the pun location task on homographic puns  Another line of research efforts related to this work is sequence labeling, such as POS tagging, chunking, word segmentation and NER  The neural methods have shown their effectiveness in this task, such as BiLSTM-CNN BIBREF13 , GRNN BIBREF29 , LSTM-CRF BIBREF30 , LSTM-CNN-CRF BIBREF14 , LM-LSTM-CRF BIBREF15  
In this work, we combine pun detection and location tasks as a single sequence labeling problem  Inspired by the work of BIBREF15 , we also adopt a LSTM-CRF with character embeddings to make labeling decisions 
Conclusion
In this paper, we propose to perform pun detection and location tasks in a joint manner from a sequence labeling perspective  We observe that each text in our corpora contains a maximum of one pun  Hence, we design a novel tagging scheme to incorporate such a constraint  Such a scheme guarantees that there is a maximum of one word that will be tagged as a pun during the testing phase  We also found the interesting structural property such as the fact that most puns tend to appear at the second half of the sentences can be helpful for such a task, but was not explored in previous works  Furthermore, unlike many previous approaches, our approach, though simple, is generally applicable to both heterographic and homographic puns  Empirical results on the benchmark datasets prove the effectiveness of the proposed approach that the two tasks of pun detection and location can be addressed by a single model from a sequence labeling perspective 
Future research includes the investigations on how to make use of richer semantic and linguistic information for detection and location of puns  Research on puns for other languages such as Chinese is still under-explored, which could also be an interesting direction for our future studies 
Acknowledgments
We would like to thank the three anonymous reviewers for their thoughtful and constructive comments  This work is supported by Singapore Ministry of Education Academic Research Fund (AcRF) Tier 2 Project MOE2017-T2-1-156, and is partially supported by SUTD project PIE-SGP-AI-2018-01 

ground_truth: Irony Classifier, Sentiment Classifier for Irony, Sentiment Classifier for Non-irony, transformation from ironic sentences to non-ironic sentences