qinchuanhui/UDA-QA huggingface dataset:
Question: Do they evaluate only on English?

Evidence:
raw_evidence: We test our proposed approach for binary classification on either sarcasm or irony, on seven benchmark datasets retrieved from different media sources. Below we describe each dataset, please see Table TABREF1 below for a summary.
Twitter: We use the Twitter dataset provided for the SemEval 2018 Task 3, Irony Detection in English Tweets BIBREF18 . The dataset was manually annotated using binary labels. We also use the dataset by BIBREF4 , which is manually annotated for sarcasm. Finally, we use the dataset by BIBREF20 , who collected a user self-annotated corpus of tweets with the #sarcasm hashtag.
Reddit: BIBREF21 collected SARC, a corpus comprising of 600.000 sarcastic comments on Reddit. We use main subset, SARC 2.0, and the political subset, SARC 2.0 pol.
Online Dialogues: We utilize the Sarcasm Corpus V1 (SC-V1) and the Sarcasm Corpus V2 (SC-V2), which are subsets of the Internet Argument Corpus (IAC). Compared to other datasets in our selection, these differ mainly in text length and structure complexity BIBREF22 .
In Table TABREF1 , we see a notable difference in terms of size among the Twitter datasets. Given this circumstance, and in light of the findings by BIBREF18 , we are interested in studying how the addition of external soft-annotated data impacts on the performance. Thus, in addition to the datasets introduced before, we use two corpora for augmentation purposes. The first dataset was collected using the Twitter API, targeting tweets with the hashtags #sarcasm or #irony, resulting on a total of 180,000 and 45,000 tweets respectively. On the other hand, to obtain non-sarcastic and non-ironic tweets, we relied on the SemEval 2018 Task 1 dataset BIBREF25 . To augment each dataset with our external data, we first filter out tweets that are not in English using language guessing systems. We later extract all the hashtags in each target dataset and proceed to augment only using those external tweets that contain any of these hashtags. This allows us to, for each class, add a total of 36,835 tweets for the Ptáček corpus, 8,095 for the Riloff corpus and 26,168 for the SemEval-2018 corpus.

Answers:
Yes
Question: What are the three different sources of data?

Evidence:
raw_evidence: We test our proposed approach for binary classification on either sarcasm or irony, on seven benchmark datasets retrieved from different media sources. Below we describe each dataset, please see Table TABREF1 below for a summary.
Reddit: BIBREF21 collected SARC, a corpus comprising of 600.000 sarcastic comments on Reddit. We use main subset, SARC 2.0, and the political subset, SARC 2.0 pol.
Twitter: We use the Twitter dataset provided for the SemEval 2018 Task 3, Irony Detection in English Tweets BIBREF18 . The dataset was manually annotated using binary labels. We also use the dataset by BIBREF4 , which is manually annotated for sarcasm. Finally, we use the dataset by BIBREF20 , who collected a user self-annotated corpus of tweets with the #sarcasm hashtag.
Online Dialogues: We utilize the Sarcasm Corpus V1 (SC-V1) and the Sarcasm Corpus V2 (SC-V2), which are subsets of the Internet Argument Corpus (IAC). Compared to other datasets in our selection, these differ mainly in text length and structure complexity BIBREF22 .

Answers:
Twitter, Reddit, Online Dialogues
Question: What type of model are the ELMo representations used in?

Evidence:
raw_evidence: The usage of a purely character-based input would allow us to directly recover and model these features. Consequently, our architecture is based on Embeddings from Language Model or ELMo BIBREF10 . The ELMo layer allows to recover a rich 1,024-dimensional dense vector for each word. Using CNNs, each vector is built upon the characters that compose the underlying words. As ELMo also contains a deep bi-directional LSTM on top of this character-derived vectors, each word-level embedding contains contextual information from their surroundings. Concretely, we use a pre-trained ELMo model, obtained using the 1 Billion Word Benchmark which contains about 800M tokens of news crawl data from WMT 2011 BIBREF24 .
Subsequently, the contextualized embeddings are passed on to a BiLSTM with 2,048 hidden units. We aggregate the LSTM hidden states using max-pooling, which in our preliminary experiments offered us better results, and feed the resulting vector to a 2-layer feed-forward network, where each layer has 512 units. The output of this is then fed to the final layer of the model, which performs the binary classification.

Answers:
A bi-LSTM with max-pooling on top of it
Question: Which morphosyntactic features are thought to indicate irony or sarcasm?

Evidence:
raw_evidence: On the other hand, deep models for irony and sarcasm detection, which are currently offer state-of-the-art performance, have exploited sequential neural networks such as LSTMs and GRUs BIBREF15 , BIBREF23 on top of distributed word representations. Recently, in addition to using a sequential model, BIBREF14 proposed to use intra-attention to compare elements in a sequence against themselves. This allowed the model to better capture word-to-word level interactions that could also be useful for detecting sarcasm, such as the incongruity phenomenon BIBREF3 . Despite this, all models in the literature rely on word-level representations, which keeps the models from being able to easily capture some of the lexical and morpho-syntactic cues known to denote irony, such as all caps, quotation marks and emoticons, and in Twitter, also emojis and hashtags.

Answers:
all caps, quotation marks, emoticons, emojis, hashtags
Question: Is the model evaluated on other datasets?

Evidence:
raw_evidence: We evaluated SDNet on CoQA dataset, which improves the previous state-of-the-art model's result by 1.6% (from 75.0% to 76.6%) overall $F_1$ score. The ensemble model further increase the $F_1$ score to $79.3\%$ . Moreover, SDNet is the first model ever to pass $80\%$ on CoQA's in-domain dataset.

Answers:
No
Question: Does the model incorporate coreference and entailment?

Evidence:
raw_evidence: Self-Attention on Question. As the question has integrated previous utterances, the model needs to directly relate previously mentioned concept with the current question. This is helpful for concept carry-over and coreference resolution. We thus employ self-attention on question. The formula is the same as word-level attention, except that we are attending a question to itself: $\lbrace {u}_i^Q\rbrace _{i=1}^n=\mbox{Attn}(\lbrace {h}_i^{Q,K+1}\rbrace _{i=1}^n, \lbrace {h}_i^{Q,K+1}\rbrace _{i=1}^n, \lbrace {h}_i^{Q,K+1}\rbrace _{i=1}^n)$ . The final question representation is thus $\lbrace {u}_i^Q\rbrace _{i=1}^n$ .

Answers:
As the question has integrated previous utterances, the model needs to directly relate previously mentioned concept with the current question. This is helpful for concept carry-over and coreference resolution.
Question: Is the incorporation of context separately evaluated?

Evidence:

Answers:
No
Question: Which frozen acoustic model do they use?

Evidence:
raw_evidence: Several recent works have investigated jointly training the acoustic model with a masking speech enhancement model BIBREF11, BIBREF12, BIBREF13, but these works did not evaluate their system on speech enhancement metrics. Indeed, our internal experiments show that without access to the clean data, joint training severely harms performance on these metrics.

Answers:
a masking speech enhancement model BIBREF11, BIBREF12, BIBREF13
Question: What features are used?

Evidence:
raw_evidence: 45 clinically interpretable features per admission were extracted as inputs to the readmission risk classifier. These features can be grouped into three categories (See Table TABREF5 for complete list of features):
Sociodemographics: gender, age, marital status, etc.
Past medical history: number of previous admissions, history of suicidality, average length of stay (up until that admission), etc.
Information from the current admission: length of stay (LOS), suicidal risk, number and length of notes, time of discharge, evaluation scores, etc.
The Current Admission feature group has the most number of features, with 29 features included in this group alone. These features can be further stratified into two groups: `structured' clinical features and `unstructured' clinical features.
Feature Extraction ::: Structured Features
Structure features are features that were identified on the EHR using regular expression matching and include rating scores that have been reported in the psychiatric literature as correlated with increased readmission risk, such as Global Assessment of Functioning, Insight and Compliance:
Global Assessment of Functioning (GAF): The psychosocial functioning of the patient ranging from 100 (extremely high functioning) to 1 (severely impaired) BIBREF13.
Insight: The degree to which the patient recognizes and accepts his/her illness (either Good, Fair or Poor).
Compliance: The ability of the patient to comply with medication and to follow medical advice (either Yes, Partial, or None).
These features are widely-used in clinical practice and evaluate the general state and prognosis of the patient during the patient's evaluation.
Feature Extraction ::: Unstructured Features
Unstructured features aim to capture the state of the patient in relation to seven risk factor domains (Appearance, Thought Process, Thought Content, Interpersonal, Substance Use, Occupation, and Mood) from the free-text narratives on the EHR. These seven domains have been identified as associated with readmission risk in prior work BIBREF14.
These unstructured features include: 1) the relative number of sentences in the admission notes that involve each risk factor domain (out of total number of sentences within the admission) and 2) clinical sentiment scores for each of these risk factor domains, i.e. sentiment scores that evaluate the patient’s psychosocial functioning level (positive, negative, or neutral) with respect to each of these risk factor domain.

Answers:
Sociodemographics: gender, age, marital status, etc., Past medical history: number of previous admissions, history of suicidality, average length of stay (up until that admission), etc., Information from the current admission: length of stay (LOS), suicidal risk, number and length of notes, time of discharge, evaluation scores, etc.
Question: Do they compare to previous models?

Evidence:
raw_evidence: To systematically evaluate the importance of the clinical sentiment values extracted from the free text in EHRs, we first build a baseline model using the structured features, which are similar to prior studies on readmission risk prediction BIBREF6. We then compare two models incorporating the unstructured features. In the "Baseline+Domain Sentences" model, we consider whether adding the counts of sentences per EHR that involve each of the seven risk factor domains as identified by our topic extraction model improved the model performance. In the "Baseline+Clinical Sentiment" model, we evaluate whether adding clinical sentiment scores for each risk factor domain improved the model performance. We also experimented with combining both sets of features and found no additional improvement.

Answers:
Yes
Question: How do they incorporate sentiment analysis?

Evidence:
raw_evidence: These unstructured features include: 1) the relative number of sentences in the admission notes that involve each risk factor domain (out of total number of sentences within the admission) and 2) clinical sentiment scores for each of these risk factor domains, i.e. sentiment scores that evaluate the patient’s psychosocial functioning level (positive, negative, or neutral) with respect to each of these risk factor domain.
These sentiment scores were automatically obtained through the topic extraction and sentiment analysis pipeline introduced in our prior work BIBREF15 and pretrained on in-house psychiatric EHR text. In our paper we also showed that this automatic pipeline achieves reasonably strong F-scores, with an overall performance of 0.828 F1 for the topic extraction component and 0.5 F1 on the clinical sentiment component.
45 clinically interpretable features per admission were extracted as inputs to the readmission risk classifier. These features can be grouped into three categories (See Table TABREF5 for complete list of features):

Answers:
features per admission were extracted as inputs to the readmission risk classifier
Question: What is the dataset used?

Evidence:
raw_evidence: The corpus consists of a collection of 2,346 clinical notes (admission notes, progress notes, and discharge summaries), which amounts to 2,372,323 tokens in total (an average of 1,011 tokens per note). All the notes were written in English and extracted from the EHRs of 183 psychosis patients from McLean Psychiatric Hospital in Belmont, MA, all of whom had in their history at least one instance of 30-day readmission.

Answers:
EHRs of 183 psychosis patients from McLean Psychiatric Hospital in Belmont, MA
Question: How do they extract topics?

Evidence:
raw_evidence: These sentiment scores were automatically obtained through the topic extraction and sentiment analysis pipeline introduced in our prior work BIBREF15 and pretrained on in-house psychiatric EHR text. In our paper we also showed that this automatic pipeline achieves reasonably strong F-scores, with an overall performance of 0.828 F1 for the topic extraction component and 0.5 F1 on the clinical sentiment component.

Answers:
 automatically obtained through the topic extraction and sentiment analysis pipeline introduced in our prior work BIBREF15
Question: What translationese effects are seen in the analysis?

Evidence:
raw_evidence: Notably, automatic translations of TED talks contain more words than the corresponding reference translation, which means that machine-translated texts of this type have also more potential tokens to enter in a coreference relation, and potentially indicating a shining through effect. The same does not happen with the news test set.
We also characterised the originals and translations according to coreference features such as total number of chains and mentions, average chain length and size of the longest chain. We see how NMT translations increase the number of mentions about $30\%$ with respect to human references showing even a more marked explicitation effect than human translations do. As future work, we consider a more detailed comparison of the human and machine translations, and analyse the purpose of the additional mentions added by the NMT systems. It would be also interesting to evaluate of the quality of the automatically computed coreferences chains used for S3.

Answers:
potentially indicating a shining through effect, explicitation effect
Question: What languages are seen in the news and TED datasets?

Evidence:
raw_evidence: As one of our aims is to compare coreference chain properties in automatic translation with those of the source texts and human reference, we derive data from ParCorFull, an English-German corpus annotated with full coreference chains BIBREF46. The corpus contains ca. 160.7 thousand tokens manually annotated with about 14.9 thousand mentions and 4.7 thousand coreference chains. For our analysis, we select a portion of English news texts and TED talks from ParCorFull and translate them with the three NMT systems described in SECREF4 above. As texts considerably differ in their length, we select 17 news texts (494 sentences) and four TED talks (518 sentences). The size (in tokens) of the total data set under analysis – source (src) and human translations (ref) from ParCorFull and the automatic translations produced within this study (S1, S2 and S3) are presented in Table TABREF20.

Answers:
English, German
Question: How are the (possibly incorrect) coreference chains in the MT outputs annotated?

Evidence:
raw_evidence: The English sources and their corresponding human translations into German were already manually annotated for coreference chains. We follow the same scheme as BIBREF47 to annotate the MT outputs with coreference chains. This scheme allows the annotator to define each markable as a certain mention type (pronoun, NP, VP or clause). The mentions can be defined further in terms of their cohesive function (antecedent, anaphoric, cataphoric, comparative, substitution, ellipsis, apposition). Antecedents can either be marked as simple or split or as entity or event. The annotation scheme also includes pronoun type (personal, possessive, demonstrative, reflexive, relative) and modifier types of NPs (possessive, demonstrative, definite article, or none for proper names), see BIBREF46 for details. The mentions referring to the same discourse item are linked between each other. We use the annotation tool MMAX2 BIBREF48 which was also used for the annotation of ParCorFull.
In the next step, chain members are annotated for their correctness. For the incorrect translations of mentions, we include the following error categories: gender, number, case, ambiguous and other. The latter category is open, which means that the annotators can add their own error types during the annotation process. With this, the final typology of errors also considered wrong named entity, wrong word, missing word, wrong syntactic structure, spelling error and addressee reference.

Answers:
allows the annotator to define each markable as a certain mention type (pronoun, NP, VP or clause), The mentions referring to the same discourse item are linked between each other., chain members are annotated for their correctness
Question: Which three neural machine translation systems are analyzed?

Evidence:
raw_evidence: We train three systems (S1, S2 and S3) with the corpora summarised in Table TABREF5. The first two systems are transformer models trained on different amounts of data (6M vs. 18M parallel sentences as seen in the Table). The third system includes a modification to consider the information of full coreference chains throughout a document augmenting the sentence to be translated with this information and it is trained with the same amount of sentence pairs as S1. A variant of the S3 system participated in the news machine translation of the shared task held at WMT 2019 BIBREF43.

Answers:
first two systems are transformer models trained on different amounts of data, The third system includes a modification to consider the information of full coreference chains
Question: Which coreference phenomena are analyzed?

Evidence:
raw_evidence: Therefore, in our analysis, we focus on the chain features related to the phenomena of shining through and explicitation. These features include number of mentions, number of chains, average chain length and the longest chain size. Machine-translated texts are compared to their sources and the corresponding human translations in terms of these features. We expect to find shining through and explicitation effects in automatic translations.

Answers:
shining through, explicitation
Question: What new interesting tasks can be solved based on the uncanny semantic structures of the embedding space?

Evidence:
raw_evidence: Based on our theoretical results, we design a general framework for data exploration on scholarly data by semantic queries on knowledge graph embedding space. The main component in this framework is the conversion between the data exploration tasks and the semantic queries. We first outline the semantic query solutions to some traditional data exploration tasks, such as similar paper prediction and similar author prediction. We then propose a group of new interesting tasks, such as analogy query and analogy browsing, and discuss how they can be used in modern digital libraries.

Answers:
 analogy query, analogy browsing
Question: What are the uncanny semantic structures of the embedding space?

Evidence:
raw_evidence: We mainly concern with the two following structures of the embedding space.
Semantic similarity structure: Semantically similar entities are close to each other in the embedding space, and vice versa. This structure can be identified by a vector similarity measure, such as the dot product between two embedding vectors. The similarity between two embedding vectors is computed as:
Semantic direction structure: There exist semantic directions in the embedding space, by which only one semantic aspect changes while all other aspects stay the same. It can be identified by a vector difference, such as the subtraction between two embedding vectors. The semantic direction between two embedding vectors is computed as:

Answers:
Semantic similarity structure, Semantic direction structure
Question: What is the general framework for data exploration by semantic queries?

Evidence:
raw_evidence: Given the theoretical results, here we design a general framework for scholarly data exploration by using semantic queries on knowledge graph embedding space. Figure FIGREF19 shows the architecture of the proposed framework. There are three main components, namely data processing, task processing, and query processing.
FLOAT SELECTED: Fig. 1. Architecture of the semantic query framework. Eclipse denotes operation, parallelogram denotes resulting data.

Answers:
three main components, namely data processing, task processing, and query processing
Question: What data exploration is supported by the analysis of these semantic structures?

Evidence:
raw_evidence: Task processing: converting data exploration tasks to algebraic operations on the embedding space by following task-specific conversion templates. Some important tasks and their conversion templates are discussed in Section SECREF5.
Query processing: executing semantic query on the embedding space and return results. Note that the algebraic operations on embedding vectors are linear and can be performed in parallel. Therefore, the semantic query is efficient.

Answers:
Task processing: converting data exploration tasks to algebraic operations on the embedding space, Query processing: executing semantic query on the embedding space and return results
Question: what are the existing models they compared with?

Evidence:
raw_evidence: Experiments ::: Baselines
For comparison, we select several public models as baselines including semantic parsing models:
BiDAF BIBREF3, an MRC model which utilizes a bi-directional attention flow network to encode the question and passage;
QANet BIBREF12, which utilizes convolutions and self-attentions as the building blocks of encoders to represent the question and passage;
BERT BIBREF23, a pre-trained bidirectional Transformer-based language model which achieves state-of-the-art performance on lots of public MRC datasets recently;
and numerical MRC models:
NAQANet BIBREF6, a numerical version of QANet model.
NAQANet+, an enhanced version of NAQANet implemented by ourselves, which further considers real number (e.g. “2.5”), richer arithmetic expression, data augmentation, etc. The enhancements are also used in our NumNet model and the details are given in the Appendix.
Syn Dep BIBREF6, the neural semantic parsing model (KDG) BIBREF22 with Stanford dependencies based sentence representations;
OpenIE BIBREF6, KDG with open information extraction based sentence representations;
SRL BIBREF6, KDG with semantic role labeling based sentence representations;
and traditional MRC models:

Answers:
Syn Dep, OpenIE, SRL, BiDAF, QANet, BERT, NAQANet, NAQANet+