qinchuanhui/UDA-QA huggingface dataset:
Question: What was the baseline?

Evidence:
raw_evidence: Classifier selection methodology. To identify the most suitable classifier for classifying the scalars associated with each text, we perform evaluations using the stochastic gradient descent, naive bayes, decision tree, and random forest classifiers. For each classifier, we split the CrowdFlower hate speech dataset into a training/validation set (75%), and a holdout set (25%). We perform 10-fold cross-validation on the training/validation set to identify the best classifier model and parameters (using a grid search). Based on the results of this evaluation, we select a 100-estimator entropy-based splitting random forest model as our classifier. tab:classifiers shows the mean accuracy and F1-score for each evaluated classifier during the 10-fold cross-validation.

Answers:
stochastic gradient descent, naive bayes, decision tree
Question: What was their system's performance?

Evidence:
raw_evidence: Real-world classifier performance. To evaluate real-world performance of our selected classifier (i.e., performance in the absence of model and parameter bias), we perform classification of the holdout set. On this set, our classifier had an accuracy and F1-score of 89.6% and 89.2%, respectively. These results show that in addition to superior accuracy during training and validation, our chosen classifier is also robust against over-fitting.
raw_evidence: Real-world classifier performance. To evaluate real-world performance of our selected classifier (i.e., performance in the absence of model and parameter bias), we perform classification of the holdout set. On this set, our classifier had an accuracy and F1-score of 89.6% and 89.2%, respectively. These results show that in addition to superior accuracy during training and validation, our chosen classifier is also robust against over-fitting.

Answers:
accuracy and F1-score of 89.6% and 89.2%, respectively
accuracy and F1-score of 89.6% and 89.2%, respectively
Question: What other political events are included in the database?

Evidence:
raw_evidence: Offensiveness over time. We find that on average 8.4% of all political comments are offensive compared to 7.8% of all apolitical comments. fig:offensive-speech-timeline illustrates the fraction of offensive political and apolitical comments made during each week in our study. We see that while the fraction of apolitical offensive comments has stayed steady, there has been an increase in the fraction of offensive political comments starting in July 2016. Notably, this increase is observed after the conclusion of the US presidential primaries and during the period of the Democratic and Republican National Conventions and does not reduce even after the conclusion of the US presidential elections held on November 8. Participants in political subreddits were 2.6% more likely to observe offensive comments prior to July 2016 but 14.9% more likely to observe offensive comments from July 2016 onwards.

Answers:
US presidential primaries, Democratic and Republican National Conventions
Question: What classifier did they use?

Evidence:
raw_evidence: Text transformation and classification. Finally, we transform text to be classified into scalars representing their distance from the constructed hate vector and use these as input to a Random Forest classifier.

Answers:
Random Forest
Question: What labels for antisocial events are available in datasets?

Evidence:
raw_evidence: We consider two datasets, representing related but slightly different forecasting tasks. The first dataset is an expanded version of the annotated Wikipedia conversations dataset from BIBREF9. This dataset uses carefully-controlled crowdsourced labels, strictly filtered to ensure the conversations are civil up to the moment of a personal attack. This is a useful property for the purposes of model analysis, and hence we focus on this as our primary dataset. However, we are conscious of the possibility that these strict labels may not fully capture the kind of behavior that moderators care about in practice. We therefore introduce a secondary dataset, constructed from the subreddit ChangeMyView (CMV) that does not use post-hoc annotations. Instead, the prediction task is to forecast whether the conversation will be subject to moderator action in the future.
Wikipedia data. BIBREF9's `Conversations Gone Awry' dataset consists of 1,270 conversations that took place between Wikipedia editors on publicly accessible talk pages. The conversations are sourced from the WikiConv dataset BIBREF59 and labeled by crowdworkers as either containing a personal attack from within (i.e., hostile behavior by one user in the conversation directed towards another) or remaining civil throughout.
Reddit CMV data. The CMV dataset is constructed from conversations collected via the Reddit API. In contrast to the Wikipedia-based dataset, we explicitly avoid the use of post-hoc annotation. Instead, we use as our label whether a conversation eventually had a comment removed by a moderator for violation of Rule 2: “Don't be rude or hostile to other users”.

Answers:
The Conversations Gone Awry dataset is labelled as either containing a personal attack from withint (i.e. hostile behavior by one user in the conversation directed towards another) or remaining civil throughout. The Reddit Change My View dataset is labelled with whether or not a coversation eventually had a comment removed by a moderator for violation of Rule 2: "Don't be rude or hostile to others users."
Question: What are two datasets model is applied to?

Evidence:
raw_evidence: To test the effectiveness of this new architecture in forecasting derailment of online conversations, we develop and distribute two new datasets. The first triples in size the highly curated `Conversations Gone Awry' dataset BIBREF9, where civil-starting Wikipedia Talk Page conversations are crowd-labeled according to whether they eventually lead to personal attacks; the second relies on in-the-wild moderation of the popular subreddit ChangeMyView, where the aim is to forecast whether a discussion will later be subject to moderator action due to “rude or hostile” behavior. In both datasets, our model outperforms existing fixed-window approaches, as well as simpler sequential baselines that cannot account for inter-comment relations. Furthermore, by virtue of its online processing of the conversation, our system can provide substantial prior notice of upcoming derailment, triggering on average 3 comments (or 3 hours) before an overtly toxic comment is posted.
raw_evidence: We consider two datasets, representing related but slightly different forecasting tasks. The first dataset is an expanded version of the annotated Wikipedia conversations dataset from BIBREF9. This dataset uses carefully-controlled crowdsourced labels, strictly filtered to ensure the conversations are civil up to the moment of a personal attack. This is a useful property for the purposes of model analysis, and hence we focus on this as our primary dataset. However, we are conscious of the possibility that these strict labels may not fully capture the kind of behavior that moderators care about in practice. We therefore introduce a secondary dataset, constructed from the subreddit ChangeMyView (CMV) that does not use post-hoc annotations. Instead, the prediction task is to forecast whether the conversation will be subject to moderator action in the future.
To test the effectiveness of this new architecture in forecasting derailment of online conversations, we develop and distribute two new datasets. The first triples in size the highly curated `Conversations Gone Awry' dataset BIBREF9, where civil-starting Wikipedia Talk Page conversations are crowd-labeled according to whether they eventually lead to personal attacks; the second relies on in-the-wild moderation of the popular subreddit ChangeMyView, where the aim is to forecast whether a discussion will later be subject to moderator action due to “rude or hostile” behavior. In both datasets, our model outperforms existing fixed-window approaches, as well as simpler sequential baselines that cannot account for inter-comment relations. Furthermore, by virtue of its online processing of the conversation, our system can provide substantial prior notice of upcoming derailment, triggering on average 3 comments (or 3 hours) before an overtly toxic comment is posted.

Answers:
 `Conversations Gone Awry' dataset, subreddit ChangeMyView
An expanded version of the existing 'Conversations Gone Awry' dataset and the ChangeMyView dataset, a subreddit whose only annotation is whether the conversation required action by the Reddit moderators.
Question: What is the CORD-19 dataset?

Evidence:
raw_evidence: We aim to address this issue. Our research goal is to develop natural language processing methods to collectively analyze the study results reported by many hospitals and medical institutes all over the world, reconcile these results, and make a holistic and unbiased conclusion regarding the correlation between radiological findings and COVID-19. Specifically, we take the CORD-19 dataset BIBREF2, which contains over 45,000 scholarly articles, including over 33,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses. We develop sentence classification methods to identify all sentences narrating radiological findings from COVID-19. Then constituent parsing is utilized to identify all noun phrases from these sentences and these noun phrases contain abnormalities, lesions, diseases identified by radiology imaging such as X-ray and computed tomography (CT). We calculate the frequency of these noun phrases and select those with top frequencies for medical professionals to further investigate. Since these clinical entities are aggregated from a number of hospitals all over the world, the population bias is largely mitigated and the conclusions are more objective and universally informative. From the CORD-19 dataset, our method successfully discovers a set of clinical findings that are closely related with COVID-19.
raw_evidence: We aim to address this issue. Our research goal is to develop natural language processing methods to collectively analyze the study results reported by many hospitals and medical institutes all over the world, reconcile these results, and make a holistic and unbiased conclusion regarding the correlation between radiological findings and COVID-19. Specifically, we take the CORD-19 dataset BIBREF2, which contains over 45,000 scholarly articles, including over 33,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses. We develop sentence classification methods to identify all sentences narrating radiological findings from COVID-19. Then constituent parsing is utilized to identify all noun phrases from these sentences and these noun phrases contain abnormalities, lesions, diseases identified by radiology imaging such as X-ray and computed tomography (CT). We calculate the frequency of these noun phrases and select those with top frequencies for medical professionals to further investigate. Since these clinical entities are aggregated from a number of hospitals all over the world, the population bias is largely mitigated and the conclusions are more objective and universally informative. From the CORD-19 dataset, our method successfully discovers a set of clinical findings that are closely related with COVID-19.

Answers:
which contains over 45,000 scholarly articles, including over 33,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses
contains over 45,000 scholarly articles, including over 33,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses
Question: How large is the collection of COVID-19 literature?

Evidence:
raw_evidence: We aim to address this issue. Our research goal is to develop natural language processing methods to collectively analyze the study results reported by many hospitals and medical institutes all over the world, reconcile these results, and make a holistic and unbiased conclusion regarding the correlation between radiological findings and COVID-19. Specifically, we take the CORD-19 dataset BIBREF2, which contains over 45,000 scholarly articles, including over 33,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses. We develop sentence classification methods to identify all sentences narrating radiological findings from COVID-19. Then constituent parsing is utilized to identify all noun phrases from these sentences and these noun phrases contain abnormalities, lesions, diseases identified by radiology imaging such as X-ray and computed tomography (CT). We calculate the frequency of these noun phrases and select those with top frequencies for medical professionals to further investigate. Since these clinical entities are aggregated from a number of hospitals all over the world, the population bias is largely mitigated and the conclusions are more objective and universally informative. From the CORD-19 dataset, our method successfully discovers a set of clinical findings that are closely related with COVID-19.
We conduct experiments to verify the effectiveness of our method. From the CORD-19 dataset, our method successfully discovers a set of clinical findings that are closely related with COVID-19.

Answers:
45,000 scholarly articles, including over 33,000 with full text
Question: Which deep learning architecture do they use for sentence segmentation?

Evidence:
raw_evidence: The CRF module achieved the best result on the Thai sentence segmentation task BIBREF8 ; therefore, we adopt the Bi-LSTM-CRF model as our baseline. This paper makes the following three contributions to improve Bi-LSTM-CRF for sentence segmentation.
raw_evidence: Several deep learning approaches have been applied in various tasks of natural language processing (NLP), including the long short-term memory BIBREF10 , self-attention BIBREF11 , and other models. Huang Z. et al. BIBREF12 proposed a deep learning sequence tagging model called Bi-LSTM-CRF, which integrates a conditional random field (CRF) module to gain the benefit of both deep learning and traditional machine learning approaches. In their experiments, the Bi-LSTM-CRF model achieved an improved level of accuracy in many NLP sequence tagging tasks, such as named entity recognition, POS tagging and chunking.
The CRF module achieved the best result on the Thai sentence segmentation task BIBREF8 ; therefore, we adopt the Bi-LSTM-CRF model as our baseline. This paper makes the following three contributions to improve Bi-LSTM-CRF for sentence segmentation.

Answers:
Bi-LSTM-CRF
Bi-LSTM-CRF
Question: What is the McGurk effect?

Evidence:
raw_evidence: For the McGurk effect, we attempt an illusion for a language token (e.g. phoneme, word, sentence) $x$ by creating a video where an audio stream of $x$ is visually dubbed over by a person saying $x^{\prime }\ne x$ . We stress that the audio portion of the illusion is not modified and corresponds to a person saying $x$ . The illusion $f(x^{\prime },x)$ affects a listener if they perceive what is being said to be $y\ne x$ if they watched the illusory video whereas they perceive $x$ if they had either listened to the audio stream without watching the video or had watched the original unaltered video, depending on specification. We call a token illusionable if an illusion can be made for the token that affects the perception of a significant fraction of people.
In this work, we attempt to understand how susceptible humans' perceptual systems for natural speech are to carefully designed “adversarial attacks.” We investigate the density of certain classes of illusion, that is, the fraction of natural language utterances whose comprehension can be affected by the illusion. Our study centers around the McGurk effect, which is the well-studied phenomenon by which the perception of what we hear can be influenced by what we see BIBREF0 . A prototypical example is that the audio of the phoneme “baa,” accompanied by a video of someone mouthing “vaa”, can be perceived as “vaa” or “gaa” (Figure 1 ). This effect persists even when the subject is aware of the setup, though the strength of the effect varies significantly across people and languages and with factors such as age, gender, and disorders BIBREF11 , BIBREF12 , BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 , BIBREF17 , BIBREF18 , BIBREF19 .
raw_evidence: In this work, we attempt to understand how susceptible humans' perceptual systems for natural speech are to carefully designed “adversarial attacks.” We investigate the density of certain classes of illusion, that is, the fraction of natural language utterances whose comprehension can be affected by the illusion. Our study centers around the McGurk effect, which is the well-studied phenomenon by which the perception of what we hear can be influenced by what we see BIBREF0 . A prototypical example is that the audio of the phoneme “baa,” accompanied by a video of someone mouthing “vaa”, can be perceived as “vaa” or “gaa” (Figure 1 ). This effect persists even when the subject is aware of the setup, though the strength of the effect varies significantly across people and languages and with factors such as age, gender, and disorders BIBREF11 , BIBREF12 , BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 , BIBREF17 , BIBREF18 , BIBREF19 .

Answers:
a perceptual illusion, where listening to a speech sound while watching a mouth pronounce a different sound changes how the audio is heard
When the perception of what we hear is influenced by what we see.
Question: what was the baseline?

Evidence:
raw_evidence: We use the publicly available OpenSubtitles2018 corpus BIBREF12 for English and Russian. For a fair comparison with previous work, we train the baseline MT system on the data released by BIBREF11. Namely, our MT system is trained on 6m instances. These are sentence pairs with a relative time overlap of subtitle frames between source and target language subtitles of at least $0.9$.
raw_evidence: The baseline model, the model used for back-translation, and the DocRepair model are all Transformer base models BIBREF15. More precisely, the number of layers is $N=6$ with $h = 8$ parallel attention layers, or heads. The dimensionality of input and output is $d_{model} = 512$, and the inner-layer of a feed-forward networks has dimensionality $d_{ff}=2048$. We use regularization as described in BIBREF15.
As a second baseline, we use the two-pass CADec model BIBREF11. The first pass produces sentence-level translations. The second pass takes both the first-pass translation and representations of the context sentences as input and returns contextualized translations. CADec requires document-level parallel training data, while DocRepair only needs monolingual training data.

Answers:
 MT system on the data released by BIBREF11
Transformer base, two-pass CADec model
Question: what phenomena do they mention is hard to capture?

Evidence:
raw_evidence: We analyze which discourse phenomena are hard to capture using monolingual data only. Using contrastive test sets for targeted evaluation of several contextual phenomena, we compare the performance of the models trained on round-trip translations and genuine document-level parallel data. Among the four phenomena in the test sets we use (deixis, lexical cohesion, VP ellipsis and ellipsis which affects NP inflection) we find VP ellipsis to be the hardest phenomenon to be captured using round-trip translations.

Answers:
Four discourse phenomena - deixis, lexical cohesion, VP ellipsis, and ellipsis which affects NP inflection.
Question: What is NER?

Evidence:
raw_evidence: Named Entity Recognition (NER) in the Biomedical domain usually includes recognition of entities such as proteins, genes, diseases, treatments, drugs, etc. Fact extraction involves extraction of Named Entities from a corpus, usually given a certain ontology. When compared to NER in the domain of general text, the biomedical domain has some characteristic challenges:
raw_evidence: Named Entity Recognition (NER) in the Biomedical domain usually includes recognition of entities such as proteins, genes, diseases, treatments, drugs, etc. Fact extraction involves extraction of Named Entities from a corpus, usually given a certain ontology. When compared to NER in the domain of general text, the biomedical domain has some characteristic challenges:

Answers:
Named Entity Recognition
Named Entity Recognition, including entities such as proteins, genes, diseases, treatments, drugs, etc. in the biomedical domain
Question: Does the paper explore extraction from electronic health records?

Evidence:
raw_evidence: BIBREF5 adopt a machine learning approach for NER. Their NER system extracts medical problems, tests and treatments from discharge summaries and progress notes. They use a semi-Conditional Random Field (semi-CRF) BIBREF6 to output labels over all tokens in the sentence. They use a variety of token, context and sentence level features. They also use some concept mapping features using existing annotation tools, as well as Brown clustering to form 128 clusters over the unlabelled data. The dataset used is the i2b2 2010 challenge dataset. Their system achieves an F-Score of 0.85. BIBREF7 is an incremental paper on NER taggers. It uses 3 types of word-representation techniques (Brown clustering, distributional clustering, word vectors) to improve performance of the NER Conditional Random Field tagger, and achieves marginal F-Score improvements.

Answers:
Yes
Question: Does jiant involve datasets for the 50 NLU tasks?

Evidence:
raw_evidence: jiant System Overview ::: jiant Components
Tasks: Tasks have references to task data, methods for processing data, references to classifier heads, and methods for calculating performance metrics, and making predictions.

Answers:
Yes
Question: Is jiant compatible with models in any programming language?

Evidence:
raw_evidence: jiant provides support for cutting-edge sentence encoder models, including support for Huggingface's Transformers. Supported models include: BERT BIBREF17, RoBERTa BIBREF27, XLNet BIBREF28, XLM BIBREF29, GPT BIBREF30, GPT-2 BIBREF31, ALBERT BIBREF32 and ELMo BIBREF33. jiant also supports the from-scratch training of (bidirectional) LSTMs BIBREF34 and deep bag of words models BIBREF35, as well as syntax-aware models such as PRPN BIBREF36 and ON-LSTM BIBREF37. jiant also supports word embeddings such as GloVe BIBREF38.

Answers:
Yes