prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What are two datasets model is applied to?
Context 1: Introduction
“Ché saetta previsa vien più lenta ”
– Dante Alighieri, Divina Commedia, Paradiso
Antisocial behavior is a persistent problem plaguing online conversation platforms; it is both widespread BIBREF0 and potentially damaging to mental and emotional health BIBREF1, BIBREF2  The strain this phenomenon puts on community maintainers has sparked recent interest in computational approaches for assisting human moderators 
Prior work in this direction has largely focused on post-hoc identification of various kinds of antisocial behavior, including hate speech BIBREF3, BIBREF4, harassment BIBREF5, personal attacks BIBREF6, and general toxicity BIBREF7  The fact that these approaches only identify antisocial content after the fact limits their practicality as tools for assisting pre-emptive moderation in conversational domains 
Addressing this limitation requires forecasting the future derailment of a conversation based on early warning signs, giving the moderators time to potentially intervene before any harm is done (BIBREF8 BIBREF8, BIBREF9 BIBREF9, see BIBREF10 BIBREF10 for a discussion)  Such a goal recognizes derailment as emerging from the development of the conversation, and belongs to the broader area of conversational forecasting, which includes future-prediction tasks such as predicting the eventual length of a conversation BIBREF11, whether a persuasion attempt will eventually succeed BIBREF12, BIBREF13, BIBREF14, whether team discussions will eventually lead to an increase in performance BIBREF15, or whether ongoing counseling conversations will eventually be perceived as helpful BIBREF16 
Approaching such conversational forecasting problems, however, requires overcoming several inherent modeling challenges  First, conversations are dynamic and their outcome might depend on how subsequent comments interact with each other  Consider the example in Figure FIGREF2: while no individual comment is outright offensive, a human reader can sense a tension emerging from their succession (e g , dismissive answers to repeated questioning)  Thus a forecasting model needs to capture not only the content of each individual comment, but also the relations between comments  Previous work has largely relied on hand-crafted features to capture such relations—e g , similarity between comments BIBREF16, BIBREF12 or conversation structure BIBREF17, BIBREF18—, though neural attention architectures have also recently shown promise BIBREF19 
The second modeling challenge stems from the fact that conversations have an unknown horizon: they can be of varying lengths, and the to-be-forecasted event can occur at any time  So when is it a good time to make a forecast? Prior work has largely proposed two solutions, both resulting in important practical limitations  One solution is to assume (unrealistic) prior knowledge of when the to-be-forecasted event takes place and extract features up to that point BIBREF20, BIBREF8  Another compromising solution is to extract features from a fixed-length window, often at the start of the conversation BIBREF21, BIBREF15, BIBREF16, BIBREF9  Choosing a catch-all window-size is however impractical: short windows will miss information in comments they do not encompass (e g , a window of only two comments would miss the chain of repeated questioning in comments 3 through 6 of Figure FIGREF2), while longer windows risk missing the to-be-forecasted event altogether if it occurs before the end of the window, which would prevent early detection 
In this work we introduce a model for forecasting conversational events that overcomes both these inherent challenges by processing comments, and their relations, as they happen (i e , in an online fashion)  Our main insight is that models with these properties already exist, albeit geared toward generation rather than prediction: recent work in context-aware dialog generation (or “chatbots”) has proposed sequential neural models that make effective use of the intra-conversational dynamics BIBREF22, BIBREF23, BIBREF24, while concomitantly being able to process the conversation as it develops (see BIBREF25 for a survey) 
In order for these systems to perform well in the generative domain they need to be trained on massive amounts of (unlabeled) conversational data  The main difficulty in directly adapting these models to the supervised domain of conversational forecasting is the relative scarcity of labeled data: for most forecasting tasks, at most a few thousands labeled examples are available, insufficient for the notoriously data-hungry sequential neural models 
To overcome this difficulty, we propose to decouple the objective of learning a neural representation of conversational dynamics from the objective of predicting future events  The former can be pre-trained on large amounts of unsupervised data, similarly to how chatbots are trained  The latter can piggy-back on the resulting representation after fine-tuning it for classification using relatively small labeled data  While similar pre-train-then-fine-tune approaches have recently achieved state-of-the-art performance in a number of NLP tasks—including natural language inference, question answering, and commonsense reasoning (discussed in Section SECREF2)—to the best of our knowledge this is the first attempt at applying this paradigm to conversational forecasting 
To test the effectiveness of this new architecture in forecasting derailment of online conversations, we develop and distribute two new datasets  The first triples in size the highly curated `Conversations Gone Awry' dataset BIBREF9, where civil-starting Wikipedia Talk Page conversations are crowd-labeled according to whether they eventually lead to personal attacks; the second relies on in-the-wild moderation of the popular subreddit ChangeMyView, where the aim is to forecast whether a discussion will later be subject to moderator action due to “rude or hostile” behavior  In both datasets, our model outperforms existing fixed-window approaches, as well as simpler sequential baselines that cannot account for inter-comment relations  Furthermore, by virtue of its online processing of the conversation, our system can provide substantial prior notice of upcoming derailment, triggering on average 3 comments (or 3 hours) before an overtly toxic comment is posted 
To summarize, in this work we:
introduce the first model for forecasting conversational events that can capture the dynamics of a conversation as it develops;
build two diverse datasets (one entirely new, one extending prior work) for the task of forecasting derailment of online conversations;
compare the performance of our model against the current state-of-the-art, and evaluate its ability to provide early warning signs 
Our work is motivated by the goal of assisting human moderators of online communities by preemptively signaling at-risk conversations that might deserve their attention  However, we caution that any automated systems might encode or even amplify the biases existing in the training data BIBREF26, BIBREF27, BIBREF28, so a public-facing implementation would need to be exhaustively scrutinized for such biases BIBREF29 
Further Related Work
Antisocial behavior  Antisocial behavior online comes in many forms, including harassment BIBREF30, cyberbullying BIBREF31, and general aggression BIBREF32  Prior work has sought to understand different aspects of such behavior, including its effect on the communities where it happens BIBREF33, BIBREF34, the actors involved BIBREF35, BIBREF36, BIBREF37, BIBREF38 and connections to the outside world BIBREF39 
Post-hoc classification of conversations  There is a rich body of prior work on classifying the outcome of a conversation after it has concluded, or classifying conversational events after they happened  Many examples exist, but some more closely related to our present work include identifying the winner of a debate BIBREF40, BIBREF41, BIBREF42, identifying successful negotiations BIBREF21, BIBREF43, as well as detecting whether deception BIBREF44, BIBREF45, BIBREF46 or disagreement BIBREF47, BIBREF48, BIBREF49, BIBREF50, BIBREF51 has occurred 
Our goal is different because we wish to forecast conversational events before they happen and while the conversation is still ongoing (potentially allowing for interventions)  Note that some post-hoc tasks can also be re-framed as forecasting tasks (assuming the existence of necessary labels); for instance, predicting whether an ongoing conversation will eventually spark disagreement BIBREF18, rather than detecting already-existing disagreement 
Conversational forecasting  As described in Section SECREF1, prior work on forecasting conversational outcomes and events has largely relied on hand-crafted features to capture aspects of conversational dynamics  Example feature sets include statistical measures based on similarity between utterances BIBREF16, sentiment imbalance BIBREF20, flow of ideas BIBREF20, increase in hostility BIBREF8, reply rate BIBREF11 and graph representations of conversations BIBREF52, BIBREF17  By contrast, we aim to automatically learn neural representations of conversational dynamics through pre-training 
Such hand-crafted features are typically extracted from fixed-length windows of the conversation, leaving unaddressed the problem of unknown horizon  While some work has trained multiple models for different window-lengths BIBREF8, BIBREF18, they consider these models to be independent and, as such, do not address the issue of aggregating them into a single forecast (i e , deciding at what point to make a prediction)  We implement a simple sliding windows solution as a baseline (Section SECREF5) 
Pre-training for NLP  The use of pre-training for natural language tasks has been growing in popularity after recent breakthroughs demonstrating improved performance on a wide array of benchmark tasks BIBREF53, BIBREF54  Existing work has generally used a language modeling objective as the pre-training objective; examples include next-word prediction BIBREF55, sentence autoencoding, BIBREF56, and machine translation BIBREF57  BERT BIBREF58 introduces a variation on this in which the goal is to predict the next sentence in a document given the current sentence  Our pre-training objective is similar in spirit, but operates at a conversation level, rather than a document level  We hence view our objective as conversational modeling rather than (only) language modeling  Furthermore, while BERT's sentence prediction objective is framed as a multiple-choice task, our objective is framed as a generative task 
Derailment Datasets
We consider two datasets, representing related but slightly different forecasting tasks  The first dataset is an expanded version of the annotated Wikipedia conversations dataset from BIBREF9  This dataset uses carefully-controlled crowdsourced labels, strictly filtered to ensure the conversations are civil up to the moment of a personal attack  This is a useful property for the purposes of model analysis, and hence we focus on this as our primary dataset  However, we are conscious of the possibility that these strict labels may not fully capture the kind of behavior that moderators care about in practice  We therefore introduce a secondary dataset, constructed from the subreddit ChangeMyView (CMV) that does not use post-hoc annotations  Instead, the prediction task is to forecast whether the conversation will be subject to moderator action in the future 
Wikipedia data  BIBREF9's `Conversations Gone Awry' dataset consists of 1,270 conversations that took place between Wikipedia editors on publicly accessible talk pages  The conversations are sourced from the WikiConv dataset BIBREF59 and labeled by crowdworkers as either containing a personal attack from within (i e , hostile behavior by one user in the conversation directed towards another) or remaining civil throughout 
A series of controls are implemented to prevent models from picking up on trivial correlations  To prevent models from capturing topic-specific information (e g , political conversations are more likely to derail), each attack-containing conversation is paired with a clean conversation from the same talk page, where the talk page serves as a proxy for topic  To force models to actually capture conversational dynamics rather than detecting already-existing toxicity, human annotations are used to ensure that all comments preceding a personal attack are civil 
To the ends of more effective model training, we elected to expand the `Conversations Gone Awry' dataset, using the original annotation procedure  Since we found that the original data skewed towards shorter conversations, we focused this crowdsourcing run on longer conversations: ones with 4 or more comments preceding the attack  Through this additional crowdsourcing, we expand the dataset to 4,188 conversations, which we are publicly releasing as part of the Cornell Conversational Analysis Toolkit (ConvoKit) 
We perform an 80-20-20 train/dev/test split, ensuring that paired conversations end up in the same split in order to preserve the topic control  Finally, we randomly sample another 1 million conversations from WikiConv to use for the unsupervised pre-training of the generative component 
Reddit CMV data  The CMV dataset is constructed from conversations collected via the Reddit API  In contrast to the Wikipedia-based dataset, we explicitly avoid the use of post-hoc annotation  Instead, we use as our label whether a conversation eventually had a comment removed by a moderator for violation of Rule 2: “Don't be rude or hostile to other users” 
Though the lack of post-hoc annotation limits the degree to which we can impose controls on the data (e g , some conversations may contain toxic comments not flagged by the moderators) we do reproduce as many of the Wikipedia data's controls as we can  Namely, we replicate the topic control pairing by choosing pairs of positive and negative examples that belong to the same top-level post, following BIBREF12; and enforce that the removed comment was made by a user who was previously involved in the conversation  This process results in 6,842 conversations, to which we again apply a pair-preserving 80-20-20 split  Finally, we gather over 600,000 conversations that do not include any removed comment, for unsupervised pre-training 
Conversational Forecasting Model
We now describe our general model for forecasting future conversational events  Our model integrates two components: (a) a generative dialog model that learns to represent conversational dynamics in an unsupervised fashion; and (b) a supervised component that fine-tunes this representation to forecast future events  Figure FIGREF13 provides an overview of the proposed architecture, henceforth CRAFT (Conversational Recurrent Architecture for ForecasTing) 
Terminology  For modeling purposes, we treat a conversation as a sequence of $N$ comments $C = \lbrace c_1,\dots ,c_N\rbrace $  Each comment, in turn, is a sequence of tokens, where the number of tokens may vary from comment to comment  For the $n$-th comment ($1 \le n \le N)$, we let $M_n$ denote the number of tokens  Then, a comment $c_n$ can be represented as a sequence of $M_n$ tokens: $c_n = \lbrace w_1,\dots ,w_{M_n}\rbrace $ 
Generative component  For the generative component of our model, we use a hierarchical recurrent encoder-decoder (HRED) architecture BIBREF60, a modified version of the popular sequence-to-sequence (seq2seq) architecture BIBREF61 designed to account for dependencies between consecutive inputs  BIBREF23 showed that HRED can successfully model conversational context by encoding the temporal structure of previously seen comments, making it an ideal fit for our use case  Here, we provide a high-level summary of the HRED architecture, deferring deeper technical discussion to BIBREF60 and BIBREF23 
An HRED dialog model consists of three components: an utterance encoder, a context encoder, and a decoder  The utterance encoder is responsible for generating semantic vector representations of comments  It consists of a recurrent neural network (RNN) that reads a comment token-by-token, and on each token $w_m$ updates a hidden state $h^{\text{enc}}$ based on the current token and the previous hidden state:
where $f^{\text{RNN}}$ is a nonlinear gating function (our implementation uses GRU BIBREF62)  The final hidden state $h^{\text{enc}}_M$ can be viewed as a vector encoding of the entire comment 
Running the encoder on each comment $c_n$ results in a sequence of $N$ vector encodings  A second encoder, the context encoder, is then run over this sequence:
Each hidden state $h^{\text{con}}_n$ can then be viewed as an encoding of the full conversational context up to and including the $n$-th comment  To generate a response to comment $n$, the context encoding $h^{\text{con}}_n$ is used to initialize the hidden state $h^{\text{dec}}_{0}$ of a decoder RNN  The decoder produces a response token by token using the following recurrence:
where $f^{\text{out}}$ is some function that outputs a probability distribution over words; we implement this using a simple feedforward layer  In our implementation, we further augment the decoder with attention BIBREF63, BIBREF64 over context encoder states to help capture long-term inter-comment dependencies  This generative component can be pre-trained using unlabeled conversational data 
Prediction component  Given a pre-trained HRED dialog model, we aim to extend the model to predict from the conversational context whether the to-be-forecasted event will occur  Our predictor consists of a multilayer perceptron (MLP) with 3 fully-connected layers, leaky ReLU activations between layers, and sigmoid activation for output  For each comment $c_n$, the predictor takes as input the context encoding $h^{\text{con}}_n$ and forwards it through the MLP layers, resulting in an output score that is interpreted as a probability $p_{\text{event}}(c_{n+1})$ that the to-be-forecasted event will happen (e g , that the conversation will derail) 
Training the predictive component starts by initializing the weights of the encoders to the values learned in pre-training  The main training loop then works as follows: for each positive sample—i e , a conversation containing an instance of the to-be-forecasted event (e g , derailment) at comment $c_e$—we feed the context $c_1,\dots ,c_{e-1}$ through the encoder and classifier, and compute cross-entropy loss between the classifier output and expected output of 1  Similarly, for each negative sample—i e , a conversation where none of the comments exhibit the to-be-forecasted event and that ends with $c_N$—we feed the context $c_1,\dots ,c_{N-1}$ through the model and compute loss against an expected output of 0 
Note that the parameters of the generative component are not held fixed during this process; instead, backpropagation is allowed to go all the way through the encoder layers  This process, known as fine-tuning, reshapes the representation learned during pre-training to be more directly useful to prediction BIBREF55 
We implement the model and training code using PyTorch, and we are publicly releasing our implementation and the trained models together with the data as part of ConvoKit 
Forecasting Derailment
We evaluate the performance of CRAFT in the task of forecasting conversational derailment in both the Wikipedia and CMV scenarios  To this end, for each of these datasets we pre-train the generative component on the unlabeled portion of the data and fine-tune it on the labeled training split (data size detailed in Section SECREF3) 
In order to evaluate our sequential system against conversational-level ground truth, we need to aggregate comment level predictions  If any comment in the conversation triggers a positive prediction—i e , $p_{\text{event}}(c_{n+1})$ is greater than a threshold learned on the development split—then the respective conversation is predicted to derail  If this forecast is triggered in a conversation that actually derails, but before the derailment actually happens, then the conversation is counted as a true positive; otherwise it is a false positive  If no positive predictions are triggered for a conversation, but it actually derails then it counts as a false negative; if it does not derail then it is a true negative 
Fixed-length window baselines  We first seek to compare CRAFT to existing, fixed-length window approaches to forecasting  To this end, we implement two such baselines: Awry, which is the state-of-the-art method proposed in BIBREF9 based on pragmatic features in the first comment-reply pair, and BoW, a simple bag-of-words baseline that makes a prediction using TF-IDF weighted bag-of-words features extracted from the first comment-reply pair 
Online forecasting baselines  Next, we consider simpler approaches for making forecasts as the conversations happen (i e , in an online fashion)  First, we propose Cumulative BoW, a model that recomputes bag-of-words features on all comments seen thus far every time a new comment arrives  While this approach does exhibit the desired behavior of producing updated predictions for each new comment, it fails to account for relationships between comments 
This simple cumulative approach cannot be directly extended to models whose features are strictly based on a fixed number of comments, like Awry  An alternative is to use a sliding window: for a feature set based on a window of $W$ comments, upon each new comment we can extract features from a window containing that comment and the $W-1$ comments preceding it  We apply this to the Awry method and call this model Sliding Awry  For both these baselines, we aggregate comment-level predictions in the same way as in our main model 
CRAFT ablations  Finally, we consider two modified versions of the CRAFT model in order to evaluate the impact of two of its key components: (1) the pre-training step, and (2) its ability to capture inter-comment dependencies through its hierarchical memory 
To evaluate the impact of pre-training, we train the prediction component of CRAFT on only the labeled training data, without first pre-training the encoder layers with the unlabeled data  We find that given the relatively small size of labeled data, this baseline fails to successfully learn, and ends up performing at the level of random guessing  This result underscores the need for the pre-training step that can make use of unlabeled data 
To evaluate the impact of the hierarchical memory, we implement a simplified version of CRAFT where the memory size of the context encoder is zero (CRAFT $-$ CE), thus effectively acting as if the pre-training component is a vanilla seq2seq model  In other words, this model cannot capture inter-comment dependencies, and instead at each step makes a prediction based only on the utterance encoding of the latest comment 
Results  Table TABREF17 compares CRAFT to the baselines on the test splits (random baseline is 50%) and illustrates several key findings  First, we find that unsurprisingly, accounting for full conversational context is indeed helpful, with even the simple online baselines outperforming the fixed-window baselines  On both datasets, CRAFT outperforms all baselines (including the other online models) in terms of accuracy and F1  Furthermore, although it loses on precision (to CRAFT $-$ CE) and recall (to Cumulative BoW) individually on the Wikipedia data, CRAFT has the superior balance between the two, having both a visibly higher precision-recall curve and larger area under the curve (AUPR) than the baselines (Figure FIGREF20)  This latter property is particularly useful in a practical setting, as it allows moderators to tune model performance to some desired precision without having to sacrifice as much in the way of recall (or vice versa) compared to the baselines and pre-existing solutions 
Analysis
We now examine the behavior of CRAFT in greater detail, to better understand its benefits and limitations  We specifically address the following questions: (1) How much early warning does the the model provide? (2) Does the model actually learn an order-sensitive representation of conversational context?
Early warning, but how early? The recent interest in forecasting antisocial behavior has been driven by a desire to provide pre-emptive, actionable warning to moderators  But does our model trigger early enough for any such practical goals?
For each personal attack correctly forecasted by our model, we count the number of comments elapsed between the time the model is first triggered and the attack  Figure FIGREF22 shows the distribution of these counts: on average, the model warns of an attack 3 comments before it actually happens (4 comments for CMV)  To further evaluate how much time this early warning would give to the moderator, we also consider the difference in timestamps between the comment where the model first triggers and the comment containing the actual attack  Over 50% of conversations get at least 3 hours of advance warning (2 hours for CMV)  Moreover, 39% of conversations get at least 12 hours of early warning before they derail 
Does order matter? One motivation behind the design of our model was the intuition that comments in a conversation are not independent events; rather, the order in which they appear matters (e g , a blunt comment followed by a polite one feels intuitively different from a polite comment followed by a blunt one)  By design, CRAFT has the capacity to learn an order-sensitive representation of conversational context, but how can we know that this capacity is actually used? It is conceivable that the model is simply computing an order-insensitive “bag-of-features”  Neural network models are notorious for their lack of transparency, precluding an analysis of how exactly CRAFT models conversational context  Nevertheless, through two simple exploratory experiments, we seek to show that it does not completely ignore comment order 
The first experiment for testing whether the model accounts for comment order is a prefix-shuffling experiment, visualized in Figure FIGREF23  For each conversation that the model predicts will derail, let $t$ denote the index of the triggering comment, i e , the index where the model first made a derailment forecast  We then construct synthetic conversations by taking the first $t-1$ comments (henceforth referred to as the prefix) and randomizing their order  Finally, we count how often the model no longer predicts derailment at index $t$ in the synthetic conversations  If the model were ignoring comment order, its prediction should remain unchanged (as it remains for the Cumulative BoW baseline), since the actual content of the first $t$ comments has not changed (and CRAFT inference is deterministic)  We instead find that in roughly one fifth of cases (12% for CMV) the model changes its prediction on the synthetic conversations  This suggests that CRAFT learns an order-sensitive representation of context, not a mere “bag-of-features” 
To more concretely quantify how much this order-sensitive context modeling helps with prediction, we can actively prevent the model from learning and exploiting any order-related dynamics  We achieve this through another type of shuffling experiment, where we go back even further and shuffle the comment order in the conversations used for pre-training, fine-tuning and testing  This procedure preserves the model's ability to capture signals present within the individual comments processed so far, as the utterance encoder is unaffected, but inhibits it from capturing any meaningful order-sensitive dynamics  We find that this hurts the model's performance (65% accuracy for Wikipedia, 59 5% for CMV), lowering it to a level similar to that of the version where we completely disable the context encoder 
Taken together, these experiments provide evidence that CRAFT uses its capacity to model conversational context in an order-sensitive fashion, and that it makes effective use of the dynamics within  An important avenue for future work would be developing more transparent models that can shed light on exactly what kinds of order-related features are being extracted and how they are used in prediction 
Conclusions and Future Work
In this work, we introduced a model for forecasting conversational events that processes comments as they happen and takes the full conversational context into account to make an updated prediction at each step  This model fills a void in the existing literature on conversational forecasting, simultaneously addressing the dual challenges of capturing inter-comment dynamics and dealing with an unknown horizon  We find that our model achieves state-of-the-art performance on the task of forecasting derailment in two different datasets that we release publicly  We further show that the resulting system can provide substantial prior notice of derailment, opening up the potential for preemptive interventions by human moderators BIBREF65 
While we have focused specifically on the task of forecasting derailment, we view this work as a step towards a more general model for real-time forecasting of other types of emergent properties of conversations  Follow-up work could adapt the CRAFT architecture to address other forecasting tasks mentioned in Section SECREF2—including those for which the outcome is extraneous to the conversation  We expect different tasks to be informed by different types of inter-comment dynamics, and further architecture extensions could add additional supervised fine-tuning in order to direct it to focus on specific dynamics that might be relevant to the task (e g , exchange of ideas between interlocutors or stonewalling) 
With respect to forecasting derailment, there remain open questions regarding what human moderators actually desire from an early-warning system, which would affect the design of a practical system based on this work  For instance, how early does a warning need to be in order for moderators to find it useful? What is the optimal balance between precision, recall, and false positive rate at which such a system is truly improving moderator productivity rather than wasting their time through false positives? What are the ethical implications of such a system? Follow-up work could run a user study of a prototype system with actual moderators to address these questions 
A practical limitation of the current analysis is that it relies on balanced datasets, while derailment is a relatively rare event for which a more restrictive trigger threshold would be appropriate  While our analysis of the precision-recall curve suggests the system is robust across multiple thresholds ($AUPR=0 7$), additional work is needed to establish whether the recall tradeoff would be acceptable in practice 
Finally, one major limitation of the present work is that it assigns a single label to each conversation: does it derail or not? In reality, derailment need not spell the end of a conversation; it is possible that a conversation could get back on track, suffer a repeat occurrence of antisocial behavior, or any number of other trajectories  It would be exciting to consider finer-grained forecasting of conversational trajectories, accounting for the natural—and sometimes chaotic—ebb-and-flow of human interactions 
Acknowledgements  We thank Caleb Chiam, Liye Fu, Lillian Lee, Alexandru Niculescu-Mizil, Andrew Wang and Justine Zhang for insightful conversations (with unknown horizon), Aditya Jha for his great help with implementing and running the crowd-sourcing tasks, Thomas Davidson and Claire Liang for exploratory data annotation, as well as the anonymous reviewers for their helpful comments  This work is supported in part by the NSF CAREER award IIS-1750615 and by the NSF Grant SES-1741441 
Context 2: Introduction
We posses a wealth of prior knowledge about many natural language processing tasks  For example, in text categorization, we know that words such as NBA, player, and basketball are strong indicators of the sports category BIBREF0 , and words like terrible, boring, and messing indicate a negative polarity while words like perfect, exciting, and moving suggest a positive polarity in sentiment classification 
A key problem arisen here, is how to leverage such knowledge to guide the learning process, an interesting problem for both NLP and machine learning communities  Previous studies addressing the problem fall into several lines  First, to leverage prior knowledge to label data BIBREF1 , BIBREF2   Second, to encode prior knowledge with a prior on parameters, which can be commonly seen in many Bayesian approaches BIBREF3 , BIBREF4   Third, to formalise prior knowledge with additional variables and dependencies BIBREF5   Last, to use prior knowledge to control the distributions over latent output variables BIBREF6 , BIBREF7 , BIBREF8 , which makes the output variables easily interpretable 
However, a crucial problem, which has rarely been addressed, is the bias in the prior knowledge that we supply to the learning model  Would the model be robust or sensitive to the prior knowledge? Or, which kind of knowledge is appropriate for the task? Let's see an example: we may be a baseball fan but unfamiliar with hockey so that we can provide a few number of feature words of baseball, but much less of hockey for a baseball-hockey classification task  Such prior knowledge may mislead the model with heavy bias to baseball  If the model cannot handle this situation appropriately, the performance may be undesirable 
In this paper, we investigate into the problem in the framework of Generalized Expectation Criteria BIBREF7   The study aims to reveal the factors of reducing the sensibility of the prior knowledge and therefore to make the model more robust and practical  To this end, we introduce auxiliary regularization terms in which our prior knowledge is formalized as distribution over output variables  Recall the example just mentioned, though we do not have enough knowledge to provide features for class hockey, it is easy for us to provide some neutral words, namely words that are not strong indicators of any class, like player here  As one of the factors revealed in this paper, supplying neutral feature words can boost the performance remarkably, making the model more robust 
More attractively, we do not need manual annotation to label these neutral feature words in our proposed approach 
More specifically, we explore three regularization terms to address the problem: (1) a regularization term associated with neutral features; (2) the maximum entropy of class distribution regularization term; and (3) the KL divergence between reference and predicted class distribution  For the first manner, we simply use the most common features as neutral features and assume the neutral features are distributed uniformly over class labels  For the second and third one, we assume we have some knowledge about the class distribution which will be detailed soon later 
To summarize, the main contributions of this work are as follows:
The rest of the paper is structured as follows: In Section 2, we briefly describe the generalized expectation criteria and present the proposed regularization terms  In Section 3, we conduct extensive experiments to justify the proposed methods  We survey related work in Section 4, and summarize our work in Section 5 
Method
We address the robustness problem on top of GE-FL BIBREF0 , a GE method which leverages labeled features as prior knowledge  A labeled feature is a strong indicator of a specific class and is manually provided to the classifier  For example, words like amazing, exciting can be labeled features for class positive in sentiment classification 
Generalized Expectation Criteria
Generalized expectation (GE) criteria BIBREF7 provides us a natural way to directly constrain the model in the preferred direction  For example, when we know the proportion of each class of the dataset in a classification task, we can guide the model to predict out a pre-specified class distribution 
Formally, in a parameter estimation objective function, a GE term expresses preferences on the value of some constraint functions about the model's expectation  Given a constraint function $G({\rm x}, y)$ , a conditional model distribution $p_\theta (y|\rm x)$ , an empirical distribution $\tilde{p}({\rm x})$ over input samples and a score function $S$ , a GE term can be expressed as follows:
$$S(E_{\tilde{p}({\rm x})}[E_{p_\theta (y|{\rm x})}[G({\rm x}, y)]])$$   (Eq  4)
Learning from Labeled Features
Druck et al  ge-fl proposed GE-FL to learn from labeled features using generalized expectation criteria  When given a set of labeled features $K$ , the reference distribution over classes of these features is denoted by $\hat{p}(y| x_k), k \in K$   GE-FL introduces the divergence between this reference distribution and the model predicted distribution $p_\theta (y | x_k)$ , as a term of the objective function:
$$\mathcal {O} = \sum _{k \in K} KL(\hat{p}(y|x_k) || p_\theta (y | x_k)) + \sum _{y,i} \frac{\theta _{yi}^2}{2 \sigma ^2}$$   (Eq  6)
where $\theta _{yi}$ is the model parameter which indicates the importance of word $i$ to class $y$   The predicted distribution $p_\theta (y | x_k)$ can be expressed as follows: $ p_\theta (y | x_k) = \frac{1}{C_k} \sum _{\rm x} p_\theta (y|{\rm x})I(x_k) $
in which $I(x_k)$ is 1 if feature $k$ occurs in instance ${\rm x}$ and 0 otherwise, $C_k = \sum _{\rm x} I(x_k)$ is the number of instances with a non-zero value of feature $k$ , and $p_\theta (y|{\rm x})$ takes a softmax form as follows: $ p_\theta (y|{\rm x}) = \frac{1}{Z(\rm x)}\exp (\sum _i \theta _{yi}x_i)  $
To solve the optimization problem, L-BFGS can be used for parameter estimation 
In the framework of GE, this term can be obtained by setting the constraint function $G({\rm x}, y) = \frac{1}{C_k} \vec{I} (y)I(x_k)$ , where $\vec{I}(y)$ is an indicator vector with 1 at the index corresponding to label $y$ and 0 elsewhere 
Regularization Terms
GE-FL reduces the heavy load of instance annotation and performs well when we provide prior knowledge with no bias  In our experiments, we observe that comparable numbers of labeled features for each class have to be supplied  But as mentioned before, it is often the case that we are not able to provide enough knowledge for some of the classes  For the baseball-hockey classification task, as shown before, GE-FL will predict most of the instances as baseball  In this section, we will show three terms to make the model more robust 
Neutral features are features that are not informative indicator of any classes, for instance, word player to the baseball-hockey classification task  Such features are usually frequent words across all categories  When we set the preference distribution of the neutral features to be uniform distributed, these neutral features will prevent the model from biasing to the class that has a dominate number of labeled features 
Formally, given a set of neutral features $K^{^{\prime }}$ , the uniform distribution is $\hat{p}_u(y|x_k) = \frac{1}{|C|}, k \in K^{^{\prime }}$ , where $|C|$ is the number of classes  The objective function with the new term becomes
$$\mathcal {O}_{NE} = \mathcal {O} + \sum _{k \in K^{^{\prime }}} KL(\hat{p}_u(y|x_k) || p_\theta (y | x_k)) $$   (Eq  9)
Note that we do not need manual annotation to provide neutral features  One simple way is to take the most common features as neutral features  Experimental results show that this strategy works successfully 
Another way to prevent the model from drifting from the desired direction is to constrain the predicted class distribution on unlabeled data  When lacking knowledge about the class distribution of the data, one feasible way is to take maximum entropy principle, as below:
$$\mathcal {O}_{ME} = \mathcal {O} + \lambda \sum _{y} p(y) \log p(y)$$   (Eq  11)
where $p(y)$ is the predicted class distribution, given by $ p(y) = \frac{1}{|X|} \sum _{\rm x} p_\theta (y | \rm x)  $ To control the influence of this term on the overall objective function, we can tune $\lambda $ according to the difference in the number of labeled features of each class  In this paper, we simply set $\lambda $ to be proportional to the total number of labeled features, say $\lambda = \beta |K|$  
This maximum entropy term can be derived by setting the constraint function to $G({\rm x}, y) = \vec{I}(y)$   Therefore, $E_{p_\theta (y|{\rm x})}[G({\rm x}, y)]$ is just the model distribution $p_\theta (y|{\rm x})$ and its expectation with the empirical distribution $\tilde{p}(\rm x)$ is simply the average over input samples, namely $p(y)$   When $S$ takes the maximum entropy form, we can derive the objective function as above 
Sometimes, we have already had much knowledge about the corpus, and can estimate the class distribution roughly without labeling instances  Therefore, we introduce the KL divergence between the predicted and reference class distributions into the objective function 
Given the preference class distribution $\hat{p}(y)$ , we modify the objective function as follows:
$$\mathcal {O}_{KL} &= \mathcal {O} + \lambda KL(\hat{p}(y) || p(y))$$   (Eq  13)
Similarly, we set $\lambda = \beta |K|$  
This divergence term can be derived by setting the constraint function to $G({\rm x}, y) = \vec{I}(y)$ and setting the score function to $S(\hat{p}, p) = \sum _i \hat{p}_i \log \frac{\hat{p}_i}{p_i}$ , where $p$ and $\hat{p}$ are distributions  Note that this regularization term involves the reference class distribution which will be discussed later 
Experiments
In this section, we first justify the approach when there exists unbalance in the number of labeled features or in class distribution  Then, to test the influence of $\lambda $ , we conduct some experiments with the method which incorporates the KL divergence of class distribution  Last, we evaluate our approaches in 9 commonly used text classification datasets  We set $\lambda = 5|K|$ by default in all experiments unless there is explicit declaration  The baseline we choose here is GE-FL BIBREF0 , a method based on generalization expectation criteria 
Data Preparation
We evaluate our methods on several commonly used datasets whose themes range from sentiment, web-page, science to medical and healthcare  We use bag-of-words feature and remove stopwords in the preprocess stage  Though we have labels of all documents, we do not use them during the learning process, instead, we use the label of features 
The movie dataset, in which the task is to classify the movie reviews as positive or negtive, is used for testing the proposed approaches with unbalanced labeled features, unbalanced datasets or different $\lambda $ parameters  All unbalanced datasets are constructed based on the movie dataset by randomly removing documents of the positive class  For each experiment, we conduct 10-fold cross validation 
As described in BIBREF0 , there are two ways to obtain labeled features  The first way is to use information gain  We first calculate the mutual information of all features according to the labels of the documents and select the top 20 as labeled features for each class as a feature pool  Note that using information gain requires the document label, but this is only to simulate how we human provide prior knowledge to the model  The second way is to use LDA BIBREF9 to select features  We use the same selection process as BIBREF0 , where they first train a LDA on the dataset, and then select the most probable features of each topic (sorted by $P(w_i|t_j)$ , the probability of word $w_i$ given topic $t_j$ ) 
Similar to BIBREF10 , BIBREF0 , we estimate the reference distribution of the labeled features using a heuristic strategy  If there are $|C|$ classes in total, and $n$ classes are associated with a feature $k$ , the probability that feature $k$ is related with any one of the $n$ classes is $\frac{0 9}{n}$ and with any other class is $\frac{0 1}{|C| - n}$  
Neutral features are the most frequent words after removing stop words, and their reference distributions are uniformly distributed  We use the top 10 frequent words as neutral features in all experiments 
With Unbalanced Labeled Features
In this section, we evaluate our approach when there is unbalanced knowledge on the categories to be classified  The labeled features are obtained through information gain  Two settings are chosen:
(a) We randomly select $t \in [1, 20]$ features from the feature pool for one class, and only one feature for the other  The original balanced movie dataset is used (positive:negative=1:1) 
(b) Similar to (a), but the dataset is unbalanced, obtained by randomly removing 75% positive documents (positive:negative=1:4) 
As shown in Figure 1 , Maximum entropy principle shows improvement only on the balanced case
Context 3: Introduction
The recently introduced BERT model BIBREF0 exhibits strong performance on several language understanding benchmarks  To what extent does it capture syntax-sensitive structures?
Recent work examines the extent to which RNN-based models capture syntax-sensitive phenomena that are traditionally taken as evidence for the existence in hierarchical structure  In particular, in BIBREF1 we assess the ability of LSTMs to learn subject-verb agreement patterns in English, and evaluate on naturally occurring wikipedia sentences  BIBREF2 also consider subject-verb agreement, but in a “colorless green ideas” setting in which content words in naturally occurring sentences are replaced with random words with the same part-of-speech and inflection, thus ensuring a focus on syntax rather than on selectional-preferences based cues  BIBREF3 consider a wider range of syntactic phenomena (subject-verb agreement, reflexive anaphora, negative polarity items) using manually constructed stimuli, allowing for greater coverage and control than in the naturally occurring setting 
The BERT model is based on the “Transformer” architecture BIBREF4 , which—in contrast to RNNs—relies purely on attention mechanisms, and does not have an explicit notion of word order beyond marking each word with its absolute-position embedding  This reliance on attention may lead one to expect decreased performance on syntax-sensitive tasks compared to RNN (LSTM) models that do model word order directly, and explicitly track states across the sentence  Indeed, BIBREF5 finds that transformer-based models perform worse than LSTM models on the BIBREF1 agreement prediction dataset  In contrast, BIBREF6 find that self-attention performs on par with LSTM for syntax sensitive dependencies in the context of machine-translation, and performance on syntactic tasks is correlated with the number of attention heads in multi-head attention 
I adapt the evaluation protocol and stimuli of BIBREF1 , BIBREF2 and BIBREF3 to the bidirectional setting required by BERT, and evaluate the pre-trained BERT models (both the Large and the Base models)  Surprisingly (at least to me), the out-of-the-box models (without any task-specific fine-tuning) perform very well on all the syntactic tasks 
Methodology
I use the stimuli provided by BIBREF1 , BIBREF2 , BIBREF3 , but change the experimental protocol to adapt it to the bidirectional nature of the BERT model  This requires discarding some of the stimuli, as described below  Thus, the numbers are not strictly comparable to those reported in previous work 
Previous setups
All three previous work use uni-directional language-model-like models 
BIBREF1 start with existing sentences from wikipedia that contain a present-tense verb  They feed each sentence word by word into an LSTM, stop right before the focus verb, and ask the model to predict a binary plural/singular decision (supervised setup) or compare the probability assigned by a pre-trained language model (LM) to the plural vs singular forms of the verb (LM setup)  The evaluation is then performed on sentences with “agreement attractors” in which at there is at least one noun between the verb and its subject, and all of the nouns between the verb and subject are of the opposite number from the subject 
BIBREF2 also start with existing sentences  However, in order to control for the possibillity of the model learning to rely on “semantic” selectional-preferences cues rather than syntactic ones, they replace each content word with random words from the same part-of-speech and inflection  This results in “coloreless green ideas” nonce sentences  The evaluation is then performed similarly to the LM setup of BIBREF1 : the sentence is fed into a pre-traiend LSTM LM up to the focus verb, and the model is considered correct if the probability assigned to the correct inflection of the original verb form given the prefix is larger than that assigned to the incorrect inflection 
BIBREF3 focus on manually constructed and controlled stimuli, that also emphasizes linguistic structure over selectional preferences  They construct minimal pairs of grammatical and ungrammatical sentences, feed each one in its entirety into a pre-trained LSTM-LM, and compare the perplexity assigned by the model to the grammatical and ungrammatical sentences  The model is “correct” if it assigns the grammatical sentence a higher probability than to the ungrammatical one  Since the minimal pairs for most phenomena differ only in a single word (the focus verb), this scoring is very similar to the one used in the two previous works  However, it does consider the continuation of the sentence after the focus verb, and also allows for assessing phenomena that require change into two or more words (like negative polarity items) 
Adaptation to the BERT model
In contrast to these works, the BERT model is bi-directional: it is trained to predict the identity of masked words based on both the prefix and suffix surrounding these words  I adapt the uni-directional setup by feeding into BERT the complete sentence, while masking out the single focus verb  I then ask BERT for its word predictions for the masked position, and compare the score assigned to the original correct verb to the score assigned to the incorrect one

ground_truth:  `Conversations Gone Awry' dataset, subreddit ChangeMyView

An expanded version of the existing 'Conversations Gone Awry' dataset and the ChangeMyView dataset, a subreddit whose only annotation is whether the conversation required action by the Reddit moderators. 

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How do they define robustness of a model?
Context 2:  An obvious reason is that maximum entropy only favors uniform distribution 
Incorporating Neutral features performs similarly to maximum entropy since we assume that neutral words are uniformly distributed  Its accuracy decreases slowly when the number of labeled features becomes larger ( $t>4$ ) (Figure 1 (a)), suggesting that the model gradually biases to the class with more labeled features, just like GE-FL 
Incorporating the KL divergence of class distribution performs much better than GE-FL on both balanced and unbalanced datasets  This shows that it is effective to control the unbalance in labeled features and in the dataset 
With Balanced Labeled Features
We also compare with the baseline when the labeled features are balanced  Similar to the experiment above, the labeled features are obtained by information gain  Two settings are experimented with:
(a) We randomly select $t \in [1, 20]$ features from the feature pool for each class, and conduct comparisons on the original balanced movie dataset (positive:negtive=1:1) 
(b) Similar to (a), but the class distribution is unbalanced, by randomly removing 75% positive documents (positive:negative=1:4) 
Results are shown in Figure 2   When the dataset is balanced (Figure 2 (a)), there is little difference between GE-FL and our methods  The reason is that the proposed regularization terms provide no additional knowledge to the model and there is no bias in the labeled features  On the unbalanced dataset (Figure 2 (b)), incorporating KL divergence is much better than GE-FL since we provide additional knowledge(the true class distribution), but maximum entropy and neutral features are much worse because forcing the model to approach the uniform distribution misleads it 
With Unbalanced Class Distributions
Our methods are also evaluated on datasets with different unbalanced class distributions  We manually construct several movie datasets with class distributions of 1:2, 1:3, 1:4 by randomly removing 50%, 67%, 75% positive documents  The original balanced movie dataset is used as a control group  We test with both balanced and unbalanced labeled features  For the balanced case, we randomly select 10 features from the feature pool for each class, and for the unbalanced case, we select 10 features for one class, and 1 feature for the other  Results are shown in Figure 3  
Figure 3 (a) shows that when the dataset and the labeled features are both balanced, there is little difference between our methods and GE-FL(also see Figure 2 (a))  But when the class distribution becomes more unbalanced, the difference becomes more remarkable  Performance of neutral features and maximum entropy decrease significantly but incorporating KL divergence increases remarkably  This suggests if we have more accurate knowledge about class distribution, KL divergence can guide the model to the right direction 
Figure 3 (b) shows that when the labeled features are unbalanced, our methods significantly outperforms GE-FL  Incorporating KL divergence is robust enough to control unbalance both in the dataset and in labeled features while the other three methods are not so competitive 
The Influence of λ\lambda 
We present the influence of $\lambda $ on the method that incorporates KL divergence in this section  Since we simply set $\lambda = \beta |K|$ , we just tune $\beta $ here  Note that when $\beta = 0$ , the newly introduced regularization term is disappeared, and thus the model is actually GE-FL  Again, we test the method with different $\lambda $ in two settings:
(a) We randomly select $t \in [1, 20]$ features from the feature pool for one class, and only one feature for the other class  The original balanced movie dataset is used (positive:negative=1:1) 
(b) Similar to (a), but the dataset is unbalanced, obtained by randomly removing 75% positive documents (positive:negative=1:4) 
Results are shown in Figure 4   As expected, $\lambda $ reflects how strong the regularization is  The model tends to be closer to our preferences with the increasing of $\lambda $ on both cases 
Using LDA Selected Features
We compare our methods with GE-FL on all the 9 datasets in this section  Instead of using features obtained by information gain, we use LDA to select labeled features  Unlike information gain, LDA does not employ any instance labels to find labeled features  In this setting, we can build classification models without any instance annotation, but just with labeled features 
Table 1 shows that our three methods significantly outperform GE-FL  Incorporating neutral features performs better than GE-FL on 7 of the 9 datasets, maximum entropy is better on 8 datasets, and KL divergence better on 7 datasets 
LDA selects out the most predictive features as labeled features without considering the balance among classes  GE-FL does not exert any control on such an issue, so the performance is severely suffered  Our methods introduce auxiliary regularization terms to control such a bias problem and thus promote the model significantly 
Related Work
There have been much work that incorporate prior knowledge into learning, and two related lines are surveyed here  One is to use prior knowledge to label unlabeled instances and then apply a standard learning algorithm  The other is to constrain the model directly with prior knowledge 
Liu et al text manually labeled features which are highly predictive to unsupervised clustering assignments and use them to label unlabeled data  Chang et al guiding proposed constraint driven learning  They first used constraints and the learned model to annotate unlabeled instances, and then updated the model with the newly labeled data  Daumé daume2008cross proposed a self training method in which several models are trained on the same dataset, and only unlabeled instances that satisfy the cross task knowledge constraints are used in the self training process 
MaCallum et al gec proposed generalized expectation(GE) criteria which formalised the knowledge as constraint terms about the expectation of the model into the objective function Graça et al pr proposed posterior regularization(PR) framework which projects the model's posterior onto a set of distributions that satisfy the auxiliary constraints  Druck et al ge-fl explored constraints of labeled features in the framework of GE by forcing the model's predicted feature distribution to approach the reference distribution  Andrzejewski et al andrzejewski2011framework proposed a framework in which general domain knowledge can be easily incorporated into LDA  Altendorf et al altendorf2012learning explored monotonicity constraints to improve the accuracy while learning from sparse data  Chen et al chen2013leveraging tried to learn comprehensible topic models by leveraging multi-domain knowledge 
Mann and McCallum simple,generalized incorporated not only labeled features but also other knowledge like class distribution into the objective function of GE-FL  But they discussed only from the semi-supervised perspective and did not investigate into the robustness problem, unlike what we addressed in this paper 
There are also some active learning methods trying to use prior knowledge  Raghavan et al feedback proposed to use feedback on instances and features interlacedly, and demonstrated that feedback on features boosts the model much  Druck et al active proposed an active learning method which solicits labels on features rather than on instances and then used GE-FL to train the model 
Conclusion and Discussions
This paper investigates into the problem of how to leverage prior knowledge robustly in learning models  We propose three regularization terms on top of generalized expectation criteria  As demonstrated by the experimental results, the performance can be considerably improved when taking into account these factors  Comparative results show that our proposed methods is more effective and works more robustly against baselines  To the best of our knowledge, this is the first work to address the robustness problem of leveraging knowledge, and may inspire other research 
We then present more detailed discussions about the three regularization methods  Incorporating neutral features is the simplest way of regularization, which doesn't require any modification of GE-FL but just finding out some common features  But as Figure 1 (a) shows, only using neutral features are not strong enough to handle extremely unbalanced labeled features 
The maximum entropy regularization term shows the strong ability of controlling unbalance 
This method doesn't need any extra knowledge, and is thus suitable when we know nothing about the corpus  But this method assumes that the categories are uniformly distributed, which may not be the case in practice, and it will have a degraded performance if the assumption is violated (see Figure 1 (b), Figure 2 (b), Figure 3 (a)) 
The KL divergence performs much better on unbalanced corpora than other methods  The reason is that KL divergence utilizes the reference class distribution and doesn't make any assumptions  The fact suggests that additional knowledge does benefit the model 
However, the KL divergence term requires providing the true class distribution  Sometimes, we may have the exact knowledge about the true distribution, but sometimes we may not  Fortunately, the model is insensitive to the true distribution and therefore a rough estimation of the true distribution is sufficient  In our experiments, when the true class distribution is 1:2, where the reference class distribution is set to 1:1 5/1:2/1:2 5, the accuracy is 0 755/0 756/0 760 respectively  This provides us the possibility to perform simple computing on the corpus to obtain the distribution in reality  Or, we can set the distribution roughly with domain expertise 
Context 3: 
For example, for the sentence:
a 2002 systemic review of herbal products found that several herbs , including peppermint and caraway , have anti-dyspeptic effects for non-ulcer dyspepsia with “ encouraging safety profiles ”   (from BIBREF1 )
I feed into BERT:
[CLS] a 2002 systemic review of herbal products found that several herbs , including peppermint and caraway , [MASK] anti-dyspeptic effects for non-ulcer dyspepsia with “ encouraging safety profiles ”   and look for the score assigned to the words have and has at the masked position 
Similarly, for the pair
the game that the guard hates is bad  
the game that the guard hates are bad  
(from BIBREF3 ), I feed into BERT:
[CLS] the game that the guard hates [MASK] bad  
and compare the scores predicted for is and are 
This differs from BIBREF1 and BIBREF2 by considering the entire sentence (excluding the verb) and not just its prefix leading to the verb, and differs from BIBREF3 by conditioning the focus verb on bidirectional context 
I use the PyTorch implementation of BERT, with the pre-trained models supplied by Google  I experiment with the bert-large-uncased and bert-base-uncased models 
The bi-directional setup precludes using using the NPI stimuli of BIBREF3 , in which the minimal pair differs in two words position, which I discard from the evaluation  I also discard the agreement cases involving the verbs is or are in BIBREF1 and in BIBREF2 , because some of them are copular construction, in which strong agreement hints can be found also on the object following the verb  This is not an issue in the manually constructed BIBREF3 stimuli due to the patterns they chose 
Finally, I discard stimuli in which the focus verb or its plural/singular inflection does not appear as a single word in the BERT word-piece-based vocabulary (and hence cannot be predicted by the model)  This include discarding BIBREF3 stimuli involving the words swims or admires, resulting in 23,368 discarded pairs (out of 152,300)  I similarly discard 680 sentences from BIBREF1 where the focus verb or its inflection were one of 108 out-of-vocabulary tokens, and 28 sentence-pairs (8 tokens) from BIBREF2  
The BERT results are not directly comparable to the numbers reported in previous work  Beyond the differences due to bidirectionality and the discarded stimuli, the BERT models are also trained on a different and larger corpus (covering both wikipedia and books) 
Code is available at https://github com/yoavg/bert-syntax 
Results
Tables 1 , 2 and 3 show the results  All cases exhibit high scores—in the vast majority of the cases substantially higher than reported in previous work  As discussed above, the results are not directly comparable to previous work: the BERT models are trained on different (and larger) data, are allowed to access the suffix of the sentence in addition to its prefix, and are evaluated on somewhat different data due to discarding OOV items  Still, taken together, the high performance numbers indicate that the purely attention-based BERT models are likely capable of capturing the same kind of syntactic regularities that LSTM-based models are capable of capturing, at least as well as the LSTM models and probably better 
Another noticeable and interesting trend is that larger is not necessarily better: the BERT-Base model outperforms the BERT-Large model on many of the syntactic conditions 
Discussion
The BERT models perform remarkably well on all the syntactic test cases  I expected the attention-based mechanism to fail on these (compared to the LSTM-based models), and am surprised by these results  The BIBREF2 and BIBREF3 conditions rule out the possibility of overly relying on selectional preference cues or memorizing the wikipedia training data, and suggest real syntactic generalization is taking place  Exploring the extent to which deep purely-attention-based architectures such as BERT are capable of capturing hierarchy-sensitive and syntactic dependencies—as well as the mechanisms by which this is achieved—is a fascinating area for future research 

ground_truth: ability to accurately classify texts even when the amount of prior knowledge for different classes is unbalanced, and when the class distribution of the dataset is unbalanced

Low sensitivity to bias in prior knowledge

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Were any of these tasks evaluated in any previous work?
Context 4:  This could be due to implementation differences, which unfortunately makes our EN results incomparable  For DE, the arguments are identified using the LTH system BIBREF16 , which gives an F1 score of 86 5% on the CoNLL 2009 DE data  The word alignments for the EN-DE parallel Europarl corpus are computed using GIZA++ BIBREF17   For high-precision, only the intersecting alignments in the two directions are kept  We define two semantic arguments as aligned if their head-words are aligned  In total we get 9 3M arguments for EN (240k CoNLL + 9 1M Europarl) and 4 43M for DE (32k CoNLL + 4 4M Europarl)  Out of these, 0 76M arguments are aligned 
Main Results
Since the CoNLL annotations have 21 semantic roles in total, we use 21 roles in our model as well as the baseline  Following garg2012unsupervised, we set the number of PRs to 2 (excluding INLINEFORM0 , INLINEFORM1 and INLINEFORM2 ), and SRs to 21-2=19  Table TABREF27 shows the results 
In the first setting (Line 1), we train and test the monolingual model on the CoNLL data  We observe significant improvements in F1 score over the Baseline (Line 0) in both languages  Using the CoNLL 2009 dataset alone, titovcrosslingual report an F1 score of 80 9% (PU=86 8%, CO=75 7%) for German  Thus, our monolingual model outperforms their monolingual model in German  For English, they report an F1 score of 83 6% (PU=87 5%, CO=80 1%), but note that our English results are not directly comparable to theirs due to differences argument identification, as discussed in section SECREF25   As their argument identification score is lower, perhaps their system is discarding “difficult” arguments which leads to a higher clustering score 
In the second setting (Line 2), we use the additional monolingual Europarl (EP) data for training  We get equivalent results in English and a significant improvement in German compared to our previous setting (Line 1)  The German dataset in CoNLL is quite small and benefits from the additional EP training data  In contrast, the English model is already quite good due to a relatively big dataset from CoNLL, and good accuracy syntactic parsers  Unfortunately, titovcrosslingual do not report results with this setting 
The third setting (Line 3) gives the results of our multilingual model, which adds the word alignments in the EP data  Comparing with Line 2, we get non-significant improvements in both languages  titovcrosslingual obtain an F1 score of 82 7% (PU=85 0%, CO=80 6%) for German, and 83 7% (PU=86 8%, CO=80 7%) for English  Thus, for German, our multilingual Bayesian model is able to capture the cross-lingual patterns at least as well as the external penalty term in BIBREF6   We cannot compare the English results unfortunately due to differences in argument identification 
We also compared monolingual and bilingual training data using a setting that emulates the standard supervised setup of separate training and test data sets  We train only on the EP dataset and test on the CoNLL dataset  Lines 4 and 5 of Table TABREF27 give the results  The multilingual model obtains small improvements in both languages, which confirms the results from the standard unsupervised setup, comparing lines 2 to 3 
These results indicate that little information can be learned about semantic roles from this parallel data setup  One possible explanation for this result is that the setup itself is inadequate  Given the definition of aligned arguments, only 8% of English arguments and 17% of German arguments are aligned  This plus our experiments suggest that improving the alignment model is a necessary step to making effective use of parallel data in multilingual SRI, for example by joint modeling with SRI  We leave this exploration to future work 
Multilingual Training with Labeled Data for One Language
Another motivation for jointly modeling SRL in multiple languages is the transfer of information from a resource rich language to a resource poor language  We evaluated our model in a very general annotation transfer scenario, where we have a small labeled dataset for one language (source), and a large parallel unlabeled dataset for the source and another (target) language  We investigate whether this setting improves the parameter estimates for the target language  To this end, we clamp the role annotations of the source language in the CoNLL dataset using a predefined mapping, and do not sample them during training  This data gives us good parameters for the source language, which are used to sample the roles of the source language in the unlabeled Europarl data  The CLVs aim to capture this improvement and thereby improve sampling and parameter estimates for the target language  Table TABREF28 shows the results of this experiment  We obtain small improvements in the target languages  As in the unsupervised setting, the small percentage of aligned roles probably limits the impact of the cross-lingual information 
Labeled Data in Monolingual Model
We explored the improvement in the monolingual model in a semi-supervised setting  To this end, we randomly selected INLINEFORM0 of the sentences in the CoNLL dataset as “supervised sentences” and the rest INLINEFORM1 were kept unsupervised  Next, we clamped the role labels of the supervised sentences using the predefined mapping from Section SECREF29   Sampling was done on the unsupervised sentences as usual  We then measured the clustering performance using the trained parameters 
To access the contribution of partial supervision better, we constructed a “supervised baseline” as follows  For predicates seen in the supervised sentences, a MAP estimate of the parameters was calculated using the predefined mapping  For the unseen predicates, the standard baseline was used 
Figures FIGREF33 and FIGREF33 show the performance variation with INLINEFORM0   We make the following observations:
[leftmargin=*]
In both languages, at around INLINEFORM0 , the supervised baseline starts outperforming the semi-supervised model, which suggests that manually labeling about 10% of the sentences is a good enough alternative to our training procedure  Note that 10% amounts to about 3 6k sentences in German and 4k in English  We noticed that the proportion of seen predicates increases dramatically as we increase the proportion of supervised sentences  At 10% supervised sentences, the model has already seen 63% of predicates in German and 44% in English  This explains to some extent why only 10% labeled sentences are enough 
For German, it takes about 3 5% or 1260 supervised sentences to have the same performance increase as 1 5M unlabeled sentences (Line 1 to Line 2 in Table TABREF27 )  Adding about 180 more supervised sentences also covers the benefit obtained by alignments in the multilingual model (Line 2 to Line 3 in Table TABREF27 )  There is no noticeable performance difference in English 
We also evaluated the performance variation on a completely unseen CoNLL test set  Since the test set is very small compared to the training set, the clustering evaluation is not as reliable  Nonetheless, we broadly obtained the same pattern 
Related Work
As discussed in section SECREF24 , our work is closely related to the crosslingual unsupervised SRL work of titovcrosslingual  The idea of using superlingual latent variables to capture cross-lingual information was proposed for POS tagging by naseem2009multilingual, which we use here for SRL  In a semi-supervised setting, pado2009cross used a graph based approach to transfer semantic role annotations from English to German  furstenau2009graph used a graph alignment method to measure the semantic and syntactic similarity between dependency tree arguments of known and unknown verbs 
For monolingual unsupervised SRL, swier2004unsupervised presented the first work on a domain-general corpus, the British National Corpus, using 54 verbs taken from VerbNet  garg2012unsupervised proposed a Bayesian model for this problem that we use here  titov2012bayesian also proposed a closely related Bayesian model  grenager2006unsupervised proposed a generative model but their parameter space consisted of all possible linkings of syntactic constituents and semantic roles, which made unsupervised learning difficult and a separate language-specific rule based method had to be used to constrain this space  Other proposed models include an iterative split-merge algorithm BIBREF18 and a graph-partitioning based approach BIBREF1   marquez2008semantic provide a good overview of the supervised SRL systems 
Conclusions
We propose a Bayesian model of semantic role induction (SRI) that uses crosslingual latent variables to capture role alignments in parallel corpora  The crosslingual latent variables capture correlations between roles in different languages, and regularize the parameter estimates of the monolingual models  Because this is a joint Bayesian model of multilingual SRI, we can apply the same model to a variety of training scenarios just by changing the inference procedure appropriately  We evaluate monolingual SRI with a large unlabeled dataset, bilingual SRI with a parallel corpus, bilingual SRI with annotations available for the source language, and monolingual SRI with a small labeled dataset  Increasing the amount of monolingual unlabeled data significantly improves SRI in German but not in English  Adding word alignments in parallel sentences results in small, non significant improvements, even if there is some labeled data available in the source language  This difficulty in showing the usefulness of parallel corpora for SRI may be due to the current assumptions about role alignments, which mean that only a small percentage of roles are aligned  Further analyses reveals that annotating small amounts of data can easily outperform the performance gains obtained by adding large unlabeled dataset as well as adding parallel corpora 
Future work includes training on different language pairs, on more than two languages, and with more inclusive models of role alignment 
Acknowledgments
This work was funded by the Swiss NSF grant 200021_125137 and EC FP7 grant PARLANCE 

ground_truth: Yes

Yes