prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Do they evaluate only on English datasets?
Context 1: Introduction
Data annotation is a major bottleneck for the application of supervised learning approaches to many problems  As a result, unsupervised methods that learn directly from unlabeled data are increasingly important  For tasks related to unsupervised syntactic analysis, discrete generative models have dominated in recent years – for example, for both part-of-speech (POS) induction BIBREF0 , BIBREF1 and unsupervised dependency parsing BIBREF2 , BIBREF3 , BIBREF4   While similar models have had success on a range of unsupervised tasks, they have mostly ignored the apparent utility of continuous word representations evident from supervised NLP applications BIBREF5 , BIBREF6   In this work, we focus on leveraging and explicitly representing continuous word embeddings within unsupervised models of syntactic structure 
Pre-trained word embeddings from massive unlabeled corpora offer a compact way of injecting a prior notion of word similarity into models that would otherwise treat words as discrete, isolated categories  However, the specific properties of language captured by any particular embedding scheme can be difficult to control, and, further, may not be ideally suited to the task at hand  For example, pre-trained skip-gram embeddings BIBREF7 with small context window size are found to capture the syntactic properties of language well BIBREF8 , BIBREF9   However, if our goal is to separate syntactic categories, this embedding space is not ideal – POS categories correspond to overlapping interspersed regions in the embedding space, evident in Figure SECREF4  
In our approach, we propose to learn a new latent embedding space as a projection of pre-trained embeddings (depicted in Figure SECREF5 ), while jointly learning latent syntactic structure – for example, POS categories or syntactic dependencies  To this end, we introduce a new generative model (shown in Figure FIGREF6 ) that first generates a latent syntactic representation (e g  a dependency parse) from a discrete structured prior (which we also call the “syntax model”), then, conditioned on this representation, generates a sequence of latent embedding random variables corresponding to each word, and finally produces the observed (pre-trained) word embeddings by projecting these latent vectors through a parameterized non-linear function  The latent embeddings can be jointly learned with the structured syntax model in a completely unsupervised fashion 
By choosing an invertible neural network as our non-linear projector, and then parameterizing our model in terms of the projection's inverse, we are able to derive tractable exact inference and marginal likelihood computation procedures so long as inference is tractable in the underlying syntax model  In sec:learn-with-inv we show that this derivation corresponds to an alternate view of our approach whereby we jointly learn a mapping of observed word embeddings to a new embedding space that is more suitable for the syntax model, but include an additional Jacobian regularization term to prevent information loss 
Recent work has sought to take advantage of word embeddings in unsupervised generative models with alternate approaches BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12   BIBREF9 build an HMM with Gaussian emissions on observed word embeddings, but they do not attempt to learn new embeddings  BIBREF10 , BIBREF11 , and BIBREF12 extend HMM or dependency model with valence (DMV) BIBREF2 with multinomials that use word (or tag) embeddings in their parameterization  However, they do not represent the embeddings as latent variables 
In experiments, we instantiate our approach using both a Markov-structured syntax model and a tree-structured syntax model – specifically, the DMV  We evaluate on two tasks: part-of-speech (POS) induction and unsupervised dependency parsing without gold POS tags  Experimental results on the Penn Treebank BIBREF13 demonstrate that our approach improves the basic HMM and DMV by a large margin, leading to the state-of-the-art results on POS induction, and state-of-the-art results on unsupervised dependency parsing in the difficult training scenario where neither gold POS annotation nor punctuation-based constraints are available 
Model
As an illustrative example, we first present a baseline model for Markov syntactic structure (POS induction) that treats a sequence of pre-trained word embeddings as observations  Then, we propose our novel approach, again using Markov structure, that introduces latent word embedding variables and a neural projector  Lastly, we extend our approach to more general syntactic structures 
Example: Gaussian HMM
We start by describing the Gaussian hidden Markov model introduced by BIBREF9 , which is a locally normalized model with multinomial transitions and Gaussian emissions  Given a sentence of length INLINEFORM0 , we denote the latent POS tags as INLINEFORM1 , observed (pre-trained) word embeddings as INLINEFORM2 , transition parameters as INLINEFORM3 , and Gaussian emission parameters as INLINEFORM4   The joint distribution of data and latent variables factors as:
DISPLAYFORM0
where INLINEFORM0 is the multinomial transition probability and INLINEFORM1 is the multivariate Gaussian emission probability 
While the observed word embeddings do inform this model with a notion of word similarity – lacking in the basic multinomial HMM – the Gaussian emissions may not be sufficiently flexible to separate some syntactic categories in the complex pre-trained embedding space – for example the skip-gram embedding space as visualized in Figure SECREF4 where different POS categories overlap  Next we introduce a new approach that adds flexibility to the emission distribution by incorporating new latent embedding variables 
Markov Structure with Neural Projector
To flexibly model observed embeddings and yield a new representation space that is more suitable for the syntax model, we propose to cascade a neural network as a projection function, deterministically transforming the simple space defined by the Gaussian HMM to the observed embedding space  We denote the latent embedding of the INLINEFORM0 word in a sentence as INLINEFORM1 , and the neural projection function as INLINEFORM2 , parameterized by INLINEFORM3   In the case of sequential Markov structure, our new model corresponds to the following generative process:
For each time step INLINEFORM0 ,
[noitemsep, leftmargin=*]
Draw the latent state INLINEFORM0
Draw the latent embedding INLINEFORM0
Deterministically produce embedding
INLINEFORM0
The graphical model is depicted in Figure FIGREF6   The deterministic projection can also be viewed as sampling each observation from a point mass at INLINEFORM0   The joint distribution of our model is: DISPLAYFORM0
where INLINEFORM0 is a conditional Gaussian distribution, and INLINEFORM1 is the Dirac delta function centered at INLINEFORM2 : DISPLAYFORM0
General Structure with Neural Projector
Our approach can be applied to a broad family of structured syntax models  We denote latent embedding variables as INLINEFORM0 , discrete latent variables in the syntax model as INLINEFORM1 ( INLINEFORM2 ), where INLINEFORM3 are conditioned to generate INLINEFORM4   The joint probability of our model factors as:
DISPLAYFORM0
where INLINEFORM0 represents the probability of the syntax model, and can encode any syntactic structure – though, its factorization structure will determine whether inference is tractable in our full model  As shown in Figure FIGREF6 , we focus on two syntax models for syntactic analysis in this paper  The first is Markov-structured, which we use for POS induction, and the second is DMV-structured, which we use to learn dependency parses without supervision 
The marginal data likelihood of our model is: DISPLAYFORM0
While the discrete variables INLINEFORM0 can be marginalized out with dynamic program in many cases, it is generally intractable to marginalize out the latent continuous variables, INLINEFORM1 , for an arbitrary projection INLINEFORM2 in Eq  ( EQREF17 ), which means inference and learning may be difficult  In sec:opt, we address this issue by constraining INLINEFORM3 to be invertible, and show that this constraint enables tractable exact inference and marginal likelihood computation 
Learning & Inference
In this section, we introduce an invertibility condition for our neural projector to tackle the optimization challenge  Specifically, we constrain our neural projector with two requirements: (1) INLINEFORM0 and (2) INLINEFORM1 exists  Invertible transformations have been explored before in independent components analysis BIBREF14 , gaussianization BIBREF15 , and deep density models BIBREF16 , BIBREF17 , BIBREF18 , for unstructured data  Here, we generalize this style of approach to structured learning, and augment it with discrete latent variables ( INLINEFORM2 )  Under the invertibility condition, we derive a learning algorithm and give another view of our approach revealed by the objective function  Then, we present the architecture of a neural projector we use in experiments: a volume-preserving invertible neural network proposed by BIBREF16 for independent components estimation 
Learning with Invertibility
For ease of exposition, we explain the learning algorithm in terms of Markov structure without loss of generality  As shown in Eq  ( EQREF17 ), the optimization challenge in our approach comes from the intractability of the marginalized emission factor INLINEFORM0   If we can marginalize out INLINEFORM1 and compute INLINEFORM2 , then the posterior and marginal likelihood of our Markov-structured model can be computed with the forward-backward algorithm  We can apply Eq  ( EQREF14 ) and obtain : INLINEFORM3
By using the change of variable rule to the integration, which allows the integration variable INLINEFORM0 to be replaced by INLINEFORM1 , the marginal emission factor can be computed in closed-form when the invertibility condition is satisfied: DISPLAYFORM0
where INLINEFORM0 is a conditional Gaussian distribution, INLINEFORM1 is the Jacobian matrix of function INLINEFORM2 at INLINEFORM3 , and INLINEFORM4 represents the absolute value of its determinant  This Jacobian term is nonzero and differentiable if and only if INLINEFORM5 exists 
Eq  ( EQREF19 ) shows that we can directly calculate the marginal emission distribution INLINEFORM0   Denote the marginal data likelihood of Gaussian HMM as INLINEFORM1 , then the log marginal data likelihood of our model can be directly written as: DISPLAYFORM0
where INLINEFORM0 represents the new sequence of embeddings after applying INLINEFORM1 to each INLINEFORM2   Eq  ( EQREF20 ) shows that the training objective of our model is simply the Gaussian HMM log likelihood with an additional Jacobian regularization term  From this view, our approach can be seen as equivalent to reversely projecting the data through INLINEFORM3 to another manifold INLINEFORM4 that is directly modeled by the Gaussian HMM, with a regularization term  Intuitively, we optimize the reverse projection INLINEFORM5 to modify the INLINEFORM6 space, making it more appropriate for the syntax model  The Jacobian regularization term accounts for the volume expansion or contraction behavior of the projection  Maximizing it can be thought of as preventing information loss  In the extreme case, the Jacobian determinant is equal to zero, which means the projection is non-invertible and thus information is being lost through the projection  Such “information preserving” regularization is crucial during optimization, otherwise the trivial solution of always projecting data to the same single point to maximize likelihood is viable 
More generally, for an arbitrary syntax model the data likelihood of our approach is: DISPLAYFORM0
If the syntax model itself allows for tractable inference and marginal likelihood computation, the same dynamic program can be used to marginalize out INLINEFORM0   Therefore, our joint model inherits the tractability of the underlying syntax model 
Invertible Volume-Preserving Neural Net
For the projection we can use an arbitrary invertible function, and given the representational power of neural networks they seem a natural choice  However, calculating the inverse and Jacobian of an arbitrary neural network can be difficult, as it requires that all component functions be invertible and also requires storage of large Jacobian matrices, which is memory intensive  To address this issue, several recent papers propose specially designed invertible networks that are easily trainable yet still powerful BIBREF16 , BIBREF17 , BIBREF19   Inspired by these works, we use the invertible transformation proposed by BIBREF16 , which consists of a series of “coupling layers”  This architecture is specially designed to guarantee a unit Jacobian determinant (and thus the invertibility property) 
From Eq  ( EQREF22 ) we know that only INLINEFORM0 is required for accomplishing learning and inference; we never need to explicitly construct INLINEFORM1   Thus, we directly define the architecture of INLINEFORM2   As shown in Figure FIGREF24 , the nonlinear transformation from the observed embedding INLINEFORM3 to INLINEFORM4 represents the first coupling layer  The input in this layer is partitioned into left and right halves of dimensions, INLINEFORM5 and INLINEFORM6 , respectively  A single coupling layer is defined as: DISPLAYFORM0
where INLINEFORM0 is the coupling function and can be any nonlinear form  This transformation satisfies INLINEFORM1 , and BIBREF16 show that its Jacobian matrix is triangular with all ones on the main diagonal  Thus the Jacobian determinant is always equal to one (i e  volume-preserving) and the invertibility condition is naturally satisfied 
To be sufficiently expressive, we compose multiple coupling layers as suggested in BIBREF16   Specifically, we exchange the role of left and right half vectors at each layer as shown in Figure FIGREF24   For instance, from INLINEFORM0 to INLINEFORM1 the left subset INLINEFORM2 is unchanged, while from INLINEFORM3 to INLINEFORM4 the right subset INLINEFORM5 remains the same  Also note that composing multiple coupling layers does not change the volume-preserving and invertibility properties  Such a sequence of invertible transformations from the data space INLINEFORM6 to INLINEFORM7 is also called normalizing flow BIBREF20  
Experiments
In this section, we first describe our datasets and experimental setup  We then instantiate our approach with Markov and DMV-structured syntax models, and report results on POS tagging and dependency grammar induction respectively  Lastly, we analyze the learned latent embeddings 
Data
For both POS tagging and dependency parsing, we run experiments on the Wall Street Journal (WSJ) portion of the Penn Treebank  To create the observed data embeddings, we train skip-gram word embeddings BIBREF7 that are found to capture syntactic properties well when trained with small context window BIBREF8 , BIBREF9   Following BIBREF9 , the dimensionality INLINEFORM0 is set to 100, and the training context window size is set to 1 to encode more syntactic information  The skip-gram embeddings are trained on the one billion word language modeling benchmark dataset BIBREF21 in addition to the WSJ corpus 
General Experimental Setup
For the neural projector, we employ rectified networks as coupling function INLINEFORM0 following BIBREF16   We use a rectified network with an input layer, one hidden layer, and linear output units, the number of hidden units is set to the same as the number of input units  The number of coupling layers are varied as 4, 8, 16 for both tasks  We optimize marginal data likelihood directly using Adam BIBREF22   For both tasks in the fully unsupervised setting, we do not tune the hyper-parameters using supervised data 
Unsupervised POS tagging
For unsupervised POS tagging, we use a Markov-structured syntax model in our approach, which is a popular structure for unsupervised tagging tasks BIBREF9 , BIBREF10  
Following existing literature, we train and test on the entire WSJ corpus (49208 sentences, 1M tokens)  We use 45 tag clusters, the number of POS tags that appear in WSJ corpus  We train the discrete HMM and the Gaussian HMM BIBREF9 as baselines  For the Gaussian HMM, mean vectors of Gaussian emissions are initialized with the empirical mean of all word vectors with an additive noise  We assume diagonal covariance matrix for INLINEFORM0 and initialize it with the empirical variance of the word vectors  Following BIBREF9 , the covariance matrix is fixed during training  The multinomial probabilities are initialized as INLINEFORM1 , where INLINEFORM2   For our approach, we initialize the syntax model and Gaussian parameters with the pre-trained Gaussian HMM  The weights of layers in the rectified network are initialized from a uniform distribution with mean zero and a standard deviation of INLINEFORM3 , where INLINEFORM4 is the input dimension  We evaluate the performance of POS tagging with both Many-to-One (M-1) accuracy BIBREF23 and V-Measure (VM) BIBREF24   Given a model we found that the tagging performance is well-correlated with the training data likelihood, thus we use training data likelihood as a unsupervised criterion to select the trained model over 10 random restarts after training 50 epochs  We repeat this process 5 times and report the mean and standard deviation of performance 
We compare our approach with basic HMM, Gaussian HMM, and several state-of-the-art systems, including sophisticated HMM variants and clustering techniques with hand-engineered features  The results are presented in Table TABREF32   Through the introduced latent embeddings and additional neural projection, our approach improves over the Gaussian HMM by 5 4 points in M-1 and 5 6 points in VM  Neural HMM (NHMM) BIBREF10 is a baseline that also learns word representation jointly  Both their basic model and extended Conv version does not outperform the Gaussian HMM  Their best model incorporates another LSTM to model long distance dependency and breaks the Markov assumption, yet our approach still achieves substantial improvement over it without considering more context information  Moreover, our method outperforms the best published result that benefits from hand-engineered features BIBREF27 by 2 0 points on VM 
We found that most tagging errors happen in noun subcategories  Therefore, we do the one-to-one mapping between gold POS tags and induced clusters and plot the normalized confusion matrix of noun subcategories in Figure FIGREF35   The Gaussian HMM fails to identify “NN” and “NNS” correctly for most cases, and it often recognizes “NNPS” as “NNP”  In contrast, our approach corrects these errors well 
Unsupervised Dependency Parsing without gold POS tags
For the task of unsupervised dependency parse induction, we employ the Dependency Model with Valence (DMV) BIBREF2 as the syntax model in our approach  DMV is a generative model that defines a probability distribution over dependency parse trees and syntactic categories, generating tokens and dependencies in a head-outward fashion  While, traditionally, DMV is trained using gold POS tags as observed syntactic categories, in our approach, we treat each tag as a latent variable, as described in sec:general-neural 
Most existing approaches to this task are not fully unsupervised since they rely on gold POS tags following the original experimental setup for DMV  This is partially because automatically parsing from words is difficult even when using unsupervised syntactic categories BIBREF29   However, inducing dependencies from words alone represents a more realistic experimental condition since gold POS tags are often unavailable in practice  Previous work that has trained from words alone often requires additional linguistic constraints (like sentence internal boundaries) BIBREF29 , BIBREF30 , BIBREF31 , BIBREF32 , acoustic cues BIBREF33 , additional training data BIBREF4 , or annotated data from related languages BIBREF34   Our approach is naturally designed to train on word embeddings directly, thus we attempt to induce dependencies without using gold POS tags or other extra linguistic information 
Like previous work we use sections 02-21 of WSJ corpus as training data and evaluate on section 23, we remove punctuations and train the models on sentences of length INLINEFORM0 , “head-percolation” rules BIBREF39 are applied to obtain gold dependencies for evaluation  We train basic DMV, extended DMV (E-DMV) BIBREF35 and Gaussian DMV (which treats POS tag as unknown latent variables and generates observed word embeddings directly conditioned on them following Gaussian distribution) as baselines  Basic DMV and E-DMV are trained with Viterbi EM BIBREF40 on unsupervised POS tags induced from our Markov-structured model described in sec:pos  Multinomial parameters of the syntax model in both Gaussian DMV and our model are initialized with the pre-trained DMV baseline  Other parameters are initialized in the same way as in the POS tagging experiment  The directed dependency accuracy (DDA) is used for evaluation and we report accuracy on sentences of length INLINEFORM1 and all lengths  We train the parser until training data likelihood converges, and report the mean and standard deviation over 20 random restarts 
Our model directly observes word embeddings and does not require gold POS tags during training  Thus, results from related work trained on gold tags are not directly comparable  However, to measure how these systems might perform without gold tags, we run three recent state-of-the-art systems in our experimental setting: UR-A E-DMV BIBREF36 , Neural E-DMV BIBREF11 , and CRF Autoencoder (CRFAE) BIBREF37   We use unsupervised POS tags (induced from our Markov-structured model) in place of gold tags  We also train basic DMV on gold tags and include several state-of-the-art results on gold tags as reference points 
As shown in Table TABREF39 , our approach is able to improve over the Gaussian DMV by 4 8 points on length INLINEFORM0 and 4 8 points on all lengths, which suggests the additional latent embedding layer and neural projector are helpful  The proposed approach yields, to the best of our knowledge, state-of-the-art performance without gold POS annotation and without sentence-internal boundary information  DMV, UR-A E-DMV, Neural E-DMV, and CRFAE suffer a large decrease in performance when trained on unsupervised tags – an effect also seen in previous work BIBREF29 , BIBREF34   Since our approach induces latent POS tags jointly with dependency trees, it may be able to learn POS clusters that are more amenable to grammar induction than the unsupervised tags  We observe that CRFAE underperforms its gold-tag counterpart substantially  This may largely be a result of the model's reliance on prior linguistic rules that become unavailable when gold POS tag types are unknown  Many extensions to DMV can be considered orthogonal to our approach – they essentially focus on improving the syntax model  It is possible that incorporating these more sophisticated syntax models into our approach may lead to further improvements 
Sensitivity Analysis
In the above experiments we initialize the structured syntax components with the pre-trained Gaussian or discrete baseline, which is shown as a useful technique to help train our deep models  We further study the results with fully random initialization  In the POS tagging experiment, we report the results in Table TABREF48   While the performance with 4 layers is comparable to the pre-trained Gaussian initialization, deeper projections (8 or 16 layers) result in a dramatic drop in performance  This suggests that the structured syntax model with very deep projections is difficult to train from scratch, and a simpler projection might be a good compromise in the random initialization setting 
Different from the Markov prior in POS tagging experiments, our parsing model seems to be quite sensitive to the initialization  For example, directed accuracy of our approach on sentences of length INLINEFORM0 is below 40 0 with random initialization  This is consistent with previous work that has noted the importance of careful initialization for DMV-based models such as the commonly used harmonic initializer BIBREF2   However, it is not straightforward to apply the harmonic initializer for DMV directly in our model without using some kind of pre-training since we do not observe gold POS 
We investigate the effect of the choice of pre-trained embedding on performance while using our approach  To this end, we additionally include results using fastText embeddings BIBREF41 – which, in contrast with skip-gram embeddings, include character-level information  We set the context windows size to 1 and the dimension size to 100 as in the skip-gram training, while keeping other parameters set to their defaults  These results are summarized in Table TABREF50 and Table TABREF51   While fastText embeddings lead to reduced performance with our model, our approach still yields an improvement over the Gaussian baseline with the new observed embeddings space 
Qualitative Analysis of Embeddings
We perform qualitative analysis to understand how the latent embeddings help induce syntactic structures  First we filter out low-frequency words and punctuations in WSJ, and visualize the rest words (10k) with t-SNE BIBREF42 under different embeddings  We assign each word with its most likely gold POS tags in WSJ and color them according to the gold POS tags 
For our Markov-structured model, we have displayed the embedding space in Figure SECREF5 , where the gold POS clusters are well-formed  Further, we present five example target words and their five nearest neighbors in terms of cosine similarity  As shown in Table TABREF53 , the skip-gram embedding captures both semantic and syntactic aspects to some degree, yet our embeddings are able to focus especially on the syntactic aspects of words, in an unsupervised fashion without using any extra morphological information 
In Figure FIGREF54 we depict the learned latent embeddings with the DMV-structured syntax model  Unlike the Markov structure, the DMV structure maps a large subset of singular and plural nouns to the same overlapping region  However, two clusters of singular and plural nouns are actually separated  We inspect the two clusters and the overlapping region in Figure FIGREF54 , it turns out that the nouns in the separated clusters are words that can appear as subjects and, therefore, for which verb agreement is important to model  In contrast, the nouns in the overlapping region are typically objects  This demonstrates that the latent embeddings are focusing on aspects of language that are specifically important for modeling dependency without ever having seen examples of dependency parses  Some previous work has deliberately created embeddings to capture different notions of similarity BIBREF43 , BIBREF44 , while they use extra morphology or dependency annotations to guide the embedding learning, our approach provides a potential alternative to create new embeddings that are guided by structured syntax model, only using unlabeled text corpora 
Related Work
Our approach is related to flow-based generative models, which are first described in NICE BIBREF16 and have recently received more attention BIBREF17 , BIBREF19 , BIBREF18   This relevant work mostly adopts simple (e g  Gaussian) and fixed priors and does not attempt to learn interpretable latent structures  Another related generative model class is variational auto-encoders (VAEs) BIBREF45 that optimize a lower bound on the marginal data likelihood, and can be extended to learn latent structures BIBREF46 , BIBREF47   Against the flow-based models, VAEs remove the invertibility constraint but sacrifice the merits of exact inference and exact log likelihood computation, which potentially results in optimization challenges BIBREF48   Our approach can also be viewed in connection with generative adversarial networks (GANs) BIBREF49 that is a likelihood-free framework to learn implicit generative models  However, it is non-trivial for a gradient-based method like GANs to propagate gradients through discrete structures 
Conclusion
In this work, we define a novel generative approach to leverage continuous word representations for unsupervised learning of syntactic structure  Experiments on both POS induction and unsupervised dependency parsing tasks demonstrate the effectiveness of our proposed approach  Future work might explore more sophisticated invertible projections, or recurrent projections that jointly transform the entire input sequence 
Context 2: Introduction
Text simplification aims to reduce the lexical and structural complexity of a text, while still retaining the semantic meaning, which can help children, non-native speakers, and people with cognitive disabilities, to understand text better  One of the methods of automatic text simplification can be generally divided into three categories: lexical simplification (LS) BIBREF0 , BIBREF1 , rule-based BIBREF2 , and machine translation (MT) BIBREF3 , BIBREF4   LS is mainly used to simplify text by substituting infrequent and difficult words with frequent and easier words  However, there are several challenges for the LS approach: a great number of transformation rules are required for reasonable coverage and should be applied based on the specific context; third, the syntax and semantic meaning of the sentence is hard to retain  Rule-based approaches use hand-crafted rules for lexical and syntactic simplification, for example, substituting difficult words in a predefined vocabulary  However, such approaches need a lot of human-involvement to manually define these rules, and it is impossible to give all possible simplification rules  MT-based approach has attracted great attention in the last several years, which addresses text simplification as a monolingual machine translation problem translating from 'ordinary' and 'simplified' sentences 
In recent years, neural Machine Translation (NMT) is a newly-proposed deep learning approach and achieves very impressive results BIBREF5 , BIBREF6 , BIBREF7   Unlike the traditional phrased-based machine translation system which operates on small components separately, NMT system is being trained end-to-end, without the need to have external decoders, language models or phrase tables  Therefore, the existing architectures in NMT are used for text simplification BIBREF8 , BIBREF4   However, most recent work using NMT is limited to the training data that are scarce and expensive to build  Language models trained on simplified corpora have played a central role in statistical text simplification BIBREF9 , BIBREF10   One main reason is the amount of available simplified corpora typically far exceeds the amount of parallel data  The performance of models can be typically improved when trained on more data  Therefore, we expect simplified corpora to be especially helpful for NMT models 
In contrast to previous work, which uses the existing NMT models, we explore strategy to include simplified training corpora in the training process without changing the neural network architecture  We first propose to pair simplified training sentences with synthetic ordinary sentences during training, and treat this synthetic data as additional training data  We obtain synthetic ordinary sentences through back-translation, i e  an automatic translation of the simplified sentence into the ordinary sentence BIBREF11   Then, we mix the synthetic data into the original (simplified-ordinary) data to train NMT model  Experimental results on two publicly available datasets show that we can improve the text simplification quality of NMT models by mixing simplified sentences into the training set over NMT model only using the original training data 
Related Work
Automatic TS is a complicated natural language processing (NLP) task, which consists of lexical and syntactic simplification levels BIBREF12   It has attracted much attention recently as it could make texts more accessible to wider audiences, and used as a pre-processing step, improve performances of various NLP tasks and systems BIBREF13 , BIBREF14 , BIBREF15   Usually, hand-crafted, supervised, and unsupervised methods based on resources like English Wikipedia and Simple English Wikipedia (EW-SEW) BIBREF10 are utilized for extracting simplification rules  It is very easy to mix up the automatic TS task and the automatic summarization task BIBREF3 , BIBREF16 , BIBREF6   TS is different from text summarization as the focus of text summarization is to reduce the length and redundant content 
At the lexical level, lexical simplification systems often substitute difficult words using more common words, which only require a large corpus of regular text to obtain word embeddings to get words similar to the complex word BIBREF1 , BIBREF9   Biran et al  BIBREF0 adopted an unsupervised method for learning pairs of complex and simpler synonyms from a corpus consisting of Wikipedia and Simple Wikipedia  At the sentence level, a sentence simplification model was proposed by tree transformation based on statistical machine translation (SMT) BIBREF3   Woodsend and Lapata BIBREF17 presented a data-driven model based on a quasi-synchronous grammar, a formalism that can naturally capture structural mismatches and complex rewrite operations  Wubben et al  BIBREF18 proposed a phrase-based machine translation (PBMT) model that is trained on ordinary-simplified sentence pairs  Xu et al  BIBREF19 proposed a syntax-based machine translation model using simplification-specific objective functions and features to encourage simpler output 
Compared with SMT, neural machine translation (NMT) has shown to produce state-of-the-art results BIBREF5 , BIBREF7   The central approach of NMT is an encoder-decoder architecture implemented by recurrent neural networks, which can represent the input sequence as a vector, and then decode that vector into an output sequence  Therefore, NMT models were used for text simplification task, and achieved good results BIBREF8 , BIBREF4 , BIBREF20   The main limitation of the aforementioned NMT models for text simplification depended on the parallel ordinary-simplified sentence pairs  Because ordinary-simplified sentence pairs are expensive and time-consuming to build, the available largest data is EW-SEW that only have 296,402 sentence pairs  The dataset is insufficiency for NMT model if we want to NMT model can obtain the best parameters  Considering simplified data plays an important role in boosting fluency for phrase-based text simplification, and we investigate the use of simplified data for text simplification  We are the first to show that we can effectively adapt neural translation models for text simplifiation with simplified corpora 
Simplified Corpora
We collected a simplified dataset from Simple English Wikipedia that are freely available, which has been previously used for many text simplification methods BIBREF0 , BIBREF10 , BIBREF3   The simple English Wikipedia is pretty easy to understand than normal English Wikipedia  We downloaded all articles from Simple English Wikipedia  For these articles, we removed stubs, navigation pages and any article that consisted of a single sentence  We then split them into sentences with the Stanford CorNLP BIBREF21 , and deleted these sentences whose number of words are smaller than 10 or large than 40  After removing repeated sentences, we chose 600K sentences as the simplified data with 11 6M words, and the size of vocabulary is 82K 
Text Simplification using Neural Machine Translation
Our work is built on attention-based NMT BIBREF5 as an encoder-decoder network with recurrent neural networks (RNN), which simultaneously conducts dynamic alignment and generation of the target simplified sentence 
The encoder uses a bidirectional RNN that consists of forward and backward RNN  Given a source sentence INLINEFORM0 , the forward RNN and backward RNN calculate forward hidden states INLINEFORM1 and backward hidden states INLINEFORM2 , respectively  The annotation vector INLINEFORM3 is obtained by concatenating INLINEFORM4 and INLINEFORM5  
The decoder is a RNN that predicts a target simplificated sentence with Gated Recurrent Unit (GRU) BIBREF22   Given the previously generated target (simplified) sentence INLINEFORM0 , the probability of next target word INLINEFORM1 is DISPLAYFORM0
where INLINEFORM0 is a non-linear function, INLINEFORM1 is the embedding of INLINEFORM2 , and INLINEFORM3 is a decoding state for time step INLINEFORM4  
State INLINEFORM0 is calculated by DISPLAYFORM0
where INLINEFORM0 is the activation function GRU 
The INLINEFORM0 is the context vector computed as a weighted annotation INLINEFORM1 , computed by DISPLAYFORM0
where the weight INLINEFORM0 is computed by DISPLAYFORM0 DISPLAYFORM1
where INLINEFORM0 , INLINEFORM1 and INLINEFORM2 are weight matrices  The training objective is to maximize the likelihood of the training data  Beam search is employed for decoding 
Synthetic Simplified Sentences
We train an auxiliary system using NMT model from the simplified sentence to the ordinary sentence, which is first trained on the available parallel data  For leveraging simplified sentences to improve the quality of NMT model for text simplification, we propose to adapt the back-translation approach proposed by Sennrich et al  BIBREF11 to our scenario  More concretely, Given one sentence in simplified sentences, we use the simplified-ordinary system in translate mode with greedy decoding to translate it to the ordinary sentences, which is denoted as back-translation  This way, we obtain a synthetic parallel simplified-ordinary sentences  Both the synthetic sentences and the available parallel data are used as training data for the original NMT system 
Evaluation
We evaluate the performance of text simplification using neural machine translation on available parallel sentences and additional simplified sentences 
Dataset  We use two simplification datasets (WikiSmall and WikiLarge)  WikiSmall consists of ordinary and simplified sentences from the ordinary and simple English Wikipedias, which has been used as benchmark for evaluating text simplification BIBREF17 , BIBREF18 , BIBREF8   The training set has 89,042 sentence pairs, and the test set has 100 pairs  WikiLarge is also from Wikipedia corpus whose training set contains 296,402 sentence pairs BIBREF19 , BIBREF20   WikiLarge includes 8 (reference) simplifications for 2,359 sentences split into 2,000 for development and 359 for testing 
Metrics  Three metrics in text simplification are chosen in this paper  BLEU BIBREF5 is one traditional machine translation metric to assess the degree to which translated simplifications differed from reference simplifications  FKGL measures the readability of the output BIBREF23   A small FKGL represents simpler output  SARI is a recent text-simplification metric by comparing the output against the source and reference simplifications BIBREF20  
We evaluate the output of all systems using human evaluation  The metric is denoted as Simplicity BIBREF8   The three non-native fluent English speakers are shown reference sentences and output sentences  They are asked whether the output sentence is much simpler (+2), somewhat simpler (+1), equally (0), somewhat more difficult (-1), and much more difficult (-2) than the reference sentence 
Methods  We use OpenNMT BIBREF24 as the implementation of the NMT system for all experiments BIBREF5   We generally follow the default settings and training procedure described by Klein et al (2017)  We replace out-of-vocabulary words with a special UNK symbol  At prediction time, we replace UNK words with the highest probability score from the attention layer  OpenNMT system used on parallel data is the baseline system  To obtain a synthetic parallel training set, we back-translate a random sample of 100K sentences from the collected simplified corpora  OpenNMT used on parallel data and synthetic data is our model  The benchmarks are run on a Intel(R) Core(TM) i7-5930K CPU@3 50GHz, 32GB Mem, trained on 1 GPU GeForce GTX 1080 (Pascal) with CUDA v  8 0 
We choose three statistical text simplification systems  PBMT-R is a phrase-based method with a reranking post-processing step BIBREF18   Hybrid performs sentence splitting and deletion operations based on discourse representation structures, and then simplifies sentences with PBMT-R BIBREF25   SBMT-SARI BIBREF19 is syntax-based translation model using PPDB paraphrase database BIBREF26 and modifies tuning function (using SARI)  We choose two neural text simplification systems  NMT is a basic attention-based encoder-decoder model which uses OpenNMT framework to train with two LSTM layers, hidden states of size 500 and 500 hidden units, SGD optimizer, and a dropout rate of 0 3 BIBREF8   Dress is an encoder-decoder model coupled with a deep reinforcement learning framework, and the parameters are chosen according to the original paper BIBREF20   For the experiments with synthetic parallel data, we back-translate a random sample of 60 000 sentences from the collected simplified sentences into ordinary sentences  Our model is trained on synthetic data and the available parallel data, denoted as NMT+synthetic 
Results  Table 1 shows the results of all models on WikiLarge dataset  We can see that our method (NMT+synthetic) can obtain higher BLEU, lower FKGL and high SARI compared with other models, except Dress on FKGL and SBMT-SARI on SARI  It verified that including synthetic data during training is very effective, and yields an improvement over our baseline NMF by 2 11 BLEU, 1 7 FKGL and 1 07 SARI  We also substantially outperform Dress, who previously reported SOTA result  The results of our human evaluation using Simplicity are also presented in Table 1  NMT on synthetic data is significantly better than PBMT-R, Dress, and SBMT-SARI on Simplicity  It indicates that our method with simplified data is effective at creating simpler output 
Results on WikiSmall dataset are shown in Table 2  We see substantial improvements (6 37 BLEU) than NMT from adding simplified training data with synthetic ordinary sentences  Compared with statistical machine translation models (PBMT-R, Hybrid, SBMT-SARI), our method (NMT+synthetic) still have better results, but slightly worse FKGL and SARI  Similar to the results in WikiLarge, the results of our human evaluation using Simplicity outperforms the other models  In conclusion, Our method produces better results comparing with the baselines, which demonstrates the effectiveness of adding simplified training data 
Conclusion
In this paper, we propose one simple method to use simplified corpora during training of NMT systems, with no changes to the network architecture  In the experiments on two datasets, we achieve substantial gains in all tasks, and new SOTA results, via back-translation of simplified sentences into the ordinary sentences, and treating this synthetic data as additional training data  Because we do not change the neural network architecture to integrate simplified corpora, our method can be easily applied to other Neural Text Simplification (NTS) systems  We expect that the effectiveness of our method not only varies with the quality of the NTS system used for back-translation, but also depends on the amount of available parallel and simplified corpora  In the paper, we have only utilized data from Wikipedia for simplified sentences  In the future, many other text sources are available and the impact of not only size, but also of domain should be investigated 
Context 3: Introduction
Data imbalance is a common issue in a variety of NLP tasks such as tagging and machine reading comprehension  Table TABREF3 gives concrete examples: for the Named Entity Recognition (NER) task BIBREF2, BIBREF3, most tokens are backgrounds with tagging class $O$  Specifically, the number of tokens tagging class $O$ is 5 times as many as those with entity labels for the CoNLL03 dataset and 8 times for the OntoNotes5 0 dataset; Data-imbalanced issue is more severe for MRC tasks BIBREF4, BIBREF5, BIBREF6, BIBREF7, BIBREF8 with the value of negative-positive ratio being 50-200 
Data imbalance results in the following two issues: (1) the training-test discrepancy: Without balancing the labels, the learning process tends to converge to a point that strongly biases towards class with the majority label  This actually creates a discrepancy between training and test: at training time, each training instance contributes equally to the objective function while at test time, F1 score concerns more about positive examples; (2) the overwhelming effect of easy-negative examples  As pointed out by meng2019dsreg, significantly large number of negative examples also means that the number of easy-negative example is large  The huge number of easy examples tends to overwhelm the training, making the model not sufficiently learned to distinguish between positive examples and hard-negative examples  The cross-entropy objective (CE for short) or maximum likelihood (MLE) objective, which is widely adopted as the training objective for data-imbalanced NLP tasks BIBREF9, BIBREF10, BIBREF11, BIBREF12, BIBREF13, BIBREF14, BIBREF15, handles neither of the issues 
To handle the first issue, we propose to replace CE or MLE with losses based on the Sørensen–Dice coefficient BIBREF0 or Tversky index BIBREF1  The Sørensen–Dice coefficient, dice loss for short, is the harmonic mean of precision and recall  It attaches equal importance to false positives (FPs) and false negatives (FNs) and is thus more immune to data-imbalanced datasets  Tversky index extends dice loss by using a weight that trades precision and recall, which can be thought as the approximation of the $F_{\beta }$ score, and thus comes with more flexibility  Therefore, We use dice loss or Tversky index to replace CE loss to address the first issue 
Only using dice loss or Tversky index is not enough since they are unable to address the dominating influence of easy-negative examples  This is intrinsically because dice loss is actually a hard version of the F1 score  Taking the binary classification task as an example, at test time, an example will be classified as negative as long as its probability is smaller than 0 5, but training will push the value to 0 as much as possible  This gap isn't a big issue for balanced datasets, but is extremely detrimental if a big proportion of training examples are easy-negative ones: easy-negative examples can easily dominate training since their probabilities can be pushed to 0 fairly easily  Meanwhile, the model can hardly distinguish between hard-negative examples and positive ones  Inspired by the idea of focal loss BIBREF16 in computer vision, we propose a dynamic weight adjusting strategy, which associates each training example with a weight in proportion to $(1-p)$, and this weight dynamically changes as training proceeds  This strategy helps to deemphasize confident examples during training as their $p$ approaches the value of 1, makes the model attentive to hard-negative examples, and thus alleviates the dominating effect of easy-negative examples 
Combing both strategies, we observe significant performance boosts on a wide range of data imbalanced NLP tasks  Notably, we are able to achieve SOTA results on CTB5 (97 92, +1 86), CTB6 (96 57, +1 80) and UD1 4 (96 98, +2 19) for the POS task; SOTA results on CoNLL03 (93 33, +0 29), OntoNotes5 0 (92 07, +0 96)), MSRA 96 72(+0 97) and OntoNotes4 0 (84 47,+2 36) for the NER task; along with competitive results on the tasks of machine reading comprehension and paraphrase identification 
The rest of this paper is organized as follows: related work is presented in Section 2  We describe different training objectives in Section 3  Experimental results are presented in Section 4  We perform ablation studies in Section 5, followed by a brief conclusion in Section 6 
Related Work ::: Data Resample
The idea of weighting training examples has a long history  Importance sampling BIBREF17 assigns weights to different samples and changes the data distribution  Boosting algorithms such as AdaBoost BIBREF18 select harder examples to train subsequent classifiers  Similarly, hard example mining BIBREF19 downsamples the majority class and exploits the most difficult examples  Oversampling BIBREF20, BIBREF21 is used to balance the data distribution  Another line of data resampling is to dynamically control the weights of examples as training proceeds  For example, focal loss BIBREF16 used a soft weighting scheme that emphasizes harder examples during training  In self-paced learning BIBREF22, example weights are obtained through optimizing the weighted training loss which encourages learning easier examples first  At each training step, self-paced learning algorithm optimizes model parameters and example weights jointly  Other works BIBREF23, BIBREF24 adjusted the weights of different training examples based on training loss  Besides, recent work BIBREF25, BIBREF26 proposed to learn a separate network to predict sample weights 
Related Work ::: Data Imbalance Issue in Object Detection
The background-object label imbalance issue is severe and thus well studied in the field of object detection BIBREF27, BIBREF28, BIBREF29, BIBREF30, BIBREF31  The idea of hard negative mining (HNM) BIBREF30 has gained much attention recently  shrivastava2016ohem proposed the online hard example mining (OHEM) algorithm in an iterative manner that makes training progressively more difficult, and pushes the model to learn better  ssd2016liu sorted all of the negative samples based on the confidence loss and picking the training examples with the negative-positive ratio at 3:1  pang2019rcnn proposed a novel method called IoU-balanced sampling and aploss2019chen designed a ranking model to replace the conventional classification task with a average-precision loss to alleviate the class imbalance issue  The efforts made on object detection have greatly inspired us to solve the data imbalance issue in NLP 
Losses ::: Notation
For illustration purposes, we use the binary classification task to demonstrate how different losses work  The mechanism can be easily extended to multi-class classification 
Let $\lbrace x_i\rbrace $ denote a set of instances  Each $x_i$ is associated with a golden label vector $y_i = [y_{i0},y_{i1} ]$, where $y_{i1}\in \lbrace 0,1\rbrace $ and $y_{i0}\in \lbrace 0,1\rbrace $ respectively denote the positive and negative classes, and thus $y_i$ can be either $[0,1]$ or $[0,1]$  Let $p_i = [p_{i0},p_{i1} ]$ denote the probability vector, and $p_{i1}$ and $p_{i0}$ respectively denote the probability that a model assigns the positive and negative label to $x_i$ 
Losses ::: Cross Entropy Loss
The vanilla cross entropy (CE) loss is given by:
As can be seen from Eq DISPLAY_FORM8, each $x_i$ contributes equally to the final objective  Two strategies are normally used to address the the case where we wish that not all $x_i$ are treated equal: associating different classes with different weighting factor $\alpha $ or resampling the datasets  For the former, Eq DISPLAY_FORM8 is adjusted as follows:
where $\alpha _i\in [0,1]$ may be set by the inverse class frequency or treated as a hyperparameter to set by cross validation  In this work, we use $\lg (\frac{n-n_t}{n_t}+K)$ to calculate the coefficient $\alpha $, where $n_t$ is the number of samples with class $t$ and $n$ is the total number of samples in the training set  $K$ is a hyperparameter to tune  The data resampling strategy constructs a new dataset by sampling training examples from the original dataset based on human-designed criteria, e g , extract equal training samples from each class  Both strategies are equivalent to changing the data distribution and thus are of the same nature  Empirically, these two methods are not widely used due to the trickiness of selecting $\alpha $ especially for multi-class classification tasks and that inappropriate selection can easily bias towards rare classes BIBREF32 
Losses ::: Dice coefficient and Tversky index
Sørensen–Dice coefficient BIBREF0, BIBREF33, dice coefficient (DSC) for short, is a F1-oriented statistic used to gauge the similarity of two sets  Given two sets $A$ and $B$, the dice coefficient between them is given as follows:
In our case, $A$ is the set that contains of all positive examples predicted by a specific model, and $B$ is the set of all golden positive examples in the dataset  When applied to boolean data with the definition of true positive (TP), false positive (FP), and false negative (FN), it can be then written as follows:
For an individual example $x_i$, its corresponding DSC loss is given as follows:
As can be seen, for a negative example with $y_{i1}=0$, it does not contribute to the objective  For smoothing purposes, it is common to add a $\gamma $ factor to both the nominator and the denominator, making the form to be as follows:
As can be seen, negative examples, with $y_{i1}$ being 0 and DSC being $\frac{\gamma }{ p_{i1}+\gamma }$, also contribute to the training  Additionally, milletari2016v proposed to change the denominator to the square form for faster convergence, which leads to the following dice loss (DL):
Another version of DL is to directly compute set-level dice coefficient instead of the sum of individual dice coefficient  We choose the latter due to ease of optimization 
Tversky index (TI), which can be thought as the approximation of the $F_{\beta }$ score, extends dice coefficient to a more general case  Given two sets $A$ and $B$, tversky index is computed as follows:
Tversky index offers the flexibility in controlling the tradeoff between false-negatives and false-positives  It degenerates to DSC if $\alpha =\beta =0 5$  The Tversky loss (TL) for the training set $\lbrace x_i,y_i\rbrace $ is thus as follows:
Losses ::: Self-adusting Dice Loss
Consider a simple case where the dataset consists of only one example $x_i$, which is classified as positive as long as $p_{i1}$ is larger than 0 5  The computation of $F1$ score is actually as follows:
Comparing Eq DISPLAY_FORM14 with Eq DISPLAY_FORM22, we can see that Eq DISPLAY_FORM14 is actually a soft form of $F1$, using a continuous $p$ rather than the binary $\mathbb {I}( p_{i1}>0 5)$  This gap isn't a big issue for balanced datasets, but is extremely detrimental if a big proportion of training examples are easy-negative ones: easy-negative examples can easily dominate training since their probabilities can be pushed to 0 fairly easily  Meanwhile, the model can hardly distinguish between hard-negative examples and positive ones, which has a huge negative effect on the final F1 performance 
To address this issue, we propose to multiply the soft probability $p$ with a decaying factor $(1-p)$, changing Eq DISPLAY_FORM22 to the following form:
One can think $(1-p_{i1})$ as a weight associated with each example, which changes as training proceeds  The intuition of changing $p_{i1}$ to $(1-p_{i1}) p_{i1}$ is to push down the weight of easy examples  For easy examples whose probability are approaching 0 or 1, $(1-p_{i1}) p_{i1}$ makes the model attach significantly less focus to them  Figure FIGREF23 gives gives an explanation from the perspective in derivative: the derivative of $\frac{(1-p)p}{1+(1-p)p}$ with respect to $p$ approaches 0 immediately after $p$ approaches 0, which means the model attends less to examples once they are correctly classified 
A close look at Eq DISPLAY_FORM14 reveals that it actually mimics the idea of focal loss (FL for short) BIBREF16 for object detection in vision  Focal loss was proposed for one-stage object detector to handle foreground-background tradeoff encountered during training  It down-weights the loss assigned to well-classified examples by adding a $(1-p)^{\beta }$ factor, leading the final loss to be $(1-p)^{\beta }\log p$ 
In Table TABREF18, we show the losses used in our experiments, which is described in the next section 
Experiments
We evaluate the proposed method on four NLP tasks: part-of-speech tagging, named entity recognition, machine reading comprehension and paraphrase identification  Baselines in our experiments are optimized by using the standard cross-entropy training objective 
Experiments ::: Part-of-Speech Tagging
Part-of-speech tagging (POS) is the task of assigning a label (e g , noun, verb, adjective) to each word in a given text  In this paper, we choose BERT as the backbone and conduct experiments on three Chinese POS datasets  We report the span-level micro-averaged precision, recall and F1 for evaluation  Hyperparameters are tuned on the corresponding development set of each dataset 
Experiments ::: Part-of-Speech Tagging ::: Datasets
We conduct experiments on the widely used Chinese Treebank 5 0, 6 0 as well as UD1 4 
CTB5 is a Chinese dataset for tagging and parsing, which contains 507,222 words, 824,983 characters and 18,782 sentences extracted from newswire sources 
CTB6 is an extension of CTB5, containing 781,351 words, 1,285,149 characters and 28,295 sentences 
UD is the abbreviation of Universal Dependencies, which is a framework for consistent annotation of grammar (parts of speech, morphological features, and syntactic dependencies) across different human languages  In this work, we use UD1 4 for Chinese POS tagging 
Experiments ::: Part-of-Speech Tagging ::: Baselines
We use the following baselines:
Joint-POS: shao2017character jointly learns Chinese word segmentation and POS 
Lattice-LSTM: lattice2018zhang constructs a word-character lattice 
Bert-Tagger: devlin2018bert treats part-of-speech as a tagging task 
Experiments ::: Part-of-Speech Tagging ::: Results
Table presents the experimental results on the POS task  As can be seen, the proposed DSC loss outperforms the best baseline results by a large margin, i e , outperforming BERT-tagger by +1 86 in terms of F1 score on CTB5, +1 80 on CTB6 and +2 19 on UD1 4  As far as we are concerned, we are achieving SOTA performances on the three datasets  Weighted cross entropy and focal loss only gain a little performance improvement on CTB5 and CTB6, and the dice loss obtains huge gain on CTB5 but not on CTB6, which indicates the three losses are not consistently robust in resolving the data imbalance issue  The proposed DSC loss performs robustly on all the three datasets 
Experiments ::: Named Entity Recognition
Named entity recognition (NER) refers to the task of detecting the span and semantic category of entities from a chunk of text  Our implementation uses the current state-of-the-art BERT-MRC model proposed by xiaoya2019ner as a backbone  For English datasets, we use BERT$_\text{Large}$ English checkpoints, while for Chinese we use the official Chinese checkpoints  We report span-level micro-averaged precision, recall and F1-score  Hyperparameters are tuned on the development set of each dataset 
Experiments ::: Named Entity Recognition ::: Datasets
For the NER task, we consider both Chinese datasets, i e , OntoNotes4 0 BIBREF34 and MSRA BIBREF35, and English datasets, i e , CoNLL2003 BIBREF36 and OntoNotes5 0 BIBREF37 
CoNLL2003 is an English dataset with 4 entity types: Location, Organization, Person and Miscellaneous  We followed data processing protocols in BIBREF14 
English OntoNotes5 0 consists of texts from a wide variety of sources and contains 18 entity types  We use the standard train/dev/test split of CoNLL2012 shared task 
Chinese MSRA performs as a Chinese benchmark dataset containing 3 entity types  Data in MSRA is collected from news domain  Since the development set is not provided in the original MSRA dataset, we randomly split the training set into training and development splits by 9:1  We use the official test set for evaluation 
Chinese OntoNotes4 0 is a Chinese dataset and consists of texts from news domain, which has 18 entity types  In this paper, we take the same data split as wu2019glyce did 
Experiments ::: Named Entity Recognition ::: Baselines
We use the following baselines:
ELMo: a tagging model from peters2018deep 
Lattice-LSTM: lattice2018zhang constructs a word-character lattice, only used in Chinese datasets 
CVT: from kevin2018cross, which uses Cross-View Training(CVT) to improve the representations of a Bi-LSTM encoder 
Bert-Tagger: devlin2018bert treats NER as a tagging task 
Glyce-BERT: wu2019glyce combines glyph information with BERT pretraining 
BERT-MRC: The current SOTA model for both Chinese and English NER datasets proposed by xiaoya2019ner, which formulate NER as machine reading comprehension task 
Experiments ::: Named Entity Recognition ::: Results
Table shows experimental results on NER datasets  For English datasets including CoNLL2003 and OntoNotes5 0, our proposed method outperforms BERT-MRCBIBREF38 by +0 29 and +0 96 respectively  We observe huge performance boosts on Chinese datasets, achieving F1 improvements by +0 97 and +2 36 on MSRA and OntoNotes4 0, respectively  As far as we are concerned, we are setting new SOTA performances on all of the four NER datasets 
Experiments ::: Machine Reading Comprehension
Machine reading comprehension (MRC) BIBREF39, BIBREF40, BIBREF41, BIBREF40, BIBREF42, BIBREF15 has become a central task in natural language understanding  MRC in the SQuAD-style is to predict the answer span in the passage given a question and the passage  In this paper, we choose the SQuAD-style MRC task and report Extract Match (EM) in addition to F1 score on validation set  All hyperparameters are tuned on the development set of each dataset 
Experiments ::: Machine Reading Comprehension ::: Datasets
The following five datasets are used for MRC task: SQuAD v1 1, SQuAD v2 0 BIBREF4, BIBREF6 and Quoref BIBREF8 
SQuAD v1 1 and SQuAD v2 0 are the most widely used QA benchmarks  SQuAD1 1 is a collection of 100K crowdsourced question-answer pairs, and SQuAD2 0 extends SQuAD1 1 allowing no short answer exists in the provided passage 
Quoref is a QA dataset which tests the coreferential reasoning capability of reading comprehension systems, containing 24K questions over 4 7K paragraphs from Wikipedia 
Experiments ::: Machine Reading Comprehension ::: Baselines
We use the following baselines:
QANet: qanet2018 builds a model based on convolutions and self-attention  Convolution to model local interactions and self-attention to model global interactions 
BERT: devlin2018bert treats NER as a tagging task 
XLNet: xlnet2019 proposes a generalized autoregressive pretraining method that enables learning bidirectional contexts 
Experiments ::: Machine Reading Comprehension ::: Results
Table shows the experimental results for MRC tasks  With either BERT or XLNet, our proposed DSC loss obtains significant performance boost on both EM and F1  For SQuADv1 1, our proposed method outperforms XLNet by +1 25 in terms of F1 score and +0 84 in terms of EM and achieves 87 65 on EM and 89 51 on F1 for SQuAD v2 0  Moreover, on QuoRef, the proposed method surpasses XLNet results by +1 46 on EM and +1 41 on F1  Another observation is that, XLNet outperforms BERT by a huge margin, and the proposed DSC loss can obtain further performance improvement by an average score above 1 0 in terms of both EM and F1, which indicates the DSC loss is complementary to the model structures 
Experiments ::: Paraphrase Identification
Paraphrases are textual expressions that have the same semantic meaning using different surface words  Paraphrase identification (PI) is the task of identifying whether two sentences have the same meaning or not  We use BERT BIBREF11 and XLNet BIBREF43 as backbones and report F1 score for comparison  Hyperparameters are tuned on the development set of each dataset 
Experiments ::: Paraphrase Identification ::: Datasets
We conduct experiments on two widely used datasets for PI task: MRPC BIBREF44 and QQP 
MRPC is a corpus of sentence pairs automatically extracted from online news sources, with human annotations of whether the sentence pairs are semantically equivalent  The MRPC dataset has imbalanced classes (68% positive, 32% for negative) 
QQP is a collection of question pairs from the community question-answering website Quora  The class distribution in QQP is also unbalanced (37% positive, 63% negative) 
Experiments ::: Paraphrase Identification ::: Results
Table shows the results for PI task  We find that replacing the training objective with DSC introduces performance boost for both BERT and XLNet  Using DSC loss improves the F1 score by +0 58 for MRPC and +0 73 for QQP 
Ablation Studies ::: The Effect of Dice Loss on Accuracy-oriented Tasks
We argue that the most commonly used cross-entropy objective is actually accuracy-oriented, whereas the proposed dice loss (DL) performs as a hard version of F1-score  To explore the effect of the dice loss on accuracy-oriented tasks such as text classification, we conduct experiments on the Stanford Sentiment Treebank sentiment classification datasets including SST-2 and SST-5  We fine-tune BERT$_\text{Large}$ with different training objectives  Experiment results for SST are shown in   For SST-5, BERT with CE achieves 55 57 in terms of accuracy, with DL and DSC losses slightly degrade the accuracy performance and achieve 54 63 and 55 19, respectively  For SST-2, BERT with CE achieves 94 9 in terms of accuracy  The same as SST-5, we observe a slight performance drop with DL and DSC, which means that the dice loss actually works well for F1 but not for accuracy 
Ablation Studies ::: The Effect of Hyperparameters in Tversky index
As mentioned in Section SECREF10, Tversky index (TI) offers the flexibility in controlling the tradeoff between false-negatives and false-positives  In this subsection, we explore the effect of hyperparameters (i e , $\alpha $ and $\beta $) in TI to test how they manipulate the tradeoff  We conduct experiments on the Chinese OntoNotes4 0 NER dataset and English QuoRef MRC dataset to examine the influence of tradeoff between precision and recall  Experiment results are shown in Table   The highest F1 for Chinese OntoNotes4 0 is 84 67 when $\alpha $ is set to 0 6 while for QuoRef, the highest F1 is 68 44 when $\alpha $ is set to 0 4  In addition, we can observe that the performance varies a lot as $\alpha $ changes in distinct datasets, which shows that the hyperparameters $\alpha ,\beta $ play an important role in the proposed method 
Conclusion
In this paper, we alleviate the severe data imbalance issue in NLP tasks  We propose to use dice loss in replacement of the standard cross-entropy loss, which performs as a soft version of F1 score  Using dice loss can help narrow the gap between training objectives and evaluation metrics  Empirically, we show that the proposed training objective leads to significant performance boost for part-of-speech, named entity recognition, machine reading comprehension and paraphrase identification tasks 

ground_truth: Yes

Unanswerable

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. what language does this paper focus on?
Context 4: Introduction
Recently, deep learning algorithms have successfully addressed problems in various fields, such as image classification, machine translation, speech recognition, text-to-speech generation and other machine learning related areas BIBREF0 , BIBREF1 , BIBREF2   Similarly, substantial improvements in performance have been obtained when deep learning algorithms have been applied to statistical speech processing BIBREF3   These fundamental improvements have led researchers to investigate additional topics related to human nature, which have long been objects of study  One such topic involves understanding human emotions and reflecting it through machine intelligence, such as emotional dialogue models BIBREF4 , BIBREF5  
In developing emotionally aware intelligence, the very first step is building robust emotion classifiers that display good performance regardless of the application; this outcome is considered to be one of the fundamental research goals in affective computing BIBREF6   In particular, the speech emotion recognition task is one of the most important problems in the field of paralinguistics  This field has recently broadened its applications, as it is a crucial factor in optimal human-computer interactions, including dialog systems  The goal of speech emotion recognition is to predict the emotional content of speech and to classify speech according to one of several labels (i e , happy, sad, neutral, and angry)  Various types of deep learning methods have been applied to increase the performance of emotion classifiers; however, this task is still considered to be challenging for several reasons  First, insufficient data for training complex neural network-based models are available, due to the costs associated with human involvement  Second, the characteristics of emotions must be learned from low-level speech signals  Feature-based models display limited skills when applied to this problem 
To overcome these limitations, we propose a model that uses high-level text transcription, as well as low-level audio signals, to utilize the information contained within low-resource datasets to a greater degree  Given recent improvements in automatic speech recognition (ASR) technology BIBREF7 , BIBREF2 , BIBREF8 , BIBREF9 , speech transcription can be carried out using audio signals with considerable skill  The emotional content of speech is clearly indicated by the emotion words contained in a sentence BIBREF10 , such as “lovely” and “awesome,” which carry strong emotions compared to generic (non-emotion) words, such as “person” and “day ” Thus, we hypothesize that the speech emotion recognition model will be benefit from the incorporation of high-level textual input 
In this paper, we propose a novel deep dual recurrent encoder model that simultaneously utilizes audio and text data in recognizing emotions from speech  Extensive experiments are conducted to investigate the efficacy and properties of the proposed model  Our proposed model outperforms previous state-of-the-art methods by 68 8% to 71 8% when applied to the IEMOCAP dataset, which is one of the most well-studied datasets  Based on an error analysis of the models, we show that our proposed model accurately identifies emotion classes  Moreover, the neutral class misclassification bias frequently exhibited by previous models, which focus on audio features, is less pronounced in our model 
Related work
Classical machine learning algorithms, such as hidden Markov models (HMMs), support vector machines (SVMs), and decision tree-based methods, have been employed in speech emotion recognition problems BIBREF11 , BIBREF12 , BIBREF13   Recently, researchers have proposed various neural network-based architectures to improve the performance of speech emotion recognition  An initial study utilized deep neural networks (DNNs) to extract high-level features from raw audio data and demonstrated its effectiveness in speech emotion recognition BIBREF14   With the advancement of deep learning methods, more complex neural-based architectures have been proposed  Convolutional neural network (CNN)-based models have been trained on information derived from raw audio signals using spectrograms or audio features such as Mel-frequency cepstral coefficients (MFCCs) and low-level descriptors (LLDs) BIBREF15 , BIBREF16 , BIBREF17   These neural network-based models are combined to produce higher-complexity models BIBREF18 , BIBREF19 , and these models achieved the best-recorded performance when applied to the IEMOCAP dataset 
Another line of research has focused on adopting variant machine learning techniques combined with neural network-based models  One researcher utilized the multiobject learning approach and used gender and naturalness as auxiliary tasks so that the neural network-based model learned more features from a given dataset BIBREF20   Another researcher investigated transfer learning methods, leveraging external data from related domains BIBREF21  
As emotional dialogue is composed of sound and spoken content, researchers have also investigated the combination of acoustic features and language information, built belief network-based methods of identifying emotional key phrases, and assessed the emotional salience of verbal cues from both phoneme sequences and words BIBREF22 , BIBREF23   However, none of these studies have utilized information from speech signals and text sequences simultaneously in an end-to-end learning neural network-based model to classify emotions 
Model
This section describes the methodologies that are applied to the speech emotion recognition task  We start by introducing the recurrent encoder model for the audio and text modalities individually  We then propose a multimodal approach that encodes both audio and textual information simultaneously via a dual recurrent encoder 
Audio Recurrent Encoder (ARE)
Motivated by the architecture used in BIBREF24 , BIBREF25 , we build an audio recurrent encoder (ARE) to predict the class of a given audio signal  Once MFCC features have been extracted from an audio signal, a subset of the sequential features is fed into the RNN (i e , gated recurrent units (GRUs)), which leads to the formation of the network's internal hidden state INLINEFORM0 to model the time series patterns  This internal hidden state is updated at each time step with the input data INLINEFORM1 and the hidden state of the previous time step INLINEFORM2 as follows: DISPLAYFORM0
where INLINEFORM0 is the RNN function with weight parameter INLINEFORM1 , INLINEFORM2 represents the hidden state at t- INLINEFORM3 time step, and INLINEFORM4 represents the t- INLINEFORM5 MFCC features in INLINEFORM6   After encoding the audio signal INLINEFORM7 with the RNN, the last hidden state of the RNN, INLINEFORM8 , is considered to be the representative vector that contains all of the sequential audio data  This vector is then concatenated with another prosodic feature vector, INLINEFORM9 , to generate a more informative vector representation of the signal, INLINEFORM10   The MFCC and the prosodic features are extracted from the audio signal using the openSMILE toolkit BIBREF26 , INLINEFORM11 , respectively  Finally, the emotion class is predicted by applying the softmax function to the vector INLINEFORM12   For a given audio sample INLINEFORM13 , we assume that INLINEFORM14 is the true label vector, which contains all zeros but contains a one at the correct class, and INLINEFORM15 is the predicted probability distribution from the softmax layer  The training objective then takes the following form: DISPLAYFORM0
where INLINEFORM0 is the calculated representative vector of the audio signal with dimensionality INLINEFORM1   The INLINEFORM2 and the bias INLINEFORM3 are learned model parameters  C is the total number of classes, and N is the total number of samples used in training  The upper part of Figure shows the architecture of the ARE model 
Text Recurrent Encoder (TRE)
We assume that speech transcripts can be extracted from audio signals with high accuracy, given the advancement of ASR technologies BIBREF7   We attempt to use the processed textual information as another modality in predicting the emotion class of a given signal  To use textual information, a speech transcript is tokenized and indexed into a sequence of tokens using the Natural Language Toolkit (NLTK) BIBREF27   Each token is then passed through a word-embedding layer that converts a word index to a corresponding 300-dimensional vector that contains additional contextual meaning between words  The sequence of embedded tokens is fed into a text recurrent encoder (TRE) in such a way that the audio MFCC features are encoded using the ARE represented by equation EQREF2   In this case, INLINEFORM0 is the t- INLINEFORM1 embedded token from the text input  Finally, the emotion class is predicted from the last hidden state of the text-RNN using the softmax function 
We use the same training objective as the ARE model, and the predicted probability distribution for the target class is as follows: DISPLAYFORM0
where INLINEFORM0 is last hidden state of the text-RNN, INLINEFORM1 , and the INLINEFORM2 and bias INLINEFORM3 are learned model parameters  The lower part of Figure indicates the architecture of the TRE model 
Multimodal Dual Recurrent Encoder (MDRE)
We present a novel architecture called the multimodal dual recurrent encoder (MDRE) to overcome the limitations of existing approaches  In this study, we consider multiple modalities, such as MFCC features, prosodic features and transcripts, which contain sequential audio information, statistical audio information and textual information, respectively  These types of data are the same as those used in the ARE and TRE cases  The MDRE model employs two RNNs to encode data from the audio signal and textual inputs independently  The audio-RNN encodes MFCC features from the audio signal using equation EQREF2   The last hidden state of the audio-RNN is concatenated with the prosodic features to form the final vector representation INLINEFORM0 , and this vector is then passed through a fully connected neural network layer to form the audio encoding vector A  On the other hand, the text-RNN encodes the word sequence of the transcript using equation EQREF2   The final hidden states of the text-RNN are also passed through another fully connected neural network layer to form a textual encoding vector T  Finally, the emotion class is predicted by applying the softmax function to the concatenation of the vectors A and T  We use the same training objective as the ARE model, and the predicted probability distribution for the target class is as follows: DISPLAYFORM0
where INLINEFORM0 is the feed-forward neural network with weight parameter INLINEFORM1 , and INLINEFORM2 , INLINEFORM3 are final encoding vectors from the audio-RNN and text-RNN, respectively  INLINEFORM4 and the bias INLINEFORM5 are learned model parameters 
Multimodal Dual Recurrent Encoder with Attention (MDREA)
Inspired by the concept of the attention mechanism used in neural machine translation BIBREF28 , we propose a novel multimodal attention method to focus on the specific parts of a transcript that contain strong emotional information, conditioning on the audio information  Figure shows the architecture of the MDREA model  First, the audio data and text data are encoded with the audio-RNN and text-RNN using equation EQREF2   We then consider the final audio encoding vector INLINEFORM0 as a context vector  As seen in equation EQREF9 , during each time step t, the dot product between the context vector e and the hidden state of the text-RNN at each t-th sequence INLINEFORM1 is evaluated to calculate a similarity score INLINEFORM2   Using this score INLINEFORM3 as a weight parameter, the weighted sum of the sequences of the hidden state of the text-RNN, INLINEFORM4 , is calculated to generate an attention-application vector Z  This attention-application vector is concatenated with the final encoding vector of the audio-RNN INLINEFORM5 (equation EQREF7 ), which will be passed through the softmax function to predict the emotion class  We use the same training objective as the ARE model, and the predicted probability distribution for the target class is as follows: DISPLAYFORM0
where INLINEFORM0 and the bias INLINEFORM1 are learned model parameters 
Dataset
We evaluate our model using the Interactive Emotional Dyadic Motion Capture (IEMOCAP) BIBREF18 dataset  This dataset was collected following theatrical theory in order to simulate natural dyadic interactions between actors

ground_truth: English

Simple English

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How are weights dynamically adjusted?
Context 4:  We use categorical evaluations with majority agreement  We use only four emotional categories happy, sad, angry, and neutral to compare the performance of our model with other research using the same categories  The IEMOCAP dataset includes five sessions, and each session contains utterances from two speakers (one male and one female)  This data collection process resulted in 10 unique speakers  For consistent comparison with previous work, we merge the excitement dataset with the happiness dataset  The final dataset contains a total of 5531 utterances (1636 happy, 1084 sad, 1103 angry, 1708 neutral) 
Feature extraction
To extract speech information from audio signals, we use MFCC values, which are widely used in analyzing audio signals  The MFCC feature set contains a total of 39 features, which include 12 MFCC parameters (1-12) from the 26 Mel-frequency bands and log-energy parameters, 13 delta and 13 acceleration coefficients The frame size is set to 25 ms at a rate of 10 ms with the Hamming function  According to the length of each wave file, the sequential step of the MFCC features is varied  To extract additional information from the data, we also use prosodic features, which show effectiveness in affective computing  The prosodic features are composed of 35 features, which include the F0 frequency, the voicing probability, and the loudness contours  All of these MFCC and prosodic features are extracted from the data using the OpenSMILE toolkit BIBREF26  
Implementation details
Among the variants of the RNN function, we use GRUs as they yield comparable performance to that of the LSTM and include a smaller number of weight parameters BIBREF29   We use a max encoder step of 750 for the audio input, based on the implementation choices presented in BIBREF30 and 128 for the text input because it covers the maximum length of the transcripts  The vocabulary size of the dataset is 3,747, including the “_UNK_" token, which represents unknown words, and the “_PAD_" token, which is used to indicate padding information added while preparing mini-batch data  The number of hidden units and the number of layers in the RNN for each model (ARE, TRE, MDRE and MDREA) are selected based on extensive hyperparameter search experiments  The weights of the hidden units are initialized using orthogonal weights BIBREF31 ], and the text embedding layer is initialized from pretrained word-embedding vectors BIBREF32  
In preparing the textual dataset, we first use the released transcripts of the IEMOCAP dataset for simplicity  To investigate the practical performance, we then process all of the IEMOCAP audio data using an ASR system (the Google Cloud Speech API) and retrieve the transcripts  The performance of the Google ASR system is reflected by its word error rate (WER) of 5 53% 
Performance evaluation
As the dataset is not explicitly split beforehand into training, development, and testing sets, we perform 5-fold cross validation to determine the overall performance of the model  The data in each fold are split into training, development, and testing datasets (8:0 5:1 5, respectively)  After training the model, we measure the weighted average precision (WAP) over the 5-fold dataset  We train and evaluate the model 10 times per fold, and the model performance is assessed in terms of the mean score and standard deviation 
We examine the WAP values, which are shown in Table 1  First, our ARE model shows the baseline performance because we use minimal audio features, such as the MFCC and prosodic features with simple architectures  On the other hand, the TRE model shows higher performance gain compared to the ARE  From this result, we note that textual data are informative in emotion prediction tasks, and the recurrent encoder model is effective in understanding these types of sequential data  Second, the newly proposed model, MDRE, shows a substantial performance gain  It thus achieves the state-of-the-art performance with a WAP value of 0 718  This result shows that multimodal information is a key factor in affective computing  Lastly, the attention model, MDREA, also outperforms the best existing research results (WAP 0 690 to 0 688) BIBREF19   However, the MDREA model does not match the performance of the MDRE model, even though it utilizes a more complex architecture  We believe that this result arises because insufficient data are available to properly determine the complex model parameters in the MDREA model  Moreover, we presume that this model will show better performance when the audio signals are aligned with the textual sequence while applying the attention mechanism  We leave the implementation of this point as a future research direction 
To investigate the practical performance of the proposed models, we conduct further experiments with the ASR-processed transcript data (see “-ASR” models in Table )  The label accuracy of the processed transcripts is 5 53% WER  The TRE-ASR, MDRE-ASR and MDREA-ASR models reflect degraded performance compared to that of the TRE, MDRE and MDREA models  However, the performance of these models is still competitive; in particular, the MDRE-ASR model outperforms the previous best-performing model, 3CNN-LSTM10H (WAP 0 691 to 0 688) 
Error analysis
We analyze the predictions of the ARE, TRE, and MDRE models  Figure shows the confusion matrix of each model  The ARE model (Fig  ) incorrectly classifies most instances of happy as neutral (43 51%); thus, it shows reduced accuracy (35 15%) in predicting the the happy class  Overall, most of the emotion classes are frequently confused with the neutral class  This observation is in line with the findings of BIBREF30 , who noted that the neutral class is located in the center of the activation-valence space, complicating its discrimination from the other classes 
Interestingly, the TRE model (Fig  ) shows greater prediction gains in predicting the happy class when compared to the ARE model (35 15% to 75 73%)  This result seems plausible because the model can benefit from the differences among the distributions of words in happy and neutral expressions, which gives more emotional information to the model than that of the audio signal data  On the other hand, it is striking that the TRE model incorrectly predicts instances of the sad class as the happy class 16 20% of the time, even though these emotional states are opposites of one another 
The MDRE model (Fig  ) compensates for the weaknesses of the previous two models (ARE and TRE) and benefits from their strengths to a surprising degree  The values arranged along the diagonal axis show that all of the accuracies of the correctly predicted class have increased  Furthermore, the occurrence of the incorrect “sad-to-happy" cases in the TRE model is reduced from 16 20% to 9 15% 
Conclusions
In this paper, we propose a novel multimodal dual recurrent encoder model that simultaneously utilizes text data, as well as audio signals, to permit the better understanding of speech data  Our model encodes the information from audio and text sequences using dual RNNs and then combines the information from these sources using a feed-forward neural model to predict the emotion class  Extensive experiments show that our proposed model outperforms other state-of-the-art methods in classifying the four emotion categories, and accuracies ranging from 68 8% to 71 8% are obtained when the model is applied to the IEMOCAP dataset  In particular, it resolves the issue in which predictions frequently incorrectly yield the neutral class, as occurs in previous models that focus on audio features 
In the future work, we aim to extend the modalities to audio, text and video inputs  Furthermore, we plan to investigate the application of the attention mechanism to data derived from multiple modalities  This approach seems likely to uncover enhanced learning schemes that will increase performance in both speech emotion recognition and other multimodal classification tasks 
Acknowledgments
K  Jung is with the Department of Electrical and Computer Engineering, ASRI, Seoul National University, Seoul, Korea  This work was supported by the Ministry of Trade, Industry & Energy (MOTIE, Korea) under Industrial Technology Innovation Program (No 10073144) 

ground_truth: One can think $(1-p_{i1})$ as a weight associated with each example, which changes as training proceeds. The intuition of changing $p_{i1}$ to $(1-p_{i1}) p_{i1}$ is to push down the weight of easy examples. For easy examples whose probability are approaching 0 or 1, $(1-p_{i1}) p_{i1}$ makes the model attach significantly less focus to them. Figure FIGREF23 gives gives an explanation from the perspective in derivative: the derivative of $\frac{(1-p)p}{1+(1-p)p}$ with respect to $p$ approaches 0 immediately after $p$ approaches 0, which means the model attends less to examples once they are correctly classified.

associates each training example with a weight in proportion to $(1-p)$, and this weight dynamically changes as training proceeds