prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How big is the difference in performance between proposed model and baselines?
Context 1: Introduction
Attempts toward constructing human-like dialogue agents have met significant difficulties, such as maintaining conversation consistency BIBREF0  This is largely due to inabilities of dialogue agents to engage the user emotionally because of an inconsistent personality BIBREF1  Many agents use personality models that attempt to map personality attributes into lower dimensional spaces (e g  the Big Five BIBREF2)  However, these represent human personality at a very high-level and lack depth  They prohibit the ability to link specific and detailed personality traits to characters, and to construct large datasets where dialogue is traceable back to these traits 
For this reason, we propose Human Level Attributes (HLAs), which we define as characteristics of fictional characters representative of their profile and identity  We base HLAs on tropes collected from TV Tropes BIBREF3, which are determined by viewers' impressions of the characters  See Figure FIGREF1 for an example  Based on the hypothesis that profile and identity contribute effectively to language style BIBREF4, we propose that modeling conversation with HLAs is a means for constructing a dialogue agent with stable human-like characteristics  By collecting dialogues from a variety of characters along with this HLA information, we present a novel labelling of this dialogue data where it is traceable back to both its context and associated human-like qualities 
We also propose a system called ALOHA (Artificial Learning On Human Attributes) as a novel method of incorporating HLAs into dialogue agents  ALOHA maps characters to a latent space based on their HLAs, determines which are most similar in profile and identity, and recovers language styles of specific characters  We test the performance of ALOHA in character language style recovery against four baselines, demonstrating outperformance and system stability  We also run a human evaluation supporting our results  Our major contributions are: (1) We propose HLAs as personality aspects of fictional characters from the audience's perspective based on tropes; (2) We provide a large dialogue dataset traceable back to both its context and associated human-like attributes; (3) We propose a system called ALOHA that is able to recommend responses linked to specific characters  We demonstrate that ALOHA, combined with the proposed dataset, outperforms baselines  ALOHA also shows stable performance regardless of the character's identity, genre of the show, and context of the dialogue  We plan to release all of ALOHA's data and code 
Related Work
Task completion chatbots (TCC), or task-oriented chatbots, are dialogue agents used to fulfill specific purposes, such as helping customers book airline tickets, or a government inquiry system  Examples include the AIML based chatbot BIBREF5 and DIVA Framework BIBREF6  While TCC are low cost, easily configurable, and readily available, they are restricted to working well for particular domains and tasks 
Open-domain chatbots are more generic dialogue systems  An example is the Poly-encoder from BIBREF7 humeau2019real  It outperforms the Bi-encoder BIBREF8, BIBREF9 and matches the performance of the Cross-encoder BIBREF10, BIBREF11 while maintaining reasonable computation time  It performs strongly on downstream language understanding tasks involving pairwise comparisons, and demonstrates state-of-the-art results on the ConvAI2 challenge BIBREF12  Feed Yourself BIBREF13 is an open-domain dialogue agent with a self-feeding model  When the conversation goes well, the dialogue becomes part of the training data, and when the conversation does not, the agent asks for feedback  Lastly, Kvmemnn BIBREF14 is a key-value memory network with a knowledge base that uses a key-value retrieval mechanism to train over multiple domains simultaneously  We use all three of these models as baselines for comparison  While these can handle a greater variety of tasks, they do not respond with text that aligns with particular human-like characteristics 
BIBREF15 li2016persona defines persona (composite of elements of identity) as a possible solution at the word level, using backpropagation to align responses via word embeddings  BIBREF16 bartl2017retrieval uses sentence embeddings and a retrieval model to achieve higher accuracy on dialogue context  BIBREF17 liu2019emotion applies emotion states of sentences as encodings to select appropriate responses  BIBREF18 pichl2018alquist uses knowledge aggregation and hierarchy of sub-dialogues for high user engagement  However, these agents represent personality at a high-level and lack detailed human qualities  LIGHT BIBREF19 models adventure game characters' dialogues, actions, and emotions  It focuses on the agent identities (e g  thief, king, servant) which includes limited information on realistic human behaviours  BIBREF20 pasunuru2018game models online soccer games as dynamic visual context  BIBREF21 wang2016learning models user dialogue to complete tasks involving certain configurations of blocks  BIBREF22 antol2015vqa models open-ended questions, but is limited to visual contexts  BIBREF23 bordes2016learning tracks user dialogues but is goal-oriented  BIBREF24 ilinykh2019meetup tracks players' dialogues and movements in a visual environment, and is grounded on navigation tasks  All of these perform well in their respective fictional environments, but are not a strong representation of human dialogue in reality 
Methodology ::: Human Level Attributes (HLA)
We collect HLA data from TV Tropes BIBREF3, a knowledge-based website dedicated to pop culture, containing information on a plethora of characters from a variety of sources  Similar to Wikipedia, its content is provided and edited collaboratively by a massive user-base  These attributes are determined by human viewers and their impressions of the characters, and are correlated with human-like characteristics  We believe that TV Tropes is better for our purpose of fictional character modeling than data sources used in works such as BIBREF25 shuster2019engaging because TV Tropes' content providers are rewarded for correctly providing content through community acknowledgement 
TV Tropes defines tropes as attributes of storytelling that the audience recognizes and understands  We use tropes as HLAs to calculate correlations with specific target characters  We collect data from numerous characters from a variety of TV shows, movies, and anime  We filter and keep characters with at least five HLA, as those with fewer are not complex enough to be correctly modeled due to reasons such as lack of data  We end up eliminating 5 86% of total characters, and end up with 45,821 characters and 12,815 unique HLA, resulting in 945,519 total character-HLA pairs  Each collected character has 20 64 HLAs on average  See Figure FIGREF1 for an example character and their HLAs 
Methodology ::: Overall Task
Our task is the following, where $t$ denotes “target":
Given a target character $c_t$ with HLA set $H_t$, recover the language style of $c_t$ without any dialogue of $c_t$ provided 
For example, if Sheldon Cooper from The Big Bang Theory is $c_t$, then $H_t$ is the set of HLA on the left side of Figure FIGREF1 
We define the language style of a character as its diction, tone, and speech patterns  It is a character specific language model refined from a general language model  We must learn to recover the language style of $c_t$ without its dialogue as our objective is to imitate human-like qualities, and hence the model must understand the language styles of characters based on their traits  If we feed $c_t$'s dialogue during training, the model will likely not effectively learn to imitate language styles based on HLAs, but based on the correlation between text in the training and testing dialogues BIBREF26 
We define character space as the character representations within the HLA latent space (see Figure FIGREF4), and the set $C = \lbrace c_1,c_2,   ,c_n\rbrace $ as the set of all characters  We define Observation (OBS) as the input that is fed into any dialogue model  This can be a single or multiple lines of dialogue along with other information  The goal of the dialogue model is to find the best response to this OBS 
Methodology ::: ALOHA
We propose a three-component system called ALOHA to solve the task (see Figure FIGREF6)  The first component, Character Space Module (CSM), generates the character space and calculates confidence levels using singular value decomposition BIBREF27 between characters $c_j$ (for $j = 1$ to $n$ where $j \ne t$) and $c_t$ in the HLA-oriented neighborhood 
The second component, Character Community Module (CCM), ranks the similarity between our target character $c_t$ with any other character $c_j$ by the relative distance between them in the character space 
The third component, Language Style Recovery Module (LSRM), recovers the language style of $c_t$ without its dialogue by training the BERT bi-ranker model BIBREF28 to rank responses from similar characters  Our results demonstrate higher accuracy at retrieving the ground truth response from $c_t$  Our system is also able to pick responses which are correct both in context as well as character space 
Hence, the overall process for ALOHA works as follows  First, given a set of characters, determine the character space using the CSM  Next, given a specific target character, determine the positive community and negative set of associated characters using the CCM  Lastly, using the positive community and negative set determined above along with a dialogue dataset, recover the language style of the target 
Methodology ::: Character Space Module (CSM)
CSM learns how to rank characters  We can measure the interdependencies between the HLA variables BIBREF29 and rank the similarity between the TV show characters  We use implicit feedback instead of neighborhood models (e g  cosine similarity) because it can compute latent factors to transform both characters and HLAs into the same latent space, making them directly comparable 
We define a matrix $P$ that contains binary values, with $P_{u,i} = 1$ if character $u$ has HLA $i$ in our dataset, and $P_{u,i} = 0$ otherwise  We define a constant $\alpha $ that measures our confidence in observing various character-HLA pairs as positive  $\alpha $ controls how much the model penalizes the error if the ground truth is $P_{u,i} = 1$  If $P_{u,i} = 1$ and the model guesses incorrectly, we penalize by $\alpha $ times the loss  But if $P_{u,i} = 0$ and the model guesses a value greater than 0, we do not penalize as $\alpha $ has no impact  This is because $P_{u,i} = 0$ can either represent a true negative or be due to a lack of data, and hence is less reliable for penalization  See Equation DISPLAY_FORM8  We find that using $\alpha =20$ provides decent results 
We further define two dense vectors $X_u$ and $Y_i$  We call $X_u$ the “latent factors for character $u$", and $Y_i$ the “latent factors for HLA $i$"  The dot product of these two vectors produces a value ($X_u^TY_i$) that approximates $P_{u,i}$ (see Figure FIGREF9)  This is analogous to factoring the matrix $P$ into two separate matrices, where one contains the latent factors for characters, and the other contains the latent factors for HLAs  We find that $X_u$ and $Y_i$ being 36-dimensional produces decent results  To bring $X_u^TY_i$ as close as possible to $P_{u,i}$, we minimize the following loss function using the Conjugate Gradient Method BIBREF30:
The first term penalizes differences between the model's prediction ($X_u^TY_i$) and the actual value ($P_{u,i}$)  The second term is an L2 regularizer to reduce overfitting  We find $\lambda = 100$ provides decent results for 500 iterations (see Section SECREF26) 
Methodology ::: Character Community Module (CCM)
CCM aims to divide characters (other than $c_t$) into a positive community and a negative set  We define this positive community as characters that are densely connected internally to $c_t$ within the character space, and the negative set as the remaining characters  We can then sample dialogue from characters in the negative set to act as the distractors (essentially negative samples) during LSRM training 
As community finding is an ill-defined problem BIBREF31, we choose to treat CCM as a simple undirected, unweighted graph  We use the values learned in the CSM for $X_u$ and $Y_i$ for various values of $u$ and $i$, which approximate the matrix $P$  Similar to BIBREF29 hu2008collaborative, we can calculate the correlation between two rows (and hence two characters) 
We then employ a two-level connection representation by ranking all characters against each other in terms of their correlation with $c_t$  For the first level, the set $S^{FL}$ is the top 10% (4582) most highly correlated characters with $c_t$ out of the 45,820 total other characters that we have HLA data for  For the second level, for each character $s_i$ in $S^{FL}$, we determine the 30 most heavily correlated characters with $s_i$ as set $S^{SL}_i$  The positive set $S^{pos}$ are the characters which are present in at least 10 $S^{SL}_i$ sets  We call this value 10 the minimum frequency  All other characters in our dialogue dataset make up the negative set $S^{neg}$  These act as our positive community and negative set, respectively  See Algorithm 1 in Appendix A for details, and Figure FIGREF11 for an example 
Methodology ::: Language Style Recovery Module (LSRM)
LSRM creates a dialogue agent that aligns with observed characteristics of human characters by using the positive character community and negative set determined in the CCM, along with a dialogue dataset, to recover the language style of $c_t$ without its dialogue  We use the BERT bi-ranker model from the Facebook ParlAI framework BIBREF32, where the model has the ability to retrieve the best response out of 20 candidate responses  BIBREF12, BIBREF19, BIBREF0 choose 20 candidate responses, and for comparison purposes, we do the same 
Methodology ::: Language Style Recovery Module (LSRM) ::: BERT
BIBREF28 is first trained on massive amounts of unlabeled text data  It jointly conditions on text on both the left and right, which provides a deep bi-directional representation of sentence inference  BERT is proven to perform well on a wide range of tasks by simply fine-tuning on one additional layer  We are interested in its ability to predict the next sentence, called Next Sentence Prediction  We perform further fine-tuning on BERT for our target character language style retrieval task to produce our LSRM model by optimizing both the encoding layers and the additional layer  We use BERT to create vector representations for the OBS and for each candidate response  By passing the first output of BERT's 12 layers through an additional linear layer, these representations can be obtained as 768-dimensional sentence-level embeddings  It uses the dot product between these embeddings to score candidate responses and is trained using the ranking loss 
Methodology ::: Language Style Recovery Module (LSRM) ::: Candidate response selection
is similar to the procedure from previous work done on grounded dialogue agents BIBREF0, BIBREF19  Along with the ground truth response, we randomly sample 19 distractor responses from other characters from a uniform distribution of characters, and call this process uniform character sampling  Based on our observations, this random sampling provides multiple context correct responses  Hence, the BERT bi-ranker model is trained by learning to choose context correct responses, and the model learns to recover a domain-general language model that includes training on every character  This results in a Uniform Model that can select context correct responses, but not responses corresponding to a target character with specific HLAs 
We then fine-tune on the above model to produce our LSRM model with a modification: we randomly sample the 19 distractor responses from only the negative character set instead  We choose the responses that have similar grammatical structures and semantics to the ground truth response, and call this process negative character sampling  This guides the model away from the language style of these negative characters to improve performance at retrieving responses for target characters with specific HLAs  Our results demonstrate higher accuracy at retrieving the correct response from character $c_t$, which is the ground truth 
Experiment ::: Dialogue Dataset
To train the Uniform Model and LSRM, we collect dialogues from 327 major characters (a subset of the 45,821 characters we have HLA data for) in 38 TV shows from various existing sources of clean data on the internet, resulting in a total of 1,042,647 dialogue lines  We use a setup similar to the Persona-Chat dataset BIBREF0 and Cornell Movie-Dialogs Corpus BIBREF33, as our collected dialogues are also paired in terms of valid conversations  See Figure FIGREF1 for an example of these dialogue lines 
Experiment ::: HLA Observation Guidance (HLA-OG)
We define HLA Observation Guidance (HLA-OG) as explicitly passing a small subset of the most important HLAs of a given character as part of the OBS rather than just an initial line of dialogue  This is adapted from the process used in BIBREF0 zhang2018personalizing and BIBREF10 wolf2019transfertransfo which we call Persona Profiling  Specifically, we pass four HLAs that are randomly drawn from the top 40 most important HLAs of the character  We use HLA-OG during training of the LSRM and testing of all models  This is because the baselines (see Section SECREF31) already follow a similar process (Persona Profiling) for training  For the Uniform Model, we train using Next Sentence Prediction (see Section SECREF12)  For testing, HLA-OG is necessary as it provides information about which HLAs the models should attempt to imitate in their response selection  Just passing an initial line of dialogue replicates a typical dialogue response task without HLAs  See Table TABREF19  Further, we also test our LSRM by explicitly passing four HLAs of `none' along with the initial line of dialogue as the OBS (No HLA-OG in Table TABREF19) 
Experiment ::: Training Details ::: BERT bi-ranker
is trained by us on the Persona-Chat dataset for the ConvAI2 challenge  Similar to BIBREF0 zhang2018personalizing, we cap the length of the OBS at 360 tokens and the length of each candidate response at 72 tokens  We use a batch size of 64, learning rate of 5e-5, and perform warm-up updates for 100 iterations  The learning rate scheduler uses SGD optimizer with Nesterov's accelerated gradient descent BIBREF34 and is set to have a decay of 0 4 and to reduce on plateau 
Experiment ::: Training Details ::: Uniform Model
is produced by finetuning the BERT bi-ranker on the dialogue data discussed in Section SECREF15 using uniform character sampling  We use the same hyperparameters as the BERT bi-ranker along with half-precision operations (i e  float16 operations) to increase batch size as recommended BIBREF7 
Experiment ::: Training Details ::: LSRM
is produced by finetuning on the Uniform Model discussed above using negative character sampling  We use the same hyperparameters as the BERT bi-ranker along with half-precision operations (i e  float16 operations) to increase batch size as recommended 
Evaluation ::: CSM Evaluation
We begin by evaluating the ability of the CSM component of our system to correctly generate the character space  To do so, during training, 30% of the character-HLA pairs (which are either 0 or 1) are masked, and this is used as a validation set (see Figure FIGREF9)  For each character $c$, the model generates a list of the 12,815 unique HLAs ranked similarly to BIBREF29 hu2008collaborative for $c$  We look at the recall of our CSM model, which measures the percentage of total ground truth HLAs (over all characters $c$) present within the top N ranked HLAs for all $c$ by our model  That is:
where $HLA_{c}^{gt}$ are the ground truth HLAs for $c$, and $HLA_{c}^{tN}$ are the top N ranked HLAs by the model for $c$  We use $N = 100$, and our model achieves 25 08% recall 
To inspect the CSM performance, we use the T-distributed Stochastic Neighbor Embedding (t-SNE) BIBREF35 to reduce each high-dimensionality data point to two-dimensions via Kullback-Leibler Divergence BIBREF36  This allows us to map our character space into two-dimensions, where similar characters from our embedding space have higher probability of being mapped close by  We sampled characters from four different groups or regions  As seen in Figure FIGREF4, our learned character space effectively groups these characters, as similar characters are adjacent to one another in four regions 
Evaluation ::: Automatic Evaluation Setup ::: Five-Fold Cross Validation
is used for training and testing of the Uniform Model and LSRM  The folds are divided randomly by the TV shows in our dialogue data  We use the dialogue data for 80% of these shows as the four-folds for training, and the dialogue data for the remaining 20% as the fifth-fold for validation/testing  The dialogue data used is discussed in Section SECREF15  This ensures no matter how our data is distributed, each part of it is tested, allowing our evaluation to be more robust to different characters  See Appendix C for five-fold cross validation details and statistics 
Evaluation ::: Automatic Evaluation Setup ::: Five Evaluation Characters
are chosen, one from each of the five testing sets above  Each is a well-known character from a separate TV show, and acts as a target character $c_t$ for evaluation of every model  We choose Sheldon Cooper from The Big Bang Theory, Jean-Luc Picard from Star Trek, Monica Geller from Friends, Gil Grissom from CSI, and Marge Simpson from The Simpsons  We choose characters of significantly different identities and profiles (intelligent scientist, ship captain, outgoing friend, police leader, and responsible mother, respectively) from shows of a variety of genres to ensure that we can successfully recover the language styles of various types of characters  We choose well-known characters because humans require knowledge on the characters they are evaluating (see Section SECREF40) 
For each of these five evaluation characters, all the dialogue lines from the character act as the ground truth responses  The initial dialogue lines are the corresponding dialogue lines to which these ground truth responses are responding  For each initial dialogue line, we randomly sample 19 other candidate responses from the associated testing set using uniform character sampling  Note that this is for evaluation, and hence we use the same uniform character sampling method for all models including ALOHA  The use of negative character sampling is only in ALOHA's training 
Evaluation ::: Baselines
We compare against four dialogue system baselines: Kvmemnn, Feed Yourself, Poly-encoder, and a BERT bi-ranker baseline trained on the Persona-Chat dataset using the same training hyperparameters (including learning rate scheduler and length capping settings) described in Section SECREF20  For the first three models, we use the provided pretrained (on Persona-Chat) models  We evaluate all four on our five evaluation characters discussed in Section SECREF28 
Evaluation ::: Key Evaluation Metrics ::: Hits@n/N
is the accuracy of the correct ground truth response being within the top $n$ ranked candidate responses out of $N$ total candidates  We measure Hits@1/20, Hits@5/20, and Hits@10/20 
Evaluation ::: Key Evaluation Metrics ::: Mean Rank
is the average rank that a model assigns the ground truth response among the 20 total candidates 
Evaluation ::: Key Evaluation Metrics ::: Mean Reciprocal Rank (MRR)
BIBREF37 looks at the mean of the multiplicative inverses of the rank of each correct answer out of a sample of queries $Q$:
where $rank_i$ refers to the rank position of the correct response for the $i$-th query, and $|Q|$ refers to the total number of queries in $Q$ 
Evaluation ::: Key Evaluation Metrics ::: @!START@$F_1$@!END@-score
equals $2 * \frac{precision*recall}{precision+recall}$  For dialogue, precision is the fraction of words in the chosen response contained in the ground truth response, and recall is the fraction of words in the ground truth response contained in the chosen response 
Evaluation ::: Key Evaluation Metrics ::: BLEU
BIBREF38 generally indicates how close two pieces of text are in content and structure, with higher values indicating greater similarity  We report our final BLEU scores as the average scores of 1 to 4-grams 
Evaluation ::: Human Evaluation Setup
We conduct a human evaluation with 12 participants, 8 male and 4 female, who are affiliated project researchers aged 20-39 at the University of [ANON]  We choose the same five evaluation characters as in Section SECREF28  To control bias, each participant evaluates one or two characters  For each character, we randomly select 10 testing samples (each includes an initial line of dialogue along with 20 candidate responses, one of which is the ground truth) from the same testing data for the automatic evaluation discussed in Section SECREF28 
These ten samples make up a single questionnaire presented in full to each participant evaluating the corresponding character, and the participant is asked to select the single top response they think the character would most likely respond with for each of the ten initial dialogue lines  See Figure FIGREF41 for an example  We mask any character names within the candidate responses to prevent human participants from using names to identify which show the response is from 
Each candidate is prescreened to ensure they have sufficient knowledge of the character to be a participant  We ask three prescreening questions where the participant has to identify an image, relationship, and occupation of the character  All 12 of our participants passed the the prescreening 
Results and Analysis ::: Evaluation Results
Table TABREF44 shows average results of our automatic and human evaluations  Table TABREF45 shows average Hits@1/20 scores by evaluation character  See Appendix F for detailed evaluation results  ALOHA is the model with HLA-OG during training and testing, and ALOHA (No HLA-OG) is the model with HLA-OG during training but tested with the four HLAs in the OBS marked as `none' (see Section SECREF17)  See Appendix G for demo interactions between a human, BERT bi-ranker baseline, and ALOHA for all five evaluation characters 
Results and Analysis ::: Evaluation Challenges
The evaluation of our task (retrieving the language style of a specific character) is challenging and hence the five-fold cross validation is necessary for the following reasons:
The ability to choose a context correct response without attributes of specific characters may be hard to separate from our target metric, which is the ability to retrieve the correct response of a target character by its HLAs  However, from manual observation, we noticed that in the 20 chosen candidate responses, there are typically numerous context correct responses, but only one ground truth for the target character (for an example, see Figure FIGREF41)  Hence, a model that only chooses dialogue based on context is distinguishable from one that learns HLAs 
Retrieving responses for the target character depends on the other candidate responses  For example, dialogue retrieval performance for Grissom from CSI, which is a crime/police context, is higher than other evaluation characters (see Table TABREF45), potentially due to other candidate responses not falling within the same crime/police context 
Results and Analysis ::: Performance: ALOHA vs  Humans
As observed from Table TABREF44, ALOHA has a performance relatively close to humans  Human Hits@1/20 scores have a mean of 40 67% and a median over characters of 40%  The limited human evaluation sample size limits what can be inferred, but it indicates that the problem is solved to the extent that ALOHA is able perform relatively close to humans on average  Notice that even humans do not perform extremely well, demonstrating that this task of character based dialogue retrieval is more difficult than typical dialogue retrieval tasks BIBREF19, BIBREF12 
Looking more closely at each character from Table TABREF45, we can see that human evaluation scores are higher for Sheldon and Grissom  This may be due to these characters having more distinct personalities, making them more memorable 
We also look at Pearson correlation values of the Hits@1/20 scores across the five evaluation characters  For human versus Uniform Model, this is -0 4694, demonstrating that the Uniform Model, without knowledge of HLAs, fails to imitate human impressions  For human versus ALOHA, this is 0 4250, demonstrating that our system is able to retrieve character responses somewhat similarly to human impressions  Lastly, for human versus the difference in scores between ALOHA and Uniform Model, this is 0 7815  The difference between ALOHA and the Uniform Model, which is based on the additional knowledge of the HLAs, is hence shown to improve upon the Uniform Model similarly to human impressions  This demonstrates that HLAs are indeed an accurate method of modeling human impressions of character attributes, and also demonstrates that our system, ALOHA, is able to effectively use these HLAs to improve upon dialogue retrieval performance 
Results and Analysis ::: Performance: ALOHA vs  Baselines
ALOHA, combined with the HLAs and dialogue dataset, achieves a significant improvement on the target character language style retrieval task compared to the baseline open-domain chatbot models  As observed from Table TABREF44, ALOHA achieves a significant boost in Hits@n/N accuracy and other metrics for retrieving the correct response of five diverse characters with different identities (see Section SECREF28) 
Results and Analysis ::: Performance: ALOHA vs  Uniform Model
We observe a noticeable improvement in performance between ALOHA and the Uniform Model in recovering the language styles of specific characters that is consistent across all five folds (see Tables TABREF44 and TABREF45), indicating that lack of knowledge of HLAs limits the ability of the model to successfully recover the language style of specific characters  We claim that, to the best of our knowledge, we have made the first step in using HLA-based character dialogue clustering to improve upon personality learning for chatbots 
ALOHA demonstrates an accuracy boost for all five evaluation characters, showing that the system is robust and stable and has the ability to recover the dialogue styles of fictional characters regardless of the character's profile and identity, genre of the show, and context of the dialogue 
Results and Analysis ::: Performance: HLA-OG
As observed from Table TABREF44, ALOHA performs slightly better overall compared to ALOHA (No HLA-OG)  Table TABREF45 shows that this slight performance increase is consistent across four of the five evaluation characters  In the case of Sheldon, the HLA-OG model performs a bit worse  This is possibly due to the large number of Sheldon's HLAs (217) compared to the other four evaluation characters (average of 93 75), along with the limited amount of HLAs we are using for guidance due to the models' limited memory  In general, HLA Observation Guidance during testing appears to improve upon the performance of ALOHA, but this improvement is minimal 
Conclusion and Future Work
We proposed Human Level Attributes (HLAs) as a novel approach to model human-like attributes of characters, and collected a large volume of dialogue data for various characters with complete and robust profiles  We also proposed and evaluated a system, ALOHA, that uses HLAs to recommend tailored responses traceable to specific characters, and demonstrated its outperformance of the baselines and ability to effectively recover language styles of various characters, showing promise for learning character or personality styles  ALOHA was also shown to be stable regardless of the character's identity, genre of show, and context of dialogue 
Potential directions for future work include training ALOHA with a multi-turn response approach BIBREF0 that tracks dialogue over multiple responses, as we could not acquire multi-turn dialogue data for TV shows  Another potential is the modeling of the dialog counterpart (e g  the dialogue of other characters speaking to the target character)  Further, performing semantic text exchange on the chosen response with a model such as SMERTI BIBREF39 may improve the ability of ALOHA to converse with humans  This is because the response may be context and HLA correct, but incorrect semantically (e g  the response may say the weather is sunny when it is actually rainy)  HLA-aligned generative models is another area of exploration  Typically, generative models produce text that is less fluent, but further work in this area may lead to better results  Lastly, a more diverse and larger participant pool is required due to the limited size of our human evaluation 
Context 2: Introduction
The use of RNNs in the field of Statistical Machine Translation (SMT) has revolutionised the approaches to automated translation  As opposed to traditional shallow SMT models, which require a lot of memory to run, these neural translation models require only a small fraction of memory used, about 5% BIBREF0   Also, neural translation models are optimized such that every module is trained to jointly improve translation quality  With that being said, one of the main downsides of neural translation models is the heavy corpus requirement in order to ensure learning of deeper contexts  This is where the application of these encoder decoder architectures in translation to and/or from morphologically rich languages takes a severe hit 
For any language pair, the efficiency of an MT system depends on two major factors: the availability and size of parallel corpus used for training and the syntactic divergence between the two languages i e morphological richness, word order differences, grammatical structure etc  BIBREF0   The main differences between the languages stem from the fact that languages similar to English are predominantly fusional languages whereas many of the morphologically rich languages are agglutinative in nature  The nature of morphologically rich languages being structurally and semantically discordant from languages like English adds to the difficulty of SMT involving such languages 
In morphologically rich languages, any suffix can be added to any verb or noun to simply mean one specific thing about that particular word that the suffix commonly represents (agglutination)  This means that there exists a lot of inflectional forms of the same noun and verb base words, conveying similar notions  For example, in Tamil, there are at least 30,000 inflectional forms of any given verb and about 5,000 forms of inflectional forms for any noun  The merged words carry information about part of speech (POS) tags, tense, plurality and so forth that are important for analyzing text for Machine Translation (MT)  Not only are these hidden meanings not captured, the corresponding root words are trained as different units, thereby increasing the complexity of developing such MT systems BIBREF1  
To add to the complexities of being a morphologically rich language, there are several factors unique to Tamil that make translation very difficult  The availability of parallel corpus for Tamil is very scarce  Most of the other models in the field of English–Tamil MT have made use of their own translation corpora that were manually created for the purposes of research  Most of these corpora are not available online for use 
Another issue specific to Tamil is the addition of suffix characters included to the words in the language for smoothness in pronunciation  These characters are of so many different types; there is a unique suffix for each and every consonant in the language  These suffixes degrade performance of MT because the same words with different such pronounciation-based suffixes will be taken as different words in training 
Also to take into consideration is the existence of two different forms of the language being used  Traditionally defined Tamil and its pronunciations aren't acoustically pleasing to use  There's no linguistic flow between syllables and its usage in verbal communication is time consuming  Therefore, there exists two forms of the language, the written form, rigid in structure and syntax, and the spoken form, in which the flow and pace of the language is given priority over syntax and correctness of spelling  This divide leads to the corpus having 2 different versions of the language that increase the vocabulary even with the same words  This can be evidently seen in the corpus between the sentences used in the Bible, which is in traditional Tamil and sentences from movie subtitles, being in spoken Tamil format 
To account for such difficulties, a trade-off between domain specificity and size of the corpus is integral in building an English–Tamil neural MT system 
Corpus
The corpus selected for this experiment was a combination of different corpora from various domains  The major part of the corpus was made up by the EnTam v2 corpus BIBREF2   This corpus contained sentences taken from parallel news articles, English and Tamil bible corpus and movie subtitles  It also comprised of a tourism corpus that was obtained from TDIL (Technology Development for Indian Languages) and a corpus created from Tamil novels and short stories from AU-KBC, Anna university  The complete corpus consisted of 197,792 sentences  Fig  FIGREF20 shows the skinny shift and heatmap representations of the relativity between the sentences in terms of their sentence lengths 
An extra monolingual Tamil corpus, collated from various online sources was used for the word2vec embedding of the Tamil target language to enhance the richness of context of the word vectors  It was also used to create the language model for the phrase-based SMT model  This corpus contained 567,772 sentences and was self-collected by combining hundreds of ancient Tamil scriptures, novels and poems by accessing the websites of popular online ebook libraries in Python using the urllib package  Since the sources had Tamil text in different encodings, the encoding scheme was standardized to be UTF-8 for the entirety of the monolingual and parallel corpora using the chardet package  The corpora were cleaned for any stray special characters, unnecessary html tags and website URLs 
Word2Vec
The word embeddings of the source and target language sentences are used as initial vectors of the model to improve contextualization  The skip gram model of the word2vec algorithm optimizes the vectors by accounting for the average log probability of context words given a source word  DISPLAYFORM0
where k is the context window taken for the vectorization, INLINEFORM0 refers to the INLINEFORM1 word of the corpus and INLINEFORM2 is the size of the training corpus in terms of the number of words  Here, the probabily INLINEFORM3 is computed as a hierarchical softmax of the product of the transpose of the output vector of INLINEFORM4 and the input vector of INLINEFORM5 for each and every pair over the entire vocabulary  The processes of negative sampling and subsampling of frequent words that were used in the original model aren't used in this experiment BIBREF3  
For the process of creating semantically meaningful word embeddings, a monolingual corpus of 569,772 Tamil sentences was used  This gave the vectors more contextual richness due to the increased size of the corpus as opposed to using just the bilingual corpus' target side sentences BIBREF3  
In the experiment, the word2vec model was trained using a vector size of 100 to ensure that the bulk of the limited memory of the GPU will be used for the neural attention translation model  It has been shown that any size over that of 150 used for word vectorization gives similar results and that a size of 100 performs close to the model with 150-sized word vectors BIBREF7   A standard size of 5 was used as window size and the model was trained over 7 worker threads simultaneously  A batch size of 50 words was used for training  The negative sampling was set at 1 as it is the nature of morphologically rich languages to have a lot of important words that don't occur more than once in the corpus  The gensim word2vec toolkit was used to implement this word embedding process BIBREF8  
Neural Translation Model
The model used for translation is the one implemented by Bahdanau et al  Bahdanau2014  A bidirectional LSTM encoder first takes the source sentence and encodes it into a context vector which acts as input for the decoder  The decoder is attention-based where the hidden states of the decoder get as input the weighted sum of all the hidden layer outputs of the encoder alongwith the output of the previous hidden layer and the previously decoded word  This provides a contextual reference into the source language sentence BIBREF4  
Neural Machine Translation models directly compute the probability of the target language sentence given the source language sentence, word by word for every time step  The model with a basic decoder without the attention module computes the log probability of target sentence given source sentence as the sum of log probabilities of every word given every word before that  The attention-based model, on the other hand, calculates: DISPLAYFORM0
where INLINEFORM0 is the number of words in the target sentence, INLINEFORM1 is the target sentence, INLINEFORM2 is the source sentence, INLINEFORM3 is the fixed length output vector of the encoder and INLINEFORM4 is the weighted sum of all the hidden layer outputs of the encoder at every time step  Both the encoder's output context vector and the weighted sum (known as attention vector) help to improve the quality of translation by enabling selective source sentence lookup 
The decoder LSTM computes: DISPLAYFORM0
where the probability is computed as a function of the decoder's output in the previous time step INLINEFORM0 , the hidden layer vector of the decoder in the current timestep INLINEFORM1 and the context vector from the attention mechanism INLINEFORM2   The context vector INLINEFORM3 for time step INLINEFORM4 is computed as a weighted sum of the output of the entire sentence using a weight parameter INLINEFORM5 : DISPLAYFORM0
where INLINEFORM0 is the number of tokens in the source sentence, INLINEFORM1 refers to the value of the hidden layer of the encoder at time step INLINEFORM2 , and INLINEFORM3 is the alignment parameter  This parameter is calculated by means of a feed forward neural network to ensure that the alignment model is free from the difficulties of contextualization of long sentences into a single vector  The feed forward network is trained along with the neural translation model to jointly improve the performance of the translation  Mathematically, DISPLAYFORM0 DISPLAYFORM1
where INLINEFORM0 is the softmax output of the result of the feedforward network, INLINEFORM1 is the hidden state value of the decoder at timestep INLINEFORM2 and INLINEFORM3 is the encoder's hidden layer annotation at timestep INLINEFORM4   A concatenation of the forward and the reverse hidden layer parameters of the encoder is used at each step to compute the weights INLINEFORM5 for the attention mechanism  This is done to enable an overall context of the sentence, as opposed to a context of only all the previous words of the sentence for every word in consideration  Fig  FIGREF12 is the general architecture of the neural translation model without the Bidirectional LSTM encoder 
A global attention mechanism is preferred over local attention because the differences in the structures of the languages cannot be mapped efficiently to enable lookup into the right parts of the source sentence  Using local attention mechanism with a monotonic context lookup, where the region around INLINEFORM0 source word is looked up for the prediction of the INLINEFORM1 target word, is impractical because of the structural discordance between the English and Tamil sentences (see Figs  FIGREF37 and FIGREF44 )  The use of gaussian and other such distributions to facilitate local attention would also be inefficient because the existence of various forms of translations for the same source sentence involving morphological and structural variations that don't stay uniform through the entire corpus BIBREF5  
The No Peepholes (NP) variant of the LSTM cell, formulated in Greff et al  greff2015lstm is used in this experiment as it proved to give the best results amongst all the variants of an LSTM cell  It is specified by means of a gated mechanism designed to ensure that the vanishing gradient problem is prevented  LSTM maintains its hidden layer in two components, the cell vector INLINEFORM0 and the actual hidden layer output vector INLINEFORM1   The cell vector is ensured to never reach zero by means of a weighted sum of the previous layer's cell vector INLINEFORM2 regulated by the forget gate INLINEFORM3 and an activation of the weighted sum of the input INLINEFORM4 in the current timestep INLINEFORM5 and the previous timestep's hidden layer output vector INLINEFORM6   The combination is similarly regulated by the input gate INLINEFORM7   The hidden layer output is determined as an activation of the cell gate, regulated by the output gate INLINEFORM8   The interplay between these two vectors ( INLINEFORM9 and INLINEFORM10 ) at every timestep ensures that the problem of vanishing gradients doesn't occur  The three gates are also formed as a sigmoid of the weighted sum of the previous hidden layer output INLINEFORM11 and the input in the current timestep INLINEFORM12   The output generated out of the LSTM's hidden layer is specified as a weighted softmax over the hidden layer output INLINEFORM13   The learnable parameters of an LSTM cell are all the weights INLINEFORM14 and the biases INLINEFORM15   DISPLAYFORM0
The LSTM specified by equations 7 through 11 is the one used for the decoder of the model  The encoder uses a bidirectional RNN LSTM cell in which there are two hidden layer components INLINEFORM0 and INLINEFORM1 that contribute to the output INLINEFORM2 of each time step INLINEFORM3   Both the components have their own sets of LSTM equations in such a way that INLINEFORM4 for every timestep is computed from the first timestep till the INLINEFORM5 token is reached and INLINEFORM6 is computed from the INLINEFORM7 timestep backwards until the first token is reached  All the five vectors of the two components are all exactly the same as the LSTM equations specified with one variation in the computation of the result  DISPLAYFORM0
Morphological Segmentation
The morphological segmentation used is a semi-supervised extension to the generative probabilistic model of maximizing the probability of a INLINEFORM0 prefix,root,postfix INLINEFORM1 recursive split up of words based on an exhaustive combination of all possible morphemes  The details of this model are specified and extensively studied in Kohonen et al  kohonen2010semi  The model parameters INLINEFORM2 include the morph type count, morph token count of training data, the morph strings and their counts  The model is trained by maximizing the Maximum A Posteriori (MAP) probability using Bayes' rule: DISPLAYFORM0
where INLINEFORM0 refers to every word in the training lexicon  The prior INLINEFORM1 is estimated using the Minimum Description Length(MDL) principle  The likelihood INLINEFORM2 is estimated as: DISPLAYFORM0
where INLINEFORM0 refers to the intermediate analyses and INLINEFORM1 refers to the INLINEFORM2 morpheme of word INLINEFORM3  
An extension to the Viterbi algorithm is used for the decoding step based on exhaustive mapping of morphemes  To account for over-segmentation and under-segmentation issues associated with unsupervised morphological segmentation, extra parameters ( INLINEFORM0 ) and ( INLINEFORM1 ) are used with the cost function INLINEFORM2 DISPLAYFORM0
where INLINEFORM0 is the likelihood of the cost function, INLINEFORM1 describes the likelihood of contribution of the annotated dataset to the cost function and INLINEFORM2 is the likelihood of the labeled data  A decrease in the value of INLINEFORM3 will cause smaller segments and vice versa  INLINEFORM4 takes care of size discrepancies due to reduced availability of annotated corpus as compared to the training corpus BIBREF2 , BIBREF6  
The Python extension to the morphological segmentation tool morfessor 2 0 was used for this experiment to perform the segmentation  The annotation data for Tamil language collated and released by Anoop Kunchukkutan in the Indic NLP Library was used as the semi-supervised input to the model BIBREF9 , BIBREF6  
Experiment
The complexities of neural machine translation of morphologically rich languages were studied with respect to English to Tamil machine translation using the RNN LSTM Bi-directional encoder attention decoder architecture  To compare with a baseline system, a phrase based SMT system was implemented using the same corpus  The Factored SMT model with source-side preprocessing by Kumar et al  kumar2014improving was used as a reference for the translation between these language pairs  Also, an additional 569,772 monolingual Tamil sentences were used for the language model of the SMT system  The model used could be split up into various modules as expanded in Fig  FIGREF17  
Bucketing
The input source and target language sentences used for training were taken and divided into bucketed pairs of sentences of a fixed number of sizes  This relationship was determined by examining the distribution of words in the corpus primarily to minimize the number of PAD tokens in the sentence  The heat map of the number of words in the English–Tamil sentence pairs of the corpus revealed that the distribution is centered around the 10–20 words region  Therefore, more buckets in that region were applied as there would be enough number of examples in each of these bucket pairs for the model to learn about the sentences in each and every bucket  The exact scheme used for the RNNSearch models is specified by Fig  FIGREF21   The bucketing scheme for the RNNMorph model, involving morphs instead of words, was a simple shifted scheme of the one used in Fig  FIGREF21 , where every target sentence bucket count was increased uniformly by 5 
Model Details
Due to various computational constraints and lack of availability of comprehensive corpora, the vocabularies for English and Tamil languages for the RNNSearch model were restricted to 60,000 out of 67,768 and 150,000 out of 340,325 respectively  The vocabulary of the languages for the RNNMorph didn't have to be restricted and the actual number of words in the corpus i
Context 3: Introduction
We posses a wealth of prior knowledge about many natural language processing tasks  For example, in text categorization, we know that words such as NBA, player, and basketball are strong indicators of the sports category BIBREF0 , and words like terrible, boring, and messing indicate a negative polarity while words like perfect, exciting, and moving suggest a positive polarity in sentiment classification 
A key problem arisen here, is how to leverage such knowledge to guide the learning process, an interesting problem for both NLP and machine learning communities  Previous studies addressing the problem fall into several lines  First, to leverage prior knowledge to label data BIBREF1 , BIBREF2   Second, to encode prior knowledge with a prior on parameters, which can be commonly seen in many Bayesian approaches BIBREF3 , BIBREF4   Third, to formalise prior knowledge with additional variables and dependencies BIBREF5   Last, to use prior knowledge to control the distributions over latent output variables BIBREF6 , BIBREF7 , BIBREF8 , which makes the output variables easily interpretable 
However, a crucial problem, which has rarely been addressed, is the bias in the prior knowledge that we supply to the learning model  Would the model be robust or sensitive to the prior knowledge? Or, which kind of knowledge is appropriate for the task? Let's see an example: we may be a baseball fan but unfamiliar with hockey so that we can provide a few number of feature words of baseball, but much less of hockey for a baseball-hockey classification task  Such prior knowledge may mislead the model with heavy bias to baseball  If the model cannot handle this situation appropriately, the performance may be undesirable 
In this paper, we investigate into the problem in the framework of Generalized Expectation Criteria BIBREF7   The study aims to reveal the factors of reducing the sensibility of the prior knowledge and therefore to make the model more robust and practical  To this end, we introduce auxiliary regularization terms in which our prior knowledge is formalized as distribution over output variables  Recall the example just mentioned, though we do not have enough knowledge to provide features for class hockey, it is easy for us to provide some neutral words, namely words that are not strong indicators of any class, like player here  As one of the factors revealed in this paper, supplying neutral feature words can boost the performance remarkably, making the model more robust 
More attractively, we do not need manual annotation to label these neutral feature words in our proposed approach 
More specifically, we explore three regularization terms to address the problem: (1) a regularization term associated with neutral features; (2) the maximum entropy of class distribution regularization term; and (3) the KL divergence between reference and predicted class distribution  For the first manner, we simply use the most common features as neutral features and assume the neutral features are distributed uniformly over class labels  For the second and third one, we assume we have some knowledge about the class distribution which will be detailed soon later 
To summarize, the main contributions of this work are as follows:
The rest of the paper is structured as follows: In Section 2, we briefly describe the generalized expectation criteria and present the proposed regularization terms  In Section 3, we conduct extensive experiments to justify the proposed methods  We survey related work in Section 4, and summarize our work in Section 5 
Method
We address the robustness problem on top of GE-FL BIBREF0 , a GE method which leverages labeled features as prior knowledge  A labeled feature is a strong indicator of a specific class and is manually provided to the classifier  For example, words like amazing, exciting can be labeled features for class positive in sentiment classification 
Generalized Expectation Criteria
Generalized expectation (GE) criteria BIBREF7 provides us a natural way to directly constrain the model in the preferred direction  For example, when we know the proportion of each class of the dataset in a classification task, we can guide the model to predict out a pre-specified class distribution 
Formally, in a parameter estimation objective function, a GE term expresses preferences on the value of some constraint functions about the model's expectation  Given a constraint function $G({\rm x}, y)$ , a conditional model distribution $p_\theta (y|\rm x)$ , an empirical distribution $\tilde{p}({\rm x})$ over input samples and a score function $S$ , a GE term can be expressed as follows:
$$S(E_{\tilde{p}({\rm x})}[E_{p_\theta (y|{\rm x})}[G({\rm x}, y)]])$$   (Eq  4)
Learning from Labeled Features
Druck et al  ge-fl proposed GE-FL to learn from labeled features using generalized expectation criteria  When given a set of labeled features $K$ , the reference distribution over classes of these features is denoted by $\hat{p}(y| x_k), k \in K$   GE-FL introduces the divergence between this reference distribution and the model predicted distribution $p_\theta (y | x_k)$ , as a term of the objective function:
$$\mathcal {O} = \sum _{k \in K} KL(\hat{p}(y|x_k) || p_\theta (y | x_k)) + \sum _{y,i} \frac{\theta _{yi}^2}{2 \sigma ^2}$$   (Eq  6)
where $\theta _{yi}$ is the model parameter which indicates the importance of word $i$ to class $y$   The predicted distribution $p_\theta (y | x_k)$ can be expressed as follows: $ p_\theta (y | x_k) = \frac{1}{C_k} \sum _{\rm x} p_\theta (y|{\rm x})I(x_k) $
in which $I(x_k)$ is 1 if feature $k$ occurs in instance ${\rm x}$ and 0 otherwise, $C_k = \sum _{\rm x} I(x_k)$ is the number of instances with a non-zero value of feature $k$ , and $p_\theta (y|{\rm x})$ takes a softmax form as follows: $ p_\theta (y|{\rm x}) = \frac{1}{Z(\rm x)}\exp (\sum _i \theta _{yi}x_i)  $
To solve the optimization problem, L-BFGS can be used for parameter estimation 
In the framework of GE, this term can be obtained by setting the constraint function $G({\rm x}, y) = \frac{1}{C_k} \vec{I} (y)I(x_k)$ , where $\vec{I}(y)$ is an indicator vector with 1 at the index corresponding to label $y$ and 0 elsewhere 
Regularization Terms
GE-FL reduces the heavy load of instance annotation and performs well when we provide prior knowledge with no bias  In our experiments, we observe that comparable numbers of labeled features for each class have to be supplied  But as mentioned before, it is often the case that we are not able to provide enough knowledge for some of the classes  For the baseball-hockey classification task, as shown before, GE-FL will predict most of the instances as baseball  In this section, we will show three terms to make the model more robust 
Neutral features are features that are not informative indicator of any classes, for instance, word player to the baseball-hockey classification task  Such features are usually frequent words across all categories  When we set the preference distribution of the neutral features to be uniform distributed, these neutral features will prevent the model from biasing to the class that has a dominate number of labeled features 
Formally, given a set of neutral features $K^{^{\prime }}$ , the uniform distribution is $\hat{p}_u(y|x_k) = \frac{1}{|C|}, k \in K^{^{\prime }}$ , where $|C|$ is the number of classes  The objective function with the new term becomes
$$\mathcal {O}_{NE} = \mathcal {O} + \sum _{k \in K^{^{\prime }}} KL(\hat{p}_u(y|x_k) || p_\theta (y | x_k)) $$   (Eq  9)
Note that we do not need manual annotation to provide neutral features  One simple way is to take the most common features as neutral features  Experimental results show that this strategy works successfully 
Another way to prevent the model from drifting from the desired direction is to constrain the predicted class distribution on unlabeled data  When lacking knowledge about the class distribution of the data, one feasible way is to take maximum entropy principle, as below:
$$\mathcal {O}_{ME} = \mathcal {O} + \lambda \sum _{y} p(y) \log p(y)$$   (Eq  11)
where $p(y)$ is the predicted class distribution, given by $ p(y) = \frac{1}{|X|} \sum _{\rm x} p_\theta (y | \rm x)  $ To control the influence of this term on the overall objective function, we can tune $\lambda $ according to the difference in the number of labeled features of each class  In this paper, we simply set $\lambda $ to be proportional to the total number of labeled features, say $\lambda = \beta |K|$  
This maximum entropy term can be derived by setting the constraint function to $G({\rm x}, y) = \vec{I}(y)$   Therefore, $E_{p_\theta (y|{\rm x})}[G({\rm x}, y)]$ is just the model distribution $p_\theta (y|{\rm x})$ and its expectation with the empirical distribution $\tilde{p}(\rm x)$ is simply the average over input samples, namely $p(y)$   When $S$ takes the maximum entropy form, we can derive the objective function as above 
Sometimes, we have already had much knowledge about the corpus, and can estimate the class distribution roughly without labeling instances  Therefore, we introduce the KL divergence between the predicted and reference class distributions into the objective function 
Given the preference class distribution $\hat{p}(y)$ , we modify the objective function as follows:
$$\mathcal {O}_{KL} &= \mathcal {O} + \lambda KL(\hat{p}(y) || p(y))$$   (Eq

ground_truth: Metric difference between Aloha and best baseline score:
Hits@1/20: +0.061 (0.3642 vs 0.3032)
MRR: +0.0572(0.5114 vs 0.4542)
F1: -0.0484 (0.3901 vs 0.4385)
BLEU: +0.0474 (0.2867 vs 0.2393)

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How were the human judgements assembled?
Context 2: e  67,768 words for English and 41,906 words for Tamil could be accommodated into the training  Words not in the vocabulary from the test set input and output were replaced with the universal INLINEFORM0 UNK INLINEFORM1 token, symbolizing an unknown word  The LSTM hidden layer size, the training batch size, and the vocabulary sizes of the languages, together, acted as a bottleneck  The model was run on a 2GB NVIDIA GeForce GT 650M card with 384 cores and the memory allotment was constrained to the limits of the GPU  Therefore, after repeated experimentation, it was determined that with a batch size of 16, the maximum hidden layer size possible was 500, which was the size used  Attempts to reduce the batch size resulted in poor convergence, and so the parameters were set to center around the batch size of 16  The models used were of 4 layers of LSTM hidden units in the bidirectional encoder and attention decoder 
The model used a Stochastic Gradient Descent (SGD) optimization algorithm with a sampled softmax loss of 512 per sample to handle large vocabulary size of the target language BIBREF10   The model was trained with a learning rate 1 0 and a decay of rate 0 5 enforced manually  Gradient clipping based on the global norm of 5 0 was carried out to prevent gradients exploding and going to unrecoverable values tending towards infinity  The model described is the one used in the Tensorflow BIBREF11 seq2seq library 
Results and Discussion
The BLEU metric parameters (modified 1-gram, 2-gram, 3-gram and 4-gram precision values) and human evaluation metrics of adequacy, fluency and relative ranking values were used to evaluate the performance of the models 
BLEU Evaluation
The BLEU scores obtained using the various models used in the experiment are tabulated in Table TABREF25  
The BLEU metric computes the BLEU unigram, bigram, trigram and BLEU-4 modified precision values, each micro-averaged over the test set sentences BIBREF7   It was observed, as expected, that the performance of the phrase-based SMT model was inferior to that of the RNNSearch model  The baseline RNNSearch system was further refined by using word2vec vectors to embed semantic understanding, as observed with the slight increase in the BLEU scores  Fig  FIGREF26 plots the BLEU scores as a line graph for visualization of the improvement in performance  Also, the 4-gram BLEU scores for the various models were plotted as a bar graph in Fig  FIGREF26
Due to the agglutinative and morphologically rich nature of the target language i e  Tamil, the use of morphological segmentation to split the words into morphemes further improved the BLEU precision values in the RNNMorph model  One of the reasons for the large extent of increase in the BLEU score could be attributed to the overall increase in the number of word units per sentence  Since the BLEU score computes micro-average precision scores, an increase in both the numerator and denominator of the precision scores is apparent with an increase in the number of tokens due to morphological segmentation of the target language  Thus, the numeric extent of the increase of accuracy might not efficiently describe the improvement in performance of the translation 
Human Evaluation
To ensure that the increase in BLEU score correlated to actual increase in performance of translation, human evaluation metrics like adequacy, precision and ranking values (between RNNSearch and RNNMorph outputs) were estimated in Table TABREF30   A group of 50 native people who were well-versed in both English and Tamil languages acted as annotators for the evaluation  A collection of samples of about 100 sentences were taken from the test set results for comparison  This set included a randomized selection of the translation results to ensure the objectivity of evaluation  Fluency and adequacy results for the RNNMorph results are tabulated  Adequacy rating was calculated on a 5-point scale of how much of the meaning is conveyed by the translation (All, Most, Much, Little, None)  The fluency rating was calculated based on grammatical correctness on a 5-point scale of (Flawless, Good, Non-native, Disfluent, Incomprehensive)  For the comparison process, the RNNMorph and the RNNSearch + Word2Vec models’ sentence level translations were individually ranked between each other, permitting the two translations to have ties in the ranking  The intra-annotator values were computed for these metrics and the scores are shown in Table TABREF32 BIBREF12 , BIBREF13  
The human evaluation Kappa co-efficient results are calculated with respect to: DISPLAYFORM0
It was observed that the ranking Kappa co-efficient for intra-annotator ranking of the RNNMorph model was at 0 573, higher that the 0 410 of the RNNSearch+Word2Vec model, implying that the annotators found the RNNMorph model to produce better results when compared to the RNNSearch + Word2Vec model 
Model Parameters
The learning rate decay through the training process of the RNNMorph model is showcased in the graph in Fig  FIGREF34   This process was done manually where the learning rate was decayed after the end of specific epochs based on an observed stagnation in perplexity The RNNMorph model achieved saturation of perplexities much earlier through the epochs than the RNNSearch + Word2Vec model  This conforms to the expected outcome as the morphological segmentation has reduced the vocabulary size of the target language from 340,325 words to a mere 41,906 morphs 
The error function used was the sampled SoftMax loss to ensure a large target vocabulary could be accommodated BIBREF10   A zoomed inset graph (Fig  FIGREF35 ) has been used to visualize the values of the error function for the RNNSearch + Word2Vec and RNNMorph models with 4 hidden layers  It can be seen that the RNNMorph model is consistently better in terms of the perplexity values through the time steps 
Attention Vectors
In order to further demonstrate the quality of the RNNMorph model, the attention vectors of both the RNNSearch with Word2Vec embedding and RNNMorph models are compared for several good translations in Figs  FIGREF37 and FIGREF44   It is observed that the reduction in vocabulary size has improved the source sentence lookup by quite an extent  Each cell in the heatmap displays the magnitude of the attention layer weight INLINEFORM0 for the INLINEFORM1 Tamil word and the INLINEFORM2 English word in the respective sentences  The intensity of black corresponds to the magnitude of the cell INLINEFORM3   Also, the attention vectors of the RNNSearch model with Word2Vec embeddings tend to attend to INLINEFORM4 EOS INLINEFORM5 token in the middle of the sentence leading to incomplete translations  This could be due to the fact that only 44% of the Tamil vocabulary and 74% of the English vocabulary is taken for training in this model, as opposed to 100% of English and Tamil words in the RNNMorph model 
Target vocabulary size
A very large target vocabulary is an inadvertent consequence of the morphological richness of the Tamil language  This creates a potential restriction on the accuracy of the model as many inflectional forms of the same word are trained as independent units  One of the advantages of morphological segmentation of Tamil text is that the target vocabulary size decreased from 340,325 to a mere 41,906  This reduction helps improve the performance of the translation as the occurrence of unknown tokens was reduced compared to the RNNSearch model  This morphologically segmented vocabulary is divided into a collection of morphological roots and inflections as individual units 
Repetitions
Some of the translations of the RNNMorph model have repetitions of the same phrases (Fig  FIGREF53 ), whereas such repetitions occur much less frequently in the RNNSearch predictions  Such translations would make for good results if the repetitions weren't present and all parts of the sentence occur just once  These repetitions might be due to the increase in the general sequence length of the target sentences because of the morphological segmentation  While it is true the target vocabulary size has decreased due to morphological segmentation, the RNNMorph has more input units (morphs) per sentence, which makes it more demanding of the LSTM's memory units and the feed forward network of the attention model  Additionally, this behavior could also be attributed to the errors in the semi-supervised morphological segmentation due to the complexities of the Tamil language and the extent of the corpus 
Model Outputs
The translation outputs of the RNNSearch + Word2Vec and Morph2Vec models for the same input sentences from the test set demonstrate the effectiveness of using a morphological segmentation tool and how the morphemes have changed the sentence to be more grammatically sound  It is also observed (from Fig  FIGREF55 ) that most of the translation sentences of the Morph2Vec model have no INLINEFORM0 UNK INLINEFORM1 tokens  They exist in the predictions mostly only due to a word in the English test sentence not present in the source vocabulary 
Related Work
Professors CN Krishnan, Sobha et al developed a machine-aided-translation (MAT) system similar to the Anusaakara English Hindi MT system, using a small corpus and very few transfer rules, available at AU-KBC website BIBREF14   Balajapally et al  balajapally2006multilingual developed an example based machine translation (EBMT) system with 700000 sentences for English to INLINEFORM0 Tamil, Kannada, Hindi INLINEFORM1 transliterated text BIBREF15 , BIBREF16   Renganathan renganathan2002interactive developed a rule based MT system for English and Tamil using grammar rules for the language pair  Vetrivel et al  vetrivel2010english used HMMs to align and translate English and Tamil parallel sentences to build an SMT system  Irvine et al  irvine2013combining tried to combine parallel and similar corpora to improve the performance of English to Tamil SMT amongst other languages  Kasthuri et al  kasthuri2014rule used a rule based MT system using transfer lexicon and morphological analysis tools  Anglabharathi was developed at IIT Kanpur, a system translating English to a collection of Indian languages including Tamil using CFG like structures to create a pseudo target to convert to Indian languages BIBREF17 , BIBREF18   A variety of hybrid approaches have also been used for English–Tamil MT in combinations of rule based (transfer methods), interlingua representations BIBREF19 , BIBREF20 , BIBREF21   The use of Statistical Machine Translation took over the English–Tamil MT system research because of its desirable properties of language independence, better generalization features and a reduced requirement of linguistic expertise BIBREF1 , BIBREF22 , BIBREF23   Various enhancement techniques external to the MT system have also been proposed to improve the performance of translation using morphological pre and post processing techniques BIBREF24 , BIBREF25 , BIBREF26  
The use of RNN Encoder Decoder models in machine translation has shown good results in languages with similar grammatical structure  Deep MT systems have been performing better than the other shallow SMT models recently, with the availability of computational resources and hardware making it feasible to train such models  The first of these models came in 2014, with Cho et al SecondOneByCho  The model used was the RNN LSTM encoder decoder model with the context vector output of the encoder (run for every word in the sentence) is fed to every decoder unit along with the previous word output until INLINEFORM0 EOS INLINEFORM1 is reached  This model was used to score translation results of another MT system  Sutskever et al  sutskever2014sequence created a similar encoder decoder model with the decoder getting the context vector only for the first word of the target language sentence  After that, only the decoded target outputs act as inputs to the various time steps of the decoder  One major drawback of these models is the size of the context vector of the encoder being static in nature  The same sized vector was expected to to represent sentences of arbitrary length, which was impractical when it came to very long sentences 
The next breakthrough came from Bahdanau et al  Bahdanau2014 where variable length word vectors were used and instead of just the context vector, a weighted sum of the inputs is given for the decoder  This enabled selective lookup to the source sentence during decoding and is known as the attention mechanism BIBREF27   The attention mechanism was further analysed by Luong et al  luong2015effective where they made a distinction between global and local attention by means of AER scores of the attention vectors  A Gaussian distribution and a monotonic lookup were used to facilitate the corresponding local source sentence look-up 
Conclusion
Thus, it is seen that the use of morphological segmentation on a morphologically rich language before translation helps with the performance of the translation in multiple ways  Thus, machine translation involving morphologically rich languages should ideally be carried out only after morphological segmentation  If the translation has to be carried out between two morphologically rich languages, then both the languages' sentences should be individually segmented based on morphology  This is because while it is true that they are both morphologically rich languages, the schemes that the languages use for the process of agglutination might be different, in which case a mapping between the units would be difficult without the segmentation 
One drawback of morphological segmentation is the increase in complexity of the model due to an increase in the average sentence lengths  This cannot be avoided as it is essential to enable a correspondence between the sentences of the two languages when one of them is a simple fusional language  Even with the increase in the average sentence length, the attention models that have been developed to ensure correctness of translation of long sequences can be put to good use when involving morphologically rich languages  Another point to note here is that morphologically rich languages like Tamil generally have lesser number of words per sentence than languages like English due to the inherent property of agglutination 
Future Work
The model implemented in this paper only includes source-side morphological segmentation and does not include a target side morphological agglutination to give back the output in words rather than morphemes  In order to implement an end-to-end translation system for morphologically rich languages, a morphological generator is essential because the output units of the translation cannot be morphemes 
The same model implemented can be further enhanced by means of a better corpus that can generalize over more than just domain specific source sentences  Also, the use of a better GPU would result in a better allocation of the hidden layer sizes and the batch sizes thereby possibly increasing the scope and accuracy of learning of the translation model 
Although not directly related to Machine Translation, the novel encoder– decoder architecture proposed in by Rocktaschel et al  rocktaschel2015reasoning for Natural Language Inference (NLI) can be used for the same  Their model fuses inferences from each and every individual word, summarizing information at each step, thereby linking the hidden state of the encoder with that of the decoder by means of a weighted sum, trained for optimization 
Acknowledgements
I would like to thank Dr  M  Anand Kumar, Assistant Professor, Amrita Vishwa Vidyapeetham for his continuous support and guidance  I would also like to thank Dr  Arvindan, Professor, SSN College Of Engineering for his inputs and suggestions 
Context 3:  13)
Similarly, we set $\lambda = \beta |K|$  
This divergence term can be derived by setting the constraint function to $G({\rm x}, y) = \vec{I}(y)$ and setting the score function to $S(\hat{p}, p) = \sum _i \hat{p}_i \log \frac{\hat{p}_i}{p_i}$ , where $p$ and $\hat{p}$ are distributions  Note that this regularization term involves the reference class distribution which will be discussed later 
Experiments
In this section, we first justify the approach when there exists unbalance in the number of labeled features or in class distribution  Then, to test the influence of $\lambda $ , we conduct some experiments with the method which incorporates the KL divergence of class distribution  Last, we evaluate our approaches in 9 commonly used text classification datasets  We set $\lambda = 5|K|$ by default in all experiments unless there is explicit declaration  The baseline we choose here is GE-FL BIBREF0 , a method based on generalization expectation criteria 
Data Preparation
We evaluate our methods on several commonly used datasets whose themes range from sentiment, web-page, science to medical and healthcare  We use bag-of-words feature and remove stopwords in the preprocess stage  Though we have labels of all documents, we do not use them during the learning process, instead, we use the label of features 
The movie dataset, in which the task is to classify the movie reviews as positive or negtive, is used for testing the proposed approaches with unbalanced labeled features, unbalanced datasets or different $\lambda $ parameters  All unbalanced datasets are constructed based on the movie dataset by randomly removing documents of the positive class  For each experiment, we conduct 10-fold cross validation 
As described in BIBREF0 , there are two ways to obtain labeled features  The first way is to use information gain  We first calculate the mutual information of all features according to the labels of the documents and select the top 20 as labeled features for each class as a feature pool  Note that using information gain requires the document label, but this is only to simulate how we human provide prior knowledge to the model  The second way is to use LDA BIBREF9 to select features  We use the same selection process as BIBREF0 , where they first train a LDA on the dataset, and then select the most probable features of each topic (sorted by $P(w_i|t_j)$ , the probability of word $w_i$ given topic $t_j$ ) 
Similar to BIBREF10 , BIBREF0 , we estimate the reference distribution of the labeled features using a heuristic strategy  If there are $|C|$ classes in total, and $n$ classes are associated with a feature $k$ , the probability that feature $k$ is related with any one of the $n$ classes is $\frac{0 9}{n}$ and with any other class is $\frac{0 1}{|C| - n}$  
Neutral features are the most frequent words after removing stop words, and their reference distributions are uniformly distributed  We use the top 10 frequent words as neutral features in all experiments 
With Unbalanced Labeled Features
In this section, we evaluate our approach when there is unbalanced knowledge on the categories to be classified  The labeled features are obtained through information gain  Two settings are chosen:
(a) We randomly select $t \in [1, 20]$ features from the feature pool for one class, and only one feature for the other  The original balanced movie dataset is used (positive:negative=1:1) 
(b) Similar to (a), but the dataset is unbalanced, obtained by randomly removing 75% positive documents (positive:negative=1:4) 
As shown in Figure 1 , Maximum entropy principle shows improvement only on the balanced case  An obvious reason is that maximum entropy only favors uniform distribution 
Incorporating Neutral features performs similarly to maximum entropy since we assume that neutral words are uniformly distributed  Its accuracy decreases slowly when the number of labeled features becomes larger ( $t>4$ ) (Figure 1 (a)), suggesting that the model gradually biases to the class with more labeled features, just like GE-FL 
Incorporating the KL divergence of class distribution performs much better than GE-FL on both balanced and unbalanced datasets  This shows that it is effective to control the unbalance in labeled features and in the dataset 
With Balanced Labeled Features
We also compare with the baseline when the labeled features are balanced  Similar to the experiment above, the labeled features are obtained by information gain  Two settings are experimented with:
(a) We randomly select $t \in [1, 20]$ features from the feature pool for each class, and conduct comparisons on the original balanced movie dataset (positive:negtive=1:1) 
(b) Similar to (a), but the class distribution is unbalanced, by randomly removing 75% positive documents (positive:negative=1:4) 
Results are shown in Figure 2   When the dataset is balanced (Figure 2 (a)), there is little difference between GE-FL and our methods  The reason is that the proposed regularization terms provide no additional knowledge to the model and there is no bias in the labeled features  On the unbalanced dataset (Figure 2 (b)), incorporating KL divergence is much better than GE-FL since we provide additional knowledge(the true class distribution), but maximum entropy and neutral features are much worse because forcing the model to approach the uniform distribution misleads it 
With Unbalanced Class Distributions
Our methods are also evaluated on datasets with different unbalanced class distributions  We manually construct several movie datasets with class distributions of 1:2, 1:3, 1:4 by randomly removing 50%, 67%, 75% positive documents  The original balanced movie dataset is used as a control group  We test with both balanced and unbalanced labeled features  For the balanced case, we randomly select 10 features from the feature pool for each class, and for the unbalanced case, we select 10 features for one class, and 1 feature for the other  Results are shown in Figure 3  
Figure 3 (a) shows that when the dataset and the labeled features are both balanced, there is little difference between our methods and GE-FL(also see Figure 2 (a))  But when the class distribution becomes more unbalanced, the difference becomes more remarkable  Performance of neutral features and maximum entropy decrease significantly but incorporating KL divergence increases remarkably  This suggests if we have more accurate knowledge about class distribution, KL divergence can guide the model to the right direction 
Figure 3 (b) shows that when the labeled features are unbalanced, our methods significantly outperforms GE-FL  Incorporating KL divergence is robust enough to control unbalance both in the dataset and in labeled features while the other three methods are not so competitive 
The Influence of λ\lambda 
We present the influence of $\lambda $ on the method that incorporates KL divergence in this section  Since we simply set $\lambda = \beta |K|$ , we just tune $\beta $ here  Note that when $\beta = 0$ , the newly introduced regularization term is disappeared, and thus the model is actually GE-FL  Again, we test the method with different $\lambda $ in two settings:
(a) We randomly select $t \in [1, 20]$ features from the feature pool for one class, and only one feature for the other class  The original balanced movie dataset is used (positive:negative=1:1) 
(b) Similar to (a), but the dataset is unbalanced, obtained by randomly removing 75% positive documents (positive:negative=1:4) 
Results are shown in Figure 4 

ground_truth: 50 human annotators ranked a random sample of 100 translations by Adequacy, Fluency and overall ranking on a 5-point scale.

adequacy, precision and ranking values

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How do they define robustness of a model?
Context 3:  As expected, $\lambda $ reflects how strong the regularization is  The model tends to be closer to our preferences with the increasing of $\lambda $ on both cases 
Using LDA Selected Features
We compare our methods with GE-FL on all the 9 datasets in this section  Instead of using features obtained by information gain, we use LDA to select labeled features  Unlike information gain, LDA does not employ any instance labels to find labeled features  In this setting, we can build classification models without any instance annotation, but just with labeled features 
Table 1 shows that our three methods significantly outperform GE-FL  Incorporating neutral features performs better than GE-FL on 7 of the 9 datasets, maximum entropy is better on 8 datasets, and KL divergence better on 7 datasets 
LDA selects out the most predictive features as labeled features without considering the balance among classes  GE-FL does not exert any control on such an issue, so the performance is severely suffered  Our methods introduce auxiliary regularization terms to control such a bias problem and thus promote the model significantly 
Related Work
There have been much work that incorporate prior knowledge into learning, and two related lines are surveyed here  One is to use prior knowledge to label unlabeled instances and then apply a standard learning algorithm  The other is to constrain the model directly with prior knowledge 
Liu et al text manually labeled features which are highly predictive to unsupervised clustering assignments and use them to label unlabeled data  Chang et al guiding proposed constraint driven learning  They first used constraints and the learned model to annotate unlabeled instances, and then updated the model with the newly labeled data  Daumé daume2008cross proposed a self training method in which several models are trained on the same dataset, and only unlabeled instances that satisfy the cross task knowledge constraints are used in the self training process 
MaCallum et al gec proposed generalized expectation(GE) criteria which formalised the knowledge as constraint terms about the expectation of the model into the objective function Graça et al pr proposed posterior regularization(PR) framework which projects the model's posterior onto a set of distributions that satisfy the auxiliary constraints  Druck et al ge-fl explored constraints of labeled features in the framework of GE by forcing the model's predicted feature distribution to approach the reference distribution  Andrzejewski et al andrzejewski2011framework proposed a framework in which general domain knowledge can be easily incorporated into LDA  Altendorf et al altendorf2012learning explored monotonicity constraints to improve the accuracy while learning from sparse data  Chen et al chen2013leveraging tried to learn comprehensible topic models by leveraging multi-domain knowledge 
Mann and McCallum simple,generalized incorporated not only labeled features but also other knowledge like class distribution into the objective function of GE-FL  But they discussed only from the semi-supervised perspective and did not investigate into the robustness problem, unlike what we addressed in this paper 
There are also some active learning methods trying to use prior knowledge  Raghavan et al feedback proposed to use feedback on instances and features interlacedly, and demonstrated that feedback on features boosts the model much  Druck et al active proposed an active learning method which solicits labels on features rather than on instances and then used GE-FL to train the model 
Conclusion and Discussions
This paper investigates into the problem of how to leverage prior knowledge robustly in learning models  We propose three regularization terms on top of generalized expectation criteria  As demonstrated by the experimental results, the performance can be considerably improved when taking into account these factors  Comparative results show that our proposed methods is more effective and works more robustly against baselines  To the best of our knowledge, this is the first work to address the robustness problem of leveraging knowledge, and may inspire other research 
We then present more detailed discussions about the three regularization methods  Incorporating neutral features is the simplest way of regularization, which doesn't require any modification of GE-FL but just finding out some common features  But as Figure 1 (a) shows, only using neutral features are not strong enough to handle extremely unbalanced labeled features 
The maximum entropy regularization term shows the strong ability of controlling unbalance 
This method doesn't need any extra knowledge, and is thus suitable when we know nothing about the corpus  But this method assumes that the categories are uniformly distributed, which may not be the case in practice, and it will have a degraded performance if the assumption is violated (see Figure 1 (b), Figure 2 (b), Figure 3 (a)) 
The KL divergence performs much better on unbalanced corpora than other methods  The reason is that KL divergence utilizes the reference class distribution and doesn't make any assumptions  The fact suggests that additional knowledge does benefit the model 
However, the KL divergence term requires providing the true class distribution  Sometimes, we may have the exact knowledge about the true distribution, but sometimes we may not  Fortunately, the model is insensitive to the true distribution and therefore a rough estimation of the true distribution is sufficient  In our experiments, when the true class distribution is 1:2, where the reference class distribution is set to 1:1 5/1:2/1:2 5, the accuracy is 0 755/0 756/0 760 respectively  This provides us the possibility to perform simple computing on the corpus to obtain the distribution in reality  Or, we can set the distribution roughly with domain expertise 

ground_truth: ability to accurately classify texts even when the amount of prior knowledge for different classes is unbalanced, and when the class distribution of the dataset is unbalanced

Low sensitivity to bias in prior knowledge