prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What metrics are used to establish that this makes chatbots more knowledgeable and better at learning and conversation? 
Context 1: Introduction
Chatbots such as dialog and question-answering systems have a long history in AI and natural language processing  Early such systems were mostly built using markup languages such as AIML, handcrafted conversation generation rules, and/or information retrieval techniques BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3   Recent neural conversation models BIBREF4 , BIBREF5 , BIBREF6 are even able to perform open-ended conversations  However, since they do not use explicit knowledge bases and do not perform inference, they often suffer from generic and dull responses BIBREF5 , BIBREF7   More recently, BIBREF8 and BIBREF9 proposed to use knowledge bases (KBs) to help generate responses for knowledge-grounded conversation  However, one major weakness of all existing chat systems is that they do not explicitly or implicitly learn new knowledge in the conversation process  This seriously limits the scope of their applications  In contrast, we humans constantly learn new knowledge in our conversations  Even if some existing systems can use very large knowledge bases either harvested from a large data source such as the Web or built manually, these KBs still miss a large number of facts (knowledge) BIBREF10   It is thus important for a chatbot to continuously learn new knowledge in the conversation process to expand its KB and to improve its conversation ability 
In recent years, researchers have studied the problem of KB completion, i e , inferring new facts (knowledge) automatically from existing facts in a KB  KB completion (KBC) is defined as a binary classification problem: Given a query triple, ( INLINEFORM0 , INLINEFORM1 , INLINEFORM2 ), we want to predict whether the source entity INLINEFORM3 and target entity INLINEFORM4 can be linked by the relation INLINEFORM5   However, existing approaches BIBREF11 , BIBREF12 , BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 solve this problem under the closed-world assumption, i e , INLINEFORM6 , INLINEFORM7 and INLINEFORM8 are all known to exist in the KB  This is a major weakness because it means that no new knowledge or facts may contain unknown entities or relations  Due to this limitation, KBC is clearly not sufficient for knowledge learning in conversations because in a conversation, the user can say anything, which may contain entities and relations that are not already in the KB 
In this paper, we remove this assumption of KBC, and allow all INLINEFORM0 , INLINEFORM1 and INLINEFORM2 to be unknown  We call the new problem open-world knowledge base completion (OKBC)  OKBC generalizes KBC  Below, we show that solving OKBC naturally provides the ground for knowledge learning and inference in conversations  In essence, we formulate an abstract problem of knowledge learning and inference in conversations as a well-defined OKBC problem in the interactive setting 
From the perspective of knowledge learning in conversations, essentially we can extract two key types of information, true facts and queries, from the user utterances  Queries are facts whose truth values need to be determined  Note that we do not study fact or relation extraction in this paper as there is an extensive work on the topic  (1) For a true fact, we will incorporate it into the KB  Here we need to make sure that it is not already in the KB, which involves relation resolution and entity linking  After a fact is added to the KB, we may predict that some related facts involving some existing relations in the KB may also be true (not logical implications as they can be automatically inferred)  For example, if the user says “Obama was born in USA,” the system may guess that (Obama, CitizenOf, USA) (meaning that Obama is a citizen of USA) could also be true based on the current KB  To verify this fact, it needs to solve a KBC problem by treating (Obama, CitizenOf, USA) as a query  This is a KBC problem because the fact (Obama, BornIn, USA) extracted from the original sentence has been added to the KB  Then Obama and USA are in the KB  If the KBC problem is solved, it learns a new fact (Obama, CitizenOf, USA) in addition to the extracted fact (Obama, BornIn, USA)  (2) For a query fact, e g , (Obama, BornIn, USA) extracted from the user question “Was Obama born in USA?” we need to solve the OKBC problem if any of “Obama, “BornIn”, or “USA" is not already in the KB 
We can see that OKBC is the core of a knowledge learning engine for conversation  Thus, in this paper, we focus on solving it  We assume that other tasks such as fact/relation extraction and resolution and guessing of related facts of an extracted fact are solved by other sub-systems 
We solve the OKBC problem by mimicking how humans acquire knowledge and perform reasoning in an interactive conversation  Whenever we encounter an unknown concept or relation while answering a query, we perform inference using our existing knowledge  If our knowledge does not allow us to draw a conclusion, we typically ask questions to others to acquire related knowledge and use it in inference  The process typically involves an inference strategy (a sequence of actions), which interleaves a sequence of processing and interactive actions  A processing action can be the selection of related facts, deriving inference chain, etc , that advances the inference process  An interactive action can be deciding what to ask, formulating a suitable question, etc , that enable us to interact  The process helps grow the knowledge over time and the gained knowledge enables us to communicate better in the future  We call this lifelong interactive learning and inference (LiLi)  Lifelong learning is reflected by the facts that the newly acquired facts are retained in the KB and used in inference for future queries, and that the accumulated knowledge in addition to the updated KB including past inference performances are leveraged to guide future interaction and learning  LiLi should have the following capabilities:
This setting is ideal for many NLP applications like dialog and question-answering systems that naturally provide the scope for human interaction and demand real-time inference 
LiLi starts with the closed-world KBC approach path-ranking (PR) BIBREF11 , BIBREF17 and extends KBC in a major way to open-world knowledge base completion (OKBC)  For a relation INLINEFORM0 , PR works by enumerating paths (except single-link path INLINEFORM1 ) between entity-pairs linked by INLINEFORM2 in the KB and use them as features to train a binary classifier to predict whether a query INLINEFORM3 should be in the KB  Here, a path between two entities is a sequence of relations linking them  In our work, we adopt the latest PR method, C-PR BIBREF16 and extend it to make it work in the open-world setting  C-PR enumerates paths by performing bidirectional random walks over the KB graph while leveraging the context of the source-target entity-pair  We also adopt and extend the compositional vector space model BIBREF20 , BIBREF21 with continual learning capability for prediction 
Given an OKBC query ( INLINEFORM0 , INLINEFORM1 , INLINEFORM2 ) (e g , “(Obama, CitizenOf, USA), which means whether Obama a citizen of USA), LiLi interacts with the user (if needed) by dynamically formulating questions (see the interaction example in Figure 1, which will be further explained in §3) and leverages the interactively acquired knowledge (supporting facts (SFs) in the figure) for continued inference  To do so, LiLi formulates a query-specific inference strategy and executes it  We design LiLi in a Reinforcement Learning (RL) setting that performs sub-tasks like formulating and executing strategy, training a prediction model for inference, and knowledge retention for future use  To the best of our knowledge, our work is the first to address the OKBC problem and to propose an interactive learning mechanism to solve it in a continuous or lifelong manner  We empirically verify the effectiveness of LiLi on two standard real-world KBs: Freebase and WordNet  Experimental results show that LiLi is highly effective in terms of its predictive performance and strategy formulation ability 
Related Work
To the best of our knowledge, we are not aware of any knowledge learning system that can learn new knowledge in the conversation process  This section thus discusses other related work 
Among existing KB completion approaches, BIBREF20 extended the vector space model for zero-shot KB inference  However, the model cannot handle unknown entities and can only work on fixed set of unknown relations with known embeddings  Recently, BIBREF22 proposed a method using external text corpus to perform inference on unknown entities  However, the method cannot handle unknown relations  Thus, these methods are not suitable for our open-world setting  None of the existing KB inference methods perform interactive knowledge learning like LiLi  NELL BIBREF23 continuously updates its KB using facts extracted from the Web  Our task is very different as we do not do Web fact extraction (which is also useful)  We focus on user interactions in this paper  Our work is related to interactive language learning (ILL) BIBREF24 , BIBREF25 , but these are not about KB completion  The work in BIBREF26 allows a learner to ask questions in dialogue  However, this work used RL to learn about whether to ask the user or not  The “what to ask aspect" was manually designed by modeling synthetic tasks  LiLi formulates query-specific inference strategies which embed interaction behaviors  Also, no existing dialogue systems BIBREF4 , BIBREF27 , BIBREF28 , BIBREF29 , BIBREF30 employ lifelong learning to train prediction models by using information/knowledge retained in the past 
Our work is related to general lifelong learning in BIBREF31 , BIBREF32 , BIBREF33 , BIBREF34 , BIBREF35 , BIBREF36   However, they learn only one type of tasks, e g , supervised, topic modeling or reinforcement learning (RL) tasks  None of them is suitable for our setting, which involves interleaving of RL, supervised and interactive learning  More details about lifelong learning can be found in the book BIBREF31  
Interactive Knowledge Learning (LiLi)
We design LiLi as a combination of two interconnected models: (1) a RL model that learns to formulate a query-specific inference strategy for performing the OKBC task, and (2) a lifelong prediction model to predict whether a triple should be in the KB, which is invoked by an action while executing the inference strategy and is learned for each relation as in C-PR  The framework improves its performance over time through user interaction and knowledge retention  Compared to the existing KB inference methods, LiLi overcomes the following three challenges for OKBC:
1  Mapping open-world to close-world  Being a closed-world method, C-PR cannot extract path features and learn a prediction model when any of INLINEFORM0 , INLINEFORM1 or INLINEFORM2 is unknown  LiLi solves this problem through interactive knowledge acquisition  If INLINEFORM3 is unknown, LiLi asks the user to provide a clue (an example of INLINEFORM4 )  And if INLINEFORM5 or INLINEFORM6 is unknown, LiLi asks the user to provide a link (relation) to connect the unknown entity with an existing entity (automatically selected) in the KB  We refer to such a query as a connecting link query (CLQ)  The acquired knowledge reduces OKBC to KBC and makes the inference task feasible 
2  Spareseness of KB  A main issue of all PR methods like C-PR is the connectivity of the KB graph  If there is no path connecting INLINEFORM0 and INLINEFORM1 in the graph, path enumeration of C-PR gets stuck and inference becomes infeasible  In such cases, LiLi uses a template relation (“@-?-@") as the missing link marker to connect entity-pairs and continues feature extraction  A path containing “@-?-@" is called an incomplete path  Thus, the extracted feature set contains both complete (no missing link) and incomplete paths  Next, LiLi selects an incomplete path from the feature set and asks the user to provide a link for path completion  We refer to such a query as missing link query (MLQ) 
3  Limitation in user knowledge  If the user is unable to respond to MLQs or CLQs, LiLi uses a guessing mechanism (discussed later) to fill the gap  This enables LiLi to continue its inference even if the user cannot answer a system question 
Components of LiLi
As lifelong learning needs to retain knowledge learned from past tasks and use it to help future learning BIBREF31 , LiLi uses a Knowledge Store (KS) for knowledge retention  KS has four components: (i) Knowledge Graph ( INLINEFORM0 ): INLINEFORM1 (the KB) is initialized with base KB triples (see §4) and gets updated over time with the acquired knowledge  (ii) Relation-Entity Matrix ( INLINEFORM2 ): INLINEFORM3 is a sparse matrix, with rows as relations and columns as entity-pairs and is used by the prediction model  Given a triple ( INLINEFORM4 , INLINEFORM5 , INLINEFORM6 ) INLINEFORM7 , we set INLINEFORM8 [ INLINEFORM9 , ( INLINEFORM10 , INLINEFORM11 )] = 1 indicating INLINEFORM12 occurs for pair ( INLINEFORM13 , INLINEFORM14 )  (iii) Task Experience Store ( INLINEFORM15 ): INLINEFORM16 stores the predictive performance of LiLi on past learned tasks in terms of Matthews correlation coefficient (MCC) that measures the quality of binary classification  So, for two tasks INLINEFORM17 and INLINEFORM18 (each relation is a task), if INLINEFORM19 [ INLINEFORM20 ] INLINEFORM21 INLINEFORM22 [ INLINEFORM23 ] [where INLINEFORM24 [ INLINEFORM25 ]=MCC( INLINEFORM26 )], we say C-PR has learned INLINEFORM27 well compared to INLINEFORM28   (iv) Incomplete Feature DB ( INLINEFORM29 ): INLINEFORM30 stores the frequency of an incomplete path INLINEFORM31 in the form of a tuple ( INLINEFORM32 , INLINEFORM33 , INLINEFORM34 ) and is used in formulating MLQs  INLINEFORM35 [( INLINEFORM36 , INLINEFORM37 , INLINEFORM38 )] = INLINEFORM39 implies LiLi has extracted incomplete path INLINEFORM40 INLINEFORM41 times involving entity-pair INLINEFORM42 [( INLINEFORM43 , INLINEFORM44 )] for query relation INLINEFORM45  
The RL model learns even after training whenever it encounters an unseen state (in testing) and thus, gets updated over time  KS is updated continuously over time as a result of the execution of LiLi and takes part in future learning  The prediction model uses lifelong learning (LL), where we transfer knowledge (parameter values) from the model for a past most similar task to help learn for the current task  Similar tasks are identified by factorizing INLINEFORM0 and computing a task similarity matrix INLINEFORM1   Besides LL, LiLi uses INLINEFORM2 to identify poorly learned past tasks and acquire more clues for them to improve its skillset over time 
LiLi also uses a stack, called Inference Stack ( INLINEFORM0 ) to hold query and its state information for RL  LiLi always processes stack top ( INLINEFORM1 [top])  The clues from the user get stored in INLINEFORM2 on top of the query during strategy execution and processed first  Thus, the prediction model for INLINEFORM3 is learned before performing inference on query, transforming OKBC to a KBC problem  Table 1 shows the parameters of LiLi used in the following sections 
Working of LiLi
Given an OKBC query ( INLINEFORM0 , INLINEFORM1 , INLINEFORM2 ), we represent it as a data instance INLINEFORM3   INLINEFORM4 consists of INLINEFORM5 (the query triple), INLINEFORM6 (interaction limit set for INLINEFORM7 ), INLINEFORM8 (experience list storing the transition history of MDP for INLINEFORM9 in RL) and INLINEFORM10 (mode of INLINEFORM11 ) denoting if INLINEFORM12 is ` INLINEFORM13 ' (training), ` INLINEFORM14 ' (validation), ` INLINEFORM15 ' (evaluation) or ` INLINEFORM16 ' (clue) instance and INLINEFORM17 (feature set)  We denote INLINEFORM18 ( INLINEFORM19 ) as the set of all complete (incomplete) path features in INLINEFORM20   Given a data instance INLINEFORM21 , LiLi starts its initialization as follows: it sets the state as INLINEFORM22 (based on INLINEFORM23 , explained later), pushes the query tuple ( INLINEFORM24 , INLINEFORM25 ) into INLINEFORM26 and feeds INLINEFORM27 [top] to the RL-model for strategy formulation from INLINEFORM28  
Inference Strategy Formulation  We view solving the strategy formulation problem as learning to play an inference game, where the goal is to formulate a strategy that "makes the inference task possible"  Considering PR methods, inference is possible, iff (1) INLINEFORM0 becomes known to its KB (by acquiring clues when INLINEFORM1 is unknown) and (2) path features are extracted between INLINEFORM2 and INLINEFORM3 (which inturn requires INLINEFORM4 and INLINEFORM5 to be known to KB)  If these conditions are met at the end of an episode (when strategy formulation finishes for a given query) of the game, LiLi wins and thus, it trains the prediction model for INLINEFORM6 and uses it for inference 
LiLi's strategy formulation is modeled as a Markov Decision Process (MDP) with finite state ( INLINEFORM0 ) and action ( INLINEFORM1 ) spaces  A state INLINEFORM2 consists of 10 binary state variables (Table 2), each of which keeps track of results of an action INLINEFORM3 taken by LiLi and thus, records the progress in inference process made so far  INLINEFORM4 is the initial state with all state bits set as 0  If the data instance (query) is a clue [ INLINEFORM5 ], INLINEFORM6 [CLUE] is set as 1  INLINEFORM7 consists of 6 actions (Table 3)  INLINEFORM8 , INLINEFORM9 , INLINEFORM10 are processing actions and INLINEFORM11 , INLINEFORM12 , INLINEFORM13 are interactive actions  Whenever INLINEFORM14 is executed, the MDP reaches the terminal state  Given an action INLINEFORM15 in state INLINEFORM16 , if INLINEFORM17 is invalid in INLINEFORM21 or the objective of INLINEFORM22 is unsatisfied (* marked the condition in INLINEFORM23 ), RL receives a negative reward (empirically set); else receives a positive reward   We use Q-learning BIBREF38 with INLINEFORM24 -greedy strategy to learn the optimal policy for training the RL model  Note that, the inference strategy is independent of KB type and correctness of prediction  Thus, the RL-model is trained only once from scratch (reused thereafter for other KBs) and also, independently of the prediction model 
Sometimes the training dataset may not be enough to learn optimal policy for all INLINEFORM0   Thus, encountering an unseen state during test can make RL-model clueless about the action  Given a state INLINEFORM1 , whenever an invalid INLINEFORM2 is chosen, LiLi remains in INLINEFORM3   For INLINEFORM4 , LiLi remains in INLINEFORM5 untill INLINEFORM6 (see Table 1 for INLINEFORM7 )  So, if the state remains the same for ( INLINEFORM8 +1) times, it implies LiLi has encountered a fault (an unseen state)  RL-model instantly switches to the training mode and randomly explores INLINEFORM9 to learn the optimal action (fault-tolerant learning)  While exploring INLINEFORM10 , the model chooses INLINEFORM11 only when it has tried all other INLINEFORM12 to avoid abrupt end of episode 
Execution of Actions  At any given point in time, let ( INLINEFORM0 , INLINEFORM1 ) be the current INLINEFORM2 [top], INLINEFORM3 is the chosen action and the current version of KS components are INLINEFORM4 , INLINEFORM5 , INLINEFORM6 and INLINEFORM7   Then, if INLINEFORM8 is invalid in INLINEFORM9 , LiLi only updates INLINEFORM10 [top] with ( INLINEFORM11 , INLINEFORM12 ) and returns INLINEFORM13 [top] to RL-model  In this process, LiLi adds experience ( INLINEFORM14 , INLINEFORM15 , INLINEFORM16 , INLINEFORM17 ) in INLINEFORM18 and then, replaces INLINEFORM19 [top] with ( INLINEFORM20 , INLINEFORM21 )  If INLINEFORM22 is valid in INLINEFORM23 , LiLi first sets the next state INLINEFORM24 and performs a sequence of operations INLINEFORM25 based on INLINEFORM26 (discussed below)  Unless specified, in INLINEFORM27 , LiLi always monitors INLINEFORM28 and if INLINEFORM29 becomes 0, LiLi sets INLINEFORM30   Also, whenever LiLi asks the user a query, INLINEFORM31 is decremented by 1  Once INLINEFORM32 ends, LiLi updates INLINEFORM33 [top] with ( INLINEFORM34 , INLINEFORM35 ) and returns INLINEFORM36 [top] to RL-model for choosing the next action 
In INLINEFORM0 , LiLi searches INLINEFORM1 , INLINEFORM2 , INLINEFORM3 in INLINEFORM4 and sets appropriate bits in INLINEFORM5 (see Table 2)  If INLINEFORM6 was unknown before and is just added to INLINEFORM7 or is in the bottom INLINEFORM8 % (see Table 1 for INLINEFORM9 ) of INLINEFORM10 , LiLi randomly sets INLINEFORM14 with probability INLINEFORM15   If INLINEFORM16 is a clue and INLINEFORM17 , LiLi updates KS with triple INLINEFORM18 , where ( INLINEFORM19 , INLINEFORM20 , INLINEFORM21 ) and ( INLINEFORM22 , INLINEFORM23 , INLINEFORM24 ) gets added to INLINEFORM25 and INLINEFORM26 , INLINEFORM27 are set as 1 
In INLINEFORM0 , LiLi asks the user to provide a clue (+ve instance) for INLINEFORM1 and corrupts INLINEFORM2 and INLINEFORM3 of the clue once at a time, to generate -ve instances by sampling nodes from INLINEFORM4   These instances help in training prediction model for INLINEFORM5 while executing INLINEFORM6  
In INLINEFORM0 , LiLi selects an incomplete path INLINEFORM1 from INLINEFORM2 to formulate MLQ, such that INLINEFORM3 is most frequently observed for INLINEFORM4 and INLINEFORM5 is high, given by INLINEFORM6   Here, INLINEFORM7 denotes the contextual similarity BIBREF16 of entity-pair INLINEFORM8   If INLINEFORM9 is high, INLINEFORM10 is more likely to possess a relation between them and so, is a good candidate for formulating MLQ  When the user does not respond to MLQ (or CLQ in INLINEFORM11 ), the guessing mechanism is used, which works as follows: Since contextual similarity of entity-pairs is highly correlated with their class labels BIBREF16 , LiLi divides the similarity range [-1, 1] into three segments, using a low ( INLINEFORM12 ) and high ( INLINEFORM13 ) similarity threshold and replaces the missing link with INLINEFORM14 in INLINEFORM15 to make it complete as follows: If INLINEFORM16 , INLINEFORM17 = “@-LooselyRelatedTo-@"; else if INLINEFORM18 , INLINEFORM19 =“@-NotRelatedTo-@"; Otherwise, INLINEFORM20 =“@-RelatedTo-@" 
In INLINEFORM0 , LiLi asks CLQs for connecting unknown entities INLINEFORM1 and/or INLINEFORM2 with INLINEFORM3 by selecting the most contextually relevant node (wrt INLINEFORM4 , INLINEFORM5 ) from INLINEFORM6 , given by link INLINEFORM7   We adopt the contextual relevance idea in BIBREF16 which is computed using word embedding BIBREF39
In INLINEFORM0 , LiLi extracts path features INLINEFORM1 between ( INLINEFORM2 , INLINEFORM3 ) and updates INLINEFORM4 with incomplete features from INLINEFORM5   LiLi always trains the prediction model with complete features INLINEFORM6 and once INLINEFORM7 or INLINEFORM8 , LiLi stops asking MLQs  Thus, in both INLINEFORM9 and INLINEFORM10 , LiLi always monitors INLINEFORM11 to check for the said requirements and sets INLINEFORM12 to control interactions 
In INLINEFORM0 , if LiLi wins the episode, it adds INLINEFORM1 in one of data buffers INLINEFORM2 based on its mode INLINEFORM3   E g , if INLINEFORM4 or INLINEFORM5 , INLINEFORM6 is used for training and added to INLINEFORM7   Similarly validation buffer INLINEFORM8 and evaluation buffer INLINEFORM9 are populated  If INLINEFORM10 , LiLi invokes the prediction model for INLINEFORM11  
Lifelong Relation Prediction  Given a relation INLINEFORM0 , LiLi uses INLINEFORM1 and INLINEFORM2 (see INLINEFORM3 ) to train a prediction model (say, INLINEFORM4 ) with parameters INLINEFORM5   For a unknown INLINEFORM6 , the clue instances get stored in INLINEFORM7 and INLINEFORM8   Thus, LiLi populates INLINEFORM9 by taking 10% (see §4) of the instances from INLINEFORM10 and starts the training  For INLINEFORM11 , LiLi uses a LSTM BIBREF40 to compose the vector representation of each feature INLINEFORM12 as INLINEFORM13 and vector representation of INLINEFORM14 as INLINEFORM15   Next, LiLi computes the prediction value, INLINEFORM16 as sigmoid of the mean cosine similarity of all features and INLINEFORM17 , given by INLINEFORM18 ) and maximize the log-likelihood of INLINEFORM19 for training  Once INLINEFORM20 is trained, LiLi updates INLINEFORM21 [ INLINEFORM22 ] using INLINEFORM23   We also train an inverse model for INLINEFORM24 , INLINEFORM25 by reversing the path features in INLINEFORM26 and INLINEFORM27 which help in lifelong learning (discussed below)  Unlike BIBREF20 , BIBREF21 , while predicting the label for INLINEFORM28 , we compute a relation-specific prediction threshold INLINEFORM29 corresponding to INLINEFORM30 using INLINEFORM31 as: INLINEFORM32 and infer INLINEFORM33 as +ve if INLINEFORM34 and -ve otherwise  Here, INLINEFORM35 ( INLINEFORM36 ) is the mean prediction value for all +ve (-ve) examples in INLINEFORM37  
Models trained on a few examples (e g , clues acquired for unknown INLINEFORM0 ) with randomly initialized weights often perform poorly due to underfitting  Thus, we transfer knowledge (weights) from the past most similar (wrt INLINEFORM1 ) task in a lifelong learning manner BIBREF31   LiLi uses INLINEFORM2 to find the past most similar task for INLINEFORM3 as follows: LiLi computes trancated SVD of INLINEFORM4 as INLINEFORM5 and then, the similarity matrix INLINEFORM6   INLINEFORM7 provides the similarity between relations INLINEFORM8 and INLINEFORM9 in INLINEFORM10   Thus, LiLi chooses a source relation INLINEFORM11 to transfer weights  Here, INLINEFORM12 is the set of all INLINEFORM13 and INLINEFORM14 for which LiLi has already learned a prediction model  Now, if INLINEFORM15 or INLINEFORM16 , LiLi randomly initializes the weights INLINEFORM17 for INLINEFORM18 and proceeds with the training  Otherwise, LiLi uses INLINEFORM19 as initial weights and fine-tunes INLINEFORM20 with a low learning rate 
A Running Example  Considering the example shown in Figure 1, LiLi works as follows: first, LiLi executes INLINEFORM0 and detects that the source entity “Obama" and query relation “CitizenOf" are unknown  Thus, LiLi executes INLINEFORM1 to acquire clue (SF1) for “CitizenOf" and pushes the clue (+ve example) and two generated -ve examples into INLINEFORM2   Once the clues are processed and a prediction model is trained for “CitizenOf" by formulating separate strategies for them, LiLi becomes aware of “CitizenOf"  Now, as the clues have already been popped from INLINEFORM3 , the query becomes INLINEFORM4 and the strategy formulation process for the query resumes  Next, LiLi asks user to provide a connecting link for “Obama" by performing INLINEFORM5   Now, the query entities and relation being known, LiLi enumerates paths between “Obama" and “USA" by performing INLINEFORM6   Let an extracted path be “ INLINEFORM7 " with missing link between ( INLINEFORM8 , INLINEFORM9 )  LiLi asks the user to fill the link by performing INLINEFORM10 and then, extracts the complete feature “ INLINEFORM11 "  The feature set is then fed to the prediction model and inference is made as a result of INLINEFORM12   Thus, the formulated inference strategy is: “ INLINEFORM13 " 
Experiments
We now evaluate LiLi in terms of its predictive performance and strategy formulation abilities 
Data: We use two standard datasets (see Table 4): (1) Freebase FB15k, and (2) WordNet INLINEFORM0   Using each dataset, we build a fairly large graph and use it as the original KB ( INLINEFORM1 ) for evaluation  We also augment INLINEFORM2 with inverse triples ( INLINEFORM3 , INLINEFORM4 , INLINEFORM5 ) for each ( INLINEFORM6 , INLINEFORM7 , INLINEFORM8 ) following existing KBC methods 
Parameter Settings  Unless specified, the empirically set parameters (see Table 1) of LiLi are: INLINEFORM0 , INLINEFORM1 , INLINEFORM2 , INLINEFORM3 , INLINEFORM4 , INLINEFORM5 , INLINEFORM6 , INLINEFORM7 , INLINEFORM8 , INLINEFORM9 , INLINEFORM10   For training RL-model with INLINEFORM11 -greedy strategy, we use INLINEFORM12 , INLINEFORM13 , pre-training steps=50000  We used Keras deep learning library to implement and train the prediction model  We set batch-size as 128, max  training epoch as 150, dropout as 0 2, hidden units and embedding size as 300 and learning rate as 5e-3 which is reduced gradually on plateau with factor 0 5 and patience 5  Adam optimizer and early stopping were used in training  We also shuffle INLINEFORM14 in each epoch and adjust class weights inversely proportional to class frequencies in INLINEFORM15  
Labeled Dataset Generation and Simulated User Creation  We create a simulated user for each KB to evaluate LiLi  We create the labeled datasets, the simulated user’s knowledge base ( INLINEFORM0 ), and the base KB ( INLINEFORM1 ) from INLINEFORM2   INLINEFORM3 used as the initial KB graph ( INLINEFORM4 ) of LiLi 
We followed BIBREF16 for labeled dataset generation  For Freebase, we found 86 relations with INLINEFORM0 triples and randomly selected 50 from various domains  We randomly shuffle the list of 50 relations, select 25% of them as unknown relations and consider the rest (75%) as known relations  For each known relation INLINEFORM1 , we randomly shuffle the list of distinct triples for INLINEFORM2 , choose 1000 triples and split them into 60% training, 10% validation and 20% test  Rest 10% along with the leftover (not included in the list of 1000) triples are added to INLINEFORM3   For each unknown relation INLINEFORM4 , we remove all triples of INLINEFORM5 from INLINEFORM6 and add them to INLINEFORM7   In this process, we also randomly choose 20% triples as test instances for unknown INLINEFORM8 which are excluded from INLINEFORM9   Note that, now INLINEFORM10 has at least 10% of chosen triples for each INLINEFORM11 (known and unknown) and so, user is always able to provide clues for both cases  For each labeled dataset, we randomly choose 10% of the entities present in dataset triples, remove triples involving those entities from INLINEFORM12 and add to INLINEFORM13   At this point, INLINEFORM14 gets reduced to INLINEFORM15 and is used as INLINEFORM16 for LiLi  The dataset stats in Table 4 shows that the base KB (60% triples of INLINEFORM17 ) is highly sparse (compared to original KB) which makes the inference task much harder  WordNet dataset being small, we select all 18 relations for evaluation and create labeled dataset, INLINEFORM18 and INLINEFORM19 following Freebase  Although the user may provide clues 100% of the time, it often cannot respond to MLQs and CLQs (due to lack of required triples/facts)  Thus, we further enrich INLINEFORM20 with external KB triples 
Given a relation INLINEFORM0 and an observed triple ( INLINEFORM1 , INLINEFORM2 , INLINEFORM3 ) in training or testing, the pair ( INLINEFORM4 , INLINEFORM5 ) is regarded as a +ve instance for INLINEFORM6   Following BIBREF18 , for each +ve instance ( INLINEFORM7 , INLINEFORM8 ), we generate two negative ones, one by randomly corrupting the source INLINEFORM9 , and the other by corrupting the target INLINEFORM10   Note that, the test triples are not in INLINEFORM11 or INLINEFORM12 and none of the -ve instances overlap with the +ve ones 
Baselines  As none of the existing KBC methods can solve the OKBC problem, we choose various versions of LiLi as baselines 
Single: Version of LiLi where we train a single prediction model INLINEFORM0 for all test relations 
Sep: We do not transfer (past learned) weights for initializing INLINEFORM0 , i e , we disable LL 
F-th): Here, we use a fixed prediction threshold 0 5 instead of relation-specific threshold INLINEFORM0  
BG: The missing or connecting links (when the user does not respond) are filled with “@-RelatedTo-@" blindly, no guessing mechanism 
w/o PTS: LiLi does not ask for additional clues via past task selection for skillset improvement 
Evaluation Metrics  To evaluate the strategy formulation ability, we introduce a measure called Coverage( INLINEFORM0 ), defined as the fraction of total query data instances, for which LiLi has successfully formulated strategies that lead to winning  If LiLi wins on all episodes for a given dataset, INLINEFORM1 is 1 0  To evaluate the predictive performance, we use Avg  MCC and avg  +ve F1 score 
Results and Analysis
Evaluation-I: Strategy Formulation Ability  Table 5 shows the list of inference strategies formulated by LiLi for various INLINEFORM0 and INLINEFORM1 , which control the strategy formulation of LiLi  When INLINEFORM2 , LiLi cannot interact with user and works like a closed-world method  Thus, INLINEFORM3 drops significantly (0 47)  When INLINEFORM4 , i e  with only one interaction per query, LiLi acquires knowledge well for instances where either of the entities or relation is unknown  However, as one unknown entity may appear in multiple test triples, once the entity becomes known, LiLi doesn’t need to ask for it again and can perform inference on future triples causing significant increase in INLINEFORM5 (0 97)  When INLINEFORM6 , LiLi is able to perform inference on all instances and INLINEFORM7 becomes 1  For INLINEFORM8 , LiLi uses INLINEFORM9 only once (as only one MLQ satisfies INLINEFORM10 ) compared to INLINEFORM11   In summary, LiLi’s RL-model can effectively formulate query-specific inference strategies (based on specified parameter values)  Evaluation-II: Predictive Performance  Table 6 shows the comparative performance of LiLi with baselines  To judge the overall improvements, we performed paired t-test considering +ve F1 scores on each relation as paired data  Considering both KBs and all relation types, LiLi outperforms Sep with INLINEFORM12   If we set INLINEFORM13 (training with very few clues), LiLi outperforms Sep with INLINEFORM14 on Freebase considering MCC  Thus, the lifelong learning mechanism is effective in transferring helpful knowledge  Single model performs better than Sep for unknown relations due to the sharing of knowledge (weights) across tasks  However, for known relations, performance drops because, as a new relation arrives to the system, old weights get corrupted and catastrophic forgetting occurs  For unknown relations, as the relations are evaluated just after training, there is no chance for catastrophic forgetting  The performance improvement ( INLINEFORM15 ) of LiLi over F-th on Freebase signifies that the relation-specific threshold INLINEFORM16 works better than fixed threshold 0 5 because, if all prediction values for test instances lie above (or below) 0 5, F-th predicts all instances as +ve (-ve) which degrades its performance  Due to the utilization of contextual similarity (highly correlated with class labels) of entity-pairs, LiLi’s guessing mechanism works better ( INLINEFORM17 ) than blind guessing (BG)  The past task selection mechanism of LiLi also improves its performance over w/o PTS, as it acquires more clues during testing for poorly performed tasks (evaluated on validation set)  For Freebase, due to a large number of past tasks [9 (25% of 38)], the performance difference is more significant ( INLINEFORM18 )  For WordNet, the number is relatively small [3 (25% of 14)] and hence, the difference is not significant 
Evaluation-III: User Interaction vs  Performance  Table 7 shows the results of LiLi by varying clue acquisition rate ( INLINEFORM0 )  We use Freebase for tuning INLINEFORM1 due to its higher number of unknown test relations compared to WordNet  LiLi’s performance improves significantly as it acquires more clues from the user  The results on INLINEFORM2 outperforms ( INLINEFORM3 ) that on INLINEFORM4   Table 8 shows the results of LiLi on user responses to MLQ’s and CLQ’s  Answering MLQ’s and CLQ’s is very hard for simulated users (unlike crowd-sourcing) as often INLINEFORM5 lacks the required triple  Thus, we attempt to analyze how the performance is effected if the user does not respond at all  The results show a clear trend in overall performance improvement when the user responds  However, the improvement is not significant as the simulated user’s query satisfaction rate (1% MLQs and 10% CLQs) is very small  But, the analysis shows the effectiveness of LiLi’s guessing mechanism and continual learning ability that help in achieving avg  +ve F1 of 0 57 and 0 62 on FB and WN respectively with minimal participation of the user 
Conclusion
In this paper, we are interested in building a generic engine for continuous knowledge learning in human-machine conversations  We first showed that the problem underlying the engine can be formulated as an open-world knowledge base completion (OKBC) problem  We then proposed an lifelong interactive learning and inference (LiLi) approach to solving the OKBC problem  OKBC is a generalization of KBC  LiLi solves the OKBC problem by first formulating a query-specific inference strategy using RL and then executing it to solve the problem by interacting with the user in a lifelong learning manner  Experimental results showed the effectiveness of LiLi in terms of both predictive quality and strategy formulation ability  We believe that a system with the LiLi approach can serve as a knowledge learning engine for conversations  Our future work will improve LiLi to make more accurate 
Acknowledgments
This work was supported in part by National Science Foundation (NSF) under grant no  IIS-1407927 and IIS-1650900, and a gift from Huawei Technologies Co Ltd 
Context 2: Introduction
Accurate language identification (LID) is the first step in many natural language processing and machine comprehension pipelines  If the language of a piece of text is known then the appropriate downstream models like parts of speech taggers and language models can be applied as required 
LID is further also an important step in harvesting scarce language resources  Harvested data can be used to bootstrap more accurate LID models and in doing so continually improve the quality of the harvested data  Availability of data is still one of the big roadblocks for applying data driven approaches like supervised machine learning in developing countries 
Having 11 official languages of South Africa has lead to initiatives (discussed in the next section) that have had positive effect on the availability of language resources for research  However, many of the South African languages are still under resourced from the point of view of building data driven models for machine comprehension and process automation 
Table TABREF2 shows the percentages of first language speakers for each of the official languages of South Africa  These are four conjunctively written Nguni languages (zul, xho, nbl, ssw), Afrikaans (afr) and English (eng), three disjunctively written Sotho languages (nso, sot, tsn), as well as tshiVenda (ven) and Xitsonga (tso)  The Nguni languages are similar to each other and harder to distinguish  The same is true of the Sotho languages 
This paper presents a hierarchical naive Bayesian and lexicon based classifier for LID of short pieces of text of 15-20 characters long  The algorithm is evaluated against recent approaches using existing test sets from previous works on South African languages as well as the Discriminating between Similar Languages (DSL) 2015 and 2017 shared tasks 
Section SECREF2 reviews existing works on the topic and summarises the remaining research problems  Section SECREF3 of the paper discusses the proposed algorithm and Section SECREF4 presents comparative results 
Related Works
The focus of this section is on recently published datasets and LID research applicable to the South African context  An in depth survey of algorithms, features, datasets, shared tasks and evaluation methods may be found in BIBREF0 
The datasets for the DSL 2015 & DSL 2017 shared tasks BIBREF1 are often used in LID benchmarks and also available on Kaggle   The DSL datasets, like other LID datasets, consists of text sentences labelled by language  The 2017 dataset, for example, contains 14 languages over 6 language groups with 18000 training samples and 1000 testing samples per language 
The recently published JW300 parallel corpus BIBREF2 covers over 300 languages with around 100 thousand parallel sentences per language pair on average  In South Africa, a multilingual corpus of academic texts produced by university students with different mother tongues is being developed BIBREF3  The WiLI-2018 benchmark dataset BIBREF4 for monolingual written natural language identification includes around 1000 paragraphs of 235 languages  A possibly useful link can also be made BIBREF5 between Native Language Identification (NLI) (determining the native language of the author of a text) and Language Variety Identification (LVI) (classification of different varieties of a single language) which opens up more datasets  The Leipzig Corpora Collection BIBREF6, the Universal Declaration of Human Rights and Tatoeba are also often used sources of data 
The NCHLT text corpora BIBREF7 is likely a good starting point for a shared LID task dataset for the South African languages BIBREF8  The NCHLT text corpora contains enough data to have 3500 training samples and 600 testing samples of 300+ character sentences per language  Researchers have recently started applying existing algorithms for tasks like neural machine translation in earnest to such South African language datasets BIBREF9 
Existing NLP datasets, models and services BIBREF10 are available for South African languages  These include an LID algorithm BIBREF11 that uses a character level n-gram language model  Multiple papers have shown that 'shallow' naive Bayes classifiers BIBREF12, BIBREF8, BIBREF13, BIBREF14, SVMs BIBREF15 and similar models work very well for doing LID  The DSL 2017 paper BIBREF1, for example, gives an overview of the solutions of all of the teams that competed on the shared task and the winning approach BIBREF16 used an SVM with character n-gram, parts of speech tag features and some other engineered features  The winning approach for DSL 2015 used an ensemble naive Bayes classifier  The fasttext classifier BIBREF17 is perhaps one of the best known efficient 'shallow' text classifiers that have been used for LID  
Multiple papers have proposed hierarchical stacked classifiers (including lexicons) that would for example first classify a piece of text by language group and then by exact language BIBREF18, BIBREF19, BIBREF8, BIBREF0  Some work has also been done on classifying surnames between Tshivenda, Xitsonga and Sepedi BIBREF20  Additionally, data augmentation BIBREF21 and adversarial training BIBREF22 approaches are potentially very useful to reduce the requirement for data 
Researchers have investigated deeper LID models like bidirectional recurrent neural networks BIBREF23 or ensembles of recurrent neural networks BIBREF24  The latter is reported to achieve 95 12% in the DSL 2015 shared task  In these models text features can include character and word n-grams as well as informative character and word-level features learnt BIBREF25 from the training data  The neural methods seem to work well in tasks where more training data is available 
In summary, LID of short texts, informal styles and similar languages remains a difficult problem which is actively being researched  Increased confusion can in general be expected between shorter pieces of text and languages that are more closely related  Shallow methods still seem to work well compared to deeper models for LID  Other remaining research opportunities seem to be data harvesting, building standardised datasets and creating shared tasks for South Africa and Africa  Support for language codes that include more languages seems to be growing and discoverability of research is improving with more survey papers coming out  Paywalls also seem to no longer be a problem; the references used in this paper was either openly published or available as preprint papers 
Methodology
The proposed LID algorithm builds on the work in BIBREF8 and BIBREF26  We apply a naive Bayesian classifier with character (2, 4 & 6)-grams, word unigram and word bigram features with a hierarchical lexicon based classifier 
The naive Bayesian classifier is trained to predict the specific language label of a piece of text, but used to first classify text as belonging to either the Nguni family, the Sotho family, English, Afrikaans, Xitsonga or Tshivenda

ground_truth: Coverage, Avg. MCC and avg. +ve F1 score

strategy formulation ability, we introduce a measure called Coverage( INLINEFORM0 ), To evaluate the predictive performance, we use Avg. MCC and avg. +ve F1 score

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which languages are similar to each other?
Context 2:  The scikit-learn multinomial naive Bayes classifier is used for the implementation with an alpha smoothing value of 0 01 and hashed text features 
The lexicon based classifier is then used to predict the specific language within a language group  For the South African languages this is done for the Nguni and Sotho groups  If the lexicon prediction of the specific language has high confidence then its result is used as the final label else the naive Bayesian classifier's specific language prediction is used as the final result  The lexicon is built over all the data and therefore includes the vocabulary from both the training and testing sets 
The lexicon based classifier is designed to trade higher precision for lower recall  The proposed implementation is considered confident if the number of words from the winning language is at least one more than the number of words considered to be from the language scored in second place 
The stacked classifier is tested against three public LID implementations BIBREF17, BIBREF23, BIBREF8  The LID implementation described in BIBREF17 is available on GitHub and is trained and tested according to a post on the fasttext blog  Character (5-6)-gram features with 16 dimensional vectors worked the best  The implementation discussed in BIBREF23 is available from https://github com/tomkocmi/LanideNN  Following the instructions for an OSX pip install of an old r0 8 release of TensorFlow, the LanideNN code could be executed in Python 3 7 4  Settings were left at their defaults and a learning rate of 0 001 was used followed by a refinement with learning rate of 0 0001  Only one code modification was applied to return the results from a method that previously just printed to screen  The LID algorithm described in BIBREF8 is also available on GitHub 
The stacked classifier is also tested against the results reported for four other algorithms BIBREF16, BIBREF26, BIBREF24, BIBREF15  All the comparisons are done using the NCHLT BIBREF7, DSL 2015 BIBREF19 and DSL 2017 BIBREF1 datasets discussed in Section SECREF2 
Results and Analysis
The average classification accuracy results are summarised in Table TABREF9  The accuracies reported are for classifying a piece of text by its specific language label  Classifying text only by language group or family is a much easier task as reported in BIBREF8 
Different variations of the proposed classifier were evaluated  A single NB classifier (NB), a stack of two NB classifiers (NB+NB), a stack of a NB classifier and lexicon (NB+Lex) and a lexicon (Lex) by itself  A lexicon with a 50% training token dropout is also listed to show the impact of the lexicon support on the accuracy 
From the results it seems that the DSL 2017 task might be harder than the DSL 2015 and NCHLT tasks  Also, the results for the implementation discussed in BIBREF23 might seem low, but the results reported in that paper is generated on longer pieces of text so lower scores on the shorter pieces of text derived from the NCHLT corpora is expected 
The accuracy of the proposed algorithm seems to be dependent on the support of the lexicon  Without a good lexicon a non-stacked naive Bayesian classifier might even perform better 
The execution performance of some of the LID implementations are shown in Table TABREF10  Results were generated on an early 2015 13-inch Retina MacBook Pro with a 2 9 GHz CPU (Turbo Boosted to 3 4 GHz) and 8GB RAM  The C++ implementation in BIBREF17 is the fastest  The implementation in BIBREF8 makes use of un-hashed feature representations which causes it to be slower than the proposed sklearn implementation  The execution performance of BIBREF23 might improve by a factor of five to ten when executed on a GPU 
Conclusion
LID of short texts, informal styles and similar languages remains a difficult problem which is actively being researched  The proposed algorithm was evaluated on three existing datasets and compared to the implementations of three public LID implementations as well as to reported results of four other algorithms  It performed well relative to the other methods beating their results  However, the performance is dependent on the support of the lexicon 
We would like to investigate the value of a lexicon in a production system and how to possibly maintain it using self-supervised learning  We are investigating the application of deeper language models some of which have been used in more recent DSL shared tasks  We would also like to investigate data augmentation strategies to reduce the amount of training data that is required 
Further research opportunities include data harvesting, building standardised datasets and shared tasks for South Africa as well as the rest of Africa  In general, the support for language codes that include more languages seems to be growing, discoverability of research is improving and paywalls seem to no longer be a big problem in getting access to published research 
Context 3: Introduction
One of the most fundamental topics in natural language processing is how best to derive high-level representations from constituent parts, as natural language meanings are a function of their constituent parts  How best to construct a sentence representation from distributed word embeddings is an example domain of this larger issue  Even though sequential neural models such as recurrent neural networks (RNN) BIBREF0 and their variants including Long Short-Term Memory (LSTM) BIBREF1 and Gated Recurrent Unit (GRU) BIBREF2 have become the de-facto standard for condensing sentence-level information from a sequence of words into a fixed vector, there have been many lines of research towards better sentence representation using other neural architectures, e g  convolutional neural networks (CNN) BIBREF3 or self-attention based models BIBREF4  
From a linguistic point of view, the underlying tree structure—as expressed by its constituency and dependency trees—of a sentence is an integral part of its meaning  Inspired by this fact, some recursive neural network (RvNN) models are designed to reflect the syntactic tree structure, achieving impressive results on several sentence-level tasks such as sentiment analysis BIBREF5 , BIBREF6 , machine translation BIBREF7 , natural language inference BIBREF8 , and discourse relation classification BIBREF9  
However, some recent works have BIBREF10 , BIBREF11 proposed latent tree models, which learn to construct task-specific tree structures without explicit supervision, bringing into question the value of linguistically-motivated recursive neural models  Witnessing the surprising performance of the latent tree models on some sentence-level tasks, there arises a natural question: Are linguistic tree structures the optimal way of composing sentence representations for NLP tasks?
In this paper, we demonstrate that linguistic priors are in fact useful for devising effective neural models for sentence representations, showing that our novel architecture based on constituency trees and their tag information obtains superior performance on several sentence-level tasks, including sentiment analysis and natural language inference 
A chief novelty of our approach is that we introduce a small separate tag-level tree-LSTM to control the composition function of the existing word-level tree-LSTM, which is in charge of extracting helpful syntactic signals for meaningful semantic composition of constituents by considering both the structures and linguistic tags of constituency trees simultaneously  In addition, we demonstrate that applying a typical LSTM to preprocess the leaf nodes of a tree-LSTM greatly improves the performance of the tree models  Moreover, we propose a clustered tag set to replace the existing tags on the assumption that the original syntactic tags are too fined-grained to be useful in neural models 
In short, our contributions in this work are as follows:
Related Work
Recursive neural networks (RvNN) are a kind of neural architecture which model sentences by exploiting syntactic structure  While earlier RvNN models proposed utilizing diverse composition functions, including feed-forward neural networks BIBREF12 , matrix-vector multiplication BIBREF5 , and tensor computation BIBREF6 , tree-LSTMs BIBREF13 remain the standard for several sentence-level tasks 
Even though classic RvNNs have demonstrated superior performance on a variety of tasks, their inflexibility, i e  their inability to handle dynamic compositionality for different syntactic configurations, is a considerable weakness  For instance, it would be desirable if our model could distinguish e g  adjective-noun composition from that of verb-noun or preposition-noun composition, as models failing to make such a distinction ignore real-world syntactic considerations such as `-arity' of function words (i e  types), and the adjunct/argument distinction 
To enable dynamic compositionality in recursive neural networks, many previous works BIBREF14 , BIBREF15 , BIBREF16 , BIBREF9 , BIBREF17 , BIBREF18 , BIBREF19 have proposed various methods 
One main direction of research leverages tag information, which is produced as a by-product of parsing  In detail, BIBREF16 ( BIBREF16 ) suggested TG-RNN, a model employing different composition functions according to POS tags, and TE-RNN/TE-RNTN, models which leverage tag embeddings as additional inputs for the existing tree-structured models  Despite the novelty of utilizing tag information, the explosion of the number of parameters (in case of the TG-RNN) and the limited performance of the original models (in case of the TE-RNN/TE-RNTN) have prevented these models from being widely adopted  Meanwhile, BIBREF9 ( BIBREF9 ) and BIBREF18 ( BIBREF18 ) proposed models based on a tree-LSTM which also uses the tag vectors to control the gate functions of the tree-LSTM  In spite of their impressive results, there is a limitation that the trained tag embeddings are too simple to reflect the rich information which tags provide in different syntactic structures  To alleviate this problem, we introduce structure-aware tag representations in the next section 
Another way of building dynamic compositionality into RvNNs is to take advantage of a meta-network (or hyper-network)  Inspired by recent works on dynamic parameter prediction, DC-TreeLSTMs BIBREF17 dynamically create the parameters for compositional functions in a tree-LSTM  Specifically, the model has two separate tree-LSTM networks whose architectures are similar, but the smaller of the two is utilized to calculate the weights of the bigger one  A possible problem for this model is that it may be easy to be trained such that the role of each tree-LSTM is ambiguous, as they share the same input, i e  word information  Therefore, we design two disentangled tree-LSTMs in our model so that one focuses on extracting useful features from only syntactic information while the other composes semantic units with the aid of the features  Furthermore, our model reduces the complexity of computation by utilizing typical tree-LSTM frameworks instead of computing the weights for each example 
Finally, some recent works BIBREF10 , BIBREF11 have proposed latent tree-structured models that learn how to formulate tree structures from only sequences of tokens, without the aid of syntactic trees or linguistic information  The latent tree models have the advantage of being able to find the optimized task-specific order of composition rather than a sequential or syntactic one  In experiments, we compare our model with not only syntactic tree-based models but also latent tree models, demonstrating that modeling with explicit linguistic knowledge can be an attractive option 
Model
In this section, we introduce a novel RvNN architecture, called SATA Tree-LSTM (Structure-Aware Tag Augmented Tree-LSTM)  This model is similar to typical Tree-LSTMs, but provides dynamic compositionality by augmenting a separate tag-level tree-LSTM which produces structure-aware tag representations for each node in a tree  In other words, our model has two independent tree-structured modules based on the same constituency tree, one of which (word-level tree-LSTM) is responsible for constructing sentence representations given a sequence of words as usual, while the other (tag-level tree-LSTM) provides supplementary syntactic information to the former 
In section 3 1, we first review tree-LSTM architectures  Then in section 3 2, we introduce a tag-level tree-LSTM and structure-aware tag representations  In section 3 3, we discuss an additional technique to boost the performance of tree-structured models, and in section 3 4, we describe the entire architecture of our model in detail 
Tree-LSTM
The LSTM BIBREF1 architecture was first introduced as an extension of the RNN architecture to mitigate the vanishing and exploding gradient problems  In addition, several works have discovered that applying the LSTM cell into tree structures can be an effective means of modeling sentence representations 
To be formal, the composition function of the cell in a tree-LSTM can be formulated as follows:
$$  \begin{bmatrix} \mathbf {i} \\ \mathbf {f}_l \\ \mathbf {f}_r \\ \mathbf {o} \\ \mathbf {g} \end{bmatrix} = \begin{bmatrix} \sigma \\ \sigma \\ \sigma \\ \sigma \\ \tanh \end{bmatrix} \Bigg ( \mathbf {W} \begin{bmatrix} \mathbf {h}_l \\ \mathbf {h}_r \\ \end{bmatrix} + \mathbf {b} \Bigg )$$   (Eq  8)
$$  \mathbf {c} = \mathbf {f}_l \odot \mathbf {c}_l + \mathbf {f}_r \odot \mathbf {c}_r + \mathbf {i} \odot \mathbf {g}\\$$   (Eq  9)
where $\mathbf {h}, \mathbf {c} \in \mathbb {R}^{d}$ indicate the hidden state and cell state of the LSTM cell, and $\mathbf {h}_l, \mathbf {h}_r, \mathbf {c}_l, \mathbf {c}_r \in \mathbb {R}^{d}$ the hidden states and cell states of a left and right child  $\mathbf {g} \in \mathbb {R}^{d}$ is the newly composed input for the cell and $\mathbf {i}, \mathbf {f}_{l}, \mathbf {f}_{r}, \mathbf {o} \in \mathbb {R}^{d}$ represent an input gate, two forget gates (left, right), and an output gate respectively  $\mathbf {W} \in \mathbb {R}^{5d\times 2d}$ and $\mathbf {b} \in \mathbb {R}^{5d}$ are trainable parameters  $\sigma $ corresponds to the sigmoid function, $\tanh $ to the hyperbolic tangent, and $\odot $ to element-wise multiplication 
Note the equations assume that there are only two children for each node, i e  binary or binarized trees, following the standard in the literature  While RvNN models can be constructed on any tree structure, in this work we only consider constituency trees as inputs 
In spite of the obvious upside that recursive models have in being so flexible, they are known for being difficult to fully utilize with batch computations as compared to other neural architectures because of the diversity of structure found across sentences  To alleviate this problem, BIBREF8 ( BIBREF8 ) proposed the SPINN model, which brings a shift-reduce algorithm to the tree-LSTM  As SPINN simplifies the process of constructing a tree into only two operations, i e  shift and reduce, it can support more effective parallel computations while enjoying the advantages of tree structures  For efficiency, our model also starts from our own SPINN re-implementation, whose function is exactly the same as that of the tree-LSTM 
Structure-aware Tag Representation
In most previous works using linguistic tag information BIBREF16 , BIBREF9 , BIBREF18 , tags are usually represented as simple low-dimensional dense vectors, similar to word embeddings  This approach seems reasonable in the case of POS tags that are attached to the corresponding words, but phrase-level constituent tags (e g  NP, VP, ADJP) vary greatly in size and shape, making them less amenable to uniform treatment  For instance, even the same phrase tags within different syntactic contexts can vary greatly in size and internal structure, as the case of NP tags in Figure 1 shows  Here, the NP consisting of DT[the]-NN[stories] has a different internal structure than the NP consisting of NP[the film 's]-NNS[shortcomings] 
One way of deriving structure-aware tag representations from the original tag embeddings is to introduce a separate tag-level tree-LSTM which accepts the typical tag embeddings at each node of a tree and outputs the computed structure-aware tag representations for the nodes  Note that the module concentrates on extracting useful syntactic features by considering only the tags and structures of the trees, excluding word information 
Formally, we denote a tag embedding for the tag attached to each node in a tree as $\textbf {e} \in \mathbb {R}^{d_\text{T}}$   Then, the function of each cell in the tag tree-LSTM is defined in the following way  Leaf nodes are defined by the following:
$$  \begin{bmatrix} \hat{\mathbf {c}} \\ \hat{\mathbf {h}} \\ \end{bmatrix} = \tanh {\left(\mathbf {U}_\text{T} \mathbf {e} + \mathbf {a}_\text{T}\right)}$$   (Eq  13)
while non-leaf nodes are defined by the following:
$$  \begin{bmatrix} \hat{\mathbf {i}} \\ \hat{\mathbf {f}}_l \\ \hat{\mathbf {f}}_r \\ \hat{\mathbf {o}} \\ \hat{\mathbf {g}} \end{bmatrix} = \begin{bmatrix} \sigma \\ \sigma \\ \sigma \\ \sigma \\ \tanh \end{bmatrix} \Bigg ( \mathbf {W_\text{T}} \begin{bmatrix} \hat{\mathbf {h}}_l \\ \hat{\mathbf {h}}_r \\ \mathbf {e} \\ \end{bmatrix} + \mathbf {b}_\text{T} \Bigg )$$   (Eq  14)
$$  \hat{\mathbf {c}} = \hat{\mathbf {f}}_l \odot \hat{\mathbf {c}}_l + \hat{\mathbf {f}}_r \odot \hat{\mathbf {c}}_r + \hat{\mathbf {i}} \odot \hat{\mathbf {g}}\\$$   (Eq  15)
where $\hat{\mathbf {h}}, \hat{\mathbf {c}} \in \mathbb {R}^{d_\text{T}}$ represent the hidden state and cell state of each node in the tag tree-LSTM  We regard the hidden state ( $\hat{\mathbf {h}}$ ) as a structure-aware tag representation for the node  $ \mathbf {U}_\text{T} \in \mathbb {R}^{2d_\text{T} \times d_\text{T}}, \textbf {a}_\text{T} \in \mathbb {R}^{2d_\text{T}}, \mathbf {W}_\text{T} \in \mathbb {R}^{5d_\text{T} \times 3d_\text{T}}$ , and $\mathbf {b}_\text{T} \in \mathbb {R}^{5d_\text{T}}$ are trainable parameters  The rest of the notation follows equations 8 , 9 , and 10   In case of leaf nodes, the states are computed by a simple non-linear transformation  Meanwhile, the composition function in a non-leaf node absorbs the tag embedding ( $\mathbf {e}$ ) as an additional input as well as the hidden states of the two children nodes  The benefit of revising tag representations according to the internal structure is that the derived embedding is a function of the corresponding makeup of the node, rather than a monolithic, categorical tag 
With regard to the tags themselves, we conjecture that the taxonomy of the tags currently in use in many NLP systems is too complex to be utilized effectively in deep neural models, considering the specificity of many tag sets and the limited amount of data with which to train  Thus, we cluster POS (word-level) tags into 12 groups following the universal POS tagset BIBREF20 and phrase-level tags into 11 groups according to criteria analogous to the case of words, resulting in 23 tag categories in total  In this work, we use the revised coarse-grained tags instead of the original ones  For more details, we refer readers to the supplemental materials 
Leaf-LSTM
An inherent shortcoming of RvNNs relative to sequential models is that each intermediate representation in a tree is unaware of its external context until all the information is gathered together at the root node  In other words, each composition process is prone to be locally optimized rather than globally optimized 
To mitigate this problem, we propose using a leaf-LSTM following the convention of some previous works BIBREF21 , BIBREF7 , BIBREF11 , which is a typical LSTM that accepts a sequence of words in order  Instead of leveraging word embeddings directly, we can use each hidden state and cell state of the leaf-LSTM as input tokens for leaf nodes in a tree-LSTM, anticipating the proper contextualization of the input sequence 
Formally, we denote a sequence of words in an input sentence as $w_{1:n}$ ( $n$ : the length of the sentence), and the corresponding word embeddings as $\mathbf {x}_{1:n}$   Then, the operation of the leaf-LSTM at time $t$ can be formulated as,
$$  \begin{bmatrix} \tilde{\mathbf {i}} \\ \tilde{\mathbf {f}} \\ \tilde{\mathbf {o}} \\ \tilde{\mathbf {g}} \end{bmatrix} = \begin{bmatrix} \sigma \\ \sigma \\ \sigma \\ \tanh \end{bmatrix} \Bigg ( \mathbf {W}_\text{L} \begin{bmatrix} \tilde{\mathbf {h}}_{t-1} \\ \mathbf {x}_t \\ \end{bmatrix} + \mathbf {b}_\text{L} \Bigg )$$   (Eq  18)
$$  \tilde{\mathbf {c}}_t = \tilde{\mathbf {f}} \odot \tilde{\mathbf {c}}_{t-1} + \tilde{\mathbf {i}} \odot \tilde{\mathbf {g}}\\$$   (Eq  19)
where $\mathbf {x}_t \in \mathbb {R}^{d_w}$ indicates an input word vector and $\tilde{\mathbf {h}}_t$ , $\tilde{\mathbf {c}}_t \in \mathbb {R}^{d_h}$ represent the hidden and cell state of the LSTM at time $t$ ( $\tilde{\mathbf {h}}_{t-1}$ corresponds to the hidden state at time $t$ -1)  $\mathbf {W}_\text{L}$ and $\mathbf {b}_\text{L} $ are learnable parameters  The remaining notation follows that of the tree-LSTM above 
In experiments, we demonstrate that introducing a leaf-LSTM fares better at processing the input words of a tree-LSTM compared to using a feed-forward neural network  We also explore the possibility of its bidirectional setting in ablation study 
SATA Tree-LSTM
In this section, we define SATA Tree-LSTM (Structure-Aware Tag Augmented Tree-LSTM, see Figure 2 ) which joins a tag-level tree-LSTM (section 3 2), a leaf-LSTM (section 3 3), and the original word tree-LSTM together 
As above we denote a sequence of words in an input sentence as $w_{1:n}$ and the corresponding word embeddings as $\mathbf {x}_{1:n}$ 

ground_truth: Nguni languages (zul, xho, nbl, ssw), Sotho languages (nso, sot, tsn)

The Nguni languages are similar to each other, The same is true of the Sotho languages

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which baselines did they compare against?
Context 3:  In addition, a tag embedding for the tag attached to each node in a tree is denoted by $\textbf {e} \in \mathbb {R}^{d_\text{T}}$   Then, we derive the final sentence representation for the input sentence with our model in two steps 
First, we compute structure-aware tag representations ( $\hat{\mathbf {h}}$ ) for each node of a tree using the tag tree-LSTM (the right side of Figure 2 ) as follows:
$$  \begin{bmatrix} \hat{\mathbf {c}} \\ \hat{\mathbf {h}} \\ \end{bmatrix} = {\left\lbrace \begin{array}{ll} \text{Tag-Tree-LSTM}(\mathbf {e}) & \text{if a leaf node} \\ \text{Tag-Tree-LSTM}(\hat{\mathbf {h}}_l, \hat{\mathbf {h}}_r, \mathbf {e}) & \text{otherwise} \end{array}\right }$$   (Eq  23)
where Tag-Tree-LSTM indicates the module we described in section 3 2 
Second, we combine semantic units recursively on the word tree-LSTM in a bottom-up fashion  For leaf nodes, we leverage the Leaf-LSTM (the bottom-left of Figure 2 , explained in section 3 3) to compute $\tilde{\mathbf {c}}_{t}$ and $\tilde{\mathbf {h}}_{t}$ in sequential order, with the corresponding input $\mathbf {x}_t$  
$$  \begin{bmatrix} \tilde{\mathbf {c}}_{t} \\ \tilde{\mathbf {h}}_{t} \\ \end{bmatrix} = \text{Leaf-LSTM}(\tilde{\textbf {h}}_{t-1}, \textbf {x}_t)$$   (Eq  24)
Then, the $\tilde{\mathbf {c}}_{t}$ and $\tilde{\mathbf {h}}_{t}$ can be utilized as input tokens to the word tree-LSTM, with the left (right) child of the target node corresponding to the $t$ th word in the input sentence 
$$  \begin{bmatrix} \check{\textbf {c}}_{\lbrace l, r\rbrace } \\ \check{\textbf {h}}_{\lbrace l, r\rbrace } \end{bmatrix} = \begin{bmatrix} \tilde{\textbf {c}}_{t} \\ \tilde{\textbf {h}}_{t} \end{bmatrix}$$   (Eq  25)
In the non-leaf node case, we calculate phrase representations for each node in the word tree-LSTM (the upper-left of Figure 2 ) recursively as follows:
$$  \check{\mathbf {g}} = \tanh {\left( \mathbf {U}_\text{w} \begin{bmatrix} \check{\mathbf {h}}_l \\ \check{\mathbf {h}}_r \\ \end{bmatrix} + \mathbf {a}_\text{w} \right)}$$   (Eq  26)
$$  \begin{bmatrix} \check{\mathbf {i}} \\ \check{\mathbf {f}}_l \\ \check{\mathbf {f}}_r \\ \check{\mathbf {o}} \end{bmatrix} = \begin{bmatrix} \sigma \\ \sigma \\ \sigma \\ \sigma \end{bmatrix} \Bigg ( \mathbf {W_\text{w}} \begin{bmatrix} \check{\mathbf {h}}_l \\ \check{\mathbf {h}}_r \\ \hat{\mathbf {h}} \\ \end{bmatrix} + \mathbf {b}_\text{w} \Bigg )$$   (Eq  27)
where $\check{\mathbf {h}}$ , $\check{\mathbf {c}} \in \mathbb {R}^{d_h}$ represent the hidden and cell state of each node in the word tree-LSTM  $\mathbf {U}_\text{w} \in \mathbb {R}^{d_h \times 2d_h}$ , $\mathbf {W}_\text{w} \in \mathbb {R}^{4d_h \times \left(2d_h+d_\text{T}\right)}$ , $\mathbf {a}_\text{w} \in \mathbb {R}^{d_h}$ , $\mathbf {b}_\text{w} \in \mathbb {R}^{4d_h}$ are learned parameters  The remaining notation follows those of the previous sections  Note that the structure-aware tag representations ( $\hat{\mathbf {h}}$ ) are only utilized to control the gate functions of the word tree-LSTM in the form of additional inputs, and are not involved in the semantic composition ( $\check{\mathbf {g}}$ ) directly 
Finally, the hidden state of the root node ( $\check{\mathbf {h}}_\text{root}$ ) in the word-level tree-LSTM becomes the final sentence representation of the input sentence 
Quantitative Analysis
One of the most basic approaches to evaluate a sentence encoder is to measure the classification performance with the sentence representations made by the encoder  Thus, we conduct experiments on the following five datasets  (Summary statistics for the datasets are reported in the supplemental materials )
MR: A group of movie reviews with binary (positive / negative) classes  BIBREF22
SST-2: Stanford Sentiment Treebank BIBREF6   Similar to MR, but each review is provided in the form of a binary parse tree whose nodes are annotated with numeric sentiment values  For SST-2, we only consider binary (positive / negative) classes 
SST-5: Identical to SST-2, but the reviews are grouped into fine-grained (very negative, negative, neutral, positive, very positive) classes 
SUBJ: Sentences grouped as being either subjective or objective (binary classes)  BIBREF23
TREC: A dataset which groups questions into six different question types (classes)  BIBREF24
As a preprocessing step, we construct parse trees for the sentences in the datasets using the Stanford PCFG parser BIBREF25   Because syntactic tags are by-products of constituency parsing, we do not need further preprocessing 
To classify the sentence given our sentence representation ( $\check{\mathbf {h}}_\text{root}$ ), we use one fully-connected layer with a ReLU activation, followed by a softmax classifier  The final predicted probability distribution of the class $y$ given the sentence $w_{1:n}$ is defined as follows,
$$\mathbf {s} = \text{ReLU}(\mathbf {W}_\text{s} \check{\mathbf {h}}_\text{root}+ \mathbf {b}_\text{s})$$   (Eq  37)
$$p(y|w_{1:n}) = \text{softmax}(\mathbf {W}_\text{c}\mathbf {s} + \mathbf {b}_\text{c})$$   (Eq  38)
where $\textbf {s} \in \mathbb {R}^{d_\text{s}}$ is the computed task-specific sentence representation for the classifier, and $\textbf {W}_\text{s} \in \mathbb {R}^{d_\text{s} \times d_h}$ , $\textbf {W}_\text{c} \in \mathbb {R}^{d_\text{c} \times d_s}$ , $\textbf {b}_\text{s} \in \mathbb {R}^{d_s}$ , $\textbf {b}_\text{c} \in \mathbb {R}^{d_c}$ are trainable parameters  As an objective function, we use the cross entropy of the predicted and true class distributions 
The results of the experiments on the five datasets are shown in table 1   In this table, we report the test accuracy of our model and various other models on each dataset in terms of percentage  To consider the effects of random initialization, we report the best numbers obtained from each several runs with hyper-parameters fixed 
Compared with the previous syntactic tree-based models as well as other neural models, our SATA Tree-LSTM shows superior or competitive performance on all tasks  Specifically, our model achieves new state-of-the-art results within the tree-structured model class on 4 out of 5 sentence classification tasks—SST-2, SST-5, MR, and TREC  The model shows its strength, in particular, when the datasets provide phrase-level supervision to facilitate tree structure learning (i e  SST-2, SST-5)  Moreover, the numbers we report for SST-5 and TREC are competitive to the existing state-of-the-art results including ones from structurally pre-trained models such as ELMo BIBREF26 , proving our model's superiority  Note that the SATA Tree-LSTM also outperforms the recent latent tree-based model, indicating that modeling a neural model with explicit linguistic knowledge can be an attractive option 
On the other hand, a remaining concern is that our SATA Tree-LSTM is not robust to random seeds when the size of a dataset is relatively small, as tag embeddings are randomly initialized rather than relying on pre-trained ones in contrast with the case of words  From this observation, we could find out there needs a direction of research towards pre-trained tag embeddings 
To estimate the performance of our model beyond the tasks requiring only one sentence at a time, we conduct an experiment on the Stanford Natural Language Inference BIBREF34 dataset, each example of which consists of two sentences, the premise and the hypothesis  Our objective given the data is to predict the correct relationship between the two sentences among three options— contradiction, neutral, or entailment 
We use the siamese architecture to encode both the premise ( $p_{1:m}$ ) and hypothesis ( $h_{1:n}$ ) following the standard of sentence-encoding models in the literature  (Specifically, $p_{1:m}$ is encoded as $\check{\mathbf {h}}_\text{root}^p \in \mathbb {R}^{d_h}$ and $h_{1:n}$ is encoded as $\check{\mathbf {h}}_\text{root}^h \in \mathbb {R}^{d_h}$ with the same encoder ) Then, we leverage some heuristics BIBREF35 , followed by one fully-connected layer with a ReLU activation and a softmax classifier  Specifically,
$$\mathbf {z} = \left[ \check{\mathbf {h}}_\text{root}^p; \check{\mathbf {h}}_\text{root}^h; | \check{\mathbf {h}}_\text{root}^p - \check{\mathbf {h}}_\text{root}^h |; \check{\mathbf {h}}_\text{root}^p \odot \check{\mathbf {h}}_\text{root}^h \right]$$   (Eq  41)
$$\mathbf {s} = \text{ReLU}(\mathbf {W}_\text{s} \mathbf {z} + \mathbf {b}_\text{s})$$   (Eq  42)
where $\textbf {z} \in \mathbb {R}^{4d_h}$ , $\textbf {s} \in \mathbb {R}^{d_s}$ are intermediate features for the classifier and $\textbf {W}_\text{s} \in \mathbb {R}^{d_\text{s} \times 4d_h}$ , $\textbf {W}_\text{c} \in \mathbb {R}^{d_\text{c} \times d_s}$ , $\textbf {b}_\text{s} \in \mathbb {R}^{d_s}$ , $\textbf {b}_\text{c} \in \mathbb {R}^{d_c}$ are again trainable parameters 
Our experimental results on the SNLI dataset are shown in table 2   In this table, we report the test accuracy and number of trainable parameters for each model  Our SATA-LSTM again demonstrates its decent performance compared against the neural models built on both syntactic trees and latent trees, as well as the non-tree models  (Latent Syntax Tree-LSTM: BIBREF10 ( BIBREF10 ), Tree-based CNN: BIBREF35 ( BIBREF35 ), Gumbel Tree-LSTM: BIBREF11 ( BIBREF11 ), NSE: BIBREF36 ( BIBREF36 ), Reinforced Self-Attention Network: BIBREF4 ( BIBREF4 ), Residual stacked encoders: BIBREF37 ( BIBREF37 ), BiLSTM with generalized pooling: BIBREF38 ( BIBREF38 ) ) Note that the number of learned parameters in our model is also comparable to other sophisticated models, showing the efficiency of our model 
Even though our model has proven its mettle, the effect of tag information seems relatively weak in the case of SNLI, which contains a large amount of data compared to the others  One possible explanation is that neural models may learn some syntactic rules from large amounts of text when the text size is large enough, reducing the necessity of external linguistic knowledge  We leave the exploration of the effectiveness of tags relative to data size for future work 
Here we go over the settings common across our models during experimentation  For more task-specific details, refer to the supplemental materials 
For our input embeddings, we used 300 dimensional 840B GloVe BIBREF39 as pre-trained word embeddings, and tag representations were randomly sampled from the uniform distribution [-0 005, 0 005]  Tag vectors are revised during training while the fine-tuning of the word embedding depends on the task  Our models were trained using the Adam BIBREF40 or Adadelta BIBREF41 optimizer, depending on task  For regularization, weight decay is added to the loss function except for SNLI following BIBREF42 ( BIBREF42 ) and Dropout BIBREF43 is also applied for the word embeddings and task-specific classifiers  Moreover, batch normalization BIBREF44 is adopted for the classifiers  As a default, all the weights in the model are initialized following BIBREF45 ( BIBREF45 ) and the biases are set to 0  The total norm of the gradients of the parameters is clipped not to be over 5 during training 
Our best models for each dataset were chosen by validation accuracy in cases where a validation set was provided as a part of the dataset  Otherwise, we perform a grid search on probable hyper-parameter settings, or run 10-fold cross-validation in cases where even a test set does not exist 
Ablation Study
In this section, we design an ablation study on the core modules of our model to explore their effectiveness  The dataset used in this experiment is SST-2  To conduct the experiment, we only replace the target module with other candidates while maintaining the other settings  To be specific, we focus on two modules, the leaf-LSTM and structure-aware tag embeddings (tag-level tree-LSTM)  In the first case, the leaf-LSTM is replaced with a fully-connected layer with a $\tanh $ activation or Bi-LSTM  In the second case, we replace the structure-aware tag embeddings with naive tag embeddings or do not employ them at all 
The experimental results are depicted in Figure 3   As the chart shows, our model outperforms all the other options we have considered  In detail, the left part of the chart shows that the leaf-LSTM is the most effective option compared to its competitors  Note that the sequential leaf-LSTM is somewhat superior or competitive than the bidirectional leaf-LSTM when both have a comparable number of parameters  We conjecture this may because a backward LSTM does not add additional useful knowledge when the structure of a sentence is already known  In conclusion, we use the uni-directional LSTM as a leaf module because of its simplicity and remarkable performance 
Meanwhile, the right part of the figure demonstrates that our newly introduced structure-aware embeddings have a real impact on improving the model performance  Interestingly, employing the naive tag embeddings made no difference in terms of the test accuracy, even though the absolute validation accuracy increased (not reported in the figure)  This result supports our assumption that tag information should be considered in the structure 
Qualitative Analysis
In previous sections, we have numerically demonstrated that our model is effective in encouraging useful composition of semantic units  Here, we directly investigate the computed representations for each node of a tree, showing that the remarkable performance of our model is mainly due to the gradual and recursive composition of the intermediate representations on the syntactic structure 
To observe the phrase-level embeddings at a glance, we draw a scatter plot in which a point represents the corresponding intermediate representation  We utilize PCA (Principal Component Analysis) to project the representations into a two-dimensional vector space  As a target parse tree, we reuse the one seen in Figure 1   The result is shown in Figure 4  
From this figure, we confirm that the intermediate representations have a hierarchy in the semantic space, which is very similar to that of the parse tree  In other words, as many tree-structured models pursue, we can see the tendency of constructing the representations from the low-level (the bottom of the figure) to the high-level (the top-left and top-right of the figure), integrating the meaning of the constituents recursively  An interesting thing to note is that the final sentence representation is near that of the phrase `, the stories are quietly moving ' rather than that of `Despite the film's shortcomings', catching the main meaning of the sentence 
Conclusion
We have proposed a novel RvNN architecture to fully utilize linguistic priors  A newly introduced tag-level tree-LSTM demonstrates that it can effectively control the composition function of the corresponding word-level tree-LSTM  In addition, the proper contextualization of the input word vectors results in significant performance improvements on several sentence-level tasks  For future work, we plan to explore a new way of exploiting dependency trees effectively, similar to the case of constituency trees 
Acknowledgments
We thank anonymous reviewers for their constructive and fruitful comments  This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF2016M3C4A7952587) 

ground_truth: Various tree structured neural networks including variants of Tree-LSTM, Tree-based CNN, RNTN, and non-tree models including variants of LSTMs, CNNs, residual, and self-attention based networks

Sentence classification baselines: RNTN (Socher et al. 2013), AdaMC-RNTN (Dong et al. 2014), TE-RNTN (Qian et al. 2015), TBCNN (Mou et al. 2015), Tree-LSTM (Tai, Socher, and Manning 2015), AdaHT-LSTM-CM (Liu, Qiu, and Huang 2017), DC-TreeLSTM (Liu, Qiu, and Huang 2017), TE-LSTM (Huang, Qian, and Zhu 2017), BiConTree (Teng and Zhang 2017), Gumbel Tree-LSTM (Choi, Yoo, and Lee 2018), TreeNet (Cheng et al. 2018), CNN (Kim 2014), AdaSent (Zhao, Lu, and Poupart 2015), LSTM-CNN (Zhou et al. 2016), byte-mLSTM (Radford, Jozefowicz, and Sutskever 2017), BCN + Char + CoVe (McCann et al. 2017), BCN + Char + ELMo (Peters et al. 2018). 
Stanford Natural Language Inference baselines: Latent Syntax Tree-LSTM (Yogatama et al. 2017), Tree-based CNN (Mou et al. 2016), Gumbel Tree-LSTM (Choi, Yoo, and Lee 2018), NSE (Munkhdalai and Yu 2017), Reinforced Self- Attention Network (Shen et al. 2018), Residual stacked encoders: (Nie and Bansal 2017), BiLSTM with generalized pooling (Chen, Ling, and Zhu 2018).