SHAPELURN : An Interactive Language Learning Game with Logical Inference (2021)

Abstract: We investigate if a model can learn natural language with minimal linguistic input through interaction. Addressing this question, we design and implement an interactive language learning game that learns logical semantic representations compositionally. Our game allows us to explore the benefits of logical inference for natural language learning. Evaluation shows that the model can accurately narrow down potential logical representations for words over the course of the game, suggesting that our model is able to learn lexical mappings from scratch successfully.

## introduction an open question in nlp research is how models can learn natural language most successfully and effectively. many state-of-the-art semantic parsers and machine learning algorithms are dependent on large data sets for successful training. this poses a problem when using nlp applications for lowresource languages or specific domains for which only little annotated training data is available if any at all @xcite . interactive nlp systems can overcome these problems as they start with a small or even empty set of training data that gets extended based on user feedback for the predictions the model makes based on its current parameters @xcite . therefore, learning mappings from natural language to formal representations through interaction with a user is an attractive approach for low-resource settings. the model parameters are optimized based on feedback and the resulting data itself can be used as training data for other models avoiding costly manual annotations. however, the interaction with a not yet fully trained model can get monotonous or can lead to frustration on the part of the users if they do not benefit from the interaction themselves @xcite . @xcite present an interactive language learning setting, called shrdlurn, in which a model learns a language by interacting with a player in a game environment, hence making the interactive learning setting more attractive and fun for users. their model is language independent and can be taught any language from scratch. here we follow @xcite and design and implement an interactive language learning game where a model learns to map natural language to logical forms in a compositional way based on the feedback provided by the player 1 . whereas wang et al. (2016)'s model learns to map instructions to executable logical forms, we aim to learn logical formulas that evaluate to truth values with respect to the current state of the game environment. this decision was taken because the additional information about the truth can be incorporated in the parsing and learning process in order to already restrict the potential logical formulas. overall, we are trying to answer the following research questions: can we implement a model that 1) can learn a natural language from scratch only from interacting with a user and 2) is not dependent on any language specific syntax and is hence language independent. ## previous work several approaches map natural language to logical form for its ability to model inferences. @xcite present a learning algorithm for mapping sentences to their lambda calculus semantic representations and automatically inducing a combinatory categorical grammar (ccg). @xcite extend this algorithm to make the grammar more flexible. @xcite also present a flexible semantic parsing framework, the floating parser, for learning mappings from natural language to logical forms in the lambda dependency-based compositional semantic language @xcite . @xcite present a framework for learning to map natural language utterances to logical forms that combines the principle of compositionality with a standard machine learning algorithm. current approaches that aim at overcoming the need for costful annotated training data include the interactive human-in-the-loop method where a human corrects annotations predicted by a machine learning system and this feedback is used to improve future predictions. @xcite use this approach for the task of entity linking and @xcite apply the approach to a ccg parser, thereby improving parsing performance. @xcite present a learning approach where a model learns to map natural language instructions to logical representations of solitaire game rules based on feedback. finally, @xcite present a game for grounded language acquisition where a human teaches an agent language from scratch in a natural language conversation. ## discussion although, the setting of our language learning game is simpler than shrdlurn, the 2-d grid environment allows to elicit different kinds of inputs that involve the composition of colors and shapes as well as relative spatial relations between objects that can be nested. in contrast to @xcite , the player task in our game is not to give instructions to reach a specified goal state from the current state but to describe some part of the current state of the game (picture). although we restrict the complexity of the descriptions in the first levels to improve the first learning phase, the player task is in general a very open task as there are many ways to describe objects in a grid and the player can freely choose the objects to describe as well as the properties used to reference them. this makes the setting particularly interactive and allows to investigate in which ways humans choose and formulate their descriptions. the design of the player task allows us to use the truth values of the parses in order to present only the true guesses to the player. hence, the number of potential denotations is limited by the number of possibilities to mark different combinations of objects in the picture. this decreases the number of denotations the player can choose from compared to @xcite where the number of potential successor states for a current state is much higher. although our design inhibits to use the number of clicks during the game as evaluation measurement, we see the overall low number of guesses as an advantage: the player spends less time on clicking through wrong guesses even in cases where the correct denotation is ranked very low what can improve the playing and success experience. we find that the informativeness of feedback plays a crucial role in interactive learning. our results indicate that making the current "knowledge" of the computer as explicit as possible, e.g. by marking all blocks mentioned in the input, could be a promising starting point as simple user feedback can provide enough information for learning. during testing we found that the learning progress depends on specific combinations of descriptions and pictures. learning appears to benefit from situations where the correct formula for the description differs in one lexical rule from other true parses: "a red circle" is more informative with respect to the meaning of "red" for a picture that shows a red circle and a circle in another color than for a picture displaying only one (red) circle. the main advantage of using only input from the player to train the model is its independence of the availability of (annotated) training data as opposed to approaches requiring large data sets such as neural approaches. hence, our approach is applicable for low resource settings. however, our model requires a grammar that covers all semantic concepts that can be part of the interaction. due to our game design, the number of concepts our grammar needs to address is very limited but extending the domain requires increasing the number of handwritten rules. therefore, scaling the model to larger domains would require the costful construction of a large-scale grammar. concerning the scalability of the game environment, the model could be easily adjusted to create more complex pictures as long as an adequate internal representation is found. a key challenge of our work was the high uncertainty of the model. the computer has no initial knowledge about language and must consider each lexicon entry for each word. additionally, the floating parser can combine the corresponding logical formulas in any order, discard tokens from the input and add additional logical forms. the number of parses is thus huge for short sentences and grows exponentially with sentence length, vastly increasing parsing and learning times. future work could use beam search at higher levels to handle this. ## conclusion we implement an interactive language learning game where the computer learns natural language based on user feedback. we find that learning interactively from scratch with a language independent model is complex due to the huge number of potential parses. our results indicate that our model is able to learn language through interaction and low-resource domains and languages could benefit from such an approach. future work will address the trade-off between increasing flexibility and increasing processing times. lexical rule: (category, logical form, weight) example word 1 (b, bf ([λb(b.@xmath0, all), w) "square" 2 (b, bf ([λb(b.@xmath1, all), w) "circle" 3 (b, bf ([λb(b.@xmath2, all), w) "triangle" 4 (b, bf ([ ], all), w) "form" 5 (n, range(1, 17), w) "a" 6 (n, [1], w) "one" 7 (n, [2], w) "two" 8 (n, [3], w) "three" 9