Explanatory Learning: Beyond Empiricism in Neural Networks

We introduce Explanatory Learning ( EL ) , an explanation-driven machine learning framework to use existing knowledge buried in symbolic sequences expressed in an unknown language . In EL , the burden of interpreting explanations is not left to humans or human-coded compilers , as done in Program Synthesis . Rather , EL calls for a learned interpreter , built upon existing explanations paired with observations of several phenomena . This interpreter can then be used to make predictions on novel phenomena , and even find an explanation for them . We formulate the EL problem as a simple binary classification task , so that common end-to-end approaches aligned with the dominant empiricist view of machine learning could , in principle , solve it . To these models , we oppose Critical Rationalist Networks ( CRNs ) , which instead embrace a rationalist view on the acquisition of knowledge . CRNs express several desired properties by construction , they are truly explainable , can adjust their processing at test-time for harder inferences , and can offer strong confidence guarantees on their predictions . As a final contribution , we introduce Odeen , a basic EL environment that simulates a small flatland-style universe full of phenomena to explain . Using Odeen as a testbed , we show how CRNs outperform empiricist end-to-end approaches of similar size and architecture ( Transformers ) for discovering new explanations of unexplained phenomena . 1 INTRODUCTION . Making accurate predictions about the future is a key ability to survive and thrive in a habitat . Living beings have evolved many systems to this end , such as memory ( McConnell , 1962 ) , and several can predict the course of complex phenomena ( Taylor et al. , 2012 ) . However , no animal comes even close to the prediction ability of humans , which stems from a unique-in-nature system . At the core of this system lies an object called explanation , formed by the proposition of a language , which has a remarkable property : it can be installed with ease into another human speaking the same language , allowing to make predictions on new phenomena without ever having experienced them . When the installation is successful , we say that the human has understood the explanation . This process is key to the success of human beings . An individual is not required to go through a painful discovery process for every phenomenon of interest , but only needs an operating system – mastering a language – and someone who communicates explanations ; then , the individual can focus on unexplained phenomena . When an explanation is found for them , it is added to the existing shared collection , which we call knowledge . How can we make machines take part in this orchestra ? With this work , we try to shed new light on this problem . Specifically , we propose a learning paradigm to allow machines ( i ) to understand existing explanations , in the sense described above , and ( ii ) create new explanations for unexplained phenomena , much like human scientists do . Our contribution in this sense is threefold : i ) We formulate the challenge of creating a machine that masters a language as a learning problem by treating explanations as simple strings , while keeping their role well separated from the role of the data . This results in the Explanatory Learning ( EL ) framework described in Sec . 2. ii ) We present Odeen , a basic environment to test EL approaches , which draws inspiration from the board game Zendo ( Heath , 2001 ) . Odeen simulates the work of a scientist in a small universe of simple geometric figures , see Figure 1 . We present it in Sec . 3 , and will release it with this work . iii ) We argue that the dominating empiricist ML approaches are not suitable for EL problems . We propose Critical Rationalist Networks ( CRNs ) , a family of models designed according to the epistemological philosophy pushed forward by Popper ( 1935 ) . Although a CRN is implemented using two neural networks , its working hypothesis does not coincide with the model parameters . Instead , the working hypothesis of CRNs is a language proposition that can only be accepted or refused in toto . We will present CRNs in Sec . 4 , and test their performance on Odeen in Sec . 5 . 2 EXPLANATORY LEARNING . Humans do not master a language from birth . A baby can not use the message “ this soap stings ” to predict the burning sensation caused by contact with the substance . Instead , the baby gradually learns to interpret such messages and make predictions for an entire universe of phenomena ( Schulz et al. , 2007 ) . We refer to this state of affairs as mastering a language , and we aim to replicate it in a machine as the result of an analogous learning process . In particular , we propose to do so without explicitly prescribing a grammar , using instead an incomplete set of example explanations in the form of plain sequences of symbols . Our focus is on learning an interpreter purely from observations . This allows not only to make predictions about phenomena for which we already have an explanation , but also to discover an explanation for unexplained phenomena . We first describe the problem setup in the sequel , comparing it to existing ML problems ; we detail our approach in Sec . 4 . Problem setup . Formally , let phenomena P1 , P2 , P3 , . . . be subsets of a universe U , which is a large set with no special structure ( i.e. , the entire data space U = { x1 , . . . , xz } ) . Over a universe U , one can define a language L as a pair ( ΣL , IL ) , where ΣL is a finite collection of short strings over some alphabet A , with |ΣL| ≫ |A| , and IL is a binary function IL : U ×ΣL → { 0 , 1 } , which we call interpreter . We say that a phenomenon Pi is explainable in a language L if there exists a string e ∈ ΣL such that , for any x ∈ U , it occurs IL ( x , e ) = 1Pi ( x ) , where 1Pi ( x ) is the indicator function of Pi . We call the string e an explanation , in the language L , for the phenomenon Pi . Our first contribution is the introduction of a new class of machine learning problems , which we refer to as Explanatory Learning ( EL ) . Consider the general problem of making a new prediction for a phenomenon P0 ⊂ U . In our setting , this is phrased as a binary classification task : given a sample x′ ∈ U , establish whether x′ ∈ P0 or not . We are interested in two instances of this problem , with different underlying assumptions : • The communication problem : we have an explanation . We are given an explanation e0 for P0 , in an unknown language L. This means that we do not have access 1The explanation Four wandering stars having their period around a principal star is adapted from the English translation of the Sidereus Nuncius ( Galilei , 2016 , page 9 ) . First sketch on the left is compatible with the rule since the fourth moon can be hidden by one of the other moons or by Jupyter itself . to an interpreter IL ; it looks like Japanese to a non-Japanese speaker . Instead , we are also given other explanations { e1 , . . . , en } , in the same language , for other phenomena P1 , . . . , Pn , as well as observations of them , i.e. , datasets { D1 , . . . , Dn } in the form Di = { ( x1,1Pi ( x1 ) ) , . . . , ( xm,1Pi ( xm ) ) } , with m ≪ |U | . Intuitively , here we expect the learner to use the explanations paired with the observations to build an approximated interpreter ÎL , and then use it to make the proper prediction for x′ by evaluating ÎL ( x′ , e0 ) . • The scientist problem : we do not have an explanation . We are given explanations { e1 , . . . , en } in an unknown language L for other phenomena P1 , . . . , Pn and observations of them { D1 , . . . , Dn } . However , we do not have an explanation for P0 ; instead , we are given just a small set of observations D0 = { ( x1,1P0 ( x1 ) ) , . . . , ( xk,1P0 ( xk ) ) } and two guarantees , namely that P0 is explainable in L , and that D0 is representative for P0 in L. That is , for every phenomenon P ̸= P0 explainable in L there should exist at least a xi ∈ D0 such that 1P0 ( xi ) ̸= 1P ( xi ) . Again , we expect the learner to build the interpreter ÎL , which should first guide the search for the missing explanation e0 based on the clues D0 , and then provide the final prediction through ÎL ( x′ , e0 ) . Several existing works fall within the formalization above . The seminal work of Angluin ( 1987 ) on learning regular sets is an instance of the scientist problem , where finite automata take the role of explanations , while regular sets are the phenomena . More recently , CLEVR ( Johnson et al. , 2017 ) posed a communication problem in a universe of images of simple solids , where explanations are textual and read like “ There is a sphere with the same size as the metal cube ” . Another recent example is CLIP ( Radford et al. , 2021 ) , where 400,000,000 captioned internet images are arranged in a communication problem to train an interpreter , thereby elevating captions to the status of explanations rather than treating them as simple labels2 . With EL , we aim to offer a unified perspective on these works , making explicit the core problem of learning an interpreter purely from observations . Relationship with other ML problems . We briefly discuss the relationship between EL and other problems in ML , pointing to Sec . 6 for additional discussion on the related work . EL can be framed in the general meta-learning framework , where the learner gains experience over multiple tasks to improve its general learning algorithm , thus requiring less data and computation on new tasks . Yet , differently from current meta-learning approaches ( Hospedales et al. , 2020 ) , we are not optimizing an explicit meta-objective . Instead , we expect the sought generality to be a consequence of learning a language from examples , rather than a goal to optimize for . EL also shares some aspects with multitask learning and few-shot learning . The core difference here resides in the additional assumption of a common language between tasks and the centrality of the interpreter learning problem . The comparison between CRN and EMP-R in the experiments shows that this additional assumption grants a disruptive increase in generalization performance over a naive multitask learning approach . Also the problem of cross-task generalization in NLP shares many aspects with the communication and scientist problem of EL , such as the common language in which the different tasks are framed , see for instance ( Ye et al. , 2021 ) . Yet , it exhibits also a crucial difference : in NLP this problem is always in a text-to-text format , i.e . in NLP the problem is framed in a single textual domain where explanations and data share the same role . To many , the concept of explanation may sound close to the concept of program ; similarly , the scientist problem may seem a rephrasing of the fundamental problem of Inductive Logic Programming ( ILP ) ( Shapiro , 1981 ) or Program Synthesis ( PS ) ( Balog et al. , 2017 ) . This is not the case . ILP has the analogous goal of producing a hypothesis from positive/negative examples accompanied by background knowledge . Yet , ILP requires data expressed as logic formulas by a human expert ; the ILP solver outputs a logic proposition , which the expert in turn interprets3 . With EL , data can be fed as-is without being translated into logic propositions , and an interpreter plays the expert ’ s role . PS also admits raw data as input , it yields a program as output , and replaces the expert with a handcrafted interpreter ; this way , the sequence of symbols produced by a PS system only makes sense to a human , not to the system itself . Instead , in EL , the interpreter is learned from scratch rather than hardcoded . An empirical comparison demonstrating the benefits of EL over PS is given in Sec . 5 . Next we introduce Odeen , an environment and benchmark to experiment with the EL paradigm . 2This shift greatly improved the performance of their model , as discussed in ( Radford et al. , 2021 , Sec . 2.3 ) . 3In this sense , ILP suffers the same problem of Explanation-Based Learning , see Minton ( 1990 )

Summary: The paper proposes a new framework for studying explanation driven machine learning problems called Explanatory Learning. The goal is to learn an interpreter model from explanations paired with observations for a particular phenomenon. The explanations might be in an unknown language, but the explanations paired with observations can be used to learn a good interpreter. Once learnt, the interpreter should be able to follow new explanation for an unseen phenomenon. They refer to this problem as the communication problem. They also define the scientist problem where explanations for the unseen phenomenon is not available. The paper also introduces an Odeen dataset to facilitate experiments with Explanatory Learning. The authors propose a neural network architecture as a solution to the scientist problem. It has two components: a conjecture generator which generates a set of candidate English explanations, and a second interpreter model. Experimental results show better generalization compared to end to end neural systems.