Explanatory Learning: Beyond Empiricism in Neural Networks

We introduce Explanatory Learning ( EL ) , an explanation-driven machine learning framework to use existing knowledge buried in symbolic sequences expressed in an unknown language . In EL , the burden of interpreting explanations is not left to humans or human-coded compilers , as done in Program Synthesis . Rather , EL calls for a learned interpreter , built upon existing explanations paired with observations of several phenomena . This interpreter can then be used to make predictions on novel phenomena , and even find an explanation for them . We formulate the EL problem as a simple binary classification task , so that common end-to-end approaches aligned with the dominant empiricist view of machine learning could , in principle , solve it . To these models , we oppose Critical Rationalist Networks ( CRNs ) , which instead embrace a rationalist view on the acquisition of knowledge . CRNs express several desired properties by construction , they are truly explainable , can adjust their processing at test-time for harder inferences , and can offer strong confidence guarantees on their predictions . As a final contribution , we introduce Odeen , a basic EL environment that simulates a small flatland-style universe full of phenomena to explain . Using Odeen as a testbed , we show how CRNs outperform empiricist end-to-end approaches of similar size and architecture ( Transformers ) for discovering new explanations of unexplained phenomena . 1 INTRODUCTION . Making accurate predictions about the future is a key ability to survive and thrive in a habitat . Living beings have evolved many systems to this end , such as memory ( McConnell , 1962 ) , and several can predict the course of complex phenomena ( Taylor et al. , 2012 ) . However , no animal comes even close to the prediction ability of humans , which stems from a unique-in-nature system . At the core of this system lies an object called explanation , formed by the proposition of a language , which has a remarkable property : it can be installed with ease into another human speaking the same language , allowing to make predictions on new phenomena without ever having experienced them . When the installation is successful , we say that the human has understood the explanation . This process is key to the success of human beings . An individual is not required to go through a painful discovery process for every phenomenon of interest , but only needs an operating system – mastering a language – and someone who communicates explanations ; then , the individual can focus on unexplained phenomena . When an explanation is found for them , it is added to the existing shared collection , which we call knowledge . How can we make machines take part in this orchestra ? With this work , we try to shed new light on this problem . Specifically , we propose a learning paradigm to allow machines ( i ) to understand existing explanations , in the sense described above , and ( ii ) create new explanations for unexplained phenomena , much like human scientists do . Our contribution in this sense is threefold : i ) We formulate the challenge of creating a machine that masters a language as a learning problem by treating explanations as simple strings , while keeping their role well separated from the role of the data . This results in the Explanatory Learning ( EL ) framework described in Sec . 2. ii ) We present Odeen , a basic environment to test EL approaches , which draws inspiration from the board game Zendo ( Heath , 2001 ) . Odeen simulates the work of a scientist in a small universe of simple geometric figures , see Figure 1 . We present it in Sec . 3 , and will release it with this work . iii ) We argue that the dominating empiricist ML approaches are not suitable for EL problems . We propose Critical Rationalist Networks ( CRNs ) , a family of models designed according to the epistemological philosophy pushed forward by Popper ( 1935 ) . Although a CRN is implemented using two neural networks , its working hypothesis does not coincide with the model parameters . Instead , the working hypothesis of CRNs is a language proposition that can only be accepted or refused in toto . We will present CRNs in Sec . 4 , and test their performance on Odeen in Sec . 5 . 2 EXPLANATORY LEARNING . Humans do not master a language from birth . A baby can not use the message “ this soap stings ” to predict the burning sensation caused by contact with the substance . Instead , the baby gradually learns to interpret such messages and make predictions for an entire universe of phenomena ( Schulz et al. , 2007 ) . We refer to this state of affairs as mastering a language , and we aim to replicate it in a machine as the result of an analogous learning process . In particular , we propose to do so without explicitly prescribing a grammar , using instead an incomplete set of example explanations in the form of plain sequences of symbols . Our focus is on learning an interpreter purely from observations . This allows not only to make predictions about phenomena for which we already have an explanation , but also to discover an explanation for unexplained phenomena . We first describe the problem setup in the sequel , comparing it to existing ML problems ; we detail our approach in Sec . 4 . Problem setup . Formally , let phenomena P1 , P2 , P3 , . . . be subsets of a universe U , which is a large set with no special structure ( i.e. , the entire data space U = { x1 , . . . , xz } ) . Over a universe U , one can define a language L as a pair ( ΣL , IL ) , where ΣL is a finite collection of short strings over some alphabet A , with |ΣL| ≫ |A| , and IL is a binary function IL : U ×ΣL → { 0 , 1 } , which we call interpreter . We say that a phenomenon Pi is explainable in a language L if there exists a string e ∈ ΣL such that , for any x ∈ U , it occurs IL ( x , e ) = 1Pi ( x ) , where 1Pi ( x ) is the indicator function of Pi . We call the string e an explanation , in the language L , for the phenomenon Pi . Our first contribution is the introduction of a new class of machine learning problems , which we refer to as Explanatory Learning ( EL ) . Consider the general problem of making a new prediction for a phenomenon P0 ⊂ U . In our setting , this is phrased as a binary classification task : given a sample x′ ∈ U , establish whether x′ ∈ P0 or not . We are interested in two instances of this problem , with different underlying assumptions : • The communication problem : we have an explanation . We are given an explanation e0 for P0 , in an unknown language L. This means that we do not have access 1The explanation Four wandering stars having their period around a principal star is adapted from the English translation of the Sidereus Nuncius ( Galilei , 2016 , page 9 ) . First sketch on the left is compatible with the rule since the fourth moon can be hidden by one of the other moons or by Jupyter itself . to an interpreter IL ; it looks like Japanese to a non-Japanese speaker . Instead , we are also given other explanations { e1 , . . . , en } , in the same language , for other phenomena P1 , . . . , Pn , as well as observations of them , i.e. , datasets { D1 , . . . , Dn } in the form Di = { ( x1,1Pi ( x1 ) ) , . . . , ( xm,1Pi ( xm ) ) } , with m ≪ |U | . Intuitively , here we expect the learner to use the explanations paired with the observations to build an approximated interpreter ÎL , and then use it to make the proper prediction for x′ by evaluating ÎL ( x′ , e0 ) . • The scientist problem : we do not have an explanation . We are given explanations { e1 , . . . , en } in an unknown language L for other phenomena P1 , . . . , Pn and observations of them { D1 , . . . , Dn } . However , we do not have an explanation for P0 ; instead , we are given just a small set of observations D0 = { ( x1,1P0 ( x1 ) ) , . . . , ( xk,1P0 ( xk ) ) } and two guarantees , namely that P0 is explainable in L , and that D0 is representative for P0 in L. That is , for every phenomenon P ̸= P0 explainable in L there should exist at least a xi ∈ D0 such that 1P0 ( xi ) ̸= 1P ( xi ) . Again , we expect the learner to build the interpreter ÎL , which should first guide the search for the missing explanation e0 based on the clues D0 , and then provide the final prediction through ÎL ( x′ , e0 ) . Several existing works fall within the formalization above . The seminal work of Angluin ( 1987 ) on learning regular sets is an instance of the scientist problem , where finite automata take the role of explanations , while regular sets are the phenomena . More recently , CLEVR ( Johnson et al. , 2017 ) posed a communication problem in a universe of images of simple solids , where explanations are textual and read like “ There is a sphere with the same size as the metal cube ” . Another recent example is CLIP ( Radford et al. , 2021 ) , where 400,000,000 captioned internet images are arranged in a communication problem to train an interpreter , thereby elevating captions to the status of explanations rather than treating them as simple labels2 . With EL , we aim to offer a unified perspective on these works , making explicit the core problem of learning an interpreter purely from observations . Relationship with other ML problems . We briefly discuss the relationship between EL and other problems in ML , pointing to Sec . 6 for additional discussion on the related work . EL can be framed in the general meta-learning framework , where the learner gains experience over multiple tasks to improve its general learning algorithm , thus requiring less data and computation on new tasks . Yet , differently from current meta-learning approaches ( Hospedales et al. , 2020 ) , we are not optimizing an explicit meta-objective . Instead , we expect the sought generality to be a consequence of learning a language from examples , rather than a goal to optimize for . EL also shares some aspects with multitask learning and few-shot learning . The core difference here resides in the additional assumption of a common language between tasks and the centrality of the interpreter learning problem . The comparison between CRN and EMP-R in the experiments shows that this additional assumption grants a disruptive increase in generalization performance over a naive multitask learning approach . Also the problem of cross-task generalization in NLP shares many aspects with the communication and scientist problem of EL , such as the common language in which the different tasks are framed , see for instance ( Ye et al. , 2021 ) . Yet , it exhibits also a crucial difference : in NLP this problem is always in a text-to-text format , i.e . in NLP the problem is framed in a single textual domain where explanations and data share the same role . To many , the concept of explanation may sound close to the concept of program ; similarly , the scientist problem may seem a rephrasing of the fundamental problem of Inductive Logic Programming ( ILP ) ( Shapiro , 1981 ) or Program Synthesis ( PS ) ( Balog et al. , 2017 ) . This is not the case . ILP has the analogous goal of producing a hypothesis from positive/negative examples accompanied by background knowledge . Yet , ILP requires data expressed as logic formulas by a human expert ; the ILP solver outputs a logic proposition , which the expert in turn interprets3 . With EL , data can be fed as-is without being translated into logic propositions , and an interpreter plays the expert ’ s role . PS also admits raw data as input , it yields a program as output , and replaces the expert with a handcrafted interpreter ; this way , the sequence of symbols produced by a PS system only makes sense to a human , not to the system itself . Instead , in EL , the interpreter is learned from scratch rather than hardcoded . An empirical comparison demonstrating the benefits of EL over PS is given in Sec . 5 . Next we introduce Odeen , an environment and benchmark to experiment with the EL paradigm . 2This shift greatly improved the performance of their model , as discussed in ( Radford et al. , 2021 , Sec . 2.3 ) . 3In this sense , ILP suffers the same problem of Explanation-Based Learning , see Minton ( 1990 )

Summary: The paper considers the problem of classification learning where each data point is accompanied by some explanation. The explanation is in some arbitrary language. The paper proposes a way to increase the performance of the classification task by learning to predict the explanation associated with a given data point, and then using that explanation along with the data point to proceed to make a prediction about the data point's label. The paper implements this pipeline using neural networks, and presents empirical results on a new benchmark dataset to demonstrate its performance.