Human-Guided Column Networks: Augmenting Deep Learning with Advice

1 INTRODUCTION . The re-emergence of Deep Learning ( Goodfellow et al. , 2016 ) has found significant and successful applications in difficult real-world domains such as image ( Krizhevsky et al. , 2012 ) , audio ( Lee et al. , 2009 ) and video processing ( Yue-Hei Ng et al. , 2015 ) . Deep Learning has also been increasingly applied to structured domains , where the data is represented using richer symbolic or graph features to capture relational structure between entities and attributes in the domain . Intuitively , deep learning architectures are naturally suited to learning and reasoning over such multi-relational domains as they are able to capture increasingly complex interactions between features with deeper layers . However , the combinatorial complexity of reasoning over a large number of relations and objects has remained a significant bottleneck to overcome . Recent work in relational deep learning has sought to address this particular issue . This includes relational neural networks ( Kazemi & Poole , 2018 ; Šourek et al. , 2015 ) , relational Restricted Boltzmann machines ( Kaur et al. , 2017 ) and neuro-symbolic architectures such as C-ILP ( França et al. , 2014 ) . In our work , we focus upon the framework of Column Networks ( CLNs ) developed by Pham et al . ( 2017 ) . Column networks are composed of several ( feedforward ) mini-columns each of which represents an entity in the domain . Relationships between two entities are modeled through edges between mini-columns . These edges allow for the short-range exchange of information over successive layers of the column network ; however , the true power of column networks emerges as the depth of interactions increases , which allows for the natural modeling of long-range interactions . Column networks are an attractive approach for several reasons : ( 1 ) hidden layers of a CLN share parameters , which means that making the network deeper does not introduce more parameters , ( 2 ) as the depth increases , the CLN can begin to model feature interactions of considerable complexity , which is especially attractive for relational learning , and ( 3 ) learning and inference are linear in the size of the network and the number of relations , which makes CLNs highly efficient . However , like other deep learning approaches , CLNs rely on vast amounts of data and incorporate little to no knowledge about the problem domain . While this may not be an issue for low-level applications such as image or video processing , it is a significant issue in relational domains , since the relational structure encodes rich , semantic information . This suggests that ignoring domain knowledge can considerably hinder generalization . It is well known that biasing learners is necessary in order to allow them to inductively leap from training instances to true generalization over new instances ( Mitchell , 1980 ) . Indeed , the inductive bias towards “ simplicity and generality ” leads to network architectures with simplifying assumptions through regularization strategies that aim to control the complexity of the neural/deep network . While deep learning does incorporate one such bias in the form of domain knowledge ( for example , through parameter tying or convolution , which exploits neighborhood information ) , we are motivated to develop systems that can incorporate richer and more general forms of domain knowledge . This is especially germane for deep relational models as they inherently construct and reason over richer representations . Such domain-knowledge-based inductive biases have been applied to a diverse array of machine learning approaches , variously known as advice-based , knowledge-based or human-guided machine learning . One way in which a human can guide learning is by providing rules over training examples and features . The earliest such approaches combined explanation-based learning ( EBL-NN , Shavlik & Towell ( 1989 ) ) or symbolic domain rules with ANNs ( KBANN , Towell & Shavlik ( 1994 ) ) . Domain knowledge as rules over input features can also be incorporated into support vector machines ( SVMs , Cortes & Vapnik ( 1995 ) ; Schölkopf et al . ( 1998 ) ; Fung et al . ( 2003 ) ; Le et al . ( 2006 ) ; Kunapuli et al . ( 2010b ) ) . Another natural way a human could guide learning is by expressing preferences and has been studied extensively within the preference-elicitation framework due to Boutilier et al . ( 2006 ) . We are inspired by this form of advice as they have been successful within the context of inverse reinforcement learning ( Kunapuli et al. , 2013 ) , imitation learning ( Odom et al. , 2015 ) and planning ( Das et al. , 2018 ) . These approaches span diverse machine learning formalisms , and they all exhibit the same remarkable behavior : better generalization with fewer training examples because they effectively exploit and incorporate domain knowledge as an inductive bias . This is the prevailing motivation for our approach : to develop a framework that allows a human to guide deep learning by incorporating rules and constraints that define the domain and its aspects . Incorporation of prior knowledge into deep learning has begun to receive interest recently , for instance , the recent work on incorporating prior knowledge of color and scene information into deep learning for image classification ( Ding et al. , 2018 ) . However , in many such approaches , the guidance is not through a human , but rather through a pre-processing algorithm to generate guidance . Our framework is much more general in that a human provides guidance during learning . Furthermore , the human providing the domain advice is not an AI/ML expert but rather a domain expert who provides rules naturally . We exploit the rich representation power of relational methods to capture , represent and incorporate such rules into relational deep learning models . We make the following contributions : ( 1 ) we propose the formalism of Knowledge-augmented Column Networks , ( 2 ) we present , inspired by previous work ( such as KBANN ) , an approach to inject generalized domain knowledge in a CLN and develop the learning strategy that exploits this knowledge , and ( 3 ) we demonstrate , across four real problems in some of which CLNs have been previously employed , the effectiveness and efficiency of injecting domain knowledge . Specifically , our results across the domains clearly show statistically superior performance with small amounts of data . As far as we are aware , this is the first work on human-guided CLNs . 2 BACKGROUND AND RELATED WORK . The idea of several processing layers to learn increasingly complex abstractions of the data was initiated by the perceptron model ( Rosenblatt , 1958 ) and was further strengthened by the advent of the back-propagation algorithm ( LeCun et al. , 1998 ) . A deep architecture was proposed by Krizhevsky et al . ( 2012 ) and have since been adapted for different problems across the entire spectrum of domains , such as , Atari games via deep reinforcement learning ( Mnih et al. , 2013 ) , sentiment classification ( Glorot et al. , 2011 ) and image super-resolution ( Dong et al. , 2014 ) . Applying advice to models has been a long explored problem to construct more robust models to noisy data ( Fung et al. , 2003 ; Le et al. , 2006 ; Towell & Shavlik , 1994 ; Kunapuli et al. , 2010a ; Odom & Natarajan , 2018 ) . For a unified view of variations of knowledge-based neural networks , we refer to Fu ( 1995 ) .The knowledge-based neural network framework has been applied successfully to various real world problems such as recognizing genes in DNA sequences ( Noordewier et al. , 1991 ) , microwave design ( Wang & Zhang , 1997 ) , robotic control ( Handelman et al. , 1990 ) and recently in personalised learning systems ( Melesko & Kurilovas , 2018 ) . Combining relational ( symbolic ) and deep learning methods has recently gained significant research thrust since relational approaches are indispensable in faithful and explainable modeling of implicit domain structure , which is a major limitation in most deep architectures in spite of their success . While extensive literature exists that aim to combine the two ( Sutskever et al. , 2009 ; Rocktäschel et al. , 2014 ; Lodhi , 2013 ; Battaglia et al. , 2016 ) , to the best of our knowledge , there has been little or no work on incorporating the advice in any such framework . Column networks transform relational structures into a deep architecture in a principled manner and are designed especially for collective classification tasks ( Pham et al. , 2017 ) . The architecture and formulation of the column network are suited for adapting it to the advice framework . The GraphSAGE algorithm ( Hamilton et al. , 2017 ) shares similarities with column networks since both architectures operate by aggregating neighborhood information but differs in the way the aggregation is performed . Graph convolutional networks ( Kipf & Welling , 2016 ) is another architecture that is very similar to the way CLN operates , again differing in the aggregation method . Diligenti et al . ( 2017 ) presents a method of incorporating constraints , as a regularization term , which are first order logic statements with fuzzy semantics , in a neural model and can be extended to collective classification problems . While it is similar in spirit to our proposed approach it differs in its representation and problem setup . 3 KNOWLEDGE-AUGMENTED COLUMN NETWORKS . Column Networks ( Pham et al. , 2017 ) allow for encoding interactions/relations between entities as well as the attributes of such entities in a principled manner without explicit relational feature construction or vector embedding . This is important when dealing with structured domains , especially , in the case of collective classification . This enables us to seamlessly transform a multi-relational knowledge graph into a deep architecture making them one of the robust relational deep models . Figure 1 illustrates an example column network , w.r.t . the knowledge graph on the left . Note how each entity forms its own column and relations are captured via the sparse inter-column connectors . Consider a graph G = ( V , A ) , where V = { ei } |V |i=1 is the set of vertices/entities . For brevity , we assume only one entity type . However , there is no such theoretical limitation in the formulation . A is the set of arcs/edges between two entities ei and ej denoted as r ( ei , ej ) . Note that the graph is multi-relational , i.e. , r ∈ R where R is the set of relation types in the domain . To obtain the equivalent Column Network C from G , let xi be the feature vector representing the attributes of an entity ei and yi its label predicted by the model1 . hti denotes a hidden node w.r.t . entity ei at the hidden layer t ( t = 1 , . . . , T is the index of the hidden layers ) . As mentioned earlier , the context between 2 consecutive layers captures the dependency of the immediate neighborhood ( based on arcs/edges/inter-column connectors ) . Thus , for entity ei , the context w.r.t . r and hidden nodes are computed as , ctir = 1 |Nr ( i ) | ∑ j∈Nr ( i ) ht−1j ; h t i = g ( bt + W tht−1i + 1 z ∑ r∈R V tr c t ir ) ( 1 ) whereNr ( i ) are all the neighbors of ei w.r.t . r in the knowledge graph G. Note the absence of context connectors between ht2 and h t 4 ( Figure 1 , right ) since there does not exist any relation between e2 and e4 ( Figure 1 , left ) . The activation of the hidden nodes is computed as the sum of the bias , the weighted output of the previous hidden layer and the weighted contexts where W t ∈ RKt×Kt1 and V tr ∈ RK t×Kt1 are weight parameters and bt is a bias for some activation function g. z is a pre-defined constant that controls the parameterized contexts from growing too large for complex relations . Setting z to the average number of neighbors of an entity is a reasonable assumption . The 1Note that since in our formulation every entity is uniquely indexed by i , we use ei and i interchangeably final output layer is a softmax over the last hidden layer . P ( yi = ` |hTi ) = softmax ( bl + Wlh T i ) ( 2 ) where ` ∈ L is the label ( L is the set of labels ) and T is the index of the last hidden layer . Following Pham et al . ( 2017 ) , we choose to formulate our approach in the context of a relationsensitive predictive modeling , specifically collective classification tasks . However , structured data is implicitly sparse since most entities in the world are not related to each other , thereby adding to the existing challenge of faithful modeling of the underlying structure . The challenge is amplified as we aim to learn in the presence of knowledge-rich , data-scarce problems . As we show empirically , sparse samples ( or targeted noise ) may lead to sub-optimal learning or slower convergence . Example 1 . Consider a problem of classifying whether a published article is about carcinoid metastasis ( Zuetenhorst & Taal , 2005 ) or is irrelevant , from a citation network , and textual features extracted from the articles themselves . There are several challenges : ( 1 ) Data is implicitly sparse due to rarity of studied cases and experimental findings , ( 2 ) Some articles may cite other articles related to carcinoid metastasis and contain a subset of the textual features , but address another topic and ( 3 ) Finally , the presence of targeted noise , where some important citations were not extracted properly by some citation parser and/or the abstracts are not informative enough . The above cases may lead to the model not being able to effectively capture certain dependencies , or converge slower , even if they are captured somewhere in the advanced layers of the deep network . Our approach attempts to alleviate this problem via augmented learning of Column Networks using human advice/knowledge . We formally define our problem in the following manner , Given : A sparse multi-relational graph G , attribute vectors xi of each entity ( sparse or noisy ) in G , equivalent Column-Network C and access to a Human-expert To Do : More effective and efficient collective classification by knowledge augmented training of C. We develop Knowledge-augmented CoLumn Networks ( K-CLN ) , that incorporates humanknowledge , for more effective and efficient learning from relational data ( Figure 2 illustrates the overall architecture ) . While knowledge-based connectionist models are not entirely new , our formulation provides - ( 1 ) a principled approach for incorporating advice specified in an intuitive encoding/language based on logic ( 2 ) a deep model for collective classification in relational data .

Summary: This work proposes a variant of the column network based on the injection of human guidance. The method does not make major changes to the network structure, but by modifying the calculations in the network. Human knowledge is embodied in a defined rule formula. The method is flexible and different entities correspond to different rules. However, the form of knowledge is limited and simple. Experiments have shown that the convergence speed and results are improved, but not significant.