Two Instances of Interpretable Neural Network for Universal Approximations

1 INTRODUCTION . Artificial neural networks ( NN ) have recently seen successful applications in many fields . Modern deep neural network ( DNN ) architecture , usually trained through the backpropagation mechanism , has been called a black-box because of its lack of interpretability . To tackle this issue , various studies have been performed to understand how a NN works ; see the following surveys Arrieta et al . ( 2020 ) ; Gilpin et al . ( 2018 ) ; Tjoa & Guan ( 2020 ) ; Wiegreffe & Marasović ( 2021 ) . This paper primarily proposes two interpretable models , namely triangularly-constructed NN ( TNN ) and Semi-Quantized Activation NN ( SQANN ) . Both possess the following three notable properties : ( 1 ) Resistance to catastrophic forgetting . ( 2 ) Mathematical proofs for arbitrarily high accuracy on training datasets ; experimentally demonstrable with python code and simple common datasets ( see supp . materials ) . ( 3 ) Detection of out-of-distribution samples through weak activations . Concept disambiguation . Several concepts have multiple possible definitions . We clarify the definitions used in this paper . 1 . Interpretability . We consider only fine-grained interpretation , i.e . we look at the meaning of each single neuron or its activation in our models . 2 . Universal approximation . Readers might be familiar with universal approximation of functions on certain conditions , e.g . compact sets . Our models can be more general , e.g . user can freely choose the interpolation function between 2 activations based on knowledge of the local manifold . The function can even be pathological . See appendix A.2.1 . 3 . Catastrophic forgetting : the tendency for knowledge of previously learned dataset to be abruptly lost as information relevant to a new dataset is incorporated . This definition is a slightly nuanced version of Kirkpatrick et al . ( 2017 ) . Hence , our models ’ resistance to catastrophic forgetting is the following . Given a training dataset D accurately modeled by an architecture M , a new dataset D′ ( especially new , out of distribution dataset ) can be incorporated into M without losing accuracy on D. See appendix A.2.2 ( analogy to biological system included ) . Related works and interpretability issues . Recent remarkable studies on universal approximators include the Deep Narrow Network by Kidger & Lyons ( 2020 ) , DeepONet universal approximation for operators by Lu et al . ( 2021 ) and the Broad Learning System by Chen et al . ( 2019 ) ; Hanin ( 2019 ) ; Park et al . ( 2021 ) ; Johnson ( 2019 ) . While insightful , they do not directly address the eXplainable Artificial Intelligence ( XAI ) issue , especially the blackbox property of the DNN . Similarly , a number of classical papers provide theoretical insights for NN as universal approximators , but interpretability , transparency and fairness issues are not their main focus . The universal approximation theorem by Cybenko ( 1989 ) asserts that a NN with a single hidden layer can approximate any function to arbitrarily small error under common conditions , proven by asserting the density of that set of NN in the function space using classic mathematical theorems . In particular , its theorem 1 uses an abstract proof by contradiction . From the proof , it is not easy to observe the internal mechanism of a NN in a straight-forward manner ; consequently modern works that depend on it ( e.g . Deep Narrow Network ) might inherit the blackbox property . Bottom-up constructions for function approximation using NN then emerged , though they also lack the interpretability ( see appendix A.3 for more related works ) . Also consider a demonstration in Nielsen ( 2015 ) that could help improve our understanding of universal approximation . Outline . This paper is arranged as the following . Section 2 shows explicit TNN construction , related results , including a pencil-and-paper example for pedagogical purpose . Likewise , section 3 shows SQANN construction , statements regarding SQANN , another pencil-and-paper example , then experimental results of its application , before we conclude the paper with limitations and future works . Python codes and clearer figures are fully available in supp . materials ( also see appendix ) . 2 TRIANGULARLY-CONSTRUCTED NN ( TNN ) . TNN is the prototype NN for our interpretable universal approximator . SQANN ( next section ) partially borrows the concept from TNN which will be useful as an easy and manageable illustration to deliver the following ideas : ( 1 ) organized activations of neurons and ( 2 ) the retrieval of α values as the outputs . The model is TNN ( x ) = αTσ ( Wx+ b ) where x ∈ [ 0 , 1 ] , α , b ∈ RN and W ∈ RN , where we use sigmoid function σ for simplicity . We start with a simple scalar function y = f ( x ) ∈ R for x ∈ [ 0 , 1 ] , thus TNN ’ s interpretability can be illustrated very clearly . Assumption : Linear Ordering . It is constructed on a linearly ordered dataset containing N samples { ( x ( k ) , y ( k ) ) ∈ Rn × R } Nk=1 such that x ( N ) < x ( N−1 ) < · · · < x ( 1 ) and y ( k ) = f ( x ( k ) ) , f the true function that TNN will approximate . The interpretability comes from the linear ordering property where higher value of x ( ≈ 1 ) will activate more neurons while lower values will activate less neurons as shown in fig 1 ( A ) . Then α values will be retrieved in a continuous way through dot product , eventually used to compute the output for prediction . In time series , such as ECG ( Electrocardiogram ) , signals can be approximated point-wise ( although it is still preferable to have a noise model during preprocessing to prevent overfitting the noise ) . Meaningful interpretation can be given , for example , by mapping PQRST segments from ECG to specific neurons within TNN , giving some neurons specific meaning and thus interpretability . For more remarks and definition of formal linear ordering etc , see appendix A.4 . Ordered activation . We would like x ( 1 ) to activate all neurons , while x ( N ) activates only 1 neuron . In other words , ideally σ ( Wx ( 1 ) + b ) = [ 1 , 1 , . . . , 1 , 1 ] T , followed by σ ( Wx ( 2 ) + b ) = [ 1 , 1 , . . . , 1 , 0 ] T and so on until σ ( Wx ( N ) + b ) = [ 1 , 0 , . . . , 0 ] T ; again , refer to fig . 1 ( A ) . With this concept , we seek to achieve interpretability by successive activations of neurons depending on the “ intensity ” of the input , with x ( 1 ) being the most intense . In general , the above can be written as σ ( k ) ≡ σ ( Wx ( k ) + b ) = [ 1 , . . . , 1︸ ︷︷ ︸ N− ( k−1 ) , 0 , . . . , 0︸ ︷︷ ︸ k−1 ] T ( 1 ) which is approximately achieved for k = 1 , . . . , N at large a ( and exactly if a→∞ ) with ( Wx ( k ) + b ) j = { ≤ −a , j ≥ N − k + 2 ≥ +a , j ≤ N − k + 1 ( 2 ) For more remarks , see appendix A.4 subsection ordered activation . TNN construction : computing weights W , b , α . How then do we compute W , b to achieve the ordered activation ? Consider first ( Wx ( 2 ) +b ) N = −a and ( Wx ( 1 ) +b ) N = a and solve them . This yields WN = 2a/∆ ( 1 ) and bN = a −WNx ( 1 ) where ∆ ( 1 ) = x ( 1 ) − x ( 2 ) . Iterating through k , i.e . solving ( Wx ( k+1 ) + b ) N−k+1 = −a and ( Wx ( k ) + b ) N−k+1 = a we obtain WN−k+1 = 2a/∆ ( k ) and bN−k+1 = a − WN−k+1x ( k ) where ∆ ( k ) = x ( k ) − x ( k+1 ) . We can rewrite the indices so that Wk = 2a/∆ ( N−k+1 ) and bk = a −Wkx ( N−k+1 ) whenever convenient . For ∆ ( N ) , we need a dummy x ( N+1 ) value or we can directly choose its value , e.g . 1NΣ N−1 k=1 ∆ ( k ) . The effect is illustrated by the value near x = 0 in fig . 2 ( A1-3 ) and should not pose any problem ; the chosen dummy value will only affect the shape at the left end of the graph . We compute α using the property of equation ( 1 ) . From fig . 1 ( A ) , this means ideally y ( 1 ) = ΣNi=1αiσ ( Wx ( 1 ) + b ) i for a → ∞ , and similarly y ( 2 ) = ΣN−1i=1 αiσ ( Wx ( 2 ) + b ) i and so on until y ( N ) = α1σ ( Wx ( N ) + b ) 1 . Putting them together as y = [ y ( 1 ) , . . . , y ( N ) ] T , we get y = Aα where A is an upper-left triangular matrix and the inverse A−1 exists . Thus , α = A−1y , a matrix such that A−1ij = 1 along the diagonal , A −1 i , i+1 = −1 and zeroes otherwise , which facilitates a convenient computation . The triangular construction is complete : TNN ( x ) = αTσ ( Wx+ b ) ( 3 ) While Nielsen ( 2015 ) provides only visual demonstration , the following result shows rigorous proof on universal approximation at work ( python code also available ) . Theorem 1 TNN achieves arbitrarily high accuracy on the training dataset . Proof : see appendix A.4.1 . Also see example results in fig . 2 . With the following proposition , test dataset that resembles training dataset will yield small error . Otherwise , there are out-of-distribution ( ood ) samples A ⊆ D′ , which can be incorporated into the training dataset . Catastrophic forgetting is not a problem during re-training because when ood samples from the test dataset are included as training points , each training point x ∈ D is still identified with a neuron . By theorem 1 , x is still approximated accurately . See appendix A.2.2 for remarks on advantage over existing methods . Proposition 1 Errors on monotonous interval . Given finite training , test datasets D , D′ , there exists A ⊆ D′ such that , using TNN constructed with D∪A , for all samples in test dataset ( x′ , y′ ) ∈ D′ , sample-wise error e = |y′−TNN ( x′ ) | has an upper bound max ( |y′− y ( k+1 ) | , |y′− y ( k ) | ) for some k. Proof : see appendix A.4.2 . There is also a mid-point property that can be exploited for generalizability to arbitrarily high accuracy , where data must be sampled such that any instance xtest lies inside either ( 1 ) the training dataset or ( 2 ) is equal to some mid-point of two neighbouring training samples ; see the proposition below . Fig . 1 ( B ) shows how the component of xmid , k at j = N − k + 1 is half-activated i.e . the activation value is 0.5 . Admittedly , this is an ideal condition for accurate generalizability . Proposition 2 Mid-point property . The mid-point xmid , k = 12 ( x ( k ) + x ( k+1 ) ) takes the value of αTσ ( Wxmid , k + b ) = 1 2 ( y ( k ) + y ( k+1 ) ) . Proof : see appendix A.4.3 . TNN pencil-and-paper example . Use TNN to fit the dataset ( x , y ) ∈ { ( 1 , 1 ) , ( 0.5 , 2 ) , ( 0 , 3 ) } . Then f ( x ) ≈ TNN ( x ) = 3σ ( 20x+ 5 ) − σ ( 20x− 5 ) − σ ( 20x− 15 ) . See appendix A.4.4 . Remarks smoothness property , special case , scalability/complexity and generalizability to higher dimensions can be found in appendix A.4.5 . We proceed with SQANN , a universal approximation inspired by TNN that allows multi-dimensional input , multi-layer stacking of neurons based on relative strength of neuron activations .

Summary: Summary:  1. This paper proposes two neural networks by construction, i.e., Triangularly-constructed Neural Network (TNN) and Semi-Quantized Activation Neural Network (SQANN).  2. These two neural networks are universal approximators, which is proven by construction.  3. These two neural networks are resistant to catastrophic forgetting.  4. For TNN, strongly activated neurons and half-activated neurons can be identified.  5. For SQANN, users can identify the samples that are likely out of distribution.   Contributions claimed by authors:  1. TNN and SQANN are proposed.  2. The universal approximation is proven, using the construction of TNN and SQANN.  3. Resistance to catastrophic forgetting is proven.  4. SQANN can identify out-of-distribution samples.