D ISENTANGLING DEEP NEURAL NETWORKS WITH RECTIFIED LINEAR UNITS USING DUALITY
Anonymous authors
Paper under double-blind review
A BSTRACT
Despite their success deep neural networks (DNNs) are still largely considered as black boxes.
The main issue is that the linear and non-linear operations are entangled in every layer, making it hard to interpret the hidden layer outputs.
In this paper, we look at DNNs with rectiﬁed linear units (ReLUs), and focus on the gating property (‘on/off’ states) of the ReLUs.
We extend the recently developed dual view in which the computation is broken path-wise to show that learning in the gates is more crucial, and learning the weights given the gates is characterised analytically via the so called neural path kernel (NPK) which depends on inputs and gates.
In this paper, we present novel results to show that convolution with global pooling and skip connection provide respectively rotational invariance and ensemble structure to the NPK.
To address ‘black box’-ness, we propose a novel interpretable counterpart of DNNs with ReLUs namely deep linearly gated networks (DLGN): the pre-activations to the gates are generated by a deep linear network, and the gates are then applied as external masks to learn the weights in a different network.
The DLGN is not an alternative architecture per se, but a disentanglement and an interpretable re-arrangement of the computations in a DNN with ReLUs
The DLGN disentangles the computations into two ‘mathematically’ interpretable linearities (i) the ‘primal’ linearity between the input and the preactivations in the gating network and (ii) the ‘dual’ linearity in the path space in the weights network characterised by the NPK.
We compare the performance of DNN DGN and DLGN on CIFAR-10 and CIFAR-100 to show that, the DLGN recovers more than 83 .
5% of the performance of state-of-the-art DNNs, i.e., while entanglement in the DNNs enable their improved performance the ‘disentangled and interpretable’ computations in the DLGN recovers most part of the performance.
We conclude by identifying several interesting future directions based on DLGN.
I NTRODUCTION
Despite their success deep neural networks (DNNs) are still largely considered as black boxes.
The main issue is that in each layer of a DNN, the linear computation, i.e., multiplication by the weight matrix and the non-linear activations are entangled.
Such entanglement has its pros and cons .
The commonly held view is that such entanglement is the key to success of DNNs, in that, it allows DNNs to learn sophisticated structures in a layer-by-layer manner.
However, in terms of interpretability, such entanglement has an adverse effect: only the ﬁnal layer is linear and amenable to a feature/weight interpretation, and the hidden layers are non-interpretable due to the non-linearities.
Prior works (Jacot et al., 2018; Arora et al., 2019; Cao & Gu, 2019) showed that training an inﬁnite width DNN with gradient descent is equivalent to a kernel method with the so called neural tangent kernel matrix.
As a pure kernel method, the NTK matrix performed better than other pure kernel methods.
However, in relation to ‘black box’-ness, there are two issues with NTK theory: (i) Issue I:
Inﬁnite width NTK matrix does not explain fully the success of DNNs because it was observed that ﬁnite width DNNs outperform their inﬁnite width NTK counterparts, and it was an open question to understand this performance gap (Arora et al., 2019), and (ii) Issue II: Since the NTK is based on the gradients, it does not offer further insights about the inner workings of DNNs even for inﬁnite width.
A dual view for DNNs with rectiﬁed linear units (ReLUs) was recently developed by Lakshminarayanan & Singh (2020) who exploited the gating property (i.e., ‘on/off’ states) of the ReLUs.
The
[Figure]
dual view is essentially linearity in the path space , i.e., the output is the summation of path contributions.
While the weights in a path are the same for each input, whether or not a path contributes to the output is entirely dictated by the gates in the path, which are ‘on/off’ based on the input.
To understand the role of the gates, a deep gated network (DGN) (see Figure 1) was used to disentangle the learning in the gates from the learning in weights .
In a DGN, the gates are generated (and learnt) in a ‘gating network’ which is a DNN with ReLUs and are applied as external signals and the weights are learnt in a ‘weight network’ consisting of gated linear units (GaLUs)
(Fiat et al., 2019).
Each
GaLU multiplies its pre-activation and the external gating signal.
Using the DGN, two important insights were provided: (i) learning in the gates is the most crucial for ﬁnite width networks to outperform inﬁnite width NTK; this addresses Issue I , and (ii) in the limit of inﬁnite width, learning the weights with ﬁxed gates, the NTK is equal to (but for a scalar) a so called neural path kernel (NPK) which is a kernel solely based on inputs and gates.
This shifts Issue II on interpretability to that of interpretability of the gates as opposed to interpretability of the gradients.
Our Contribution.
We extend the dual view to address ‘black box’-ness by completely disentangling the ‘gating network’ and the ‘weight network’.
Our contributions are listed below.
• Disentangling Gating Network.
For this, we propose a novel Deep Linearly Gated Network (DLGN) as a mathematically interpretable counterpart of a DNN with ReLUs (see Figure 1).
In a DLGN, the gating network is a deep linear network, i.e., there is disentanglement because of the absence of non-linear activations.
The gating network is mathematically interpretable, because, the transformations from input to the pre-activations are entirely linear; we call this primal linearity .
• Dual View (Section 3.1).
We present an unnoticed insight in prior work on fully connected networks that the NPK is a product kernel and is invariant to layer permutations.
We present new results to show that (i) the NPK is rotationally invariant for convolutional networks with global average pooling, and (ii) the NPK is an ensemble of many kernels in the presence of skip connections.
• Disentangling Weight Network.
We then argue via theory and experiments that the weight network is disentangled in the path space , i.e., it learns path-by-path and not layer-by-layer.
For this, in Section 3.2 we show via experiments that destroying the layer-by-layer structure by permuting the layers and providing a constant 1 as input (see DLGN in Figure 1) do not degrade performance.
These counter intuitive results are difﬁcult to reconcile using the commonly held ‘sophisticated structures are learnt in layers’ interpretation.
However, these experimental results follow from the theory in Section 3.1.
In other words, it is useful to think that the learning in the weight network happens path-by-path; we call this dual linearity , which (for inﬁnite width) is interpreted via the NPK.
Message.
The DLGN is not an alternative architecture per se, but a disentanglement and an interpretable re-arrangement of the computations in a DNN with ReLUs.
The DLGN disentangles the computations into two ‘mathematically’ interpretable linearities (i
the
‘primal’ linearity
and (ii) the
‘dual’ linearity interpreted via the NPK
Using
the facts
that
the NPK is
based
on input and the
gates, and
in a DLGN
the pre-activations in the gating network are ‘primal’ linear,
we
have complete disentanglement
We
compare the performance of DNN, DGN and DLGN on CIFAR-
and CIFAR-
100 to
show that
the DLGN recovers more than
% of the performance of
state
-of
the-
art
DNNs ,
i.
e while
entanglement in the
DNNs enable
their
improved
performance
the ‘disentangled and interpretable’ computations in the DLGN recovers
most
part of the performance
This
brings us
to an interesting
question: ‘ Is DLGN a universal spectral approximator? ’
(see Section 3.2).
Related Works.
We now compare our work with the related works.
• Kernels.
Several works have examined theoretically as well as empirically two important kernels
associated with a DNN namely its NTK based on the correlation of the gradients and the conjugate kernel based on the correlation of the outputs
(Fan & Wang, 2020; Geifman et al., 2020; Liu et al., 2020; Chen et al., 2020; Xiao et al., 2020; Jacot et al., 2018; Arora et al., 2019; Novak et al., 2018;
Lee et al., 2017; 2020b)
In contrast, the NPK is based on the correlation of the gates.
We do not build pure-kernel method with NPK, but use it as an aid to disentangle ﬁnite width DNN with ReLUs.
• ReLU, Gating, Dual Linearity.
A spline theory based on max-afﬁne linearity was proposed in (Balestriero et al., 2018; Balestriero & Baraniuk, 2018) to show that a DNN with ReLUs performs hierarchical, greedy template matching.
In contrast, the dual view exploits the gating property to simplify the NTK into the NPK.
Gated linearity was studied in (Fiat et al., 2019) for single layered networks, along with a non-gradient algorithm to tune the gates.
In contrast, we look at networks of any depth, and the gates are tuned via standard optimisers.
The main novelty in our work in contrast to the above is that in DLGN the feature generation is linear.
The gating in this paper refers to the gating property of the ReLU itself and has no connection to (Srivastava et al., 2015) where gating is a mechanism to regulate information ﬂow.
Also, the soft-gating used in our work and in (Lakshminarayanan & Singh, 2020) enables gradient ﬂow via the gating network and is different from Swish (Ramachandran et al., 2018), which is the multiplication of pre-activation and sigmoid.
• Finite vs Inﬁnite Width.
Lee et al.
(2020a) perform an extensive comparison of ﬁnite versus inﬁnite width DNNs.
An aspect that is absent in their work, but present in the dual view is the disentanglement of gates and weights, and the fact that the learning in gates is crucial for ﬁnite width network to outperform inﬁnite width DNNs.
In our paper, we make use of theory developed for inﬁnite width DNNs to provide empirical insights into inner workings of ﬁnite width networks.
• Capacity.
Our experiments on destruction of layers, and providing constant 1 input are direct consequences of the insights from dual view theory.
These are not explained by mere capacity based studies showing DNNs are powerful to ﬁt even random labelling of datasets (Zhang et al.,
P RIOR W ORK : N EURAL T ANGENT K ERNEL AND D UAL V IEW
In this section, we will focus on the dual view (Lakshminarayanan & Singh, 2020) and how the dual view helps to address the open question in the NTK theory.
We begin with a brief summary of NTK.
NTK.
An important kernel associated with a DNN is its neural tangent kernel (NTK), which, for a pair of input examples x, x (cid:48) ∈ R d in , and network weights Θ ∈ R d net , is given by:
[Equation]
where ˆ y Θ ( · ) ∈ R is the DNN output.
Prior works (Jacot et al., 2018; Arora et al., 2019; Cao & Gu, 2019) have shown that, as the width of the DNN goes to inﬁnity, the NTK matrix converges to a limiting deterministic matrix NTK ∞ , and training an inﬁnitely wide DNN is equivalent to a kernel method with NTK ∞ .
While, as a pure kernel NTK ∞ performed better than prior kernels by more than 10% ,
Arora et al.
(2019) observed that on CIFAR-10:
[Equation]
≤
[Equation]
30 % where, CNN-GAP is a convolutional neural network with global average pooling and CNTK-GAP is its corresponding NTK ∞ matrix.
Due to this performance gap of about 5 − 6% , they concluded that
NTK ∞ does not explain fully the success of DNNs, and explaining this gap was an open question .
2. D UAL V IEW F OR DNN S WITH R E LU S : C HARACTERISING THE ROLE OF GATES
In the dual view, the computations are broken down path-by-path.
The input and the gates (in each path) are encoded in a neural path feature vector and the weights (in each path) are encoded in a neural path value vector, and the output is the inner product of these two vectors.
The learning in the gates and the learning in the weights are separated in a deep gated network (DGN) setup, which leads to the two main results of dual view presented in Section 2.1.2 and Section 2.1.3, wherein, the neural path kernel, the Gram matrix of the neural path features will play a key role.
2.1. N EURAL P ATH F EATURE , V ALUE , K ERNEL AND D EEP G ATED N ETWORK
Consider a fully connected DNN with ‘ d ’ layers and ‘ w ’ hidden units in each layer.
Let the DNN accept input x ∈ R d in and produce an output ˆ y Θ ( x ) ∈ R .
[Figure]
Deﬁnition 2.1.
A path starts from an input node, passes through a weight and a hidden unit in each layer and ends at the output node.
We deﬁne the following quantities for a path p :
Activity :
A Θ ( x, p ) is the product of the ‘ d − 1 ’ gates in the path.
Value : v Θ ( p ) is the product of the ‘ d ’ weights in the path.
Feature : φ Θ ( x, p ) is the product of the signal at the input node of the path and A Θ ( x, p ) .
The neural path feature (NPF) given by φ Θ ( x ) = (cid:0) φ Θ ( x, p ) , p = 1 , . . .
, P fc (cid:1) , ∈ R P fc and the neural path value (NPV) given by v Θ = (cid:0) v Θ ( p ) , p = 1 , . .
, P fc (cid:1) , ∈ R P fc , where P fc =
d in w ( d − 1) is the total number of paths.
Proposition 2.1.
The output of the DNN is then the inner product of the NPF and NPV:
[Equation]
Subnetwork Interpretation of DNNs with ReLUs.
A path is active only if all the gates in the path are active.
This gives a subnetwork interpretation, i.e., for a given input x ∈ R d in , only a subset of the gates and consequently only a subset of the paths are active, and the input to output computation can be seen to be produced by this active subnetwork.
The following matrix captures the correlation of the active subnetworks for a given pair of inputs x, x (cid:48) ∈ R d in .
Deﬁnition 2.2 (Overlap of active sub-networks) .
The total number of ‘active’ paths for both x and x (cid:48) that pass through input node i is deﬁned to be:
[Equation]
Lemma 2.1 (Neural Path Kernel (NPK)) .
Let D ∈ R d
in be a vector of non-negative entries and for u, u (cid:
) ∈ R d in , let (cid:104) u, u (cid:48) (cid:105) D
= (cid:80) d in i =
1 D ( i ) u ( i ) u (cid:48)
( i ) .
Then the neural path kernel (NPK) is given by:
[Equation]
Deep Gated Network (DGN) is a setup to separate the gates from the weights.
Consider a DNN with ReLUs with weights Θ ∈ R d net .
The DGN corresponding to this DNN (left diagram in Figure 2) has two networks of identical architecture (to the DNN) namely the ‘gating network’ and the ‘weight network’ with distinct weights Θ f ∈ R d net and Θ v ∈ R d net .
The ‘gating network’ has ReLUs which turn ‘on/off’ based on their pre-activation signals, and the ‘weight network’ has gated linear units (GaLUs)
(Fiat et al., 2019; Lakshminarayanan & Singh, 2020), which multiply their respective pre-activation inputs by the external gating signals provided by the ‘gating network’.
Since both the networks have identical architecture, the ReLUs and GaLUs in the respective networks have a one-to-one correspondence.
Gating network realises φ Θ f ( x ) by turning ‘on/off’ the corresponding GaLUs in the weight network.
The weight network realises v Θ v and computes the output
ˆ y DGN ( x ) =
(cid:104)
φ Θ f ( x ) , v Θ v (cid:105) .
The gating network is also called as the feature network since it realises the neural path features, and the weight network is also called as the value network since it realises the neural path value.
2.1. L EARNING W EIGHTS W ITH F IXED G ATES = N EURAL P ATH K ERNEL
During training, a DNN learns both φ Θ ( x ) as well as v Θ simultaneously, and a ﬁnite time characterisation of this learning in ﬁnite width DNNs is desirable.
However, this is a hard problem.
An easier problem is to understand in a DGN, how the weights in the value network are learnt when the gates are ﬁxed in the feature network, i.e., how ˆ y DGN ( x ) =
(cid:104)
φ Θ f ( x ) , v Θ v (cid:105) is learnt by learning v Θ v with ﬁxed φ Θ f ( x ) .
While ˆ y DGN ( x ) = (cid:104) φ Θ f ( x ) , v Θ v (cid:105) is linear in the dual variables, it is still non-linear in the value network weights Θ v .
However, Lakshminarayanan & Singh (2020) showed that the dual linearity is characterised by the NPK in the inﬁnite width regime.
We state the assumption followed by Theorem 5.1 in (Lakshminarayanan & Singh, 2020), wherein, the
NTK ( x, x (cid:48) )
= (cid:104)∇ Θ v ˆ y Θ DGN 0 ( x ) , ∇ Θ v ˆ y Θ DGN
( x (cid:48) )
(cid:105) is due to the gradient of ˆ y DGN with respect to the value network weights, and the NPK ( x, x (cid:48) )
= (cid:104)
φ Θ f 0 ( x ) , φ Θ f 0 ( x (cid:48) )
(cid:105) is due to the feature network weights.
Assumption 2.1.
Θ v 0 i.i.d ∼ Bernoulli ( 12 ) over {− σ, + σ } and statistically independent of Θ f 0 .
Theorem 2.1 (Theorem 5.1 in (Lakshminarayanan & Singh, 2020)) .
Under Assumption 2.1 for a fully connected DGN :
[Equation]
as w
[Equation]
Remark.
In the fully connected case, overlap ( i, x, x (cid:48) ) is identical for all i = 1 , . .
, d in , and hence (cid:104) x, x (cid:48) (cid:105) overlap ( · ,x,x (cid:48) ) in Lemma 2.1 becomes (cid:104) x, x (cid:48) (cid:105) · overlap ( x, x (cid:48) ) in Theorem 2.1.
It follows from
NTK theory that an inﬁnite width DGN with ﬁxed gates is equivalent to kernel method with NPK.
2.1. L EARNING IN G ATES K EY F OR F INITE W IDTH T O B E B ETTER T HAN I NFINITE W IDTH
The ﬁxed gates setting is an idealised setting, in that, it does not theoretically capture the learning of the gates, i.e., the neural path features φ Θ ( x ) .
However, the learning in the gates can be empirically characterised by comparing ﬁxed learnt (FL) gates coming from a pre-trained DNN and ﬁxed random (FR) gates coming from randomly initialised DNN, and the inﬁnite width NTK.
Using a
CNN-GAP and its corresponding DGN, Lakshminarayanan & Singh (2020) showed on CIFARthat (see Figure 2 for details on DGN training and the CNN-GAP architecture):
[Equation]
% based on which it was concluded that learning in the gates (i.e., neural path features) is crucial for ﬁnite width CNN-GAP to outperform the inﬁnite width CNTK-GAP.
It was also shown that the DGN can be trained standalone (as shown in Figure 2) and is only marginally poor to the DNN.
D EEP L INEARLY G ATED N ETWORKS : C OMPLETE D ISENTANGLEMENT DLGN :
[Equation]
→ Primal Linear → Pre-
[Equation]
Gates
[Equation]
The deep linearly gated network (DLGN) has two ‘mathematically’ interpretable linearities, the ‘primal’ and the ‘dual’ linearities.
The primal linearity is ensured in via construction and needs no theoretical justiﬁcation.
Once the pre-activations triggers gates, φ Θ f ( x ) gets realised in the value network by activating the paths.
Now, the value network itself is ‘dual’ linear, i.e., it simply computes/learns the inner product
ˆ y ( x ) = (cid:104) φ Θ f ( x ) , v Θ v (cid:105) .
Gating lifts the ‘primal’ linear computations in the feature network to ‘dual’ linear computations in the value network.
Dual linearity is characterised by the NPK (for inﬁnite width) which in turn depends on the input and gates, and the fact that the pre-activations to the gates are primal linear implies complete disentanglement and interpretability.
Dual linearity is mathematically evident due to the inner product relationship, however, adopting it has the following conceptual issue: it is a commonly held view that ‘sophisticated features are learnt in the layers’, that is, given that the input x ∈ R d in is presented to the value network (as in Figure 2), it could be argued that the GaLUs and linear operations are entangled which in turn enable learning of sophisticated features in the layers.
In what follows, we demystify this layer-by-layer view via theory (inﬁnite width case) in Section 3.1 , and experiments (on ﬁnite width networks) in Section 3.2, and then study the performance of DLGN in Section 3.2.
The layer-by-layer view is demystiﬁed by showing that (i) a constant 1 input can be given to the value network, (ii) layer-by-layer structure can be destroyed.
The constant 1 input is meant to show that if the input is not given to the value network then it is not possible to learn sophisticated structures ‘from the input’ in a layer-by-layer manner.
In terms of the dual linearity, providing a constant 1 input has only a minor impact, in that, the neural path feature becomes φ ( x, p ) = 1 · A ( x, p ) , i.e., it still encodes the path activity which is still input dependent.
Since φ ( x ) depends only on gates, the NPK will depend only on the overlap matrix; results in Section 3.1 captures this in theory.
Now, it could be argued that, despite a constant 1 input, the gates are still arranged layer-by-layer, due to which, the value network is still able to learn sophisticated structures in its layers.
Section 3.1 has theory that points out that as long as the correlation of the gates is not lost, the layer-by-layer structure can be destroyed.
3. D UAL L INEARITY : N EW I NSIGHTS AND N EW R ESULTS
We saw in Section 2.1.2 that dual linearity is characterised by the NPK for inﬁnite width case.
In this section, we: (i) cover standard architectural choices namely convolutions with global-average-pooling and skip connections in Theorems 3.2 and 3.3; the prior result Theorem 2.1 is only for the fully connected case, (ii) present new insights on Theorem 2.1 by restating it explicitly in terms of the gates in Theorem 3.1, and (iii) discuss how the NPK structure helps in demystifying the layer-by-layer view.
Note: Results in this section are about the value network and hold for both DGN and DLGN.
3.1. F ULLY C ONNECTED : P RODUCT OF L AYER W ISE B ASE K ERNELS
Theorem 3.1.
Let G l ( x ) ∈
[0 , 1] w denote the gates in layer l ∈ { 1 , . . .
, d − 1 } for input x ∈ R d in .
Under Assumption 2.1 ( σ = c scale √ w ) as w → ∞ , we have for fully connected DGN/DLGN:
[Equation]
• Product Kernel : Role of Depth and Width.
Theorem 3.1 is mathematically equivalent to Theorem 2.1, which follows from the observation that overlap
[Equation]
While this observation is very elementary in itself, it is signiﬁcant at the same time
Theorem
provides the most simplest kernel expression that characterises the information in the gates.
From Theorem 3.1 it is evident that the role of width is averaging (due to the division by w ).
Each layer therefore corresponds to a base kernel (cid:104)
G l ( x ) ,G l ( x
(cid:48) )
(cid:105) w which measures the correlation of the gates .
The role of depth is to provide the product of kernels.
To elaborate, the feature network provides the gates G l ( x ) and the value network realises the product kernel in Theorem 3.1 by laying out the
GaLUs depth-wise, and connecting them to form a deep network.
The depth-wise layout is important: for instance, if we were to concatenate the gating features as ϕ ( x )
=
( G l ( x ) , l = 1 , . . .
, d − 1) ∈ { 0 , 1 } ( d − 1) w , it would have only resulted in the kernel (cid:104) ϕ
( x ) , ϕ ( x (cid:48) )
(cid:105) =
(cid:80) d − 1 l =1 (cid:104) G l ( x ) , G l ( x (cid:48) )
(cid:105) , i.e., a sum (not product) of kernels.
• Constant 1 Input.
This has a minor impact, in that, the expression on right hand side of Theorem 3. becomes d · c 2( d − 1) scale · d in · Π d − 1 l =1 (cid:104)
G l ( x ) ,G l ( x (cid:48) )
(cid:105) w ,
i
e., the kernel still has information of the gates.
• Destroying structure by permuting the layers.
Π d − 1 l =1 (cid:104) G l ( x ) ,G l ( x (cid:48) )
(cid:105) w is permutation invariant, and hence permuting the layers has no effect.
3.1. C ONVOLUTION G LOBAL A VERAGE P OOLING : R OTATIONALLY I NVARIANT K ERNEL
We consider networks with circular convolution and global average pooling (architecture and notations are in the Appendix).
In Theorem 3.2, let the circular rotation of vector x ∈ R d in by ‘ r ’ co-ordinates be deﬁned as rot ( x, r )( i ) =
x ( i + r ) , if i + r ≤ d in and rot ( x, r )( i ) =
x ( i + r − d in ) if i + r > d in .
Theorem 3.2.
Under Assumption 2.1, for a suitable β cv (see Appendix for expansion of β cv ):
NTK CONV ( x, x (cid:48) )
→ β cv d in 2 · d in − 1 (cid:88) r = (cid:104) x, rot ( x (cid:48) , r ) (cid:105) overlap ( · ,x,rot ( x (cid:48) ,r )) , as w → ∞ • (cid:80) d in − 1 r =0 (cid:104) x, rot ( x (cid:48) , r ) (cid:105)
[Equation]
where the inner ‘ Σ ’ is the inner product between x and rot ( x (cid:48) , r ) weighted by overlap and the outer
‘ Σ
’ covers
all possible rotations, which in addition to the fact that all the variables internal to the network rotate as the input rotates, results in the rotational invariance.
It was observed by Arora et al.
(2019) that networks
with
global average-pooling
are better than
vanilla convolutional networks
The rotational invariance
holds
for convolutional architectures
only
in the presence of global-pooling.
So, this result explains why global-average-pooling helps.
That said, rotational invariance is not a new observation; it was shown by Li et al.
(2019) that prediction using CNTK-GAP is equivalent to prediction using CNTK without GAP but with full translation data augmentation (same as rotational invariance) with wrap-around at the boundary (same as circular convolution).
However, Theorem 3. is a necessary result, in that, it shows rotational invariance is recovered in the dual view as well.
• The expression in Theorem 3.2 becomes β cv d in 2 · (cid:80) d in − 1 r =0 (cid:80) d in i =1 overlap ( i, x, rot ( x (cid:48) , r )) for a constant 1 input .
The key novel insight is that the rotational invariance is not lost and overlap matrix measures the correlation of the paths which in turn depends on the correlation of the gates.
• Destroying structure by permuting the layers does not destroy the rotational invariance in Theorem 3.2.
This is because, due to circular convolutions all the internal variables of the network rotate as the input rotates.
Permuting the layers only affects the ordering of the layers
and does not affect the fact that the gates rotate if the input rotates, and correlation in the gates is not lost.
3.1. R ESIDUAL N ETWORKS W ITH S KIP C ONNECTIONS (R ES N ET ): E NSEMBLE O F K ERNELS
We consider a ResNet with ‘ ( b + 2) ’ blocks and ‘ b ’ skip connections between the blocks.
Each block is a fully connected (FC) network of depth ‘ d blk ’ and width ‘ w ’.
There are 2 b many sub-FCNs within this ResNet (see Deﬁnition 3.1).
Note that the blocks being fully connected is for expository purposes, and the result continue to hold for any kind of block.
Deﬁnition 3.1.
[Sub FCNs] Let 2 [ b ] denote the power set of [ b ] and let J ∈ 2 [ b ] denote any subset of [ b ] .
Deﬁne the‘ J th ’ sub-FCN of the ResNet to be the fully connected network obtained by (i) including block j , ∀ j ∈ J and (ii) ignoring block j , ∀ j / ∈ J .
Theorem 3.3.
Let NPK FC J be the NPK of the J th sub-FCN, and β J fc (see Appendix for expansion of β J fc ) be the associated constant.
Under Assumption 2.1, we have:
[Equation]
• Ensemble.
To the best of our knowledge, this is the ﬁrst theoretical result to show that ResNets have an ensemble structure, where each kernel in the ensemble,
i
e., NPK
FC
J corresponds to one of the 2 b sub-architectures (see Deﬁnition 3.1).
The ensemble behaviour of ResNet and presence of 2 b architectures was observed by Veit et al.
(2016), however without any concrete theoretical formalism.
• Effect of constant 1 input is as before for kernels NPK FC J and translates to the ensemble NTK RES .
• Destroying structure.
The ResNet inherits the invariances of the block level kernel.
In addition, the ensemble structure allows to even remove layers.
Veit et al.
(2016) showed empirically that removing single layers from ResNets at test time does not noticeably affect their performance, and yet removing a layer from architecture such as VGG leads to a dramatic loss in performance.
Theorem 3.
can
be
seen to provide a theoretical justiﬁcation for this empirical result.
In other words, due to the ensemble
structure a ResNet
is
capable of
dealing
with failure of components.
While failure of component itself does not occur unless one makes them fail purposefully as done in (Veit et al., 2016), the insight is that even if one or many of the kernels in the ensemble are corrupt and the good ones can compensate.
3. N UMERICAL E XPERIMENTS
Section 3.1 presented theoretical results which demystiﬁed the layer-by-layer view in value network, in this section we will verify these theoretical results in experiments.
We then show that DLGN recovers major part of performance of state-of-the-art DNNs on CIFAR-10 and CIFAR-100.
Setup Details.
We consider 3 DNN architectures, C4GAP, VGG-16 and Resnet-110, and their
DGN and DLGN counterparts.
Here C4GAP is a simple model (achieves about 80% accuracy on
CIFAR-10), mainly used to verify the theoretical insights in Section 3.1.
VGG-16 and Resnet-110 are chosen for their state-of-the-art performance on CIFAR-10 and CIFAR-100.
All models are trained using off-the-shelf optimisers (for more details, see Appendix D).
The DGN and DLGN are trained from scratch, i.e., both the feature and value network are initialised at random and trained.
In DGN and DLGN, we use soft gating (see Figure 2) so that gradient ﬂows through the feature network and the gates are learnt (we chose β = 10 ).
In what follows, we use the notation DGN ( x f , x v ) and
DLGN ( x f , x v ) where x f and x v denote the input to the value and feature networks respectively.
For instance, DLGN ( x, x ) means that both the value and feature network of the DLGN is provided the image as input, and DLGN ( x, 1 ) will mean that the feature network is given the image as input and the value network is given a constant 1 as input.
DGN ( x, x ) and DGN ( x, 1 ) notation works similarly.
Disentangling Value Network.
We show that destroying the layer-by-layer structure via permutations and providing a constant 1 input do not degrade performance.
Since our aim here is not state-of-the-art performance, we use C4GAP with 4 convolutional layers which achieves only about 80% test accuracy on CIFAR-10, however, enables us to run all the 4!
= 24 layer permutations.
The C4GAP, DGN and DLGN with layer permutations are shown in Figure 3.
Once a permutation is chosen, it is ﬁxed during both training and testing.
The results in Table I of Figure 3 show that there is no signiﬁcant difference in performance between DGN ( x, x ) vs DGN ( x, 1 ) , and DLGN ( x, x ) vs DLGN ( x, 1 ) , i.e., constant 1 input does not hurt.
Also, there is no signiﬁcant difference between the models without permutations and the models with permutations.
These counter intuitive and surprising results are difﬁcult to explain using the commonly held ‘sophisticated features are learnt layer-by-layer’ view.
However, neither the permutations or the constant 1 input destroys the correlation in the gates, and are not expected to degrade performance as per the insights in Section 3.1.
This veriﬁes Claim I.
DLGN Performance.
For this we choose VGG-16 and Resnet-110.
The results in Table II of Figureshow that the DLGN recovers more than 83 .
5% (i.e., 83 .
78% in the worst case) of the performance of the state-of-the-art DNN.
While entanglement in the DNNs enable their improved performance, the ‘disentangled and interpretable’ computations in the DLGN recovers most part of the performance.
[Figure]
[Table]
FC
GAP
[Figure]
[Table]
[Figure]
[Table]
ReLU
[Table]
ReLU
[Table] [Table]
[Table] [Table]
[Table]
3. I NTERESTING F UTURE D IRECTIONS
Motivated by the success of DLGN we further break it down into DLGN-
Shallow Features (DLGNSF), wherein, the feature network is a collection of shallow single matrix multiplications.
We compare a shallow DNN with ReLU called C1GAP against DLGN-SF of C4GAP (see Figure 4) and VGG- (see Appendix).
The results are in Figure 4, based on which we observe the following:
• Power of depth in value network and lifting to dual space.
Both C1GAP and C4GAP-DLGN-SF were trained with identical batch size, optimiser and learning rate (chosen to be the best for C1GAP).
The performance of C1GAP at 200 epochs is ∼ 10% lower than that of C4GAP-DLGN-SF.
After 2000 epochs of training and ensembling 16 such C1GAPs as C1GAP- 16-ENS closes the gap within ∼ 3% on C4GAP-DLGN-SF.
Yet, a deeper architecture VGG-16-DLGN-SF is ∼ 10% better than C1GAP- 16-ENS .
Note that both VGG-16-DLGN-SF and C1GAP- 16-ENS have gates for 16 layers produced in a shallow manner.
While in a C1GAP- 16-ENS , ‘16’ C1GAPs are ensembled, in VGG16-DLGN-SF these gates for 16 layers are used as gating signals to turn ‘on/off’
the GaLUs laid depth-wise as 16 layers of the value network, which helps to lift the computations to the dual space.
Thus, using the gates to lift (instead of ensembling) the computations to the dual space in the value network is playing a critical role, investigating which is an important future work.
• Power of depth in feature network.
By comparing CIFAR-100 performance of VGG-16-DLGNSF in Figure 4 and that of VGG-16-DLGN in Figure 3, we see ∼ 6% improvement if we have a deep linear network instead of many shallow linear networks as the feature network.
This implies depth helps even if the feature network is entirely linear, investigating which is an important future work.
• DGN vs DLGN In Table I and II of Figure 3, the difference between DGN and DNN is minimal (about 3% ), however, the difference between DLGN and DNN is signiﬁcantly large.
Thus, it is important to understand the role of the ReLUs in the feature network of DGN.
It is interesting to know whether this is simpler than understanding the DNN with ReLUs itself.
• Is DLGN a Universal Spectral Approximator?
The value network realises the NPK
which in general is an ensemble (assuming skip connections).
The NPK is based on the gates whose preactivations are generated linearly.
It is interesting to ask whether the DLGN via its feature network learns the right linear transformations to extract the relevant spectral features (to tigger the gates) and via its value network learns the ensembling of kernels (based on gates) in a dataset dependent manner.
C ONCLUSION
Entanglement of the non-linear and the linear operation in each layer of a DNN makes them uninterpretable.
This paper proposed a novel DLGN which disentangled the computations in a DNN with
ReLUs into two mathematically interpretable linearities, the ‘primal’ linearity from the input to the pre-activations that trigger the gates, and the ‘dual’ linearity in the path space.
DLGN recovers more than 83 .
5% of performance of state-of-the-art DNNs on CIFAR-10 and CIFAR-100.
Based on this success of DLGN, the paper concluded by identifying several interesting future directions.
A F ULLY C ONNECTED
Here, we present the formal deﬁnition for the neural path features and neural path values for the fully connected case in Deﬁnition A.1.
The layer-by-layer way of expressing the computation in a DNN of width ‘ w ’ and depth ‘ d ’ is given below.
Input Layer :
[Equation]
Pre-Activation
[Equation]
z x, Θ ( i in , l − 1)
Gating :
G x, Θ ( i out , l ) =
1 { q x, Θ ( i out ,l ) > 0 } Hidden Layer Output :
z x, Θ ( i out , l ) =
q x, Θ ( i out , l ) · G x, Θ
( i out , l )
Final Output : ˆ y Θ ( x ) =
(cid:80) i in Θ( i in , i out , d ) · z x, Θ ( i in , d − 1)
Notation A.1.
Index maps identify the nodes through which a path p passes.
The ranges of index maps I f l , I l , l ∈ [ d − 1] are [ d in ] and [ w ] respectively.
I d ( p ) = 1 , ∀ p ∈
[ P fc ] .
Deﬁnition A.1.
Let x ∈ R d
in be the input to the DNN.
For this input,
(i) A Θ ( x, p ) ∆ =
Π d − 1 l =1 G x, Θ
( I l ( p ) , l ) is the activity of a path.
(ii) φ Θ ( x ) ∆ =
(cid:16) x
( I f 0 ( p ))
A Θ ( x, p ) , p ∈
[ P fc ] (cid:17) ∈ R P fc is the neural path feature (NPF).
(iii) v Θ ∆ = (cid:0) Π dl =1 Θ( I l − 1 ( p ) , I l ( p ) , l ) , p ∈
[ P fc ] (cid:1) ∈
R P fc is the neural path value (NPV).
B C ONVOLUTION W ITH G LOBAL A VERAGE P OOLING
In this section, we deﬁne NPFs and NPV in the presence of convolution with pooling.
This requires three key steps (i) treating pooling layers like gates/masks (see Deﬁnition B.2) (ii) bundling together the paths that share the same path value (due to weight sharing in convolutions, see Deﬁnition B.3), and (iii) re-deﬁning the NPF and NPV for bundles (see Deﬁnition B.4).
Weight sharing due to convolutions and pooling makes the NPK rotationally invariant Lemma B.1.
We begin by describing the architecture.
Architecture: We consider (for sake of brevity) a 1 -dimensional 1 convolutional neural network with circular convolutions, with d cv convolutional layers ( l = 1 , . . .
, d cv ), followed by a global-averagepooling layer ( l = d cv + 1 ) and d fc ( l = d cv + 2 , . . .
d cv + d fc + 1 ) fully connected layers.
The convolutional window size is w cv < d in , the number of ﬁlters per convolutional layer as well as the width of the FC is w .
Indexing:
Here i in /i out are the indices (taking values in [ w ] ) of the input/output ﬁlters.
i cv denotes the indices of the convolutional window taking values in [ w cv ] .
i fout denotes the indices (taking values in [ d in ] , the dimension of input features) of individual nodes in a given output ﬁlter.
The weights of layers l ∈ [ d cv ] are denoted by Θ( i cv , i in , i out , l ) and for layers l ∈ [ d fc ] + d cv are denoted by Θ( i in , i out , l ) .
The pre-activations, gating and hidden unit outputs are denoted by q x, Θ ( i fout , i out , l ) ,
G x, Θ ( i fout , i out , l ) , and z x, Θ ( i fout , i out , l ) for layers l = 1 , . . .
, d cv .
Deﬁnition B.1 (Circular Convolution) .
For x ∈ R d in , i ∈
[ d in ] and r ∈ { 0 , . . .
, d in − 1 } , deﬁne :
(i) i ⊕ r
= i
+ r , for i + r ≤ d in
and i
⊕ r
=
i + r − d in
, for i
+
r
>
d in .
(ii) rot ( x, r )( i ) =
x ( i ⊕ r ) , i ∈
[ d in ] .
(iii) q x, Θ ( i fout , i out , l ) =
(cid:80) i cv ,i in Θ( i cv , i in , i out , l ) ·
z x, Θ ( i fout ⊕ ( i cv − 1) , i in , l − 1)
Deﬁnition B.2 (Pooling) .
Let G pool x, Θ ( i fout , i out , d cv + 1) denote the pooling mask, then we have
[Equation]
where in the case of global-average-pooling G pool x,
[Equation]
( i fout ,
[Equation] [Equation]
+ 1) =
[Equation]
∀
[Equation]
∈ [ w ] ,
[Equation]
fout ∈
[ d in ] .
Input Layer : z x, Θ ( · , 1 , 0) =
x
Convolutional Layers, l ∈ [ d cv ]
[Table]
: q x, Θ ( i fout ,i out ,l ) = (cid:80) i cv ,i in Θ( i cv ,i in ,i out ,l ) · z x, Θ ( i fout ⊕ ( i cv − 1) ,i in ,l − 1)
Gating Values :
G x, Θ ( i fout ,i out ,l ) = 1 { q x, Θ ( i fout ,i out ,l ) > 0 } Hidden
[Table]
: z x, Θ ( i fout ,i out ,l ) =
q x, Θ ( i fout ,i out ,l ) · G x, Θ
( i fout ,i out ,l )
GAP Layer, l = d cv +
[Table]
: z x, Θ ( i out ,d cv + 1) =
(cid:80) i fout
z x, Θ ( i fout ,i out ,d cv ) · G pool x, Θ ( i fout ,i out ,d cv + 1)
Fully Connected Layers, l ∈ [ d fc ] + ( d cv + 1)
[Table]
: q x, Θ ( i out ,l ) =
(cid:80) i in Θ( i in ,i out ,l ) · z x, Θ ( i in ,l − 1)
[Table]
Values :
G x, Θ ( i out ,l ) =
1 { ( q x, Θ ( i out ,l )) > 0 }
[Table]
: z x, Θ ( i out ,l ) =
q x, Θ ( i out ,l ) · G x, Θ
( i out ,l )
[Table]
: ˆ y Θ ( x ) = (cid:80) i in Θ( i in ,i out ,d ) · z x, Θ ( i in ,d − 1)
Table 2: Shows the information ﬂow in the convolutional architecture described at the beginning of
Appendix B.
B. N EURAL P ATH F EATURES , N EURAL P ATH V ALUE
Proposition B.1.
The total number of paths in a CNN is given by P cnn =
d in ( w cv w ) d cv w ( d fc − 1) .
Notation B.1 (Index Maps) .
The ranges of index maps I f l , I cv l , I l are [ d in ] , [ w cv ] and [ w ] respectively.
Deﬁnition B.3 (Bundle Paths of Sharing Weights) .
Let ˆ P cnn
= P cnn d in , and { B 1 , . . .
, B ˆ P cnn } be a collection of sets such that ∀ i, j ∈ [ ˆ P cnn ] , i (cid:54) =
j we have B i ∩ B j = ∅ and ∪ ˆ P cnn i =
1 B i =
[ P cnn ] .
Further, if paths p, p (cid:48) ∈ B i , then I cv l ( p ) =
I cv l ( p (cid:48) ) , ∀ l = 1 , . . .
d cv and I l ( p ) =
I l ( p (cid:48) ) , ∀ l = 0 , . . .
, d cv .
Proposition B.2.
There are exactly d in paths in a bundle.
Deﬁnition
B.4.
Let x ∈
R d in be the input to the
CNN.
For this input,
[Equation]
B. R OTATIONAL I NVARIANT K ERNEL
Lemma B.1.
[Equation]
Proof.
For the CNN architecture considered in this paper
[Equation]
each bundle has exactly d in number of paths, each one corresponding to a distinct input node.
For a bundle b ˆ p , let b ˆ p ( i ) , i ∈
[ d in ] denote the path starting from input node i .
[Equation]
In what follows we re-state Theorem 3.2.
Theorem B.1.
Let σ cv = c scale √ ww cv for the convolutional layers and σ fc =
c scale √ w for FC layers.
Under
Assumption 2.1, as w → ∞ , with β cv = (cid:16)
d cv σ 2( d cv − 1) cv
[Equation]
fc (cid:17) we have:
[Equation]
Proof.
Follows from Theorem 5.1 in [13].
C R ESIDUAL N ETWORKS WITH S KIP CONNECTIONS
As a consequence of the skip connections, within the ResNet architecture there are 2 b sub-FC networks (see Deﬁnition 3.1).
The total number of paths P res in the ResNet is equal to the summation of the paths in these 2 b sub-FC networks (see Proposition C.1).
Now, The neural path features and the neural path value are P res dimensional quantities, obtained as the concatenation of the NPFs and
NPV of the 2 b sub-FC networks.
Proposition C.1.
The total number of paths in the ResNet is P res
[Equation]
Lemma C.1 (Sum of Product Kernel) .
Let NPK RES Θ be the NPK of the ResNet, and NPK J Θ be the
NPK of the sub-FCNs within the ResNet obtained by ignoring those skip connections in the set J .
Then,
[Equation]
Proof.
Proof is complete by noting that the NPF of the ResNet is a concatenation of the NPFs of the 2 b distinct sub-FC-DNNs within the ResNet architecture.
We re-state Theorem 3.
Theorem C.1.
Let σ = c scale √ w .
Under Assumption 2.1, as w → ∞ , for β J res = ( |J | + 2) · d blk · σ
[Equation]
Proof.
Follows from Theorem 5.1 in [13].
D N UMERICAL E XPERIMENTS : S ETUP D ETAILS
We now list the details related to the numerical experiments which have been left out in the main body of the paper.
• Computational Resource.
The numerical experiments were run in Nvidia-RTX 2080 TI GPUs and Tesla V100 GPUs.
• All the models in Table I of Figure 3 we used Adam (Kingma & Ba, 2014) with learning rate of 3 × 10 − 4 , and batch size of 32.
• We tried several values of β in the range from 1 to 100 , and found the range 4 to 10 to be suitable.
We have chosen β = 10 throughout the experiments.
• In Section 3.2, the codes for experiments based on VGG-16 and Resnet-110 were refactored from following repository: “https://github.com/gahaalt/resnets-in-tensorﬂow2".
• For VGG-16-DLGN in Figure 3 and DLGN-SF in Figure 4, the max -pooling were replaced by average pooling so as to ensure that the feature network is entirely linear.
For the comparison to be fair, we replaced the max pooling in VGG-16 reported in Figure 3 by average pooling.
Batch normalisation layers were retained in VGG-16, VGG-16-DLGN and VGG-16-DLGN-SF (all three are shown in Figure 8).
• DLGN of Resnet-110 was derived from Resnet-110 in a similar manner.
• All the VGG-16, Resnet-110 (and their DGN/DLGN) models in Table II of Figure 3 we used SGD optimiser with momentum 0 .
9 and the following learning rate schedule (as suggested in “https://github.com/gahaalt/resnets-in-tensorﬂow2") : for iterations [0 , 400) learning rate was 0 .
01 , for iterations [400 , 32000) the learning rate was 0 .
1 , for iterations [32000 , 48000)
the learning rate was 0 .
01 , for iterations [48000 , 64000)
the learning rate was 0 .
The batch size was 128 .
The models were trained till 32 epochs.
• The VGG-16-DLGN-SF in Table III of Figure 4 uses the same optimiser, batch size and learning rate schedule as the models in Table II of Figure 3 as explained in the previous point.
• For C1GAP and C4GAP in Table III of Figure 4, we used Adam (Kingma & Ba, 2014) with learning rate of 10 − 3 , and batch size of 32.
This learning rate is best among the set { 10 − 1 , 10 − 2 , 10 − 3 , 3 × 10 − 4 } for C1GAP.
[Table]
[Table]
GA P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v {+G 4 G 3 G 2 G G i 4 G i 
 G+}
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC GA P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v G 4 G 3 G 2 G
G i 4 G i
G i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v[-G 4 G 3 G 2 G G i 4 G i 
 G-]
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
[Table]
v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
C f 4 C f 3 C f 2 C f 1 x f ˆ y ( x ) FC
GA
P G a L U C v 4 G a L U C v 3 G a L U C v 2 G a L U C v 1 x v
G 4 G 3 G 2 G
G i 4 G i
G
i 2 G i
[Table]
C
BN
G
C
Avg.
Pool
BN
G
C
BN
G
Cx f ˆ y ( x )
FC
GlobalAvg.
Pool
GaLU+Avg.
Pool
BN
C
GaLU
BN
C
GaLU
BN
C
GaLU+Avg.
Pool
BN
C
GaLU
BN
C
GaLU
BN
C
GaLU+Avg.
Pool
BN
C
GaLU
BN
C
GaLU
BN
C
GaLU+Avg.
Pool
BN
C
GaLU
BN
C
GaLU+Avg.
Pool
BN
C
GaLU
BN
Cx v
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
BN
G
C
Avg.
Pool
Avg.
Pool
Avg.
Pool
Avg.
Pool x f
F C ONTINUAL L EARNING
In this section we compare DLGN and Gated Linear Network (GLN) of Veness et al.
We highlight two closely related aspects of the GLN (as a result of the context based gating) namely (i) context induced increased capacity(CIIC) and (ii) context induced sparse activity (CISA) .
These two aspects were present in the work by Veness et al.
(2019), yet they were not highlighted because these two become apparent only from the equivalent ﬂattened computational graph (see
Appendix F.3) of the GLN (which was not present in Veness et al.
While Veness et al.
(2019) attributed ‘ cosine similarity ’ for the success of GLN in continual learning tasks, we show that cosine similarity itself is not sufﬁcient and context induced increased capacity and context induced sparse activity are also necessary.
We show via experiments that :
(i) GLN with CIIC and CISA performs well in continual learning.
(ii) GLN without CIIC and CISA performs poorly in continual learning.
(iii) DLGN-SF with increased capacity and sparse activity performs well in continual learning.
(iv) DLGN-SF without increased capacity and sparse activity performs poorly in continual learning.
Continual Learning Task.
We compare the continual learning capabilities of the GLN and the
DLGN-SF on the permuted MNIST dataset.
We ﬁrst generate 8 different permuted MNIST datasets and then use the same set of 8 tasks to study the continual learning capabilities of all the models.
F. N ON -C OMPARABLE A SPECTS OF DLGN AND GLN
We ﬁrst list and discuss the non-comparable aspects between the two works.
• Training Algorithm:
The DLGN in our paper is a close cousin of the DNN with ReLUs and is trained by backpropagation.
The GLN on the other hand is conceived and motivated from the experts setting and every node of the network is trained in a backpropagation free manner.
• Learning in the Gates: In our paper, the parameters of the feature network (whose layers trigger the gates) are learnt.
In the GLN, the gating is ﬁxed and not learnt.
This is a very important difference because it was shown by Lakshminarayanan & Singh (2020) that the learning in the gates is a very important aspect of the DNNs with ReLUs.
• Gating Mechanism:
In the case of DLGN, gating mechanism exists for fully connected, convolution, residual networks.
In the GLN, there are no (equivalents of) convolutional/residual architectures, and as such we can say the GLN is equivalent to a fully connected architecture.
Now, even in the case of fully connected architecture, the gating between DLGN and GLN differ quite a bit.
In the case of DLGN, each unit is controlled by a gate which corresponds to a hyperplane.
In the case of GLN, for each unit, the number of hyperplanes is equal to the so called context-dimension , and the binary encoding of these hyperplanes is mapped into one of 2 context-dimension contexts which is activated/selected.
F. C OMPARABLE A SPECTS OF DLGN AND GLN
Despite the aforementioned differences, DLGN and the GLN share the following commonalities (1) separate gating, (2) both models are data dependent linear networks .
For the purpose of comparison to be close enough, let us consider fully connected DLGN-SF (shallow features) whose feature network is initialised at random and not trained.
This way the DLGN-SF is similar to the GLN in that (a) the gates are ﬁxed and (b) gates are generated via random hyperplanes.
We now look at the data dependent linearity part by comparing the computational graphs of the DLGN-SF and GLN.
F. F LATTING THE COMPUTATIONAL GRAPH OF GLN AND COMPARING IT WITH DLGN-SF
We now compare the computational graphs of GLN and the value network of DLGN-SF in Figure 9.
For the sake of lucidity, in the Figure 9 as well in the discussion below, we have chosen input dimension to be 2 , layers to be 2 and units per layer (i.e., width) to be 2 .
In Figure 9 we have omitted the gating part which will be discussed in the text below, along with other relevant details.
w c 1 iw c 2 iw c 3 iw c 4 iiw c 1 jw c 2 jw c 3 jw c 4 jjw c 1 jw c 2 jw c 3 jw c 4 jjw c 1 iw c 2 iw c 3 iw c 4 iix (1) x (2) i+
i c 1i c 2i c 3i c 4w c 1 iw c 2 iw c 3 iw c 4 i+ jj c 1j c 2j c 3j c 4w c 1 jw c 2 jw c 3 jw c 4 ji+ i c
1i c 2i c 3i c 4w c 1 iw c 2 iw c 3 iw c 4 i+ jj c 1j c 2j c 3j c 4w c 1 jw c 2 jw c 3 jw c 4 jx (1) x (2) ≡ ijw iw jx (1) x (2) ijw
iw j
[Figure]
For sake of clarity we have ommitted the details about gating which we have described in
Here x = ( x (1) , x (2))
∈ R 2 is the
Each
‘ w ’s is a weight vector with 2 components.
The bold lines connecting the units with their weight vectors denote their one-to-one correspondence.
The lines emanating from each w
Left Most Diagram (Figure 9) depicts (without gating) the GLN in a manner similar to Figure 1 of
Veness et al.
Here, i 1 and j in layer and and
The ‘
’s are vectors in
The does not the
, where each C · : x → { c 1 , c 2 , c 3 , c 4 } .
For example, if C j 1 ( x ) = c 2 then unit j 1 uses weight w c 2 j 1 , and its output will be w c 2 j 1 x .
Middle Diagram (Figure 9) is a ﬂattened version of the GLN in the left most diagram.
The aim of ﬂattening is to depict the diagram in the left in the form of a standard computation graph.
Flattening is achieved by expanding the units i 1 , j 1 , i 2 , j 2 context wise into corresponding sub-units i c k 1 , j c k 1 , i c k 2 , j c k 2 , k = 1 , 2 , 3 , 4 .
The function of the context maps can be achieved by one-hot encoded gating signals (this is not shown in Figure 9),
G i ck 1 ( x ) , G j ck 1 ( x ) , G
i ck 2 ( x ) , G j ck 2 ( x ) , k = 1 , 2 , 3 , 4 .
For instance, if C i 2 ( x ) = c 3 , then we will have G i c
( x ) = 1 and G i ck 2 ( x ) = 0 , ∀ k (cid:54)
= 3 .
The output of unit i c 2 1 is G i c 21 ( x ) · w c 2 i 1 x .
The output of unit i 1 is the summation of the outputs of i c 1 1 , i c 2 1 , i c 3 1 , i c 4 1 .
Right Most Diagram (Figure 9) shows the value network of a DLGN.
The gating signals
G i 1 , G j 1 , G i 2 , G j 2 have been left out in the diagram.
The output of unit i 1 is G
i 1 ( x ) · w i 1 x .
F. C ONTEXT I NDUCED I NCREASE I N C APACITY AND S PARSE A CTIVITY
Two notable aspects about the GLN are the following:
• Context Induced Increase In Capacity.
Note that due to the presence of contexts, after ﬂattening, we can see that the width of the GLN gets multiplied by a factor equal to no.of.contexts = 2 context-dimension .
In Figure 9, the width of the GLN is 2 (in the left more diagram) and after ﬂattening it becomes 2 × 2 2 (in the middle diagram).
The number of parameters in a GLN is equal to the number of parameters of the DLGN-SF times the same factor no.of.contexts = 2 context-dimension , i.e., say in the ﬁrst layer, instead of w i 1 and w j 1 in the right most diagram, in the middle diagram we have w c k i 1 , w c k j 1 , k = 1 , 2 , 3 , 4 .
• Context Induced Sparse Activity: Eventhough the ﬂattened width of the GLN gets multiplied by the factor equal to no.of.contexts = 2 context-dimension , in each of the ﬂattened layers only
[Table]
units are active for any given input.
F. C ONTINUAL LEARNING IN GLN WITH AND WITHOUT CIIC AND CISA
The main reason for the success of GLN in continual learning (in permuted MNIST task) that is highlighted by Veness et al.
(2019) is “inputs close in terms of cosine similarity will give rise to similar data dependent weight matrices.
Since each task-speciﬁc cluster of examples is far from each other in signature space, the amount of interference between tasks is signiﬁcantly reduced, with the gating essentially acting as an implicit weight hashing mechanism".
However, we show in the experiments below that cosine similarity is not sufﬁcient, context induced increased capacity and context induced sparse activity are also necessary.
Experimental Setup.
We trained two GLN models namely GLNM-CD4 and GLNM-CD1 each of which have 10 one-vs-all GLNs each with 4 layers with 128 units in the ﬁrst 3 layers, and one unit in the ﬁnal layer which serves as the output unit.
The context dimension of GLNM-CD4 is equal to 4 and the context dimension of GLNM-CD1 is equal to 1 .
The code for the GLN (GLN training) is from Basu & Kuhnle (2020).
While it is mentioned in Veness et al.
(2019) that a 98% accuracy is achieved (on a single MNIST taksk), we obtain only 94 .
There are two reasons for this (i) in order to keep the comparison between GLN and DLGN-SF fair, we did not de-skew the dataset (which is the default option while using the GLN in Basu & Kuhnle (2020)), and if skewing is added the accuracy improves to about 96 .
5% and (ii) we also conﬁrmed with one of the authors of Veness et al.
(2019) that 128 units might not be sufﬁcient to achieve 98% .
However, since our aim here is to understand the trend in continual learning we accept 94 .
32% as a reasonable ballpark performance.
We now compare GLNM-CD4 and GLNM-CD1 in terms of CIIC and CISA.
CIIC (GLNM-CD4 vs GLNM-CD1).
In GLNM-CD4, the ﬂattened width is 128 × 16 .
In GLNMCD1, the ﬂattened width is 128 × 2 .
CISA (GLNM-CD4 vs GLNM-CD1).
GLNM-CD4 the only 128 out of the 128 × 16 units are active for any given input, i.e., only 116 of the units are active.
In GLNM-CD1, 128 out of the 256 units are active, i.e., 12 of the units are active.
Thus we can see GLNM-CD4 is a GLN with CIIC and CISA, and GLNM-CD1 is a GLN without
CIIC and CISA.
The results are in Tables A, and B where the entry in i th row and j th column stands for the test accuracy in Task-
j while training Task- i .
All the results are based on 3 runs.
[Table] [Table]
Observation.
From Tables A and B, we can observe that the GLNM-CD4 with CIIC and CISA performs well and GLNM-CD1 without CIIC and CISA performs poorly, from which we can conclude that cosine similarity is not sufﬁcient, context induced increased capacity and context induced sparse activity are also necessary.
F. C ONTINUAL L EARNING IN DLGN-SF WITH AND WITHOUT CAPACITY AND SPARSE ACTIVITY
Cosine Similarity holds in DLGN-SF.
Even in DLGN-SF, the gates are nothing but hyperplanes and similar inputs should trigger similar gates and activate similar data dependent linear networks.
Therefore cosine similarity holds true for DLGN-SF as well.
Increasing Capacity in DLGN-SF.
In order to make capacity of DLGN-SF comparable to that of
GLN with w GLN and a given context-dimension, we let the width of the DLGN-SF to be w DLGN-SF = w GLN × 2 context-dimension .
Thus we consider two models namely DLGN-SF-128x16 which is comparable to GLNM-CD4 and DLGN-SF-128x2 which is comparable to GLNM-CD1.
DLGN-SF-128x16 has 10 one-vs-all DLGN-SFs, each with 4 layers with 128 × 16 units in the ﬁrst three layers, and one unit in the last layer which serves as the output unit.
DLGN-SF-128x2 has 10 one-vs-all DLGN-SFs, each with 4 layers with 128 × 2 units in the ﬁrst three layers, and one unit in the last layer which serves as the output unit.
Inducing Sparse Activity in DLGN-SF.
The gates in a DLGN-SF are triggered by the feature network which contains w DLGN-SF hyperplanes.
Since we are considering DLGN-SF with randomly initialised (from a symmetric distribution) and ﬁxed feature network, it is reasonable to expect that ∼ w DLGN-SFof the gates are active/on.
Thus in order to induce sparse activity we sort the pre-activation signals of the gates and activate the gates corresponding to the top
[Table]
-dimension pre-activation signals.
Thus, for the case of DLGN-SF-128x16 to ensure sparse activity we activate only the top 128 × 16=gates.
For the case of DLGN-SF-128x2, since we are comparing it with GLNM-CD1 whose context dimension is 1 , we need ∼ w DLGN-SF 2to be active, and hence we just let the pre-activations to directly trigger the gates.
Experimental Setup.
We trained with SGD, batch size of 32 and learning rate of 0 .
32 (this is equivalent to a learning rate of 10 − 2 and batch size of 1 ).
All the results are based on 3 runs.
Table C shows DLGN-SF-128x16 with sparse activiy
[Table]
and Table D shows DLGN-SF-128x16 without sparse activity.
Table E shows DLGN-SF-128x2 which does not sparse activity
[Table]
[Table] [Table]
[Table] [Table]
Observation.
From Tables C with Tables D and E
[Table]
we can observe that the DLGN-SF-128x with CIIC and CISA performs well and DLGN-SF-128x16 without CIIC and CISA as well as
DLGN-SF-128x2 without CIIC and CISA perform poorly.
F. I MPROVING C ONTINUAL L EARNING IN DLGN-SF BY I NTRODUCING S PARSE A CTIVITY
We now show that DLGN-SF-128x2 can be made to perform well if we introduce sparse activity.
For this we sort the pre-activations to the gates and then trigger only the top 16 gates.
The results are shown in Table F.
[Table]
Observation The difference between Table E and Table F is that the model in Table F has sparse activity
[Table]
and we observe that it sparse activity ensures good performance
[Table]
F.
C ONCLUSION
We conclude this section on continual learning by saying DLGN-SF with increased capacity and sparse activity performs well in continual learning.
A comparison of the continual learning capabilities of GLN and DLGN to ﬁnd out which is the optimal model out of the two will require a more thorough study which we defer to future work.
Further, to perform well in continual learning, the models should be capable of learning many tasks, which naturally requires the models to have more capacity.
In the models that perform well above, we note that capacity has been built in two ways (i) via themodels due to ‘one-vs-all’ and (ii) the context in the GLN increases the capacity by a factor equal to no.of.contexts, and in the DLGN-SF we ensure capacity increase by increasing the width.
Sparsity helps to ensure that the different parts of networks are responsible for different contexts and they do not interfere with one another.
High capacity and interference minimisation have also been mentioned in Cheung et al.
in the context of continual learning.