A Chain Graph Interpretation of Real-World Neural Networks
Authors: Yuesong Shen
Daniel Cremers
Venue:  (2021)
Keywords: neural network interpretation
chain graph
deep learning theory
probabilistic graphical model

Abstract: The last decade has witnessed a boom of deep learning research and applications achieving state-of-the-art results in various domains. However, most advances have been established empirically, and their theoretical analysis remains lacking. One major issue is that our current interpretation of neural networks (NNs) as function approximators is too generic to support in-depth analysis. In this paper, we remedy this by proposing an alternative interpretation that identifies NNs as chain graphs (CGs) and feed-forward as an approximate inference procedure. The CG interpretation specifies the nature of each NN component within the rich theoretical framework of probabilistic graphical models, while at the same time remains general enough to cover real-world NNs with arbitrary depth, multi-branching and varied activations, as well as common structures including convolution / recurrent layers, residual block and dropout. We demonstrate with concrete examples that the CG interpretation can provide novel theoretical support and insights for various NN techniques, as well as derive new deep learning approaches such as the concept of partially collapsed feed-forward inference. It is thus a promising framework that deepens our understanding of neural networks and provides a coherent theoretical formulation for future deep learning research.

Under review as a conference paper at ICLR 2021
A CHAIN GRAPH INTERPRETATION OF REAL-WORLD NEURAL NETWORKS
An Anonymous authors Paper under double-blind review
ABSTRACT
The last decade has witnessed boom of deep learning research and applications achieving state-of-the-arl results in various domains. However, most advances have been established empirically, and their theoretical analysis remains lacking. One major issue is that our current interpretation of neural networks (NNs) as function approximators is too generic to support in-depth analysis. In this pa- per, we remedy this by proposing an alternative interpretation that identifies NNs as chain graphs (CGs) and feed-forward as an approximate inference procedure. The CG interpretation specifies the nature of each NN component within the rich theoretical framework of probabilistic graphical models, while at the same time remains general enough to cover real-world NNs with arbitrary depth, multi- branching and varied activations, as well as common structures including convo- lution / recurrent layers, residual block and dropout. We demonstrate with con- crete examples that the CG interpretation can provide novel theoretical support and insights for various NN techniques, well derive deep learning ap- proaches such as the concept of partially collapsed feed-forward inference. It is thus a promising framework that deepens our understanding of neural networks and provides a coherent thcoretical formulation for future deep learning research.
INTRODUCTION
During the last decade, deep learning (Goodfellow et al. 2016). the study of neural networks (NNs), has achieved ground-breaking results in diverse areas such as computer vision (Krizhevsky et al.)
2012
He et al. 2016, Long al. 2015) Chen et al. 2018), natural language processing (Hinton
jet al. 2012; Vaswani al. 2017 Devlin et al. 2211). generative modeling (Kingma & WWeliigg Goodfellow et al. 2014) and reinforcement learning (Mnih et al. 2015) Silver et al. 20166) and various network designs have been proposed. However, neural networks have been treated largely as "black-box" function approximators, and their designs have chiefly been found via trial- and-error, with little ysis is the current overly generic modeling of neural networks as function approximators: simply interpreting neural network as a composition of parametrized functions provides little insight to decipher the nature of its components or its behavior during the learning process.
2014
theoretical justification. A major cause that hinders the theoretical anal-
In this paper, we show that a neural network can actually be interpreted as a probabilistic graphical model (PGM) called chain graph (CG) (Koller& & Friedman] 2009). and feed-forward as an efficient approximate probabilistic inference on it. This offers specific interpretations for various neural network components, allowing for in-depth theoretical analysis and derivation of new approaches.
1.1 RELATED WORK
In terms of theoretical understanding of neural networks, well known result based on the function approximator view is the universal approximation theorem (Goodfellow et al.] 2016), however it only establishes the representational power of NNs. Also, there have been many efforts on alternative NN interpretations. One prominent approach identifies infinite width NNs Gaussian processes (Neal,
1996 Lee et al. 2018), enabling kernel method analysis (Jacot et al.) 2018). Other works also employ theories such as optimal transport (Genevay al. 2017 Chizat & Bach 2018) or mean field (Mei et al., 2019). These approaches lead to interesting findings, however they tend to only hold under limited unrealistic settings and have difficulties interpreting practical real-world NNs.

---

Under review as a conference paper at ICLR 2021
Rect. Gauss, constant
Rect. Geuss, sigmold
Rect. Gouss. tanh leaky
Rect. Gauss, tanh
Softplus
ReLU
Leakty
Figure I: Neural networks can be interpreted as layered chain graphs where activation functions are determined node distributions. Left: An example neural network interpreted as chain graph with three chain components which represent its layers; Right: A variety of activation functions (softplus, ReLU, leaky ReLU) approximated by nodes following rectified Gaussian distributions (e, q as in Eq.
We visualize the approximations stochastically by averaging over 200 samples.
Alternatively, some existing works study the post-hoc interpretability (Lipton 2018), proposing methods to analyze the empirical behavior of trained neural networks: activation maximization (Erhan et al. 2009), typical input synthesis (Nguyen et al. 2016). deconvolution (Zeiler & Fergus, 2014), layer-wise relevance propagation (Bach et 2015). etc. These methods can offer valuable insights to the practical behavior of neural networks, however they represent distinct approaches and focuses, and are all limited within the function approximator view.
Our work links neural networks to probabilistic graphical models (Koller & Friedman 2009), a rich theoretical framework that models and visualizes probabilistic systems composed of random vari- ables (RVs) and their interdependencies. There are several types of graphical models. The chain graph model (also referred to as L.F chain graph model) (Koller & Friedman, 2009: Lauritzen & Wermuth 1989 Frydenberg 1990) used in work is general form that unites directed and undirected variants, visualized as partially directed acyclic graph (PDAG). Interestingly, there exists series of works constructing hierarchical graphical models for data-driven learning prob- lems, such as sigmoid belief network (Neal) 1992), deep belief network (Hinton et al.) 20066), deep Boltzmann machine (Salakhutdinov Hinton (002) and sum product network (Poon & Domingos) 2011]. As alternatives to neural networks, these models have shown promising potentials for gen- erative modeling and unsupervised learning. Nevertheless, they are yet demonstrate competitive performances over neural network for discriminative learning.
Neural networks and graphical models have far been treated as distinct approaches in general. Existing works that combine them (Zheng et al. 2015) Chen et al. 2018 Lample et al. (0166) mainly treat either neural networks as function approximators for amortized inference, or graphical models as post-processing steps. Tang & Salakhutdinov (2013) create hybrid model, the stochastic feed- forward neural network (SFNN), by concatenating deterministic neurons with stochastic Bernoulli random variables, in order to represent multimodal distributions. Some also consider neural net- works as graphical models with deterministic hidden nodes (Buntine 1994). However this is an atypical degenerate regime. To the best of our knowledge, our work provides the first rigorous and comprehensive formulation of (non-degenerate) graphical model interpretation for neural networks in practical use.
1.2 OUR CONTRIBUTIONS
The main contributions of our work are summarized as follows:
We propose layered chain graph representation of neural networks, interpret feed-forward as an approximate probabilistic inference procedure, and show that this interpretation provides an extensive coverage of practical NN components (Section
2

---

Under review as a conference paper at ICLR 2021
To illustrate its advantages, show with concrete examples (residual block, RNN, dropout) that the chain graph interpretation enables coherent and in-depth theoretical support, and provides additional insights to various empirically established network structures (Section
Furthermore, we demonstrate the potential of the chain graph interpretation for discovering new approaches by using it derive novel stochastic inference method named partially collapsed feed-forward, and establish experimentally its empirical effectiveness (Section
2
CHAIN GRAPH INTERPRETATION NEURAL NETWORKS
Without further delay, derive the chain graph interpretation of neural networks in this section. We will state and discuss the main results here and leave the proofs in the appendix.
2.1 THE LAYERED CHAIN GRAPH REPRESENTATION
We start by formulating the so called layered chain graph that corresponds to neural networks we use in practice: Consider a system represented by layers random variables (x' ,",,,. where X is the i-th variable node in the 1-th layer, and denote N' the number of nodes in layer 1. We assume that nodes X in the same layer / have the same distribution type characterized by feature function T' that can be multidimensional. Also, we assume that the layers are ordered topologically and denote Pa(2 X') the parent layers of X To ease our discussion, we assume that X is the input layer and X the output layer (our formulation can easily extend to multi-input/output cases). A layered chain graph is then defined as follows:
Definition 1. A layered chain graph that involves layers of random variables (X chain graph that encodes the overall distribution P(X² ..... x such that:
X4) is
can be factored into layerwise chain components P(X Pa(X))) following the topological or- der, and nodes X within each chain component P(X (x*)) are conditionally independent given their parents (this results in bipartite chain components), thus allowing for further decom- position into nodewise conditional distributions PP This means have
I Pa(X'))
L N
P(X2 .... X P((
IIII P P(X 1=
2. For each layer with parent layers P P XP Pi...., Pr. {1...., 1}. its nodewise conditional distributions Pa are modeled by pairwise conditional random fields (CRFs) with with unary (b/) and pairwise (WF))) weights (as we will see, they actually correspond to biases and weights in NN layers):
P(X fl (T'
TPi (XPL) )..))....
T (P*)))
(2)
Pa
with (T (XP1) ) ..... T TX*(x*)) bj
W J. (X?).
p=p1j=1
Figure Left illustrates an example three-layer network as layered chain graph and its chain com- ponent factorization. In Eq. (2). f is an arbitrary function that represents probability distri- bution. For exponential family distributions (Koller& & Friedman 2009). Eq. (2) simply becomes
P(XX))((X x exp (X)) c (X*1) T (XP*))
Note that layered chain graph has globally directed graph structure and has an equivalent modeling based on directed graphical model (Bayesian network) (Koller & Friedman 2009), we elaborate on
this point for interested readers in Appendix A
2.2 FEED-FORWARD AS APPROXIMATE PROBABILISTIC INFERENCE
To identify layered chain graphs with real-world neural networks, need show that they behave the same way during inference and learning. For this, we establish the fact that forw-fo actually be seen performing approximate probabilistic inference layered chain graph:
3

---

Under review as a conference paper at ICLR 2021
Given input sample x², consider the problem of inferring the marginal distribution Q node X and its expected features q. defined as
of
(1(1/(x))) = P(X)
- x
[X; (q' = =
(4)
Consider non-input layer with parent layers Pl..... Pn. the independence assumptions encoded by the layered chain graph lead to the following recursive expression for marginal distributions Q:
(-( = Eq Qp. [P(x:(Pa(X))) X
(5)
However, the above expression is general intractable, as it integrates over the entire admissible states of all parents nodes in Pa (x To proceed further, simplifying approximations are needed. Interestingly, by using linear approximations, we can obtain the following results (in case of discrete random variable the integration in Eq. 7 replaced by summation):
Proposition 1. If we make the assumptions that the corresponding expressions are approximately linear w.r.t parent features TP:
Tpn (XP*), ), we obtain the following approximations:
(((x((5))= = f (T' (ot) .("")))"'.)))))))
(6)
dt = / (' (T' ( .....
......."))))) )d == == ((cc(qq)......q"
Especially, Eq. 77 is a seed-forward expression for expected Seattrr qt with activation fanction g' determined by T and f'. i.e. the distribution type of random variable nodes in layer l.
The proof is provided in Appendix B.I This allows us to identify feed-forward as an approximate probabilistic inference procedure for layered chain graphs. For learning, the loss function typically function of (Q", q²) obtainable via network parameter update using stochastic gradient descent and backpropagation. Thus we are able to replicate the exact neural network training process with this layered chain graph framework.
and we can follow the same classical neural
The following corollary provides concrete examples of some common activation functions g (we emphasize their names in bold, detailed formulations and proofs are given in Appendix B.2):
Corollary 2. We have the following node distribution - activation function correspondences:
1. Binary nodes taking values {a, B} results in sigmoidal activations, especially, obtain sigmoid (a,, 3 are interchangeable);
with a = 0, 3 = 1 and tanh with a = -1, 3
Multilabel nodes characterized by label indicator features result in the softmax activation:
3. Variants of (leaky) rectified Gaussian distributions (T!) X max(cY; ; Y?) with Y! ~
N (el, (s((ee))2)) can approximate activations such as softplus (6 = 0, s R 1.7761) and e-leaky rectified linear unit (ReLU) (s) = tanh(e!)) including ReLU (c (c= 0) and identity (c = 1).
Figure D Right illustrates activation functions approximated by various rectified Gaussian variants. We also plotted (in orange) alternative approximation of ReLU with gegmoid-modulaten standard deviation proposed by Nair & Hinton (011)) which is less accurate around the kink at the origin.
The linear approximations, needed for feed-forward, is coarse and only accurate for small pairwise weights (i) [| < 1) or already linear regions. This might justify weight decay beyond the general "anti-overfit" argument and the empirical superiority of piecewise linear activations like ReLU (Nair & Hinton| 2010). Conversely, as a source of eиror, it might explain some "failure cases" of neural networks such as their vulnerability against adversarial samples, see e.g., Goodfellow al. (2015).
2.3 GENERALITY OF THE CHAIN GRAPH INTERPRETATION
The chain graph interpretation formulated in Sections 2.1 and 2.2 is a general framework that can describe many practical network structures. To demonstrate this, we list here a wide range of neural network designs (marked in bold) that are chain graph interpretable.
In terms of network architecture, is clear that the chain graph interpretation can model net- works of arbitrary depth, and with general
structures such as inception modules (Szegedy et al.] 2013) or residual blocks (He et al.) 2016 He et al. 2016) discussed in Sec- Also, possible built up recurrent neural networks (RNNs) for sequential data
tion 3.1
4

---

Under review as a conference paper at ICLR 2021
learning, as we will see in Section 3.2 Furthermore, the modularity of chain components justifies transfer learning via partial reuse pre-trained networks, e.g., backbones trained for image classification be reused for segmentation (Chen et al. 2018))
In terms of layer structure, we are free to employ sparse connection patterns and shared/fixed obtain not only dense connections, but also connections like convolu-
weight, that tion, average pooling or skip connections. Moreover, as shown in Section 3.3 dropout can be reproduced by introducing and sampling from auxiliary random variables, and normalization layers like batch normalization (loffe & Szegedy, 2015) can be seen as reparametrizations of node distributions and fall within the general form (Eq. (2). Finally, we can extend the layered chain graph model to allow for intra-layer connections, which enables non-bipartite CRF layers which are typically used on output layers for structured prediction tasks like image segmenta- tion (Zheng et al. 2015, Chen et al.] 2018) or named entity recognition (Lample et al. 2016)
However, sced-forward is no longer applicable through these intra-connected layers.
Node distributions can be chosen freely, leading variety of nonlincarities (c.g., Corollary
SELECTED CASE STUDIES OF EXISTING NEURAL NETWORK DESIGNS
The proposed chain graph interpretation offers detailed description of the underlying mechanism of neural networks. This allows us to obtain novel theoretical support and insights for various net- work designs which are consistent within a unified framework. We illustrate this with the following concrete examples where perform in-depth analysis based on the chain graph formulation.
3.1 RESIDUAL BLOCK AS REFINEMENT MODULE
The residual block, proposed originally in He et al. (2016) and improved later (He et al. (016) with the preactivation form, is an effective design for building up very deep networks. Here we show that a preactivation residual block corresponds to refinement module within chain graph. We use modules to refer to encapsulations of layered chain subgraphs as input-output mappings without specifying their internal structures. refinmmen module defined as follows:
Definition 2. Given a base submodule from layer X to layer X refinement module augments this base submodule with side branch that chains copy of the base submodule (sharing weight with its original) from X!- to duplicated layer X!, and then a refining submodule from X to X
(-11
W,b
W.b
X
Tx
X!
W,b
Figure 2: Example of refinement module (left) and its corresponding computational graph (right), composed of a base submodule X > X (blue background) and a refining submodule X + z' > X (red background). In the computational graph each W. b represents a linear connection (Eq. (3) and a an activation function. Same color identifies corresponding parts the two graphs. We see that this refinement module corresponds exactly to preactivation residual block.
Proposition 3. A refinement module corresponds a preactivation residual block.
We provide a proof in Appendix B.3 and illustrate this correspondence in Figure 2 An interesting remark that the refinement process can be recursive: the base submodule of refinement module can be refinement module itself. This results in sequence of consecutive residual blocks.
While a vanilla layered chain component encodes a generalized linear model during ferw-rd (c.f. Eqs. 3). ) the refinement process introduces nonlinear extension term to the previously lin- ear output preactivation, effectively increasing the representational power. This provides a possible explanation the empirical improvement generally observed when using residual blocks.
Note that it is also possible to interpret the original postactivation residual blocks, however a somewhat artificial manner, as it requires defining identity connections with manually fixed weights.
5

---

Under review as a conference paper at ICLR 2021
3.2 RECURRENT NEURAL NETWORKS
Recurrent neural networks (RNNs) Goodfellow et al. 2016) are widely used for handling sequential data. An unrolled recurrent neural network can be interpreted as dynamic layered chain graph constructed as follows: a given base layered chain graph is copied for each time step, then these copies are connected together through recurrent chain components following the Markov assumption KKller & Friedman 2009): each recurrent layer X at time t is connected by its corresponding layer X from the previous time step parent layers of X in the base chain graph, we can easily interpret the following two variants:
Especially, denoting Pa' (X..)) the non-recurrent
Proposition 4. Given recurrent chain component that encodes P(X'')Pa' (X!.), x
1. It corresponds to simple (or vanilla / Elman) recurrent layer Goodfellow et al. 2016 if the
connection from X to X dense;
2. It corresponds to independently RNN (IndRNN) (Li et al. 2018) layer if the conditional inde- pendence assumptions among the nodes X!" within layer I are kept through time:
Vi E (',,...NN),, We provide proof in Appendix B.4 and illustrates both variants in Figure
P(X (, x,x(+=))=++++))))))))))) P |Pa (x!+)
(8)
Figure 3: Comparison of an simple recurrent layer (left) v.s. an IndRNN (right) recurrent layer. IndRNN, the better variant, enforces the intra-layer conditional independence through time.
The simple recurrent layer, despite its exhaustive dense recurrent connection, is known to suffer from vanishing/exploding gradient and can not handle long sequences. The commonly used long-short term memory (Hochreiter & Schmidhuber 1997) and gated recurrent unit (Cho et al. 2014) alleviate this issue via long term memory cells and gating. However, they tend to result in bloated structures, and still cannot handle very long sequences (Li et al., 2018). On the other hand, IndRNNs can process much longer sequences and significantly outperform not only simple RNNs, but also LSTM- based variants (Li et al. 2018 2019). This indicates that the assumption of intra-layer conditional independence through time, analogue to the local receptive fields of convolutional neural networks, could be an essential sparse network design tailored for sequential modeling.
3.3 DROPOUT
Dropout Srivastava et al., 2014] is practical stochastic regularization method commonly used especially for regularizing fully connected layers. As we see in the following proposition, from the chain graph point of view, dropout corresponds to introducing Bernoulli auxiliary random variables that serve as noise generators for feed-forward during training:
Proposition 5. Adding dropout with drop rate 1 1pp' p' to layer I corresponds the following chain graph construction; for each node X in layer I we introduce an auxiliary Bernoulli random variable D' ~ Bernoullily)) and multiply it with the pairwise interaction terms all preuctivations (Eq. (3) involving X as parent (this makes D a parent of all child nodes of X! and extend their pairwise interactions with X to ternary ones). The behavior of dropout is reproduced exactly
During training, we sample auxiliary nodes D during feed This results in dropping each activation d. of node X with probability 1 ppp;
At test time, we marginalize auxiliary nodes D during each feed-forward. This leads to deter- ministic evaluations with a constant scaling of y for the node activations d.
Note that among other things, this chain graph interpretation of dropout provides a theoretical justification of the constant scaling at test time. This was originally proposed heuristic in Srivastava et al. (2014) maintain consistent behavior after training.
We provide proof in Appendix B.5

---

Under review as a conference paper at ICLR 2021
4 PARTIALLY COLLAPSED FEED-FORWARD
The theoretical formulation provided by the chain graph interpretation can also be used to derive new approaches for neural networks. allows us to create new deep learning methods following coherent framework that provides specific semanties to the building blocks of neural networks. Moreover, we can make use of the abundant existing work from the PGM field, which also serves as a rich source of inspiration. As concrete example, we derive in this section new stochastic inference procedure called partially collapsed feed-forward (PCFF) using the chain graph formulation.
4.1 PCFF: CHAIN GRAPH FORMULATION
A layered chain graph, which can represent neural network, is itself probabilistic graphical model that encodes an overall distribution conditioned on the input. This means that, to achieve stochastic behavior, can directly draw samples from this distribution, instead of introducing additional "noise generators" like in dropout. In fact, given the globally directed structure of layered chain graph, and the fact that the conditioned input nodes are ancestral nodes without parent, it is well- known PGM result that we can apply forward sampling (or ancestral sampling) (Koller & Friedman 2009) to efficiently generate samples: given input sample x we follow the topological order and sample each non-input node X using its nodewise distribution (Eq. (2)) conditioned on the samples (xx) (x.).... xx") of its parents. Compared to reed-forward, forward sampling also performs a single forward pass, but generates instead an unbiased stochastic sample estimate.
While general an unbiased estimate is preferable and the stochastic behavior can also introduce regularization during training (Srivastava et al.) 2014)) forward sampling can not directly replace feed-forward, since the sampling operation is not differentiable and will jeopardize the gradient flow during backpropagation. To tackle this, one idea is to apply the reparametrization trick (Kingma & Welling 2014) on continuous random variables (for discrete RVs the Gumbel softmax trick (Jang al. 2017) can be used but requires additional continuous relaxation). An alternative solution is to only sample part of the nodes as in the case dropout.
The proposed partially collapse feed-forward follows the second idea: we simply "mix up" feed- forward and forward sampling, so that for each forward inference during training, we randomly select portion of nodes to sample and the rest to compute deterministically with wordadd. Thus for a node X with parents (XP:,.. "e.,,,,.,"", XP*), its forward inference update becomes
[g'(e( (e (PT)) d + (T' x
if collapsed (feed- forword);;::;
(9)
fr (T' (X)), (X)))e((a))) e((
)) if uncollapsed (forward sampling).
Following the collapsed sampling (Koller & Friedman, 2009) terminology, we call this method the partially collapsed feed-forward (PCFF). PCFF is a generalization over feed-forward and forward sampling, which can be seen as its fully collapsed / uncollapsed extremes. Furthermore, it offers bias-variance trade-off, and can be combined with the reparametrization trick to achieve unbiased estimates with full sampling, while simultaneously maintaining the gradient flow.
Relation to stochastic feedforward neural network While PCFF can also seen as a stochastic generalization of the feed-forward inference, it represents a substantially distinct approach compared to SFNN: Apart from the clear difference that PCFF uses forward sampling and SFNN uses impor- tance sampling, major dissimilarity is that SFNN makes clear distinction between deterministic neurons and stochastic random variables, whereas PCFF identifies neurons with random variables thanks to the layered chain graph interpretation. This is why PCFF can freely choose different sub- set of nodes to sample during each forward pass. From the chain graph interpretation perspective, SFNN can be seen layered chain graph having a fixed subset of nodes with stochastic behavior, and it performs a hybrid of feed-forward and importance sampling for inference.
4.2 PCFF: EXPERIMENTAL VALIDATION
In the previous sections, have been discussing existing approaches whose empirical evaluations have been thoroughly covered by prior work. The novel PCFF approach proposed this section, however, requires experiments to check its practical effectiveness. For this we conduct here series
7

---

Under review as a conference paper at ICLR 2021
of experiments Our emphasis is to understand the behavior of PCFF under various contexts and not to achieve best result for any specific task. We only use chain graph interpretable components, and adopt the reparamcterization trick (Kingma & Welling 2014) for ReLU PCFF samples.
The following experiments show that PCFF overall an effective stochastic regularization method. Compared to dropout, it tends produce more consistent performance improvement, and can some- times outperform dropout. This confirms that our chain graph based reasoning has successfully found an interesting novel deep learning method.
MNIST
FashonMiiST
0.125-
Test error (log scale)
Testerrrr error
0.070
Regulaiiation Cropout PCFF
Regulanzat
Drapout
None
0.000
0.025
Activation function
Drop/Sample
Figure 4: Comparison of stochastic methods (None/Dropou/PCFFF) in terms of image classification test errors (lower is better) under various settings. Left: MNIST/FashioMMNIST datasets with simple dense network and tanb/ReLU activation functions; Right: CIFAR-I0 dataset with ResNet20 and varying drop/sample rates. All reported results are average values of three runs. Compared to dropout, PCFF can achieve comparable results, and tend to deliver more consistent improvements.
Simple dense network We start with simple network with two dense hidden layers of 1024 nodes to classify MNIST (Lecun et al. 1998) and FashionMNIST (Xiao et al.) (017) images. We use PyTorch (Paszke et al., 2017). train with stochastic gradient descent (learning rate 0.01, mo- mentum 0.9), and set up 20% of training data validation set for performance monitoring and carly-stopping. We set drop rate to 0.5 for dropout, and for PCFF we set the sample rate to 0.4 for tanh and 1.0 (full sampling) for ReLU. Figure 4 Left reports the test errors with different activation functions and stochastic regularizations.
We that dropout and PCFF are overall comparable, and both improve the results in most Also, the ReLU activation consistently produces better results that tanh. Additional experiments show that PCFF and dropout be used together, which sometimes yields improved performance.
Convolutional residual network To figure out the applicability of PCFF in convolutional residual networks, we experiment CIFAR-10 (Krizhessky) (209) image classification. For this we adapt an existing implementation (Idelbayev) to use preactivation variant. We focus on the ResNer20 structure, and follow the original learning rate schedule except for setting up validation set of 10% training data to monitor training performance. Figure Right summarizes the test errors under different drop/sample rates.
We observe that in this case PCFF can improve the performance over a wide range of sample rates, whereas dropout is only effective with drop rate 0.1, and large drop rates in this case significantly deteriorate the performance. We also observe clear trade-off of the PCFF sample rate, where a partial sampling of 0.3 yields the best result.
We complete our empirical evaluations PCFF with an RNN test case. For this we used IndRNNs with 6 layers to solve the sequential/permuter MNIST classification problems based on an existing Implementation provided provvddd by the authors of IndRNN (Li al. 2018 2019).
Independently RNN
We tested over dropout with drop rate 0.1 and PCFF with sample rate 0.1 and report the average test accuracy of three runs. We notice that, while in the permuted MNIST case both dropout (0.9203) and PCFF (0.9145) improves the result (0.9045), in the sequential MNIST case, dropout (0.9830) seems to worsen the performance (0. 9841) whereas PCFF (0.9842) delivers comparable result.
"Implementation available at: (Github link placeholder, provided as supplementary material.) https:: //i thub. com/ htt.b./.w...ssmmnrrraaaaaaaaaana
8

---

Under review as a conference paper at ICLR 2021
5 CONCLUSIONS AND DISCUSSIONS
In this work, we show that neural networks can be interpreted as layered chain graphs, and that feed- forward can be viewed an approximate inference procedure for these models. This chain graph interpretation provides a unified theoretical framework that elucidates the underlying mechanism of real-world neural networks and provides coherent and in-depth theoretical support for wide range of empirically established network designs. Furthermore, it also offers solid foundation to derive new deep learning approaches, with the additional help from the rich existing work on PGMs. thus promising alternative neural network interpretation that deepens our theoretical understanding and unveils new perspective for future deep learning research.
In the future, we plan to investigate number of open questions that stem from this work, especially:
Is the current chain graph interpretation sufficient to capture the full essence of neural networks? Based on the current results, we are reasonably optimistic that the proposed interpretation can cover an essential part of the neural network mechanism. However, compared to the function approximator view, it only covers subset of existing techniques. Is this subset good enough? On a related note: can we find chain graph interpretations for other important network designs (or otherwise some chain graph interpretable alternatives with comparable or better performance)?
The current work provides a good start, but it is by no means an exhaustive study.
Finally, what other new deep learning models and procedures can build up based on the chain graph framework? The partially collapsed feed-forward inference proposed in this work is just simple illustrative example, and we believe that many other promising deep learning techniques can be derived from the proposed chain graph interpretation.
REFERENCES
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Rabur-Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10(7):1-45 07 2015.
Wray L. Buntine. Operations for learning with graphical models. J. Artif. Int. Res., 2(1):159-225, December 1994.
Liang-Chieh Chen, George Papandreou, lasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected erfs. TPAMI, 40(4):834-848, 2018.
Lenaic Chizat and Francis Bach. On the global convergence of gradient descent for over- parameterized models using optimal transport. In NeurlPS, 2018.
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder- for statistical machine translation. In EMNLP. 2014.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of deep network. 2009.
Morten Frydenberg. The chain graph markov property. Scandinavian Journal of Statistics, pp. 333-353, 1990.
Aude Genevay, Gabriel Peyré, and Marco Cuturi. GAN and VAE from optimal transport point of view. arXiv preprint 1706. 01807, 2017.
I. J. Goodfellow, J. Pouget-Abadic M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks. In NIPS, 2014.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.

---

Under review as a conference paper at ICLR 2021
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
K. He, X. Zhang, Ren, and J. Sun. Deep residual learning for image recognition. CVPR, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. ECCV, 2016.
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury. Deep neural networks for acoustic modeling in speech recogni- tion: The shared views of four research groups. IEEE Signal Processing Magazine, (96)):2-97, 2012.
Geoffrey E. Hinton, Simon Osindero, and Yee- Tee-Whye Teh. A fast learning algorithm for deep belief nets. Neural Comput., 18(7):1227-1554, 2006.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., 9(8):1735- 1780, November 1997.
Yerlan Idelbayev. Proper ResNet implementation for CIFARIO/CIFARION in PyTorch. https: //sptttww..co//kkkattttttrrrrrre
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. NeurIPS, 2018
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with sumtel-softmax. In ICLR, 2017.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
Daphne Koller and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, 2012.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural architectures for named entity recognition. NACCL, 2016.
Steffen I Lilholt Lauritzen and Nanny Wermuth. Graphical models for associations between variables, some of which are qualitative and some quantitative. The annals of Statistics, pp. 31-57, 1989.
Y. Lecun, L.. Bottou, Y. Bengio, and P. Haffner. tradieet-based learning applied to document recog- nition. Proceedings of the IEEE, 86(1 86(11))2278-2324 1998.
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Pennington, and Jascha Sohl-Dickstein. Deep neural networks as gaussian processes. In ICLR, 2018.
Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, and Yanbo Gao. Independently recurrent neural network (indmn): Building longer and deeper RNN. In CVPR, 2018
Shuai Li, Wanqing Li, Chris Cook, Yanbo Gao, and Ce Zhu. Deep independently recurrent neural network (indmn). arXiv preprint arXiv: 190000690 2019.
Zachary C. Lipton. The mythos of model interpretability. Commun. ACM, 61(())33611 September 2018.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
10

---

Under review as a conference paper at ICLR 2021
Song Mei, Theodor Misiakiewiez, and Andrea Montanari. Mean-field theory of two-layers neural networks: dimension- free bounds and kernel limit. Proceedings of Machine Learning Research, (99), 2019.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Belle- mare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonogiou, Helen King, Dharshan Kumaran, Daan Wier- stra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning- Nature, 518(7540))529- 533, February 2015.
Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
Radford M. Neal. Connectionist learning of belief networks. AI, (1056660011-111 July 1992.
Radford M. Neal. Bayesian Learning for Neural Networks. Springer-Verlag 1996.
Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In NIPS, 2016.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS- W, 2017.
Hoifung Poon and Pedro Domingos. Sum-product networks: A new deep architecture. In UAI, 2011.
Ruslan Salakhutdinov and Geoffrey Hinton. An efficient learning procedure for deep boltzmann machines. Neural Comput.. 24(8):1677-2066, August 2012.
Yuesong Shen, Tao Wu, Csaba Domokos, and Daniel Cremers. Probabilistic discriminative learning with layered graphical models. CoRR, his/1992.00057, 2019.
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George den Driessche, Julian Schrittwieser, loannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587::8444889 January 2016.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. JMLR, 15(56): 2014.
C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, 2015.
Charlie Tang and Russ R Salakhutdinov. Learning stochastic feedforward neural networks. In Advances in Neural Information Processing Systems, pp. 530-538, 2013.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. NIPS, 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmark- ing machine learning algorithms. CoRR, abs/1708.07747, 2017.
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In ECCV, 2014.
Shuai Zheng, Sadeep Jayasumana, Bernardino Comera-Paredess Vibhav Vineet, Zhizhong Su, Da- long Du, Chang Huang, and Philip H. S. Torr. Conditional random fields as recurrent neural networks. In ICCV, 2015.
11

---

Under review as a conference paper at ICLR 2021
A GLOBALLY DIRECTED STRUCTURE OF LAYERED CHAIN GRAPH
With the absence of (undirected) intra-layer connection the chain components P( Pa(X))) the layered chain graph defined in Definition [ has globally directed structure. This means that cquivalently it can also be modeled by a directed graphical model (Bayesian network) which admits the same nodewise decomposition
L IIII P(X 1-
P(X2
(10)
and whose factorized nodewise conditional distributions P(X: are additionally modeled by pairwise CRFs:
=
(X T (X**))
(1I)
The expressions Eq. Top. are identical to Eq. II. 2 which shows the equivalence. This is rather straightforward result as directed graphical model is just special case of chain graph. We employ the more general chain graph modeling this paper out of two concerns:
1. Neural network designs rely heavily on the notion of layers. The layered chain graph formulation provides us with the notion of chain component that can correspond exactly neural network layer. This is missing in directed graphical model formulation:
As we have discussed in Section 2.3 using the layered chain graph formulation allows for a straightforward extension from the classical neural network case (layered chain graph) to the more complex case with connecedd layers that corresponds to general chain graphs with non-bipartite CRF chain components, which can be useful for, e.g., structured prediction tasks.
B PROOFS
B.
PROOF OF PROPOSITION
The main idea behind the proof is that for linear function, its expectation can moved inside and directly applied on its arguments. With this in mind let's start the actual deductions:
To obtain Eq. (6) we start from Eqs.
5 and (
(x .Bgr.....ue P( Pa(()))
(12)
= BEqm.....u [/ (T' (x)), (
TT". (XP))))) )))
(13)
Now, make the assumption that the following mapping approximately linear:
(V1...., v..) + f (T!()) v((v),...,,,,)), This allows move the expectation inside, resulting in (racall the definition of Eq.
(14)
((||(²)) ~ of (X.)) e (Equ (XP)), Eqr, Th (XP1) Px )])
(15)
f' ( (((a") ."*.
(16)
To obtain Eq. summations):
we go through a similar procedure (for discrete RVs we replace integrations by
From Eqs.
and
we have
q
E
T (X!)
(17)
/ T'
(18)
mf
[ T Te (X*)) Tpa (x |dy
Q
(19)
EEq.....n 1/ T'a))) (T )...,... ... (X do
(20)
12

---

Under review as a conference paper at ICLR 2021
Defining the activation function g' (thus ef corresponds to the preactivation)
g'(v) = / Tz)) (2.), v)de!,
(21)
then have
q = Egrr..... [g' (c) (TP) (X")) T (XP"))))
(22)
Again, we make another assumption that the following mapping is approximately linear:
(V1...., vn. ト g' (e) Vn)))
(23)
This leads to the following approximation in similar fashion:
q g (E@o) [T (X""), ..... Eq T (XPr)
(24)
g (q ....,.-)))
(25)
B.2 PROOF (AND OTHER DETAILS) OF COROLLARY 2
Let's consider node X connected by parent layers X ..... XPx, To lessen the notations use the shorthands et for e/ (T (XP1)..... Thx T (X"*)) and oi for c! ..m)...)....
1. For the binary case we have
T'X:) = X {),
(26)
PaX = (() Z(Pu(X!)) exp(X) et)
1
(27)
with the partition function
P(X ((Pa(X')) = exp(a ct) + exp(Be)
(28)
that makes sure P(X Pa(X)) )) is normalized. This means that, since x can either a or 3 equivalently write (a I + 1/(1 exp(-x)) denotes the sigmoid function)
( 8) et)
if =o((2r) 33) ct).
(29)
[a((3 - a) c)) of =
Using the feed-forward expression (Eq. 77), we have
of Rz
Σ
d (x) al(a 3) et) + 3 Bo(( a) E
(30)
AE(a.
- tanh 3 05-a
a * B
+
(31)
2
Especially,
When a 0, = 1, we have of R o
(32)
When -1 1, we have of R tanh(e)).
(33)
Furthermore, the roles of and 3 are interchangeable. For the multilabel case, let's assume that the node X can take one of the labels {1....,c} this case, we have an indicator feature function which outputs length-c feature vector
c}. In
T' (xx) (1x:=I......
(34)
This means that for any given label j E (I,...,c) c). T'()) is one-hot vector indicating the j-th position. Also, of and of will both be vectors of length e, and we denote e/ and ei their j-th entries. We have then
= P(X/)Pa(X')
e/) = Z(Pa(X!)) (x!(T'(X.) e
(35)
13

---

Under review as a conference paper at ICLR 2021
with the normalizer (i.e. partition function)
Z(Pa(X))) = Σ exp(era)
(36)
==1
This means that
Vi =
= (softmax(é........ ((,)),
(37)
and, using the feed-forward expression (Eq. (7)) we have
d ~ Σ (xT e) (x)) = softmax(e i...... ,,,),, T
(38)
softmax(e,11 ............................. cl,c),
(39)
i.c the expected features qf of the multi-labeled node X3 is a length-c vector that encodes the result of sofumax activation.
3. The analytical forms of the activation functions are quite complicated for rectified Gaussian nodes. Luckily, is straight-forward to sample from rectified Gaussian distributions (get Gaus- sian samples, then rectify), meaning that we can easily evaluate them numerically with sample averages. resulting visualization is displayed in Figure Right. Specifically:
The ReLU nonlinearity can be approximated reasonably well by rectified Gaussian node with no leak (c = 0) and fanh-modulane standard deviation (s² = tanh(e))), as shown by the red plot in Figure I Right;
Similar to the ReLUU case, the leaky ReLU nonlinearity can be approximated by a leaky (e # O) rectified Gaussian node with tanh-modulated standard deviation (sf te) See the
green plot in Figure Right which depict the case with leaky factor e = 1/3;
We discover that a rectified Gaussian node with no leak (c 0) and an appropriately constant standard deviation si can closely approximate the softplus nonlinearity (see the blue plot in Figure Right). We numerically evaluate s 1.776091849724477 to minimize the maximum pointwise approximation error.
Averaging over more samples would of course lead to more accurate (visually thinner) plots, however Figure Right we deliberately only average over 200 samples, because we also want to visualize their stochastic behaviors: the perceived thickness of a plot can provide hint to the output sample variance given the preactivation e/.
B.3 PROOF OF PROPOSITION 3
Given a refinement module (c.f. Definition 2 that augments a base submodule m from layer X to layer X using a refining submodule r from X to layer X denote g' the activation function corresponding to the distribution of nodes in layer X we assume that these two submodules alone would represent the following mappings during foed-forward
q' = g'"""" ()) (base submodule)
(40)
qp - g'e
(refining submodule)
where e and e" represent the output preactivations of the base submodule and the refining sub- module respectively. Then, given an input activation q'-1 from X the output preactivation of the overall refinement module should sum up contributions from both the main and the side branches fc.f. Eq. (3), meaning that the refinement module computes the output as
q' g'(e" (q'-1) + e"()))
(41)
with q' the output of the duplicated base submodule with shared weight, given by
q' = g (e
(42)
We have thus (Id denotes the identity function)
q' = (Id+e" te" g') (q
(43)
where the function Id +e" og describes a preactivation residual block that arises naturally from the refinement module structure.
14

---

Under review as a conference paper at ICLR 2021
B.4 PROOF OF PROPOSITION 4
To match the typical deep learning formulations and ease the derivation, we assume that the base layered chain graph has sequential structure, meaning that Pa' (X(.t) contains only X and we have
Pa x,xx"))) = P(X) x x( ef
Pa
(44)
(T'( ee
T' ).)
(45)
1. When the connection from X to X is dense, we have that for each i E [1....,N}}
N
N
('(x(-1)) -
Σ w
T
U T' (x
))+ b/- (46)
Ri
j=1
Thus the feed-forward update for layer at time is
N-1 w
N
Vi E (I,...,,N'}, N'), of g'(
Σ
11-1
+b)
(47)
which corresponds to the update of simple recurrent layer.
The assumption of intra-layer conditional independence through time means that we have
P( Pa' (x'+). x P(X x xx," xx!"'), )
(48)
which in terms of preactivation function means that for each E (1,...,N'),
(x ('( o/ (x' T (x!
(49)
N
Σ w + (x!" b/.
(50)
In this case the feed-forward update for layer I at time is
NI-1
Vi E (1.....N)), o = g'(
w W 1-11
U/q 111-1 + b!)
(51)
which corresponds to the update of an independently RNN layer (c.f. Eq. (2) of et 2018
B.5 PROOF OF PROPOSITION 5
Again, to match the typical deep learning formulations and ease the derivation, we assume that layered chain graph has sequential structure, meaning that X is only the parent layer of X With the introduction of the auxiliary Bernoulli RVs D the 1-th chain component represents
N
P(X x D')
II j=1
r (T+ o 44 (T'('))
(52)
with
NI-1
(T')) D') = Σ (w bb).
(53)
For feed-forward pass during training, we draw sample di for each Bernoulli RV D/. and the feed-forward update layer 7 becomes
Vj E (1 No+1), a ag g
Σ dw d +b,).
(54)
Since d: = 1 with probability p' and df = 0 with probability pl, each activation qj will be "dropped out" (i.e. d/g. 0) with probability pd (this affects all a, 4+1 simultaneously).
15

---

Under review as a conference paper at ICLR 2021
For feed-forward pass at test time, marginalize each Bernoulli RV DJ. Since have
vi (1....,N') N'), E[D p',
(55)
the feed-forward update for layer / + in this case becomes
N p W 1. +
Vj E fl...., N }. afth =g"
(56)
where we see the appearance of the constant scale p'.
C REMARK ON INPUT MODELING
technical detail that was not elaborated in the discussion of chain graph interpretation (Section 2 the input modeling: How encode an input data sample? Ordinarily, an input data sample is treated as a sample drawn from some distribution that represents the input. In our case however, since the feed-forward process only pass through feature expectations, we can also directly interpret an input data sample as a feature expectation, meaning as a resulting average rather than a single sample. Using this fact, [Shen et al. (2019) propose the "soft clamping" approach to encode real valued input taken from interval, such pixel intensity, simply an expected value of binary node which chooses between the interval boundary values.
This said, since only the conditional distribution P(X².... X X is modeled, our discriminative
setting actually do not require specifying an input distribution P(X.).
16


Reviews:
[{"review_id": "KC5znrpUtcX", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "###############################################################################\n\nSummary\n\nSummarize what the paper claims to contribute. Be positive and generous.\n\nThe authors propose an interpretation of feed-forward neural networks and recursive neural networks as chain graphs (CGs). They claim this new interpretation can provide novel theoretical support an insights to existing techniques, as well allow for the derivation of new approaches.\n\n###############################################################################\n\nPros and cons\n\nPros:\n - the text is well-written\n - the attempt at studying neural networks under a graphical probabilistic model perspective is praisable\n\nCons:\n - the chain graph interpretation put forward by the authors is superfluous, as in reality the definition found in the apper is that of DAGs with particular parametric constraints\n - most of the results presented by the authors do not rely on the CG interpretation, and can already be found in the litterature\n - the paper is missing a connexion to stochastic feedforward neural networks (SFNNs), to which the proposed interpretation is extremely similar\n\n###############################################################################\n\nRecommendation\n\nI recommend rejection of the paper, for two reasons. First, I believe most of the contributions proposed in the paper are not novel, as they can already be found in published form. See my detailed comments below. Second, the CG interpretation put forward by the authors is trivial and superfluous, since the authors in reality only consider CGs restricted to DAGs. As such, the whole claim of the paper that CGs can give a new, relevant perspective to NNs, is not credible.\n\n###############################################################################\n\nQuestions to authors\n\nI would like the authors to comment on the fact that their CG interpretation is indeed the LWF-CG interpretation, restricted to DAGs (singleton chain components). Furthermore, I would appreciate if the authors could relate their approach to the SFNN model, and what differs from that interpretation.\n\n###############################################################################\n\nDetailed comments:\n\np.1 §3: an efficient approximate probabilistic inference -> What is meant here by efficient ? Is it an unbiased estimator ? What is meant by inference ? Computing $p(y|x)$ ? $\\arg \\max_y p(y|x)$ ?\n\np.2 §2: The chain graph model -> There exists at least 4 chain graph interpretations. See\nDrton, Mathias. „Discrete Chain Graph Models“. In: Bernoulli 15 (Sept. 2009),\npp. 736–753.\nAmong those, there are two which subsume UGs and DAGs, while the two others subsume BGs and DAGs. If you follow the CG interpretation from Koller and Friedman then you assume NNs are LWF-CGs.\nI strongly suggest that you make it explicit which chain graph interpretation you follow, for the sake of clarity. These interpretations are not equivalent.\n\np.2 §2: there exists a series of works [...] -> I believe the list is much bigger than that. You forget very popular models which are direct instanciation of PGMs, such as VAEs, HMMs, LDAs, GMMs, CRFs, and all of their variants.\n\np.2 §4: an approximate probabilistic inference procedure -> What is meant by that ?\n\np.2 §4: provides additional insights -> such as ?\n\np.3 §2: layered chain graph -> A chain graph is always layered into chain components. The name \"layered chain graph\" is confusing, as it seems to imply that some chain graphs may not be layered. Since you are using the LWF-CG interpretation, you might point to the relevant papers where the graphical model was introduced, and simply re-use the establihsed LWF-CG name from the litterature, instead of inventing a new one.\nLauritzen, Steffen L. and Wermuth, N. „Graphical Models for Associations\nbetween Variables, some of which are Qualitative and some Quantitative“. In:\nThe Annals of Statistics 17.1 (Mar. 1989), pp. 31–57\nFrydenberg, M. (1990). The chain graph Markov property. Scand. J. Statist. 17 333–353. MR1096723\n\np.3 eq.1: This factorization is not that of a CG, but that of a DAG. Since you assume no undirected connection between variables in the same layer, then all chain component in your chain graph actually contain a single variable. What is the point of introducing the conecpt of CGs then, if you only consider CGs which are restricted to DAGs ?\n\np.3 eq.2: The LWF-CG interpretation, which you seem to follow since you refer to Koller and Friedman, does not imply this factorization for the graphs you consider. Since each chain component $K$ in your graph is a singleton, each $X_i$ and its parents form a clique in the closure graph $K$. As such, the CG structure does not imply any further factorization for p(x|pa(x))... It does seem your interpretation of NNs is not as general CGs, but as very specific DAGs with parametric constraints. Furthermore, you do not define what are $T$ and $f$ here.\n\np.3 Section 2.2: I believe your CG interpretation is simply a reformulation of stochastic neural networks, for which there already exists a great body of work. See, eg:\n\nEric Jang Shixiang Gu Ben Poole. Categorical Reparameterization with Gumbel-Softmax. ICLR (2017)\n\nYoshua Bengio, Nicholas Leonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013.\n\nYichuan Tang, Ruslan Salakhutdinov. Learning Stochastic Feedforward Neural Networks. NIPS 2013\n\np.4 Proposition 2.1: This result seems very limited, since it assumes linear layers, and in the end a linear NN.\n\np.4 Corollary 2: What are alpha and beta here ? How do they relate to $f$, $e$ or $T$ from Definition 1 ? What are $e_i^l$ and $s_i^l$ here ?\n\np.4 §6: the modularity of chain components justifies transfer learning via partial reuse of pre-trained networks -> I do not understand this argument. What is meant here by \"modularity\" ?\n\np.4 §7: However, feed-forward is no longer applicable through these intra-connected layers. -> You finally give an example here that corresponds to non-trivial chain graphs (with chain components greater than 1), and recognize that your theoretical results do not apply any more. This contradicts your claim that CGs offer a general interpretation of NNs with theoretical support. I believe the CG interpretation is not justified, nor required to derive the results you present in this paper.\n\np.5 Definition 2: I fail to see the point of introducing this new concept, which is a re-definition of residual blocks.\n\np.5 §4: While a vanilla layered [...] -> I fail to understand where the CG interpretation fits in this argument.\n\np.5 Proposition 4: This result seems trivial to me, and does not require the concept of a CG. Once you show that a linear layer followed by a specific activation can be interpreted as a parametric model for $p(\\textit{out}|\\textit{in})$, then any NN that unrolls as an acyclic directed graph can be interpreted as a probabilistic model.\n\np.6 §1: The simple recurrent layer, [...] -> Again, I do not see where the CG interpretation gives any insight here.\n\np.6 Section 3.3: This result, again, does not require the CG interpretation. It is also already know. See, e.g., Pierre Baldi and Peter J. Sadowski. Understanding Dropout. NIPS 2013\n\np.6 Section 4: It seems to me you are reinventing SFFNs. See, Yichuan Tang and Ruslan Salakhutdinov. Stochastic Feedforward Neural Network. NIPS 2013\n\np.7 §2: the sampling operation is not differentiable -> This is not true. See, e.g.,  the reparameterization trick for VAEs. You mention this in the next sentence...", "extra_scores": {}, "extra_text": {}}, {"review_id": "JkUt_OmaXIJ", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update: after reading the feedback and discussing with the other reviewers, I decide to keep my score unchanged.\n\nOriginal comments:\nIn this paper, the authors provide new interpretation of neural networks via chain graphs, which can be used as a new theoretical framework to understand the behavior of neural networks.\n\nPros.\n\n1. Although both PMG and Neural networks are based on graphs, very little research has been proposed to bring the two fields together. This paper provides an interesting result to bridge the two previously unrelated fields, which equips the community with useful insights to spark new research directions.\n\n2. This paper helps the community to fully utilize the existing works from PGM to solve current obstacles in deep learning.\n\n3. The open questions are also interesting. I would be particular interested in the third question.\n\nOverall, I think this is an interesting paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "9A3BfoaoUYc", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper tries to interpret neural networks with chain graphs that provides theoretical analysis on various neural network components. Furthermore, this chain graph interpretation has been used to propose a new approach (architecture), which is a partially collapsed feed-forward. A layered chain graph representation is adopted to formulate the neural networks with layered chain graphs. This further establishes to interpret feed-forward as an approximate probabilistic inference with using linear approximations. Some concrete examples are shown to be analyzed  based on the chain graph formulation. \n\nThe overall context (analysis) seems straightforward to interpret the neural networks with chain graphs, but it is hard to achieve some meaningful information from this new interpretation to improve the current neural network models in terms of learning procedure or optimization.  The proposed partially collapsed feed-forward is a good example to come up with a new approach based on the chain graph interpretation. However, in terms of performance and complexity, it is practically not showing impressive improvements compared to the baseline methods. Moreover, it seems quite similar to previous works as far as I remember and one similar work is 'stochastic feedforward neural networks'. I fully agree the future works (open questions) in the conclusion and discussion section that this work still needs more investigations although this paper is a good initiative work.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "0qta0j1HReC", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors look for ways to frame neural networks in terms of probabilistic models. This is a worthy goal and I believe the authors should pursue this path with enthusiasm. The theme is relevant, however the novel ideas that are in the paper seem to have relatively small significance. The translation from neural networks to chain graphs is not surprising; actually if I understand the translation we basically get a Bayesian network out of the directed acyclic graph encoding the connections in a neural network (I must say that the translation could be presented in a more didactic fashion. In any case, given this translation, the authors are able to frame some techniques from neural networks as techniques from chain graphs. But this feels too forced a translation, because neural networks do have a natural interpretation and usage as function approximators; why should one go all this ways to find a probabilistic translation that explains a few well known things, and opens the possibility of some probabilistic algorithms that could anyway be derived in the context of neural networks? One example: given the proposed translation, it is obvious that sequential models will appear as dynamic chain graphs, why is this useful in any way? \n\nOn top of this, I find it troublesome that the authors \"prove\" results by saying things like \"approximately linear\" and \"approximately\" this and that. How can these results be proven given this level of informal justification. I really think this should be fixed before publication. \n\nSome sentences are a bit confusing. In Page 3, for instance, the authors say that a model with P(X|Pa(X)) is modeled by a CRF; usually a CRF models a discriminative model, is this the case here? What exactly is going on? Also the last paragraph of Section 3.2 is very hard to parse (the main point there is not clear at all). ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors draw connections between probabilistic graphical models (specifically LWF chain graphs) and neural network models. There was general agreement amongst the reviewers that this is an interesting topic that merits further study, and would be of interest to the ICLR audience. At the same time, all of the reviewers have read the author response and there is a consensus that the novelty and significance of this work is limited. The connections between CGs and NNs are somewhat standard and well-known, and the significance of the results has not been convincingly demonstrated.

Decision: Reject