White Box Network: Obtaining a right composition ordering of functions
Authors: Eun saem Lee
Hyung Ju Hwang
Venue:  (2020)
Keywords: white box
black box
function composition
neural network
ordering functions
reverse engineering
programmable logic controller
plc
white box network
WBN

Abstract: Neural networks have significantly benefitted real-world tasks. The universality of a neural network enables the approximation of any type of continuous functions. However, a neural network is regarded as a non-interpretable black box model, and this is fatal to reverse engineering as the main goal of reverse engineering is to reveal the structure or design of a target function instead of approximating it. Therefore, we propose a new type of a function constructing network, called the white box network. This network arranges function blocks to construct a target function to reveal its design. The network uses discretized layers, thus rendering the model interpretable without disordering the function blocks. Additionally, we introduce an end-to-end PathNet structure through this discretization by considering the function blocks as neural networks

Under review as a conference paper at ICLR 2020
WHITE Box NETWORK: OBTAINING A RIGHT COMPOSITION ORDERING OF FUNCTIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Neural networks have significantly benefitted real-world tasks. The universality of a neural network enables the approximation of any type of continuous functions. However, neural network is regarded as non-interpretable black box model, and this is fatal to reverse engineering as the main goal of reverse engineering is to reveal the structure or design of a target function instead of approximating it. Therefore, we propose a new type of a function constructing network, called the white box network. This network arranges function blocks to construct target function to reveal its design. The network uses discretized layers, thus rendering the model interpretable without disordering the function blocks. Additionally, we introduce an end-to-end PathNet structure through this discretization by consider- ing the function blocks as neural networks
INTRODUCTION
Reverse engineering can be defined as the deconstruction of an object to reveal its design or ar- chitecture, to extract knowledge from the object [Eldad, 2005]. Typically, in machine learning, obtaining the internal functions a device only with its input and output values a regression problem. Before machine learning developed, devices were disassembled to amalyze them. However, advances in deep learning have allowed us to approximate various functions in regression tasks; therefore, we now solve this regression problem without actually looking inside the ob- ject.
While this development in deep learning (Bengio 2009 Bengio al. 2013) has benefitted reverse engineering significantly, some limitations still exist. First, deep learning is black box model; therefore, the results may be interpreted. In this case, information regarding the object such as what it does and how it works may not be extracted. This causes field engineers to not entirely trust the results of deep learning even the correct values were reflected. Next, deep learning ap- proximates the target function and does not reveal the actual design and architecture of the device. artificial neural network mathematical simulation of a neural network composed of human neurons and synapses that simulate the manner in which the human brain recognizes patterns. How- ever, a device typically comprises an electronic circuit and block functions; therefore, may not fit perfectly with the neural network structure, which cause large errors when certain data are used. We herein propose a white box network (WBN) with a selection layer, specially designed for reverse engineering. This network continuously composites function blocks create target function. We assume that all types of function blocks that can constitute the target function are provided in ad- vance. This differs from the normal neural network because the WBN reveals the exact functions with the correct inputs and their ordering to construct the target function, instead of merely ap- proximating them. Our WBN is different from normal neural networks as the model architecture is discretized using a discretized matrix called the selection layer. Because discretization is inter- pretable (Chen al. 2016), our WBN can reconstruct the target function.
The WBN is motivated by the questions, Can we benefit from knowing the functions that are used in certain fields?. For example, in a programmable logic controller (PLC), target function can be obtained through ladder logic diagram comprising well-known function blocks. Therefore, can obtain a target function by ordering those function blocks with the correct inputs. As such, this study, create WBN that can automate reverse engineering. For the experiment, imitated the PLC data and verified whether our model could obtain their ladder logic diagram. Additionally, conducted other experiments validate the selection layer. We assumed that

---

Under review as a conference paper at ICLR 2020
may not know the functions used in certain field. Therefore, regarded the given function blocks as neural networks. Subsequently, the network was ordered and connected to the appropriate neural networks to obtain the target function that resembles PathNet (Fernando et al., 2017). Using this concept, we create an end-to-end PathNet structure and perform experiments to verify whether it functions like PathNet.
2 RELATED WORKS
In machine learning, regression a task of obtaining a real-valued function y = f(x) from the hypothesis set (X, Y). However, typically, actual data may contain noise or exhibit data loss owing measurement limitations other causes. Therefore, regression is often considered process and exact function may not be obtained. For example, in Gaussian process regression [Williams and Ras-mussen, 2006] and support vector regression (SVR) (Smola & Schölkopt 2004), reproduced kernel and obtained approximated value instead of obtaining function. An- other typical regression method is using a multilayered neural network (Bengio 2009 Bengio et al., 2013), which approximates function by using the human neuron model.
black box
The aim of reverse engineering is to obtain the exact structure of a function rather than merely ap- proximating it. In some fields, we may know the type of function forms that typically appear: hence, we can use that information to construct a function. An equation learner (EQL) (Martius & Lampert 2017 [ahoo al. 2018) is designed for regression task that can be described by analytical expres- sions of mechanical systems such as pendulum or robotic arm. An EQL has been shown more effective at solving dynamics than conventional neural nets; additionally. reduces extrapolation errors. Furthermore, PDE-net (Long et al. 2018b a) accurately predicts the dynamics of complex systems and uncovers the underlying hidden partial differential equation models using a convolution kernel filter. This type of model renders regression or processes that appear in a specific field are known.
effective if certain types of function blocks
A discrete variable be used variously, e.g., learning probabilistic latent representations (Kingma jt al. 2014), image region (Xu et al. 2015), and memory access (Graves et al. 2014 2016). Espe- cially for the memory access, it resembles attention mechanism that can help interpret the focus of the model. The end-to-end memory network (Kingma et al., 2014) uses softmax to access the memory in questions and answering task. This allows refers to when it answers a specific question. The neural Turing machine (NTM) (Graves et al.) 2014) and differentiable neural computer (Graves et al.) (2066) use sharpening method instead of merely using softmax. As such, those models select information from the memory without disor- dering information, thus rendering them more interpretable. Therefore, we can determine how the model functions. For example, we can reconstruct the Copy processes of the NTM and create pseudocode by referring to the headers in the model. Therefore, discretization can be used such that the model is more interpretable (Chen et al. 2016), and it is sometimes more efficient for comput- ing (Rae et al. 2016).
determine the sentence that the network
PathNet (Fernando et al. 2017) is a modular deep neural network with pathways. It uses genetic algorithm to obtain the best pathway for learning. The pathways are represented as integer-valued matrices; therefore, the modules can be selected by accessing those matrices. PathNet reuses the pathways for different task as a transfer learning paradigm and shows the potential in selecting the appropriate modules to obtain the target results.
METHODS
3.1
WHITE BOX NETWORK ARCHITECTURE
The WBN is multilayered feed forward network. For each hidden layer, it consists selection layer followed by function block mappings. In this study, we assumed that all function blocks constituting the target function were known before we built the model. However, the exact order of the function blocks with the correct inputs must be obtained. Each function block can be multivalued with multiple inputs; however, for simplicity, we assume that the function blocks are unitary and binary operations.
Suppose that the model is L-layered, and we have m(l) unitary functions and r(l) binary functions in the Ith layer. In the selection layer, select inputs enter into each function block. The
2

---

Under review as a conference paper at ICLR 2020
y(
y(2)
2
z(1
input
output
Selection Layer
Selection Layer
Selection Layer
Figure 1: Architecture of white box network for two layers (T. 2) with each layer containing two unitary operations f2) and two binary operations {gI 92 as its function blocks.
selection layer can be written
20) Wiy
Here, y (1-1) is the output of the previous layer and W is the selection matrix. each row of ((( has only one and the remaining Os, then, it selects one element the (1-1) as an element of (1), To make (1 differentiable, we approximate (! by
W softmax (W()) /T, axis=-1)
Here, 7 > is temperature parameter that renders V sharper as + can be chosen as both constant and a learnable parameter.
After multiplying the selection matrix, z goes through the function block mappings. Let 1 : m()) and gi :-..-99(1) be the given unitary and binary function blocks the Ith layer. Then, our new output (t) is
(!)
3 (1) (=1)(=
(n (m))))
), 91 (1) (fm()++1 =m())))) 9x(1)) (= ( (n(1))2n(()-111 (m(()+=+))())
As such, when we select the function blocks of each layer, the dimensions of the variables are fixed. For example, (") represents (m(!) + (((())+(()))dmmma)) (ef())ddimesiionll vectors and (1) represents (m(l) + 2n(())- dimensional vectors.
Finally, we convert the last layer's output to fit the target output dimension. Therefore, we use the selection layer again to match with the output dimension.
W
Here, the final output a composition of function blocks from each layer with selective inputs.
3.2 END-TO-END PATHNET ARCHITECTURE
Suppose do have any information regarding the target function. In that case, we can select our function blocks as a single-layered neural network followed by nonlinear activation. Then, our model becomes modular neural network that selects the right modules find the best pathway to obtain the target value. This concept is the same as that of previous study, i.c., PathNet (Fernando al. (017).
To apply one key component of our WBN to existing PathNet architecture, need to switch the or- der of the selection layer and the function block mapping for each hidden layer. Therefore, function block mapping is proceeded by the selection layer. In function block mapping, the function blocks are one-layered neural networks with learnable parameters. Each them is assigned a single input and yields single output, which is typically multidimensional.
Now, suppose that our model is a modular deep neural network with L layers with each layer con- taining M neural network modules. Let N be the number of maximum distinct modules per layer

---

Under review as a conference paper at ICLR 2020
P1
input
yy
@
yx88)
cutput
Selection Layer
Salection Layer
Linear boyer
Reduce Sum
Raclue
+s
Pg
iPo
Figure 2: Architecture of end-to-end PathNet comprising two-layer (L. network with six modules (M = 6) in each layer. The number of maximum distinct modules per layer that are permitted in the pathway is three (NV 3), and the final layer comprises fully connected linear layer. The activated path and modules are colored in red.
that can be selected pathway.
Let " @ be neural network modules in the /th layer and let A(()) be the dimension( (imener number of channels) of hidden output variables. Then, the function mapping layer is written as
0 = (0) (y ()) 02 (y (1 ()
(y
))
Here, z is M by 100 matrix. Then, selection layer selects the neural network outputs
{0 (! (()
The selection matrix ( N x M with each row of W containing only one almost and the remaining almost Os; (1) can be approximated in the manner described previously. Now, we add all rows of (1) such that the hidden output (d) follows the existing PathNet.
( reduce_ sum ( (1) axis=0)
thhee a linear layer that can be used to convert the last hidden layer to the output value.
W (4) +
This architecture is inspired by the PathNet in which selection matrix instead of genetic algorithm is used to select pathway. It is differentiable, and hence we can easily train the network by gradient descent.
3.3 TRAINING DETAILS
We used the stochastic gradient descent algorithm with bini-batches and Adam (Kingma & Ba 2015) to update the parameters. The free parameter of the WBN is 0 (W(1) W or possibly 0 = (W111) ...., W (11) ...., -(4).. Furthermore, we used the mean square error loss function for the regression task and an Jn regularization term for (1)sss
[D]
Loss (D)
Σ [(ő(z;)
vill + Σ (
D
which is Lasso-like (Tbsshirani 1996). For the WBN, L1 regularization is highly important in optimization. Without the regularization term, the absolute value of elements of ( can become extremely large. This causes (i((1) to be extremely sparse and sharp, and it can be obstacle for shifting the function blocks from one ordering to another. As expected, the model typically fails to obtain the exact ordering of function blocks when set 0. Hence, the regularization parameter a should be set to more than O. Typically, we set a E (0, 5] at the beginning of training to prevent the problem.
4

---

Under review as a conference paper at ICLR 2020
input signal output signa
Figure 3: Three ladder logic diagrams (LLDs) for the experiment.
However, another problem occurs with the scale of a. We typically set a be more than the beginning; however, after the model has discovered the exact ordering, W cannot be sharpened enough because of the large regularization restriction. Therefore, we change the regularization pa- rameter a during training time; we set a = 0 after enough learning time. Two methods exist for O: one is by threshold and the other is by iteration number. For
selecting when to the former, if the MSE is smaller than a certain threshold (typically 0.001), we set a = For the latter, if the iteration reaches certain number (typically t T/2, where T is the total number of iterations), we set a = Occasionally, used the temperature learning strategy such that the temperature of each layer can be learned during the optimization. We used 7(> 0) because we wished to sharpen our selection matrix W however if r is extremely small at the early stage of training, the model may be pre- vented from selecting the exact order of function blocks. Because we want our т to approach zero as learning progresses,
#00
7(0
TO + softplus(7(1)
exp(ro + softplus(r()))
w = softmax (1()) 700,, axis=-1)
where To is constant determining upper bound of f.
4 EXPRIMENTS
4.1 LADDER LOGIC DIAGRAM
The ladder logic diagram (LLD) is a programming language that represents a program by graphical diagram based the circuit diagrams of relay logic hardware. It is especially used in developing software for PLCs. Typically, PLC device black boxed before it is released as that the core part of the technology. In fact, an attempt to model a PLC using recurrent neural networks has been reported (Abdelhameed & Darabi 2005). However, the tracking and analysis of the learning process were inadequate; theretore, the internal structure of the controller could be determined. Our white box model is designed to reveal this internal structure, and our goal in this experiment is restore the logic diagram through reverse engineering using the input and output signals of PL.C device.
Three LLDs were used in our experiment, as shown in Figure These LLDs range from simple to complex. The first one is a single output problem and the last two are multiple output problems. Furthermore, in the last two diagrams, the output signals become input signals that create other outputs; therefore, the outputs are highly related to each other. The maximum number of inputs required was six, but set the input signals as eight dimensional validate whether the network could find the LLDs even if nonmeaningful input values existed.
The inputs and outputs Boolean numbers, which are either 0 or 1. Therefore, the function
5

---

Under review as a conference paper at ICLR 2020
Table 1: Mean squared errors for ladder logic diagrams
LLDI
LLD2
LLD3
WBN ANN
(7.404 = 0.166) 10-
(2.425 = 0.160)
10
(3.312 = 0.108) 10
(2.729 + 0.287) x 10-6
(7.422 = 0.280) X 10
(9.619 + 0.216) X 10
SVR
1.181 x 10-2
1.861 x 10
7.123 x 10
HDDI sample31
sampled1
(a)
I101 samplet?
1
(b)
(c)
Figure 4: Sample structures for three LLDs suggested by WBN. (a) LLDI sample #1 constructed by WBN (b) LLDI sample #2 constructed by WBN (c) LLD2 sample #1 constructed by WBN
blocks used in LLDs are nondifferentiable Boolean discrete functions. To solve this problem, we continuously extended the function blocks such that gradient descent could be applied. The four function blocks used in this experiment are as follows.
IDENTITY (a:)
NOT(x) = x
AND(E, 5) = F X OR(x, 1) x
x x y
The first two are unitary operations and the last two are binary. Therefore, the number of unitary 2. We used these functions multiple
functions m = 2 and the number of binary functions times by duplicating them. Henee, the number of functions was set to m + E {12, 16, 20} and the number of layers L {3, 4, 5, 6) was selected in our experiment. The initial L1 regularization parameter a was set to a E (0.1, 1,5).
We prepared multilayered artificial neural network (ANN) and support vector regression (SVR) to compare with our model. For the ANN, the number of layers were set as that of our model and the number of hidden units l (5, 10, 20) per layer. For SVR, used radial basis function kernel of width 7y E (0.05, 0.1, 1.0} and set two hyperparameters C E (0.01, 0.1, 1, 10, 100) meaningful as our model not only reduces the error, but also obtains the actual structure of the target function, it clearly shows how our model effectively reduces the loss after obtaining the structure of the target function.
and E (0.01, 0.1, 1}. Although comparing losses
Three samples of LLDs suggested by our model are shown in Figure 4 (a) a sample from LLD1, which is equivalent to LLDI. Two equivalent LLDs imply that, for every possible input signal, the corresponding output signals from two L..Ds are the same. Therefore, we verified that the WBN may not suggest the exact diagram but equivalent diagram (it sometimes produces the exact LLD). (b) another sample of LLDI that also equivalent to LLDI. We discovered that we set the number of layers larger than the required number, the WBN will likely generate inefficient LLD. (b) is sample of LLD2; is equivalent to LLD2. Both 1/1 and 3/2 cannot be obtained efficiently using this diagram. However, it is clear that y/s is of simpler form than the original LLD2; additionally, it is equivalent to real 3/3 in LLD2 because NOT(y2) and Φ1 cannot occur simultancously. We included more samples of I. LLDss including samples from LLD33 in the Appendix. See Figure

---

Under review as a conference paper at ICLR 2020
binary
binary
Without transfer (Pedi)
60
Without transfer (PeeS)
After transfer (Tvi3 Poodh
After transfer (4us7 (
Frequency
Frequency
-0
20-
20
4b
80
100
20
40
100
Generations achieve 0.998 accuracy
Generations achieve 0.998 accuracy
(b)
Figure 5: Binary MNIST classification results. (a) transfer result for Ivs3 - 7vs8. Total speed-up
radio is 1.18. (b) transfer result for 4vs7. 2vs5. Total speed-up radio is 1.13
4.2 BINARY MNIST CLASSIFICATION
We performed experiments to verify the end-to-end PathNet structure. We used the selection matrix instead of the genetic algorithm, which is different from the original PathNet. Experiments were conducted to determine whether the selection matrix could select the appropriate path. We per- formed the experiments using the transfer learning paradigm suggested in the original PathNet and observed positive transfer by using our network.
binary MNIST classification is a task distinguishing other. In our experiment, we conducted two tasks for the transfer learning paradigm. First, we trained binary classification task until we obtained 0.998 training accuracy. After we achieved the accuracy, we set the best-fit pathway, that is, the parameters within the pathway were no longer allowed to change. Subsequently, we initialized the remaining parameters including the last linear layer and trained another binary classification task B until obtained 0.998 training accuracy. In this experiment, we adopted the original PathNet structure to construct our model. The end-to- end PathNet structure comprises L 3 layers; each layer contains M 10 linear units with 20 neurons each followed by rectified linear units. We set modules permissible in pathway for each layer.
classes of MNIST digits from each
as the maximum number of distinct
We created two different datasets for transfer learning. One is for transferring from classifying and classifying 7 and 8 (Ivs3 to 7vs8). The other is for transferring from elassifying 4 and 7 to classifying 2 and 5 (4vs7 to 2vs5). We created these datasets because, in our opinion, transferring 1vs3 to 7vs8 is more reasonable than transferring 4vs7 to 2vs5 because I, are numbers with only straight lines, whereas 2, 5, 8 are numbers with curved lines. We expect that the model that is learned to classify and can classify 7 and easily after transfer. The experiment was performed 500 times for each dataset to analyze the statistics.
Figure 5 shows the results positive transfer. For classifying 7 and the number generations to obtain a 0.998 training accuracy was 49.728 and 42.112 on average before and after transfer, respectively. The speed-up ratio was 1.18 for this dataset. For classifying 2 and 5, the number of generations to obtain 0.998 training accuracy transfer, respectively. The speed-up ratio was 1.13 for this dataset. We discovered that the learning speed of the task enhanced for both datasets. Furthermore, is clear that the transfer from v to 7vs8 is better than that the other by comparing the speed-up ratios. This implies that the selection matrix successfully discovered the best way pathway transfer. The loss decreases quickly the early stage of the training for the post-transferred task compared with the nontransferred task. See Figure Din in appendix.
55.324 and 48.808 average before and after
4.3 CIFAR CLASSIHICATION
The CIFARIO 10 dataset contains images from 10 different classes, We performed an experiment using this dataset to determine whether our network could be used for classification problems. In the original PathNet study, a similar experiment
performed, in which information was transferred between CIFAR and SVHN datasets. One is for the classification of 10 different objects and the other that of 10 different digits. In opinion, using dataset not suitable for transfer learning as

---

Under review as a conference paper at ICLR 2020
they need to focus different items when classifying them. Therefore, divided the CIFAR dataset into two classes: class 1, which comprises data from airplanes, automobiles, birds, cats, and deer; and class 2, which comprises data from dogs, frogs, horses, ships, and trucks.
In our experiment, we used a transfer paradigm similar to that the previous experiment. However, trained the task according to fixed iteration number instead of obtaining certain accuracy. We trained one class 5000 times and then fixed the best-fit pathway. Subsequently, we initialized the remaining parameters and trained the other class 5000 times to verify the accuracy with their test sets.
The end-to-ent PathNet structure we used comprises L 3 layers: each layer contains M 16 linear units with 20 neurons each followed by rectified linear units. We set N 4 as the maximum number of distinct modules permissible a pathway for each layer.
CIARR (Class
(Class
160
140
140
DansterCCL*CCu
Frequency
100
Frequency
100
40-
15
16
Accuracy(%6 of test set of 500
Accuracy(%) of test set of 500
Figure 6: CIFAR classification results. (a) Transfer results for class 1. On average, test accuracy increased from 51.28% to 52. 52.42% after transfer. (b) Transfer results for class On average, test accuracy increased from 58.50% to 59.73% after transfer.
Figure F shows the results of positive transfer in the experiment. We conducted the experiment 500 times for each task and obtained the test accuracy. The test accuracy enhanced after a transfer was performed for both classes. For the first class, the total accuracy increased from 51.28% on average; for the second class, the total accuracy increased from 58.50 58.50% to 59.73% on average. Furthermore, we verified that the task was learned quickly the early stage of the training for the post-transferred task compared with the nontransferred task. See Figure (a), (b) in appendix.
52.42%
5 CONCLUSION
We presented a new model called the WBN, which obtains the exact order and correct inputs of function blocks to compose them for constructing target functions. It is especially suitable for re- verse engineering, e.g., for revealing PLC devices. It is different from other neural networks as it not only approximates a target function, but also constructs and reveals its structure. The WBN is differentiable with discretized variable-named selection matrix; therefore, gradient descent can be used for its training. We used the L₁ regularization with a variable parameter temperature learn- ing during training to optimize our network.
Using this network, we discovered that it could reveal LLDs in devices, e.g., logic devices such as PLCs. This network provides the target logic, which may be helpful in auto-reverse engineering for reconstructing PLC-like devices. A WBN provides an equivalent logic rather than an exact one. Furthermore, we discovered that if the layer was too large, then a WBN may yield an inefficient logic. This will be investigated in future studies.
Additionally, we created another structure to verify the efficacy of the selection layer. Assuming that the function blocks were not known, we created an end-to-end PathNet structure using the selection layers. We performed few experiments using this network and the transfer learning paradigm; consequently, positive transfer results were shown. This demonstrated that the selection layer could successfully select networks or functions during training.
However, the hyperparameters and their scales must be investigated. The efficient initial regulariza- tion parameter and temperature based on the data scale remain unclear. This will be investigated in future studies as it is important topic.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Magdy M. Abdelhameed and Houshang Darabi. Diagnosis and debugging of programmable logic controller control programs by neural networks. In IEEE International Conference on Automation Science and Engineering, CASE 2005, Edmonton, Alberta, Canada, August 1-2, 2005, pp. 313- 318, 2005.
Yoshua Bengio. Learning deep architectures for Al. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.
Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8))1798-1888))) 2013.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems 29: Annual Conference Neural Informa- tion Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 2172-2180, 2016.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. CoRR, abs/1701.0873. 2017. URL http:://rxixl wrg//bs//700
08734
Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, abs/1410.5401, 2014.
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwinska, Sergio Gomez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomênech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. Hybrid computing using neural network with dynamic external memory. Nature, 538(7626): 471-476, 2016.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Diederik Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi- supervised learning with deep generative models. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pp. 3581-3589 2014.
Zichao Long, Yiping Lu, and Bin Dong. Pde-net 2.0: Learning pdes from data with A numeric- symbolic hybrid deep network. CoRR, abs/18 hts/18112 142.0422, 2018a. URL /att:://aaxxia org/ abs/181 22..2.45 04426
Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. Pde-net: Learning pdes from data. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stock- holmsmässan, Stockholm, Sweden, July 10-15, 2018, pp. 3214-3222, 2018b.
Georg Martius and Christoph H. Lampert. Extrapolation and learning equations. In 5th Interna- tional Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings, 2017.
Jack W. Rae, Jonathan J. Hunt, Ivo Danihelka, Timothy Harley, Andrew W. Senior, Gregory Wayne, Alex Graves, and Tim Lillicrap. Scaling nemory-augmente neural networks with sparse reads and writes. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spuin, pp. 3621- 3629, 2016.
Subham S. Sahoo, Christoph H. Lampert, and Georg Martius. Learning equations for extrapolation and control. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmüssann Stockholm, Sweden, July 10-15, 2018, pp. 4439-4441 2018

---

Under review as a conference paper at ICLR 2020
Alexander J. Smola and Bernhard Schölkopf. A tutorial support vector regression. Statistics and Computing, 14(3):19--22 222, 2004.
R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society (Series B), 58:267-288, 1996.
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the 32nd International Conference Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, pp. 2048-2057, 2015.
10

---

Under review as a conference paper at ICLR 2020
A
APPENDIX
LLD1 samplet
LLD2 sample#2
LLD1 samplees
(c)
LLD3 samole#1
LLD3 sample#2
(d)
(e)
Figure 7: Additional sample structures for three L..Ds suggested by WBN. (a), (b) are from LLD1, (c) is from LLD2 and (d), (c) are from LLD3.
Equatiion WBW 1.
y1
( (x4 x6)) (x6xx1)) 0x3r (XLxx213) x213)
y1 x
(x1xx))
y2 (03) x21v (X1, x4)
y3
(x4xx4))> (1X3: X21 (clxx)))
Expected Equation by MEN 15
1044.300 1V (1))
(X22 (XC3) X22
V3 (20d xi), (K5: X51) (1)) 1- (xc. XD0 1)3
14 (O0s X6), <s (X3 )1: & (11X6)) (>x5))s (13, X2)), (34-X11013
Figure 8: Sample target functions for three LLDs suggested by WBN, (a) sample equation for LLD1 larget function. (b) sample equation for LLD2 target function. (c) sample equation for LLD3 target function
11

---

Under review as a conference paper at ICLR 2020
Test accuracy
Accuracy(%)
Without transfer(PvsB)
After transfertIve3- =vvs)
Iteration number
Figure 9: Learning curve for binary MNSIT classification (Ivs3 - 7vs8). We conduct experiments 10 times to get the learning curves.
accuracy
Accuracy(%o)
Wehoout transfer (C3)
After
1000
2000
3000
4000
5000
Iteration number
accuracy
Accuracy(%))
Without transfer (C2) After (CL-xC2)
0
1000
2000
3000
4000
5000
Iteration number
(b)
Figure 10: Learning curve for CIFAR classification. (a) learning curve for class 2. (b) learning curve for class 1. We conduct experiments 10 times to get the learning curves.
12


Reviews:
[{"review_id": "SyliaCJBYr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper investigates the question of identifying concise equations from data to understand the functional relations. In particular, a set of base functions are given in hand and the goal is to obtain the right composition of these functions which fits the target function. The main contribution of the paper is to introduce a selection layer, which enhances sparse connections in the network. Several experiments are conducted to show the effectiveness of the method. \n\nMy main concern of the paper is about the novelty and the lack of comparison of existing methods. The framework of finding functional relations is set up in [1,2], the main contribution of the paper is a refine architecture with the introduction of the selection layer. However, this selection layer is nothing but incorporating a softmax function. The idea of combining softmax functions in the hidden layers is not novel neither, which could be found in [3,4]. As a result, I find the contribution of the paper very limited, which could be summarized as applying an existing technique on a specific problem. Moreover, in the experimental section, there is a lack of comparison with existing methods such as EQL[1,2] and I consider it a major omission. \n\nOverall, due to the novelty concern and the lack of comparison, I do not support publication of the paper.\n\n[1] Sahoo et al.  Learning Equations for Extrapolation and Control\n[2] Martius et al. Extrapolation and learning equations\n[3] Graves et al.  Neural turing machines\n[4] Graves et al.  Hybrid computing using a neural network with dynamic external memory", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HkxJIK0TFS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "My understanding of this paper is that it proposes a combination of simple logical blocks that can efficiently learn logic rules implemented by a logic function. I think the authors define interpretability as the possibility to exactly express an unknown function in a composition of blocks, but despite several reading of this paper, I am not sure. A strong assumption in this paper seems to be that the target function of a supervised task can be exactly expressed via some compositional blocks. I would suggest a significant revision to clearly explain why WBNs are more interpretable and avoid any vague terminology.\n\nI will list below some of my concerns:\n\n- After reading several times this paper, I do not understand why this method is \"more interpretable\". The explanation of the papers are quite verbose. I tried to phrase my concern as this: could a standard CNN be more interpretable than this WBN because it simply uses linear operation? I'd like to see the reaction of the authors to this questions\n\nFor instance:\n\"This differs from the normal neural network because the WBN reveals the exact functions with the correct inputs and their ordering to construct the target function, instead of merely approximating them.\"\nIf the target function is precisely a cascade of linear operators and ReLU, then the objective of learning would be to recover exactly the linear operators and wouldn't consist in an approximation. Are the authors trying to tackle the nature of the objective functions? It is very unclear to me.\n\n\"It is different from other neural networks as it not only approximates a target function, but also constructs and reveals its structure.\"\nI do not understand why the structure is less opaque than in standard CNNs or how it is revealed.\n\nThe authors must clearly define the notion of interpretability in the text, in an explicit and simple manner. As an active researcher in this field, I believe that this is quite difficult because everybody has its own interpretation of interpretability. Here, I would suggest to significantly rephrase this.\n\n- I am quite confused in the notation.. For instance, sometimes \\hat W is used, sometimes W...\n\n- It is claimed that $\\ell^1$ minimisation will allow to avoid... sparse and sharp operators: \"This causes Wˆ (l) to be extremely sparse and sharp, and it can be an obstacle for shifting the function blocks from one ordering to another.\" This goes against my intuitions/knowledge, could the authors point me to a reference?\n\n- Table 1: Do the mean square errors indicate an exact learning? If yes, this should be commented. Also, the Table is not discussed in the text...\n\n- I do not understand why the CIFAR and MNIST experiments are relevant to this paper. Furthermore, the accuracy are very low.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "Bklz76psYH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents White Box Network (WBN), which allows for composing function blocks from a given set of functions to construct a target function. The main idea is to introduce a selection layer that only selects one element of the previous layer as an input to a function block. This allows for both introducing function priors as well as interpreting the learned function. The paper also presents a setting where each function block is a neural network that can be learned end-to-end using a PathNet style setting and shows positive transfer across MNIST and CIFAR classification tasks.\n\nThis presents an interesting technique to enforce learning composition of function blocks while learning the target function. This is important for both interpretability of the learned function as well as for introducing prior information about useful functions in a given domain. The extension to learnable functions (in the form of neural networks) for learning pathways for different tasks is also promising.\n\nThe biggest weakness of the paper is that it does not compare both theoretically as well as empirically with several closely related techniques such as RoutingNetworks[1], Modular Networks[2], Neural RAM[3], Compositional Recursive Learner[4], etc. (please find the references below). Routing Networks allow for selecting among a set of function blocks given some inputs. Module Networks similarly introduce modular layer that determines the appropriate modules given the inputs from the previous layers. Neural RAM learns to compose differentiable functions to learn a target function (similar to learning the LLD programs).\n\nIt would be good to describe the differences between the proposed approach in WBN and these approaches, as they all seem to propose a similar solution of learning target functions by learning to compose function blocks and reusing the learnt computations for transfer learning. It would also be important to empirically evaluate the related approaches to better understand the pros/cons of WBN compared to these approaches.\n\nWhat is the biggest size LLD programs that can be learned by WBNs? It would be interesting to evaluate the scalability of the approach to understand the limits of learning complex target functions.\n\nI was also curious if instead of providing the four pre-defined function block (Identity, not, and, or), what would the behavior be if they were all neural networks and also learnt in an end-to-end fashion somewhat similar to [4].\n\nFor the MNIST and CIFAR classification tasks, the function blocks are neural networks themselves. After training them using the PathNet like training, is the learned network more interpretable? It might be interesting to see if the selection layer and the function blocks learned some semantic information that might be easier to distill.\n\nFor a better comparison with Neural RAM, it might also be interesting to empirically evaluate the performance of WBNs on algorithm induction tasks such as the one used in [3].\n\n[1] Clemens Rosenbaum, Tim Klinger, Matthew Riemer. Routing Networks: Adaptive Selection of Non-Linear Functions for Multi-Task Learning. ICLR 2018\n\n[2] Louis Kirsch, Julius Kunze, David Barber. Modular Networks: Learning to Decompose Neural Computation. NeurIPS 2018\n\n[3] Karol Kurach, Marcin Andrychowicz, Ilya Sutskever. Neural Random-Access Machines. ICLR 2016\n\n[4] Michael Chang, Abhishek Gupta, Sergey Levine, Thomas Griffiths. Automatically Composing Representation Transformations as a Means for Generalization. ICLR 2019\n\nMinor:\n\npage1: ?. -> ?\npage 2: questions and answering --> question answering\npage 3: y^(l) represents (m(l)+n(l))-dimensional vectors --> should this be m(l) + 2*n(l)?", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents White Box Network (WBN), which allows for composing function blocks from a given set of functions to construct a target function. The main idea is to introduce a selection layer that only selects one element of the previous layer as an input to a function block.  The reviewers were unanimous in their opinion that the paper is not suitable for publication at ICLR in its current form.  There were significant concerns about the clarity in writing, and reviewers have provided detailed discussion should the authors wish to improve the paper.
Decision: Reject