Improved Modeling of Complex Systems Using Hybrid Physics/Machine Learning/Stochastic Models
Authors: Anand Ramakrishnan
Warren B. Jackson
Kent Evans
Venue:  (2020)
Keywords: Composition
extrapolation
boosting
autocorrelation
systematic errors

Abstract: Combining domain knowledge models with neural models has been challenging.  End-to-end trained neural models often perform better (lower Mean Square Error) than domain knowledge models or domain/neural combinations, and the combination is inefficient to train.  In this paper, we demonstrate that by composing domain models with machine learning models, by using extrapolative testing sets, and invoking decorrelation objective functions, we create models which can predict more complex systems. The models are interpretable, extrapolative, data-efficient, and capture predictable but complex non-stochastic behavior such as unmodeled degrees of freedom and systemic measurement noise.  We apply this improved modeling paradigm to several simulated systems and an actual physical system in the context of system identification.   Several ways of composing domain models with neural models are examined for time series, boosting, bagging, and auto-encoding on various systems of varying complexity and non-linearity.  Although this work is preliminary, we show that the ability to combine models is a very promising direction for neural modeling.

Under review as a conference paper at ICLR 2020
IMPROVED MODELING OF COMPLEX SYsTEMs USiNG HYBRID PHYSICS/MACHINE LEARNING/STOCHASTU MODELS
Anonymous authors Paper under double-bling review
ABSTRACT
Combining domain knowledge models with neural models has been challenging. End-to-end trained neural models often perform better (lower Mean Square Error) than domain knowledge models or domain/neural combinations, and the combi- nation is inefficient to train. In this paper, we demonstrate that by composing domain models with machine learning models, by using extrapolative testing sets, and invoking decorrelation objective functions, we create models which can pre- dict more complex systems. The models are interpretable, extrapolative, data- efficcent, and capture predictable but complex non-stochastic behavior such unmodeled degrees freedom and systemic measurement noise. We apply this improved modeling paradigm to several simulated systems and actual physical system in the context of system identification. Several ways of composing domain models with neural models are examined for time series, boosting, bagging, and auto-encoding on various systems of varying complexity and non-linearity. Al- though this work is preliminary, show that the ability to combine models is very promising direction for neural modeling.
INTRODUCTION
Modeling has been used for many years explain, predict, and control the real world. Traditional models include science/math equations, algorithms, simulations, parametric models which capture domain knowledge, and interpolative models such as cubic splines or polynomial least squares among others which do not have explanatory value but can interpolate between known values well. The nonpredictable part the signal is captured by a stochastic noise model.
The domain/physical models predict /-dimensional output vectors, Y R2xl given n k- dimen- sional input vectors, X E g=*k with adjustable parameters, used to obtain the best fit (first term in Eq. 1). The unmodeled non-deterministic part of the data is often attributed to random noise fit to various stochastic models. / (0)) with parameters & (2nd term, Eq. (1)). This traditional approach has been very successful. The advantages of good model include high data efficiency, interpretable, the ability to extrapolate to predict outputs from inputs beyond the range of the training input data, and composable (multiple models can be combined solve more complex problems).
Y f(x 0) + No() NN((;W)
However, this traditional approach has limitations. Complex systems often have degrees of freedom which are not modeled by the traditional models. These unmodeled degrees of freedom or sys- tematic errors of the measurement are not modeled adequately by the noise model. In addition, the parameters the physical models, 0 can be error be time dependent. In these cases the behavior of unmodeled part of the system is not random and thus the usual combined deterministic-stochasti model is inadequate.
Neural models NN(X; W), e.g. neural network models, are fundamentally just another form of parametric models where are the inputs and W are the weight parameters. However, neural modes have unique properties to exploit. First, neural models handle high dimensional input- output relations with complex patterns. Second, like interpolative models such as cubic splines or polynomials, NNs sufficiently expressive fit many possible relations but often not good

---

Under review as a conference paper at ICLR 2020
extrapolation, (see below) Trask et al. (0018) Third, neural models do not require handcrafting basis functions. Hence, neural models have the potential for describing unmodeled degrees of freedom, systematic errors, and nonstationary behavior.
In this paper, neural modeling is combined with traditional modeling to achieve the advantages of both traditional and neural models and compensate for the problems mentioned above using follow- ing steps. (1) Composing Hybrid Models. We examine several ways of creating hybrid models: boosting, ensemble, and cyclical autoencoder (Fig. D.. Combining domain models and neural mod- els requires assumptions about relationship between various system and noise. For example, Eq. (1) makes an implicit assumption that the models are composed by addition but there are many other possible assumptions. Unlike most boosting approaches, we use different model classes and loss functions for the various stages. (2) Extrapolation Testing. An extension to the traditional machine learning approach of dividing the data set into test and training portions is extended to include both interpolative and extrapolative testing sets as a stringent test of modeling power. (3) Stochastic Loss. Unlike previous approaches, the quality of the hybrid models to produce truly stochastic residuals is enforced using novel loss functions that enforce appropriate correlation of residuals.
In this work, this paradigm is applied to system-identificationn (SysID) for simulated and real sys- tems. The results demonstrate that these models decompose into deterministic, predictable, and stochastic components and can handle more complex systems
2 RELATED AND PREVIOUS WORK
The use of combination of traditional models and neural models has been investigated before. Psi-
chogios & Ungar 1992) used an extended Kalman filtering and a multilayer perceptron. Talmoudi
et al. (2008) used Khonen neural networks form of cluster analysis to help in traditional models.
Zhang (2003) used hybrid ARIMA neural network to explain data. Pathak et al. (0018) investi-
gated a liquid state plus physics model hybrid. Most of these papers do not use the latest machine learning approaches in particular, boosting as a general principle for a modeling paradigm or the
most effective current neural models. In this earlier work, the models are evaluated by MSE over an interpolative testing set (discussed below). Thus in previous work, the flexibility of neural models to interpolate with enough parameters and training has resulted in the conclusion that end-to-end neural models without domain/physies modeling nearly always exhibit the lower MSE and there- fore there is no advantage for incorporating domain knowledge. In addition, the important issues of how to compose the hybrid models, evaluate their ability to incorporate prior knowledge, to provide interpretability in terms of modeled, unmodeled predictable, and unpredictable components, and to evaluate their ability extrapolate vs interpolation, were not adequately addressed. These relatively new issues are addressed in this work.
Various approaches for composing models include following. Jacovi al. (201)) generated super- vised data using a blackbox physics model to fit neural model used to back propagate the errors to the input models. [Innes al. (219)) [nnes (2018) propagate the errors through parametric physics models using automatic differentiation. This works for small models but doesn't scale well, solve the more complex domain problems, address extrapolation or exhibit noise model consistency, [Sa- hoo et al. (2018) introduces the idea of specific functional neurons computing model function which slow computation by mixing NN and nonlinear least squares as well as limited modeling capabil- ity. Amos & Kolter (2017) constructed a nonlinear least squares layer to fit several interactions of a model as part of a larger neural network. Other work Zhou et al. (017) incorporated a physics model by adapting the inputs to the model using neural network to adjust the outputs to be the desired results. None of these address the issue of training protocols, interpreting the neural com- ponent, investigating the statistical properties of the residuals. [Bengio et al. (2006) looked at boosting one stage of a neural layer and Finally, Huang et al.) (2017) has investigated the problem of composing through boosting. Like most boosting, the model class and loss function where the same for each stage and no attempt to connect with the larger modeling paradigm was attempted.
3 COMPOSING HYBRID MODELS
To form hybrid models, we consider a few promising ways combine a domain model with neural networks and stochastic models: a) sequential, b) parallel, and e) cyclic training as shown in Fig.
2

---

Under review as a conference paper at ICLR 2020
1.) Sequential (boosting)
X[t+1]
Syster
error
Lutt xit]
Trad. Model
Model 2
Model
[t+1]
X(t=1]
Asjust Sammetee
Aduat parameters
2.J Parsitel (barging)
(ul Xit
System
X((+1)
Trad. Model
[(++1)
X(te1)
Systern
Ut:
Model (Encode)
Model Decode)
(4:))
(Ul
xit]
Model
X(te1
Model 3
Figure 1:
Various ways of creating hybrid domain/neural models. 1) Sequential boosting. 2) Parallel (bagging or ensemble) and 3) Cyclic (variational autoencoding training
3.1 SEQUENTIAL METHOD
The sequential boosting method generalizes the traditional modeling splitting the output into deterministic and stochastic components. In this model composition procedure, the parameters, of one more domain/physics models are trained by selecting a loss function discussed in the next section. The next stage is implemented by selecting another model using residuals and and previous inputs as the new input and selecting a loss boost (train) the next stage parameters. This process can be iterated a number of times. Normally boosting uses one model class such as trees but importantly in our case, the weak learner is the domain model(s) for the first stages followed neural model boosting create hybrid. The final residuals are ascribed to stochastic models. More precisely, the boosting method is implemented as follows. A sequence of M learners is used to refine the predicted output. The weak base models consist of ppysics/determinisit are followed by NN models to capture the predictable but complex portion
models which
M
f - jM (x)
(x)
(x)
==0
where M is the number of iterations, fo is the initial guess and Si are the boosts.
fa (x) - f-)1()) + he (z, 0.)
(3)
where
0: = arg min Ve fy. (x) + he (z. 0))
(4)
where (-. is a loss function for stage discussed below. In summary, the sequential method involves a number of stages where a model fitted to the residuals using stage specific loss function. Generally, the domain models are applied first followed by the neural models.
3.2 ENSEMBLE METHOD
In the parallel (ensemble) modeling (Fig. 2 the various models are trained one stage to min- imize the loss between the system and the model. Both 0 and W are trained simultaneously. This has the advantage finding a global optimum but is more compute intensive.
3.3 CYCLIC METHOD
In this hybrid method, for each training stage, model is selected with loss function in se- quential, but second decoder NN model learned as well which provides a cyclical loss term to replicate the input. This cyclical training is performed for each stage. So the method is similar to the sequential method in terms stages of domain then NN learning. This method often improves training and provides decoder/inverse model of use inverse modeling and is related the implicit deep equilibrium layers Bai et al.] (2019). There are of course other ways compose models but these methods provide direct connection with the additive decomposition of Eq. (1).
3

---

Under review as a conference paper at ICLR 2020
Interpolative Extrapolative Training
Training set
Interpolating test set
Extrapolating set
E
E
Training and interpolating test sampled from convex set
Extrapolating test sets
Figure 2: Interpolating training and test sets, and extrapolating test sets defined by a complex hull
4 STOCHASTIC MODELING
In order to fit or the parameters, the loss function, V(...) for each learning/iiting stage must be selected. For the domain model training, the ensemble model or the first stages, the usual MSE function
vt (v. f
Σ =0=00
(5)
sufficient where pj is the ith lag of the autocorrelation function of the residual
T y/i - f(:))
In general however, the MSE does not insure that the residuals after training compatible with the stochastic model. The stochastic model normally assumes independent, identical distributed (IID) random variables. The residuals are compared the stochastic model using autocorrelation function usually zero mean Gaussian with variance σ². Introducing a new objective namely a Ljung-Box (LJB) loss function
n p
Te (v) n(n + 22
Σ
ensures uncorrelated residuals and compatibility with the stochastic model. L is hyper parameter which should be larger than possible correlations. If residual correlations are expected, the expected correlation can built into the LJB function. Thus, to create and train a hybrid model, domain and NN models selected. For each stage of learning, loss function is selected to drive the outputs towards a desired goal. Decorrelation loss functions are used to control the correlations of the residuals.
INTERPOLATIVE TRAINING/EXTRAPOLAIIVV TESTING
Finally, having selected models and loss function, the training and testing data must be selected. For the most part, previous work has sampled the testing set from the same input region as the train- ing set. Because of the interpolating ability of large parameter neural models, the models achieve a lower MSE than domain and hybrid domain/neural models. But this measure of quality does not measure the extrapolation ability of domain models. Thus, a new method for creating testing data used. (Fig. 2). An interpolating test set is created by sampling from the convex hull different points than the training set. An extrapolating test set is created by sampling outside the training convex hull. Extrapolative testing identifies those models which generalize larger input domains and discriminates against those models which memorize the data. Indeed, in the results below, high parameter models look good for interpolative but not for extrapolative data.
The alternative of performing end-to-end of all the models at once is an significant alternative. However, this procedure often very inefficient because the switch from neural models nonlinear

---

Under review as a conference paper at ICLR 2020
Pyllo
Resta wetrr-uus Pesta londen
Pryli Pyea beii
Pyda
Peska bocddw
Rlyk Ganti Hyeaaduli Bao
Pacingg dee
Figure 3: RMSE in db for the inverse pendulum problem using an interpolative test set for various models, model hybrids, and loss functions during training as described the text
domain models is an efficient use of GPUs. Morcover, one can guide the output of each stage and maintain intepretability by training in stages.
In summary, the modeling procedure selects one or more models, combines them in parallel or sequentially, selects a training set, and uses loss function to train this stage. The process is repeated, preferable with complementary models the previous ones and another loss function, fit to the residuals or outputs of the previous stage. A decorrelation loss function insures consistent Eq. (1) decomposition.
RESULTS
The above process for modeling complex phenomena is applied to system identification for a number of different mechanical problems time series prediction, had as much success Makridakis et al. (2018).
where machine learning has not
In particular, problems from the OpenAl gym (Brockman al. (016)), the inverse pendulum and a double pendulum are simulated as function of time with Gaussian noise added. Finally, the procedures applied to actual DC motor system with backlash. In order create system-id data set, series of actuations excite the system, and the resulting response simulated or measured. Training and testing data sets consist of the previous state æt + 1). the actuation u ut), and resid- uals r(t). The outputs are the next time step residual values r(t + 1). The state of each system includes various degrees of freedom such as the angle, angular velocity, and acceleration. The ac- tuation time series include training set and an interpolative testing set from restricted range of actuations restricting amplitude and/or frequency. The extrapolative test is created by increasing the amplitude and/or the frequency of actuation for the various models and the actual physical motor. The training time series is used to fit/train various models including a physics model, linear state space model, dense neural network, recursive neural network (RNN) and cyclical model. The physics model and the linear state space model are also used as the first stage in hybrid models and RNN, dense NN and cyclical NN are used to fit the residuals with either MSE LjungBox (LJB) as the objective function.
6.1 INVERSE PENDULUM
The inverted pendulum swingup problem is classic problem in the control literature. To create physics model we introduce some errors into the model (mass of the pendulum was changed from I and length change to 10 from 1) to insure error to be captured by other models. Otherwise, the physics model fits nearly perfectly. Model inputs cos Ot)), sin O(t), é(t), (tt) (Applied Torque) and outputs cos O 1), sin O 1), Ott +1)
5

---

Under review as a conference paper at ICLR 2020
Pypoodrenddts Pesko hondoes
Pylosioon-8 Rylootyt--boottt
Drandion-er
Figure 4: RMSE in db for the inverse pendulum problem using extrapolative test set for various models, model hybrids, and loss functions during training as described the text
We train our models on a constrained training dataset of size 25000 points with control actions between -1.0 to 1.0. The performance of different model compositions on interpolative validation dataset is shown Fig. 3
We then create an extrapolation dataset of size 25000 utilizing the entire space of control action space i.e. between
2. The performances the models on this dataset is shown in Fig.
In Fig. 3] the box plot of root mean square residuals (RMSE) between the predicted state and the observed state in db = 20log1o| RSM is plotted for various models showing the median, 25/75 percentile and the 5/95 percentile along with the arithmetic mean (triangle) over the testing time period. As expected, the dense NN and the RNN do very well compared with the nonlinear physics model. Boosting the physics model with linear state space model helps reduce the median but the variation (error bars) is still large. Boosting with a dense NN and in particular, an RNN greatly reduces the median and variance of the residuals. The cyclical autocorrelation also does good job reducing the interpolative testing errors. The LJB loss boosting results in a worse MSE as expected because the LJB attempts minized the residual correlations.
For the extrapolative testing set, the models exhibit greater variation(Fig.. the physics models do better than the linear model, and the various single neural network models, Dense and RNN. These results validate the claim that physics models even with incorrect values extrapolate better than NN models. Because db is log, small plotted changes are actually quite significant. In (Fig. 4. even when the NNs or linear are boosted, the median errors are larger. Because the problem is nonlinear, the extrapolated linear model prediction errors are rather large. Only the physics model boosted with RNN and trained with a LBB loss function competes with the other physics models. Thus, the combination of the physics plus the neural network models improve on the pure NN models and on the physics models well. We also note that in general, the MSE for the extrapolative data sets are larger than the interpolative data sets as expected. In summary, the results for the inverted pendulem confirm that hybrid domain(physics) models augemented with NN models results in better fits especially for the extrpolative data sets.
Except for State 2,
6.2 DOUBLE PENDULUM
Next we look at the double pendulem to further demonstrate that the hybrid modeling can capture unmodeled degrees of freedom. We create a simulation of a double pendulum in free fall under gravity. The model system consists of large pendulum with small pendulum at the end starting at 90 degrees and oscillates without actuation. The data interpolative because the testing is within the range training data. Model inputs are: (t),, (t)), Ôt) and outputs are O + 1), Ô + 1), Õ(t+ 1). The physics, dense and RNN fit the state of pendulum if it were single large pendulum. The RMSE for the physics model is significantly reduced by boosting the physics model with neural models, in particular the RNN reduces the error (see Fig. 9in appendix). Fig, Showss how the hybrid improves the autocorrelation function of the residuals. In the first set, the autocorrelation function for only the physics model for single pendulum fit to the double pendulum residuals are plotted.

---

Under review as a conference paper at ICLR 2020
10
NussCemmiiion
-32
100
5-
120
180
10
1200
200
1000
2000
4000
160
Physics Model
Physics Boosted RNN Model
Physics Boosted RNN Model LB Cost Function
Figure 5: AutoCorrelation the double pendulum physics model-LSSE loss function (left), physics model boosted by RNN (middle) and physics model boosted by RNN trained with Ljung-Bor Box LJB) loss function.
meos Seaterd
Uneer Desse
Liber Goost-Nt
Linean Up dosss ROH
Uneandydl BOGST-N
Dasaa
Llesan Li Boatt
Figure 6: RMSE in db for a real DC motor with backlash for an interpolative for various models
Because the second pendulum motion is not modeled, the residual autocorrelation function exhibits the oscillation of the unmodeled motion. In the second panel, the physics model boosted by the RNN with MSE as the loss function shows that the RNN successfully modeling the oscillation of the second smaller pendulum When the LJB loss function is used for the RNN boosting phase, the autocorrelation function (with zero lag suppressed) is very close to uncorrelated and the variance of the residuals added to the simulation. Thus, we have successfully captured the large motion with physics model, the second pendulum (the unmodeled degrees of freedom) with the RNN, and the unpredictable signal components with consistent IID stochastic model. This result confirms the ability to capture structured disturbances.
the autocorrelation function exhibits reduced amplitude oscillations.
fact the noise
6.3
DC MOTOR
Finally, we apply the hybrid modeling to actual physical systems. A DC motor was coupled rotary encoder with a 3D-printed shaft. The shaft was divided into two pieces and these two pieces
for State
-50
to-
Figure 7: RMSE in db for a real DC motor with backlash for an exterpolative test set for various models

---

Under review as a conference paper at ICLR 2020
AutoCamstatio
14
100
03-
1550
Linear Model
Linear Boosted RNN Model
Linear Boosted RNN Modell LIB Cost Function
Figure 8: AutoCorrelation residuals for real DC motor with backlash a linear(physics) state space model trained with a LSE loss (left), a linear(physics) state space model boosted by an RNN with LSE loss (middle) and a linear(physics) state space model boosted by an RNN with Ljung Box loss
were coupled through gearing that purposely exhibited substantial backlash. The voltage was con- trolled through Pulse Width Modulation (PWM) with polarity which allowed for range of speeds and direction changes. Position and velocity data of the shaft was collected from the encoder using a National Instruments DAQ board. For data collection, the Arduino controller was programmed to randomly vary the speed and polarity of the motor collecting data inputs: ( (t)) (t) (Volt- age.) paired with model outputs: Next time step O + 1), Ô(1 + 1). This input,output pair data was fit to a linear state space model, (i.e. physics model) where the linear coefficients are interpretable in terms of motor/load moment of inertia, friction, winding resistance, and motor inductance. The data was also modeled using RNN, dense NN, and various hybrid models. Fig. shows that the dense NN and RNN are able to capture some of the backlash. The linear/physice model boosted by an RNN using cyclic training does the best for the interpolative data set with 10 Hz excitation. For the extrapolative data in Fig. 77, the frequency of excitation pulses were 100 Hz. As expected, the RMSE is larger for the extrapolated data set. Boosting by a NN, particularly the RNN, captures the delay of the backlash and the MSE much smaller. In Fig. the modeling shows how the domain model picks up some of the behavior(left), the RNN captures the backlash (middle), and the LJB loss function forces the residual match the IID assumptions (right). Thus, these results demonstrate that for both simulated and real systems, the boosting of domain models with neural models utilizing whitening loss functions, results consistent, interpretable, extrapolating models.
CONCLUSIONS AND FUTURE DIRECTIONS
Combining neural models with physics(domain) and stochastic models greatly expand the ability to model complex phenomena particularly for control. The expanded hybrid models incorporate the domain knowledge, interpretability, data efficiency and extrapolability of domain models with neural models which can model complex, high dimension, but predictable uncontrolled, unmodeled degrees of freedom of the system and of the measurement system (systematic noise). Using boosting, novel whitening objective functions, and extrapolative/interpoaat capture the behavior of more complex models a meaningful decomposition. These models help solve the problems of unmodeled non-stochastic components of system behavior. For future work, measures such as signal-to-noie toiise,,, eтror rates etc can be generalized to the case of non-stochastic neural modeled behavior. The combined modeling solves problems with structure noise. In the field of control, stability bounds for control systems can be implemented by using these models in the context of robust control.
testing sets, these hybrid models
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, al. Tensorflow: system for large- scale machine learning. In /2th {USENIX) Symposium on Operating Systems Design and Imple- mentation ((OSDI) 16), pp. 265-283, 2016.
Brandon Amos and Zico Kolter. Optnet: Differentiable optimization as a layer in neural networks. International Conference on Machine Learning, pp. 136-145, 2017. URL http:://axxix
org/abs/ 00443
Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. Deep equilibrium models, 2019.
Yoshua Bengio, Nicolas L. Roux, Pascal Vincent, Olivier Delalleau, and Patrice Marcotte. Convex neural networks. In Advances in neural information processing systems, pp- 123-130. 2006.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint arXiv:16606.01540.00 2016.
François Chollet et al. Keras, 2015.
Furong Huang, Jordan T. Ash, John Langford, and Robert E. Schapire. Learning deep resnet blocks sequentially using boosting theory. CoRR, abs//aa... 2017. URI. httl/ttt////xxiv....
abs/1706.. 04964
Michael Innes. Don't umroll adjoint: Differentiating ssa-form programs. CoRR, abs/1810.07951, 2018. URLtttt://arxxtt
org/ http:///w....rr/////1111....0... 07951
Mike Innes, Alan Edelman, Keno Fischer, Christopher Rackauckas, Elliot Saba, Viral B. Shah, and Will Tebbutt. A differentiable programming system to bridge machine learning and scientific computing. CoRR, abs/1907.07587 2019. URL/att:://axxi otg.a/iii..ooroaaaooooa 1ttp//197..0333
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint 102.v3/500.03166 2015.
Alon Jacovi, Guy Hadash, Einat Kermany, Boaz Carmeli, Ofer Lavi, George Kour, and Jonathan Berant. Neural network gradient- baaseentt-bbdddddeee learning of black-box function interfaces. arXiv preprint arXiv: 19099999. 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. Statistical and machine learning forecasting methods: Concerns and ways forward. PloS one, 13(3):0014488) 2018
Jaideep Pathak, Alexander Wikner, Rebeckah Fussell, Sarthak Chandra, Brian R Hunt, Michelle Girvan, and Edward Ott. Hybrid forecasting of chaotic processes: Using machine learning in conjunction with knowledge-based model. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28(4):041 101, 2018
Dimitris C Psichogios and Lyle H Ungar. A hybrid neural network-fing first principles approach to process modeling. AIChE Journal, 14(10))1499-155 1992.
Subham S. Sahoo, Christoph H. Lampert, and Georg Martius. Learning equations for extrapolation and control. CoRR, 0tss/800.072555 2018. URL http: httt:ttpp//aaeet http///xxvvvvrr..//. /ttp::://///cx wttp:////..////////00... 07259
Talmoudi, K Abderrahim, R Ben Abdennour, and M Ksouri. Multimodel approach using neural networks for complex systems modeling and identification. Nonlinear Dynamics and Systems Theory, 833):2993316 2008.
Andrew Trask, Felix Hill, Scott E. Reed, Jack W. Rae, Chris Dyer, and Phil Blunsom. Neural arith- metic logic units. CoRR, abs/1808.00500 0000000 2018. URL http: arxiv. org. wwtp://ww.w//bb///....// 00508

---

Under review as a conference paper at ICLR 2020
RAASE fu 500055
-30
-100
myaks
Hoyio-yydk cycl--cckooaaoooo
FAUSE Stat-t
Phinks
Beess BM
Peylo- Unoer
Paylts Bonk Denns
Phalsboot-3
RUSE Sorte2
1100
Figure 9: RMSE in db for the double pendulum problem using the interpolative test set for various models, model hybrids, and loss functions during training as described the text. The problem is treated single pendulum with the second pendulum unmodeled degree of freedom
Peter Zhang. Time series forecasting using a hybrid arima and neural network model. Neurocom- puting, 50: 10:-511, 2003.
Siqi Zhou, Mohamed Helwa, and Angela P Schoellig. Design of deep neural networks add- on blocks for improving impromptu trajectory tracking. 56th IEEE Conference on Decision and Control, arXiv preprint arXiv:170s. 2017.
Appendices
A NEURAL NET MODEL DETAILS
Hyperparameter discussion are presented here. All models are trained on Keras (Chollet et al. 2015)) with TensorFlow (Abadi et al. (2016)) as the backend. Trained models saved based on performance on a validation dataset created from the Interpolative convex set. The initial learn- ing rate is set at 0.01 and the learning rate is halved if the validation loss plateaus for 10 consecutive epochs. Early Stopping is effected the models do not show any reduction in validation loss for 30 consecutive epochs. Optimizer used is Adam (Kingma & Ba (2014)) and models
trained with mini batch of size 512. Batch Normalization (Toffe & Szegedy (2015)) is used at the input stage of all models. All trained Dense Models have 10 Hidden Layers with 256 weights in each layer. All trained RNN models have 10 Hidden layers with 50 recurrent weight blocks in each layer.
B DOUBLE PENDULUM RMSE
The interpolative double pendulum results are shown in Fig.
10


Reviews:
[{"review_id": "rye7ynORtB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents approaches for combining neural network (NN) with non-NN models to predict behavior of complex physical systems.  \nI found this paper very hard to read, with a lot of details and key pieces of information missing or vaguely stated.  Following are some specific instances in no particular order:\na) From the results it seems that often approaches using ‘cyclic-boost’ perform the best.  However the description of cyclic boost in Section 3.3 lacks any precise description of what the cyclic approach does.  For instance, the NN that predicts the inputs from model outputs, how is that used and under what objective is that learnt?\nb) The main claim of the paper is that combining physics models with NNs is better than NNs alone, esp. when tested on ‘extrapolative data’.  However, if cyclic-boost is the best approach, the paper should include results with cyclic-boost-RNN combination, these do not seem to be there to assess value add of the physics models in the best performing system.\nc) Section 3.2 states a parallel ensemble of NN and non-NN models has the advantage of finding ‘global optimum’ … this is a strong statement and needs to be demonstrated.\nd) The description of new loss function in Section 4 is unclear.  For instance, what is ’n’ in (7)?\ne) In Section 3.1 I’m assuming that ‘h_t’ denotes the model being trained in iteration ’t’, and some of these are NN whereas others are domain models?", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "rkequUYRFB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present a new hybrid models that incorporates traditional models with neural networks by boosting and reducing the residuals in stages. They show 3 different boosting schemes: sequential boosting, parallel boosting and cyclical boosting (inspired by variational autoencoders). Furthermore the authors present the new Ljung-Box (LJB) loss function which reduces autocorrelation. They test on 2 simulated systems and on a physical DC Motor and put an emphasis on testing their result on extrapolated (unseen) data.\n\nStrengths:\n- The authors combine traditional models with deep learning approaches.\n- The models can extrapolate to unseen data better than conventional deep learning approaches.\n- The results show that the authors found a loss function that can reduce autocorrelation.\n\nWeaknesses: \n- The paper lacks an exact explanation for every model that was used. This is made worse by a lack of consistency, e.g. the model Physics-Boost-Dense doesn't explain which of the 3 boosting schemes is used.\n- The authors propose a new Ljung-Box (LJB)  loss function. It contains hyper parameter L \"which should be larger than possible correlations\". It is not mentioned how to find such a value or what the value for the experiments for. They show that this loss function can reduce autocorrelation, but it comes at the expense of higher RMSE. It is not clear in what scenario this is useful.\n- The methodology is difficult to follow. E.g. there is a mathematical explanation in 3.1, but not for 3.2 or 3.3. In Fig. 1(1) and Fig. 1(2), they incorporate a traditional model, but not in Fig. 1(3). Later they show results for a cyclic method that incorporates a physical model, but it's not clear how that was done.\n- The authors claim that they are using \"a new method for creating testing data\". However, testing on extrapolating data is not a new method.\n\nAdditional Comments: \n- Equation (2): I assume the hat should only be above f, not the entire f(x). If not, I don't see this symbol explained or used anywhere else in the paper.\n- h_t in equation (3) and (4) are not explained in the text\n- Fig. 1(1) and Fig. 1(3) are not mentioned in the text\n- Figure 3 and 4 are hard to understand. Instead of State 0/1/2, it should say angle / angular velocity / acceleration. There should be a table to accompany this figure, so that it's easier to compare the models. As is, it's very hard to compare e.g. Physics-Ensemble-Dense with Dense-Ensemble-RNN.\n- There are plenty of typos and a general lack of clarity. E.g. this sentence: \"The  alternative  of  performing  end-to-end  of  all  the  models  at  once  is  an  significant  alternative.\"\n- Add reference for original Ljung-Box test\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1gffMK5YS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper conducts several experiments to compare the extrapolative predictions of various hybrid models (sequential, ensemble, and cyclic), which compose physical models, neural networks and stochastic models. \n\nUnmodeled dynamics is a bottleneck for model learning, model-based reinforcement learning and sim-to-real transfer. This paper tries to tackle this important research challenge. However, this research is at its early-stage and the results are preliminary. I would not recommend accepting the paper at this time, and encourage the authors to continue this promising research direction. First, the conclusion of the paper is not clear to me. Which is the best way to compose models? Is it physics+RNN+cyclic LJB loss? It would be helpful to put all the visualizations of the results together and organize it in a way that can easily reveal conclusions across different experiments. If the conclusions are inconsistent across different experiments, detailed analysis and explanations are expected. Second, it is not sufficient to just analyze the regression errors of different hybrid models for an ICLR paper. The paper would be much stronger if the it could demonstrate that with the more accurate hybrid model, model-based learning performs better or transfer learning is more straightforward. For example, it could change Section 6.2 to control a double inverted pendulum using only a rough physics model of a single pendulum. Similarly, it could augment Section 6.3 to control a real inverted pendulum with a DC motor with large backlash.\n\nIn addition, I have a few questions/comments about the clarity of the writing:\n1) Are \\theta in eq. (1) the parameters of the physical system, such as mass, length of the rod for the inverted pendulum case?\n2) I am not familiar with LJB loss function. If it is commonly-used, please add a reference. If not, more explanations would be helpful. Is LJB loss function used alone or combined with the MSE loss?\n3) Is there any particular reason that cos\\theta and sin\\theta are chosen as states for Inverted Pendulum instead of \\theta? In Double Pendulum, \\theta is used.\n4) In DC Motor, \"This input, output pair data was fit to a linear state space model, (i.e. a physics model)...\" Is the output linear to the states (\\theta, \\dot{\\theta}) or linear to the physical parameters (inertia, friction, etc.)? In previous examples, such as Figure 4, physics model and linear model are separate. It is confusing to me that in this example, the linear model seems to be the physics model. Is the dynamics equation of the DC motor linear? Am I missing something?", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agree that the paper is to be rejected, provided strong claims that were not answered. In this form (especially with such a title) it could not be published (it is more of a technical/engineering interest).
Decision: Reject