Introduction to Regression Modeling

Long a standard tool of engineers and machine learning researchers, the Kalman filter (Kalman 1960) is a powerful algorithm for problems that can be cast as stochastic models indexed by time. In the context of linear Gaussian state-space models, the Kalman filter allows for efficient prediction, filtering, and parameter estimation. However, as pointed out by Meinhold and Singpurwalla (1983), most published literature about the Kalman filter appears in engineering journals, using notation and language that can mask its relevance to statistical problems. Nearly 25 years later, this remains the case, with a few exceptions (like the Meinhold and Singpurwalla article) that present the Kalman filter from a Bayesian perspective. This book proposes to remedy this by offering a self-contained exploration of the Kalman filter. Eubank derives the Kalman recursions from mathematical principles applied to a simplified state-space model, then extends the machinery to the general state-space case. Each chapter carefully builds on and refines what was presented before. Chapter 1 starts by describing the use of the Gramm–Schmidt and Cholesky algorithms with general signal-plus-noise models, then treats the special case of state-space models. Chapter 2 derives of the covariance relationship between the so-called innovations (the discrepancies between the actual observations and the predicted observations) and the state vectors in state-space models. The author uses this relationship in Chapter 3 to obtain a computationally efficient version of the Cholesky decomposition, which leads directly to the forward and backward Kalman filter recursions in Chapters 4 and 5. With this background, Eubank then treats particular questions and cases, such as how to initialize the state vector for the recursions (Chap. 6), how to take advantage of the Kalman recursions when the underlying state-space model is Gaussian (Chap. 7), and how to extend the results to a general formulation of the state-space model (Chap. 8). The book is well suited for people who want to “look under the hood” of the Kalman filter to understand how it works at a mathematically rigorous level. Although not specifically designed as a textbook, the book could be used to guide a special topics seminar for graduate students. A background in matrix algebra and introductory graduate-level statistics should be sufficient prerequisites. Eubank includes pseudocode for many of the algorithms presented in the text, making it easier to implement the methods. However, people looking for a less detailed introduction to the Kalman filter to understand and implement it quickly might do better initially with a guide like that of Welch and Bishop (2003). Although A Kalman Filter Primer is quite comprehensive, a few “cosmetic” issues impair its accessibility. In particular, the index is quite spare and thus not very helpful, and poor typesetting (including strange hyphenation decisions like “C-holesky” and “parameter-s”) distracts from the text. Overall, though, the book certainly achieves Eubank’s goal of a “self-contained, ‘no frills,’ mathematically rigorous derivation of all the basic Kalman filter recursions from first principles.”

Field of study: Mathematics