Exploring Multivariate Data with the Forward Search

This book discusses forward search (FS), a method using graphs to explore and model continuous multivariate data that was introduced by Atkinson and Riani (2000). Its viewpoint is toward applications, and it demonstrates the merits of FS using a variety of examples, with a thorough discussion of statistical issues and interpretation of results. The book requires knowledge of multivariate statistical methods, because it provides only basic background information on the methods considered (although with excellent references for further reading at the end of each chapter). Each chapter also includes exercises with solutions. FS algorithms are available in S–PLUS, R, and STATA at websites specified in the book. Tables of data are given in an appendix. Chapter 1, “Examples of Multivariate Data,” introduces the basic concepts of FS using simulated and real data. FS is a search that starts from a small subset of the observations excluding “outliers” and moves forward through the data—adding observations—until there are no “clean” data remaining outside the subset. The chapter discusses a method for finding such an initial subset, and also provides further references for identifying robust bivariate centers of the data. Clearly, distance/proximity measures serve as the bases for outlier and cluster generation, because they indicate how patterns “look similar” to each other. FS uses Mahalanobis distances (Mds) as a main diagnostic tool. The motivation is that Mds would correctly order the data, with outliers being most remote, when starting with subsets that are outlier-free. As a result, when Mds are plotted for each subset, and “unusual” observations begin entering the subset, Mds start changing noticeably and allow identification of outliers, clusters, and any “unsuspected” influential data. As illustrated, Mds from all observations, as well as deletion of observations (backward methods), may fail to reveal such features. Chapter 2, “Multivariate Data and Forward Search,” gives theoretical results related to FS, including the exact distribution of squared Mds (a scaled beta distibution, with an alternative derivation due to Wilks given as a reference). The second part of the chapter focuses on application of FS in regression. As demonstrated, monitoring parameter estimates can provide additional useful information about model fitting. The authors suggest using parameter estimates that minimize the median of the squared residuals (least median squares (LMS)). Their motivation is that LMS allows almost half of the data to be outliers or to come from other models and will still provide an unbiased estimate of the regression line (i.e., a “breakdown point” of approximately .5). The authors also derive a t test for testing a new variable in a model that is unaffected by the ordering of observations in the search. The “breakdown point”—the maximum proportion of outliers such that a model no longer fits the majority of data, if the proportion is higher than that—is a significant concept. For example, for the least squares estimator it is 0. Only one outlier needs to be moved toward infinity to cause an arbitrarily large change in the parameter estimates. Pison and Van Aelst (2004) discussed a graphical tool that detects influential points and classified observations using empirical influence functions and S-estimators (one-step reweighted minimum covariance determinant estimators ). For S-estimators, there is an easy-to-compute positive breakdown value. It would be interesting to see in the literature a comprehensive study on breakdown points of different estimators, which might illuminate the choice of (possibly) a class of estimators that is most appropriate for a problem. Chapter 3, “Data from One Multivariate Distribution,” demonstrates via examples that multivariate normal is a useful model in FS and that a transformation to achieve multivariate normality should be performed, if needed. When a larger number of variables is involved (28 in the given example) using univariate analysis of outliers based on boxplots and selecting variables that have distinctive patterns of outliers is recommended. But, as noted, such method tends to select highly correlated variables in which the same units are outlying; as is also known, lower-dimensional projections may fail to detect higher-dimensional outliers. To reduce the dimension of a problem, the authors suggest using principal components (PCs), but any dimension-reduction technique would be helpful, because obviously a larger number of variables complicates FS analysis. Nevertheless, all of the given examples illustrate the wide variety of information that can be obtained by fitting a single multivariate normal distribution to data, when this is combined with the diagnostic plots from FS. Chapter 4, “Multivariate Transformations to Normality,” discusses in detail the contribution of FS in establishing a transformation that is satisfactory for the bulk of the data while revealing the influential outliers and illustrates the beneficial effects of a transformation that moves the data closer to normality. In their analysis, the authors use the multivariate version of the parametric Box and Cox family of power transformations. They demonstrate how monitoring quantities during FS, such as parameter estimates for Box–Cox transformation, likelihood ratio tests, and approximate score tests for testing the value of a single transformation parameter can be used to choose a transformation. Because influential observations may be evident only for some transformations of the data but not for others, they suggest using FS on untransformed data as well as on data subject to various transformations. Chapter 5, “Principal Components Analysis,” argues that what is needed for outlier detection is a form of PC in which the major components are unaffected by outliers while the minor components may respond strongly to outliers. Their idea is to perform PC analysis on each subset of the FS and “forward-plot” the percentage of variation explained by each component (along with other quantities of interest in the PC problem). As illustrated by examples, such analysis might not necessarily lead to new discoveries about the data, or only few observations might have a clear effect on PC analysis, but multivariate normality clearly improves PC analysis. Evidently, FS can illuminate the effect of individual units on the structure of PCs. Chapter 6, “Discriminant Analysis,” suggests applying FS by performing discriminant analysis on each subset using two approaches: by considering each group separately for the ordering of Mds (constrained search) and by ignoring group membership and ordering Mds accordingly. Difference in the results of the two types of searches might be caused by the order in which the units enter the subset, because in the constrained search the units for the group with smaller variance tend to enter earlier. Forward plots of Mds and sample covariance quantities are recommended as diagnostics in examining differences in