  Most scientific publications follow the familiar recipe of (i) obtain data,
(ii) fit a model, and (iii) comment on the scientific relevance of the effects
of particular covariates in that model. This approach, however, ignores the
fact that there may exist a multitude of similarly-accurate models in which the
implied effects of individual covariates may be vastly different. This problem
of finding an entire collection of plausible models has also received
relatively little attention in the statistics community, with nearly all of the
proposed methodologies being narrowly tailored to a particular model class
and/or requiring an exhaustive search over all possible models, making them
largely infeasible in the current big data era. This work develops the idea of
forward stability and proposes a novel, computationally-efficient approach to
finding collections of accurate models we refer to as model path selection
(MPS). MPS builds up a plausible model collection via a forward selection
approach and is entirely agnostic to the model class and loss function
employed. The resulting model collection can be displayed in a simple and
intuitive graphical fashion, easily allowing practitioners to visualize whether
some covariates can be swapped for others with minimal loss.
