  Model comparison is the cornerstone of theoretical progress in psychological
research. Common practice overwhelmingly relies on tools that evaluate
competing models by balancing in-sample descriptive adequacy against model
flexibility, with modern approaches advocating the use of marginal likelihood
for hierarchical cognitive models. Cross-validation is another popular approach
but its implementation has remained out of reach for cognitive models evaluated
in a Bayesian hierarchical framework, with the major hurdle being prohibitive
computational cost. To address this issue, we develop novel algorithms that
make variational Bayes (VB) inference for hierarchical models feasible and
computationally efficient for complex cognitive models of substantive
theoretical interest. It is well known that VB produces good estimates of the
first moments of the parameters which gives good predictive densities
estimates. We thus develop a novel VB algorithm with Bayesian prediction as a
tool to perform model comparison by cross-validation, which we refer to as
CVVB. In particular, the CVVB can be used as a model screening device that
quickly identifies bad models. We demonstrate the utility of CVVB by revisiting
a classic question in decision making research: what latent components of
processing drive the ubiquitous speed-accuracy tradeoff? We demonstrate that
CVVB strongly agrees with model comparison via marginal likelihood yet achieves
the outcome in much less time. Our approach brings cross-validation within
reach of theoretically important psychological models, and makes it feasible to
compare much larger families of hierarchically specified cognitive models than
has previously been possible.
