NIMBLE: A language for algorithms for graphical models embedded in R Problem-specific hierarchical statistical models (graphical models) are used by domain experts in many scientific fields: • State-space or hidden Markov models for time-series data • Random field models for spatial data • Generalized linear mixed models for complex designed studies • Capture-recapture models • Non-parametric regression and distribution models • Combinations of these and many more ideas There are many unmet statistical challenges for hierarchical models: • Efficient Markov chain Monte Carlo (MCMC) for parameter estimation • Maximum likelihood (or empirical Bayes) estimation when the likelihood requires integration. • Approximation of normalizing constants (likelihood or marginal likelihood) for model comparisons. • Tools for model selection, an unresolved area of Bayesian methodology. • Tools for model averaging. • Tools for assessing/validating model fit or assumptions. Perry de Valpine1, Christopher J. Paciorek1, Daniel Turek2, Nick Michaud1, & Duncan Temple Lang3 1University of California, Berkeley. 2Williams College. 3University of California, Davis. Previous software typically follows one of two basic designs: 1. Define a single model or family of models and provide specific methods for them. • Typical of R packages. • E.g. Generalized linear mixed models (GLMMS; lme4, MCMCglmm). • E.g. Generalized additive mixed models (mgcv). • E.g. Spatial models (spBayes, INLA). • E.g. Dirichlet process type models (dpPackage) • User cannot extend the model. 2. Provide a domain-specific language for writing general models and one or a few black-box algorithms. • E.g. MCMC is provided by BUGS (WinBUGS, OpenBUGS, JAGS), Stan, PyMC and others. • E.g. particle filtering is provided by BiiPS, LibBI, pomp, and others. • User cannot write new algorithms for the models. New computational statistics methods for general models Software for statistically savvy domain scientists Statisticians and computer scientists publish many new methods that are not accessible via software to domain scientists. • Model-specific methods can be distributed as R packages. • Model-generic methods are hard to code because the model must be abstracted. There has been no general system for model-generic programming. Model flexibility Algorithm flexibility BUGS/JAGS PyMC ADMB Stan Typical R packages NIMBLE = BUGS language + Algorithm programming 1. Domain-specific language (DSL) for statistical models • We adopt and extend the widely-used BUGS language 2. Domain-specific language embedded within R for model-generic algorithms 3. Code-generator (compiler) that generates C++ from the model and algorithms DSLs. • C++ objects are managed from R by dynamically-generated interface classes 4. Algorithm library (MCMC, SMC, etc.) metropolis_hastings_sampler <- nimbleFunction( contains = sampler_BASE, setup = function(model, mvSaved, targetNode, scale) { calcNodes <- model$getDependencies(targetNode) }, run = function() { model_lp_current <- model$getLogProb(calcNodes) proposal <- rnorm(1, model[[targetNode]], scale) model[[targetNode]] <<- proposal model_lp_prop <- model$calculate(calcNodes) log_MH_ratio <- model_lp_prop - model_lp_initial if(decide(log_MH_ratio)) jump <- TRUE else jump <- FALSE if(jump) copy(from = model, to = mvSaved, row = 1, nodes = calcNodes, logProb = TRUE) else copy(from = mvSaved, to = model, row = 1, nodes = calcNodes, logProb = TRUE) }) The gap between methods and software Limitations of previous software designs NIMBLE enables algorithm programming for general graphical models Four components of NIMBLE Numerical Inference for statistical Models using Bayesian and Likelihood Estimation Existing and future (in colors) NIMBLE processing flows Why R? Benefits of embedding NIMBLE in R: • R is widely used in applied statistics. • The BUGS language uses extremely R-like syntax that can be natively parsed in R. • R handles code as an object. • NIMBLE constructs and evaluat