performance: An R Package for Assessment, Comparison and Testing of Statistical Models

<p>A crucial part of statistical analysis is evaluating a model's quality and fit, or performance. During analysis, especially with regression models, investigating the fit of models to data also often involves selecting the best fitting model amongst many competing models. Upon investigation, fit indices should also be reported both visually and numerically to bring readers in on the investigative effort. While functions to build and produce diagnostic plots or to compute fit statistics exist, these are located across many packages, which results in a lack of a unique and consistent approach to assess the performance of many types of models. The result is a difficult-to-navigate, unorganized ecosystem of individual packages with different syntax, making it onerous for researchers to locate and use fit indices relevant for their unique purposes. The performance package in R fills this gap by offering researchers a suite of intuitive functions with consistent syntax for computing, building, and presenting regression model fit statistics and visualizations.</p>

section_content: While functions to build and produce diagnostic plots or to compute fit statistics exist, these are located across many packages, which results in a lack of a unique and consistent approach to assess the performance of many types of models. The result is a difficult-to-navigate, unorganized ecosystem of individual packages with different syntax, making it onerous for researchers to locate and use fit indices relevant for their unique purposes. The performance package in R fills this gap by offering researchers a suite of intuitive functions with consistent syntax for computing, building, and presenting regression model fit statistics and visualizations. 

performance is part of the easystats ecosystem, which is a collaborative project focused on facilitating simple and intuitive usage of R for statistical analysis (Ben-Shachar et al., 2020; Lüdecke, Ben-Shachar, Patil, Waggoner, et al., 2020; Lüdecke, Ben-Shachar, Patil, & Makowski, 2020; Makowski et al., 2019 Makowski et al.,, 2020)).
section_name: Statement of Need

section_content: Compared to other packages (e. g., lmtest (Zeileis & Hothorn, 2002), MuMIn (Barton, 2020), car (Fox & Weisberg, 2019), broom (Robinson et al., 2020) ), the performance package offers functions for checking validity and model quality systematically and comprehensively for many regression model objects such as (generalized) linear models, mixed-effects models, and Bayesian models. performance also offers functions to compare and test multiple models simultaneously to evaluate the best fitting model to the data.
section_name: Comparison to other Packages

section_content: Beyond validity and quality checks, performance also includes plotting functions via the see package (Lüdecke, Ben-Shachar, Patil, Waggoner, et al., 2020). 1
section_name: Features

section_content: Inferences made from regression models such as significance tests or interpretation of coefficients require meeting several assumptions, which vary based on the type of model. performance offers a collection of functions to check if assumptions are met. To demonstrate the efficiency of the package, we provide examples for a few functions, followed by a broader function that runs a comprehensive suite of checks in a single call. 

For example, linear (Gaussian) models assume constant error variance (homoscedasticity). We can use check_heteroscedasticity() from performance to check if this assumption has been violated. For another example, Poisson regression models assume equidispersion. Violating this assumption leads to overdispersion, which occurs when the observed variance in the data is higher than the expected variance from the model. We can call check_overdispersion() to check if overdispersion is an issue. library(glmmTMB) data(Salamanders) model <-glm(count ~spp + mined, family = poisson, data = Salamanders) check_overdispersion(model)

In addition to tests for checking assumptions, performance also provides convenience functions to visually assess these assumptions of regression models. performance's visual checks detect the type of model passed to the function call, and return the appropriate visual checks for each model type. At present, there are many supported regression models, such as linear models, linear mixed-effects models or their Bayesian equivalents. Inspect the package documentation for a complete listing. 

For example, consider the visual checks from a simple linear regression model. For example, consider a simple linear model. 

m1 <-lm(mpg ~wt + cyl, data = mtcars) model_performance(m1) ----------------------------------------------------# > 156. 010 | 161. 873 | 0. 830 | 0. 819 | 2. 444 | 2. 568 Next, consider a mixed-effects model. ----------------------------------------------------------------------#
section_name: Checking Model Assumptions

section_content: While comparing these indices is often useful, making a decision such as whether to keep or drop a model, can often be difficult as some indices can give conflicting suggestions. Additionally, it may be unclear which index to favour in different contexts. This difficulty is one of the reasons why tests are often useful as they facilitate decisions via "significance" indices like p-values (in a frequentist framework) or Bayes Factors (in a Bayesian framework). 

The generic test_performance() function computes the appropriate test(s) based on the supplied input. For instance, the following example shows results from Vuong's Test (Vuong, 1989). 

test_performance(lm1, lm2, lm3, lm4) -----------------------------------------------------------#
section_name: Testing Models

section_content: performance is licensed under the GNU General Public License (v3. 0), with all source code stored at GitHub (https://github. com/easystats/performance), and with a corresponding issue tracker for bug reporting and feature enhancements. In the spirit of honest and open science, we encourage requests, tips for fixes, feature updates, as well as general questions and concerns via direct interaction with contributors and developers.
section_name: Licensing and Availability

section_name: Acknowledgments

section_content: performance is part of the collaborative easystats ecosystem. Thus, we thank the members of easystats as well as the users.
section_name: Acknowledgments

section_name: Comparing Multiple Models

section_content: For multiple models, users can inspect a table of these indices by calling, compare_perform ance(). data(iris) lm1 <-lm(Sepal. Length ~Species, data = iris) lm2 <-lm(Sepal. Length ~Species + Petal. Length, data = iris) lm3 <-lm(Sepal. Length ~Species * Sepal. Width, data = iris) lm4 <-lm(Sepal. Length ~Species * Sepal. Width + Petal. Length + Petal. Width, data = iris) compare_performance(lm1, lm2, lm3, lm4) -------------------------------------------------------------------#
section_name: Comparing Multiple Models