TITLE: Workflow for beta-range forest plots, bootstrap ridgeline plots, and bootstrap violin plots
AUTHORS: Jonathan Fries, Sandra Oberleiter, Jakob Pietschnig

[DESCRIPTION]
Regression is a widely used statistical method in various research areas, such as educational psychology, and it is common to display regression coefficients in tables. Although tables contain dense information, they can be difficult to read and interpret when they are extensive. To address this issue, we present three innovative visualizations that enable researchers to present a large number of regression models in a single plot. We demonstrate how to transform simulated data and plot the results, which produce visually appealing representations of regression results that are both efficient and intuitive. These visualizations can be applied for screening models in the selection stage or reporting results in research papers. Our method is reproducible using the provided code and can be implemented using free and open-source software routines in R.

[GUIDELINES]
Get R:
The statistical programming environment can be downloaded free-of-charge at https://cran.r-project.org.




[STEPS]
SECTION: Install and load packages
2. The following workflow requires the statistical programming environment R (at least version 4.2.2). Before running any command prompts, some packages need to be installed. To this end, we create a vector that contains the respective package names and run a for loop that only installs packages that are not already installed.

 

Next, we load the required packages.

 

SECTION: Data simulation of decathlon results
9. To demonstrate the bootstrap-based data visualization techniques bootstrap ridgeline plot and bootstrap violin plot, we use simulated data.
SECTION: Data simulation of decathlon results
9.3. We create a dataframe that contains names and distribution parameters for the variables that we intend to simulate. Here, we chose to demonstrate our method using a fictional dataset of decathlon results. Decathlon is a combination athletics discipline. Over the course of two days, each athlete competes in ten track and field events (100 meters race, long jump, shot-put, high jump, 400 meters race, 110 meters hurdles, discus, pole-vault, javelin, and 1500 meters race).

 

SECTION: Data simulation of decathlon results
9.2. A random seed is then defined; this step ensures that the results are exactly replicable. If replicability is of no concern, this command may be removed.

 

SECTION: Data simulation of decathlon results
9.4. The simulated decathlon results are expected to be intercorrelated. Thus, we define a matrix of correlations between the variables that we intend to simulate and convert it to a covariance matrix.

 

SECTION: Data simulation of decathlon results
9.5. Using this covariance matrix, we now generate correlated decathlon results for 10,000 fictional athletes.

 

SECTION: Data simulation of decathlon results
9.6. Some of the generated values are outside the range  of human physical capability. To address this, we truncate the results to stay within reasonable boundaries. The custom helper function we defined above is applied within a for loop.

 

SECTION: Data simulation of decathlon results
9.7. Simulation of athletics results is now completed. Next, some predictors are needed. We chose to use fictional data of seven hematological indices (ferritin, haptoglobin, hematocrit, hemoglobin, iron, red blood cell count, and transferrin) that have been demonstrated to be associated with sports performance. For each hematological index, we simulate normally distributed data that exhibit pre-defined correlations with some of the decathlon events.

 

SECTION: Data simulation of decathlon results
9.8. Data simulation is complete. To tidy up the data, we re-order the dataframe columns.

 

SECTION: Regression for simulated decathlon data
10. Next, we intend to predict decathlon outcomes from hematological indices. We model these relationships using a single-predictor linear regression framework.
SECTION: Regression for simulated decathlon data
10.1. The dataset needs to be converted to long format. The conversion is carried out using the melt command from the reshape2 package.

 

SECTION: Regression for simulated decathlon data
10.2. Now, we run regressions for each unique combination of outcome and predictor in batch and extract parameters using data.table, resulting in 70 regression models stored in a separate dataframe.

 

SECTION: Bootstrap violin plot
13. To address the issue of overlapping density plots, we developed an alternative representation of the distributions of bootstrapped beta coefficients by making use of the violin plot. To create a bootstrap violin plot, the bootstrap dataset from before can be used without modifications.

 

 
SECTION: Computing bootstrapped regression parameter estimates
11. The purpose of the demonstrated plots is the representation of the uncertainty associated with regression effect sizes. We achieve this by computing bootstrapped point estimates and subsequently plotting the resulting distributions.
SECTION: Computing bootstrapped regression parameter estimates
11.1. First, we set a random seed.

 

SECTION: Computing bootstrapped regression parameter estimates
11.2. Next, we define a helper function for bootstrapping. 

 

SECTION: Computing bootstrapped regression parameter estimates
11.3. We create a container dataframe for the botstrapped estimates which we add in the next step.

 

SECTION: Computing bootstrapped regression parameter estimates
11.4. Now we compute bootstrap parameter estimates for all 70 regression models from above (models_decathlon). CAUTION: This computation is fairly lengthy.

 

SECTION: Computing bootstrapped regression parameter estimates
11.5. Some minor formatting operations.

 

SECTION: Computing bootstrapped regression parameter estimates
11.6. We generate factor variables with reverse-ordered levels. This is necessary because the coordinate system of the plots is flipped. Thus, factor levels also need to be "flipped".

 

SECTION: Bootstrap ridgeline plot
12. Now the bootstrapped data are ready for plotting. We create a bootstrap ridgeline plot using ggplot2.

 

  

SECTION: Importing modified large-scale educational assessment data
3. For demonstrating the first data visualization technique (beta-range forest plot), we modified data from the 2019 TIMSS (Trends in Mathematics and Science Study) assessment of 8th graders from 39 countries (n = 87,547). These data were linked to macroeconomic indicators provided by the world bank (https://data.worldbank.org). The data were modified because publication of the original datasets is prohibited. The modified dataset ("timss_modified.xlsx") can be downloaded at: 

Open Science Framework repository

SECTION: Importing modified large-scale educational assessment data
4. We import the dataset. Please select the file "timss_modified.xlsx".


 

SECTION: Importing modified large-scale educational assessment data
4.1. Now the dataset is converted to long format. We use the melt function from the reshape2 package. 

 

SECTION: Regression for modified TIMSS data
5. Now, we run linear regressions in batch for each unique combination of outcome and predictor. In the same step, we extract selected parameters into a data.table. CAUTION: May incur substantial computation time due to the mixed-effects models. 

 

SECTION: Data modification of TIMSS regression dataset
6. Now, we perform some data modifications on the resulting regression dataset.


 

SECTION: Compute estimate ranges, CI ranges, and pooled effect estimates
7. The next step is arguably the most important for plotting the data. Here, we compute the parameters that are subsequently displayed in the plot. 
SECTION: Compute estimate ranges, CI ranges, and pooled effect estimates
7.1. We create an empty container dataframe to append pooled effect estimates to (calculated in loop below).

 

SECTION: Compute estimate ranges, CI ranges, and pooled effect estimates
7.2. Next, lower and upper limits of point estimate ranges as well as CI ranges are stored in a dataframe. In addition, we calculate pooled effect estimates using the procedure laid out by Rubin (1987). 

 

SECTION: Compute estimate ranges, CI ranges, and pooled effect estimates
7.3. Some formatting operations need to be performed.

 

SECTION: Compute estimate ranges, CI ranges, and pooled effect estimates
7.4. Now, we recode predictor and domain variables for nicer display in the plot. Then, the dataframe is reordered.


 

SECTION: Beta-range forest plot
8. The data is now ready for plotting; we use the graphics environment ggplot2. 

 

    

SECTION: Compute estimate ranges, CI ranges, and pooled effect estimates
7.5. Some final formatting operations are necessary. We create factor variables with reverse-ordered levels. This is necessary because the coordinate system of the plots is flipped. Thus, factor levels also need to be "flipped". 

 

SECTION: Recommendations and links
1. We recommend running the code in this protocol from the text file ("regression_plots.R"). All necessary data and code files can be downloaded at this repository:

Open Science Framework repository

SECTION: Data simulation of decathlon results
9.1. A custom helper function for the data simulation process is defined beforehand.