Levels of variation and intraclass correlation
Channel: Mikko Rönkkö
Date: 2019-07-27

When working with multiple data sets, it is important to analyze the level at which individual observations vary. This can be done by calculating intraclass correlation (ICC) to quantify the variances. Using profitability data as an example, there are three levels of variation that can affect a data set: year-to-year variation within companies, variation between companies, and variation between industries.

In a small data set, graphical analysis can be used to identify patterns, while in larger data sets with manageable numbers of clusters, box plots can be used to understand between and within variances. The ICC(1) is calculated as the variance between groups divided by the total variance, helping to determine how much of the variation in the data is attributed to the groups and how much is within the groups. ICC(1) values close to 0 indicate no meaningful clustering, and values close to 1 indicate no variance within clusters. In cases where ICC(1) is around 0.5, multi-level modeling may be needed to account for the levels in the analysis.

Link to the slides: https://osf.io/fjumc