Detecting latent mean differences between non-invariant groups using ordered categorical variables
Authors:   , Zhang Ti 
Published: April 2021

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] Though more and more applied researchers have begun to treat response options as ordered-categorical variables when conducting measurement invariance (MI) testing, little is known about the role of ordered-categorical variables when comparing latent means between groups. Therefore, this study simulated ordered-categorical data to specifically examine the detection of latent mean differences between non-invariant groups across a variety of conditions, including the number of items, population latent mean differences, etc. The purpose of this study was to investigate the relative parameter bias, power rates, and Type I error rates that may arise when ignoring various types of MI in both the configural invariance and metric invariance models. In summary, the most important contributors to relative bias of the true latent mean difference estimates were a) the number of items and the size of the factor loadings in the configural invariance model, b) the size of the factor loading and threshold differences in the metric invariance model that ignored group parameter differences, and c) the number of items in the metric invariance model that addressed the group parameter differences. Thus, in order to reduce the bias in estimating the true latent mean difference between groups, practitioners should identify and address the non-invariance and use a test instrument with more items. The dominant effect on the power to identify whether the latent mean difference was different from 0, in both the configural invariance model and the metric invariance model that ignored true group differences, was the population latent mean difference. In the metric invariance model that addressed the group differences, the most important effects were a) population latent mean differences, and b) loading and threshold differences. When the latent mean difference was at least moderate or the large threshold difference was ignored, the power rate was inflated to be above .90. Applied researchers should know that it will be easier to detect relatively large latent mean differences if both the loading and threshold differences are free to differ between groups. The dominant effect on Type I error rate in the configural invariance model was the number of items. In the metric invariance model that ignored the group parameter differences, the most important effects were a) the size of threshold differences, b) the loading and threshold differences, and c) the number of items. In the metric invariance model that addressed the group parameter differences, the most important effect on Type I error was the number of noninvariant items, which also significantly interacted with the number of items. Often, applied researchers assume their groups are equal, and may not concern themselves with detecting the true latent mean differences. Of course, true population differences cannot be known, so it is recommended that researchers should still conduct a MI analysis. It is especially important to note that in the metric invariance model that addressed group parameter differences, the Type I error rate was below .05. This result suggests that conducting MI testing will help applied researchers detect the true latent mean difference regardless of the magnitude of that difference (i.e., 0, .2 and .5 in this study).