The effect of different operationalizations of affect and preprocessing choices on power-based sample size recommendations in intensive longitudinal research
Creators: name: Lafit, Ginette
name: Cloos, Leonie
name: Revol, Jordan
name: Kuppens, Peter
name: Ceulemans, Eva
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Quantitative Psychology
subject: Social and Behavioral Sciences
subject: Psychology
subject: FOS: Psychology
subject: Affect dynamics
subject: Intensive Longitudinal designs
subject: Power analysis
subject: Replicability
subject: Sample size selection

description: Intensive longitudinal (IL) designs have become indispensable to the investigation of how affect fluctuates in daily life. The data collected with IL designs contains repeated measurements taken from the same individuals over a period of time, which are then usually analyzed with multilevel regression models that account for the serial dependence in the data. These models allow to characterize within-person affective processes, and how they differ between persons. They have for example been widely used to study emotional inertia (Kuppens et al., 2010), or emotional responses to positive and negative events (Dejonckheere et al., 2021). Furthermore, differences between people in emotional inertia or emotional reactivity to events have been linked to depression (Houben et al., 2015; Dejonckheere et al., 2018), as well as personality traits (Hisler et al., 2020). The popularity of IL research has, however, also exposed the multitude of decisions that have to be made concerning study design, data curation, and analyses (Kirtley et al., 2021; Trull &amp; Ebner-Priemer, 2020). The choices made obviously have consequences for the findings, their replicability and comparability across studies. Among the first choices when designing a study is to determine the number of persons and the number of measurement occasions to include. These sample size decisions are ideally based on statistical power considerations. In recent years, a number of methods for conducting a priori power analysis for multilevel models for IL data have been developed (see Lafit et al., 2021 for an overview). However, a major challenge when conducting such power analyses is the lack of a unifying approach for computing standardized effect sizes (Rights &amp; Sterba, 2019). Consequently, power analyses are run by specifying the population values of all model parameters. This is a daunting task, given the large number of model parameters. Therefore, these values are usually set based on data from previous studies (Lane and Hennes, 2018). The use of previous studies for determining the parameter values for a priori power analysis is problematic, however, when data comes from small or unrepresentative samples (Albers &amp; Lakens, 2018; Gelman &amp; Carlin, 2014). Furthermore, there are no standardized procedures to measure affect in daily life (Brose et al., 2020; Cloos et al., n.d.; Horstmann &amp; Ziegler, 2020). In affect dynamic research it is common practice to combine an ad hoc selection of specific emotion items to construct positive or negative affect scores, but what items are used and how they are combined differs across studies (Dejonckheere et al., 2019; Eisele et al., 2021a; 2022; Weermeijer et al., 2022) This may lead to different values for the model parameters and effect sizes which is troublesome for power analyses. Statistical power based on estimates from a previous study may no longer be useful if the planned study uses a different operationalization of momentary affect. Similarly, preprocessing choices that lead to exclusion of measurement occasions within participants due to too short or too long response times, or the exclusion of participants, based on low compliance (see e.g., Eisele et al., 2021b; Eisele et al., 2022; Weermeijer et al., 2022) change the data used to determine the model parameters for computing statistical power, which in turn may also impact the value of the parameter estimates and their uncertainty. In sum, when basing power analyses on previous study results, the outcome could be heavily affected by particular elements of study design and/or data preprocessing (which can be intended or not, sometimes convenience based, or even random). Knowing the impact of such decisions could significantly help to guide well-powered research. Therefore, in this paper, we will showcase the impact of differences in affect measurement and preprocessing choices related to data exclusion on statistical power. We will also provide a hands-on tutorial that allows researchers to implement the proposed analyses for their research questions. The Current Study Using the data of a recent study that includes distinct types of items to measure positive and negative affect (i.e., standard discrete emotions, and unipolar single items), we can investigate this issue in more detail. More specifically, we consider 12 different operationalizations of momentary affective constructs (i.e., positive and negative affect). In addition, we also look at data exclusion procedures related to response time and compliance, and investigate how they impact statistical power when evaluating (1) associations between momentary positive and negative affect and person-level predictors such as neuroticism and depression; (2) within-person associations between momentary positive and negative affect and event intensity; and (3) within-person lagged effects in momentary affect (i.e., auto- and cross-regressive effects between positive and negative affect). Specifically, we expect that the 12 different measurement operationalizations in combination with the exclusion choices will yield different values of the model parameters, and thus, would result in different sample size recommendations. Expected Conclusions The proposed study aims to shed light on the effect of different operationalizations of momentary affect and preprocessing choices related to data exclusion on sample size recommendations based on power analysis. Due to a lack of general recommendations on which operationalization of affect or preprocessing choice to use, it is crucial to investigate to which extent such decisions influence statistical power, and consequently the replicability of findings in daily life affective research.