# Longitudinal measurement invariance assessment at the item level with Rasch family models: performances of the ROSALI algorithm for response shift detection

Myriam Blanchin, Yseulys Dubuy, Priscilla Brisson, Karima Hammas, Odile

Stahl, Véronique Sébille

► To cite this version:

Myriam Blanchin, Yseulys Dubuy, Priscilla Brisson, Karima Hammas, Odile Stahl, et al. Longitudinal measurement invariance assessment at the item level with Rasch family models: performances of the ROSALI algorithm for response shift detection. 10th European Congress of Methodology [EAM 2023], Jul 2023, Ghent, Belgium. . [hal-04779802](https://hal.science/hal-04779802v1)

HAL Id: hal-04779802

https://hal.science/hal-04779802v1

Submitted on 13 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Longitudinal measurement invariance assessment at the item level with Rasch family models: performance of the ROSALI algorithm for response shift detection

SPHERE
methodS in
Patient-centered outcomes
& Health Research
http://sphere-nantes.fr/

Blanchin M, Dubuy Y, Brisson P, Hammas K, Stahl O, Sébille V

Nantes
Université

Nantes Université, Université de Tours, INSERM, MethodS in Patient-centered outcomes and Health Research, SPHERE, Nantes, France

myriam.blanchin@univ-nantes.fr

## LONGITUDINAL MEASUREMENT INVARIANCE IN PATIENT-REPORTED OUTCOME RESEARCH

Patients might not respond to patient-reported outcomes (PRO) consistently over time because their level of the target construct (e.g., quality of life) has changed but also because their interpretation of the items aiming at measuring the construct has changed.

Lack of longitudinal measurement invariance is a concern as it can bias the estimation of PRO change. This phenomenon theorized as response shift (RS) in PRO research is also viewed as a meaningful effect to quantify for a better understanding of patients' adaptation.

### ROSALI (Hammas et al., 2020)

<figure><img src="image_2.png" /></figure>

## RESPONSE SHIFT DETECTION AT THE ITEM LEVEL

### "RespOnse Shift ALgorithm at the Item level" (ROSALI)

* iterative procedure

* Bonferroni correction (multiple testing)

* based on Rasch models

* **DIF** and **RS** detection (recalibration: change in internal standards)

* identification of affected items

* In studies comparing 2 or 3 groups: assessment of heterogeneity of RS

* group and time effects estimation (differences in latent variable means) adjusted for DIF and RS

Stata module on SSC

Freely accessible on https://pro-online.net/

## SIMULATION STUDY

**Objective:** assess the performances of ROSALI in studies comparing two groups of patients in terms of

* RS detection at the item level

* group and time effect estimations

**Scenarios:**

* no RS

* RS in only one group

* RS affecting both groups similarly (same type, direction and magnitude)

* RS affecting both groups in a different way (same type, different direction and magnitude)

<table><thead><tr><th>Simulation parameters</th><th>Values</th></tr></thead><tbody><tr><td>Sample size (2 equal group sizes)</td><td>200; 300; 500</td></tr><tr><td>Number of items</td><td>4; 7</td></tr><tr><td>Number of answer categories</td><td>4; 7</td></tr><tr><td>Group effect</td><td>0; 0,2</td></tr><tr><td>Time effect</td><td>0; 0,3</td></tr><tr><td>Group by time interaction</td><td>0</td></tr><tr><td>Number of items affected by RS</td><td>1 (central item or item with the highest location)</td></tr><tr><td>Type of RS</td><td>Uniform (same shift for all thresholds over time); Non-uniform</td></tr></tbody></table>

624 combinations, 500 replications

**Criteria:**

* no simulated RS: rates of **false detection** of RS (detection of RS on at least one item)

* simulated RS:

* rates of **detection of RS** (detection of RS on at least one item)

* rates of **perfect RS detection** (correct identification of type of RS, and of items and groups affected by RS)

* **biases** of estimated differences in latent variable means

Sprangers MA, Schwartz CE. Soc Sci Med. 1999;48(11):1507-15.
Vanier A, et al. Qual Life Res. 2021;30(12):3309-22.
Sprangers MAG, et al. Qual Life Res. 2021;30(12):3299-308.
Blanchin M, Brisson P. ROSALI. Statistical Software Components. 2020.
Hammas K, et al. Front Psychol. 2020;11:3704.
Blanchin M, et al. Methods. 2022;204:327-39.

## RESULTS

Rates of **false detection** of RS are low (between 2% and 7%). None of the simulation parameters seems to have an impact on this rate.

Rates of **detection of RS** range from 87% to 100% when uniform RS was simulated. They are lower when non-uniform RS was simulated (25%-100%). The lowest rates are observed for the lowest sample size, the highest number of items and answer categories.

Rates of **perfect RS detection** show the same trends with higher rates when uniform RS was simulated. The effect of simulation parameters are summarized below for datasets with 7 items.

<figure><img src="image_3.png" /></figure>

M: number of answer categories, N: total sample size, RS: response shift

**ROSALI is able to detect RS and identify the item and group(s) affected when RS is uniform.** Uniform RS is more easily detected than non-uniform RS as it affects all item thresholds the same way. Performances are also better when both groups are affected similarly by RS.

**In the least favorable scenarios, at least 500 patients are required to perform RS detection using ROSALI.**