# Associations of clinical, psychological, and sociodemographic characteristics and ecological momentary assessment completion in the 10-week Hypo-METRICS study: Hypoglycaemia MEasurements ThResholds and ImpaCtS

Natalie Zaremba, Gilberte Martine-edith, Patrick Divilly, Uffe Søholm, Melanie Broadley, Namam Ali, Evertine J Abbink, Bastiaan de Galan, Monika Cigler, Julia K Mader, et al.

<figure><img src="image_1.png" /></figure>

► To cite this version:

Natalie Zaremba, Gilberte Martine-edith, Patrick Divilly, Uffe Søholm, Melanie Broadley, et al. Associations of clinical, psychological, and sociodemographic characteristics and ecological momentary assessment completion in the 10-week Hypo-METRICS study: Hypoglycaemia MEasurements ThResholds and ImpaCtS. Diabetic Medicine, In press, 10.1111/dme.15345. hal-04583301

HAL Id: hal-04583301

https://hal.science/hal-04583301

Submitted on 22 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
RESEARCH ARTICLE

Care Delivery

DIABETIC
Medicine

# Associations of clinical, psychological, and sociodemographic characteristics and ecological momentary assessment completion in the 10-week Hypo-METRICS study: Hypoglycaemia MEasurements ThResholds and ImpaCtS

Natalie Zaremba¹ ID

Gilberte Martine-Edith¹ ID

Patrick Divilly¹ ID

Uffe Søholm¹,²,³ ID

Melanie Broadley³ ID

Namam Ali⁴ ID

Evertine J. Abbink⁴ ID

Bastiaan de Galan⁴,⁵,⁶ ID

Monika Cigler⁷ ID

Julia K. Mader⁷,⁸ ID

Julie Brosen⁹ ID

Ulrik Pedersen-Bjergaard⁹,¹⁰ ID

Allan Vaag¹¹,¹²,¹³ ID

Mark Evans¹⁴ ID

Eric Renard¹⁵,¹⁶ ID

Rory J. McCrimmon¹⁷ ID

Simon Heller¹⁸ ID

Jane Speight³,¹⁹,²⁰ ID

Frans Pouwer³,¹⁹,²¹ ID

Stephanie A. Amiel¹ ID

Pratik Choudhary¹,²² ID

for the Hypo-RESOLVE Consortium

**Correspondence**

Natalie Zaremba, Department of
Diabetes, School of Life Course
Sciences, Faculty of Life Sciences and
Medicine, King's College London,
London SE5 9RJ, UK.

Email: natalie.zaremba@kcl.ac.uk

**Funding information**

Leona M. and Harry B. Helmsley
Charitable Trust; European
Commission; Innovative Medicines
Initiative

**Abstract**

**Introduction:** Reporting of hypoglycaemia and its impact in clinical studies is often retrospective and subject to recall bias. We developed the Hypo-METRICS app to measure the daily physical, psychological, and social impact of hypoglycaemia in adults with type 1 and insulin-treated type 2 diabetes in real-time using ecological momentary assessment (EMA). To help assess its utility, we aimed to determine Hypo-METRICS app completion rates and factors associated with completion.

**Methods:** Adults with diabetes recruited into the Hypo-METRICS study were given validated patient-reported outcome measures (PROMs) at baseline. Over 10 weeks, they wore a blinded continuous glucose monitor (CGM), and were asked to complete three daily EMAs about hypoglycaemia and aspects of daily functioning, and two weekly sleep and productivity PROMs on the bespoke Hypo-METRICS app. We conducted linear regression to determine factors associated with app engagement, assessed by EMA and PROM completion rates and CGM metrics.

**Results:** In 602 participants (55% men; 54% type 2 diabetes; median(IQR) age 56 (45–66) years; diabetes duration 19 (11–27) years; HbA1c 57 (51–65) mmol/mol),

For affiliations refer to page 9.

This is an open access article under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

© 2024 The Authors. *Diabetic Medicine* published by John Wiley & Sons Ltd on behalf of Diabetes UK.
# Associations of clinical, psychological, and sociodemographic characteristics and ecological momentary assessment completion in the 10-week Hypo-METRICS study: Hypoglycaemia MEasurements ThResholds and ImpaCtS

Natalie Zaremba¹ ID | Gilberte Martine-Edith¹ ID | Patrick Divilly¹ ID |
Uffe Søholm¹,²,³ ID | Melanie Broadley³ ID | Namam Ali⁴ ID | Evertine J. Abbink⁴ |
Bastiaan de Galan⁴,⁵,⁶ ID | Monika Cigler⁷ ID | Julia K. Mader⁷,⁸ ID | Julie Brosen⁹ ID |
Ulrik Pedersen-Bjergaard⁹,¹⁰ ID | Allan Vaag¹¹,¹²,¹³ ID | Mark Evans¹⁴ ID | Eric Renard¹⁵,¹⁶ |
Rory J. McCrimmon¹⁷ ID | Simon Heller¹⁸ ID | Jane Speight³,¹⁹,²⁰ ID |
Frans Pouwer³,¹⁹,²¹ ID | Stephanie A. Amiel¹ ID | Pratik Choudhary¹,²² ID |
**for the Hypo-RESOLVE Consortium**

**Correspondence**

Natalie Zaremba, Department of
Diabetes, School of Life Course
Sciences, Faculty of Life Sciences and
Medicine, King's College London,
London SE5 9RJ, UK.

Email: natalie.zaremba@kcl.ac.uk

**Funding information**

Leona M. and Harry B. Helmsley
Charitable Trust; European
Commission; Innovative Medicines
Initiative

**Abstract**

**Introduction:** Reporting of hypoglycaemia and its impact in clinical studies is often retrospective and subject to recall bias. We developed the Hypo-METRICS app to measure the daily physical, psychological, and social impact of hypoglycaemia in adults with type 1 and insulin-treated type 2 diabetes in real-time using ecological momentary assessment (EMA). To help assess its utility, we aimed to determine Hypo-METRICS app completion rates and factors associated with completion.

**Methods:** Adults with diabetes recruited into the Hypo-METRICS study were given validated patient-reported outcome measures (PROMs) at baseline. Over 10 weeks, they wore a blinded continuous glucose monitor (CGM), and were asked to complete three daily EMAs about hypoglycaemia and aspects of daily functioning, and two weekly sleep and productivity PROMs on the bespoke Hypo-METRICS app. We conducted linear regression to determine factors associated with app engagement, assessed by EMA and PROM completion rates and CGM metrics.

**Results:** In 602 participants (55% men; 54% type 2 diabetes; median(IQR) age 56 (45–66) years; diabetes duration 19 (11–27) years; HbA1c 57 (51–65) mmol/mol),
median(IQR) overall app completion rate was 91 (84–96)%, ranging from 90 (81–96)%, 89 (80–94)% and 94(87–97)% for morning, afternoon and evening checks, respectively. Older age, routine CGM use, greater time below 3.0 mmol/L, and active sensor time were positively associated with app completion.

**Discussion:** High app completion across all app domains and participant characteristics indicates the Hypo-METRICS app is an acceptable research tool for collecting detailed data on hypoglycaemia frequency and impact in real-time.

### KEYWORDS

continuous glucose monitoring, ecological momentary assessment, Hypoglycaemia, Mobile applications, type 1 diabetes, type 2 diabetes

# 1 | INTRODUCTION

Hypoglycaemia is a serious complication of insulin treatment for both type 1 diabetes (T1D) and insulin-treated type 2 diabetes (T2D), affecting both short and long-term health outcomes. It can be a key barrier to optimal diabetes self-management.¹⁻³ In addition to being a serious health risk, self-treated hypoglycaemia is a frequent burden on the daily lives of people with diabetes, impacting daily routines and behaviours and reducing quality of life.⁴,⁵

In most clinical studies, hypoglycaemia and its impact on daily life are typically reported retrospectively using self-report questionnaires and recording cumulative impacts of hypoglycaemia over weeks or months.⁶ This methodology is susceptible to recall bias and is unable to measure the impact of individual episodes of hypoglycaemia.⁷ Furthermore, where questionnaires are administered at hospital clinics or during clinical trials, these may not be representative of usual daily diabetes experience and thus lack ecological validity.⁸⁻¹⁰

Ecological momentary assessment (EMA) is a method of data collection that addresses these issues by using repeated sampling of participants' current experiences or behaviour, typically over a short duration (7–14 days), allowing real-time data collection about participant experiences in their daily lives and environments.¹¹ Prior research has used EMA to collect data about health-related behaviours and symptoms in various conditions, including T1D and T2D.¹²⁻¹⁵ A recent systematic review of health-related EMA studies in non-clinical populations found the highest EMA completion to occur in studies with student populations, event-contingent sampling, and a smartphone delivery system,¹² while other evidence suggests that the type of EMA sampling and study protocol has an effect on study completion in non-clinical, but not clinical populations.¹⁶ Investigations are needed to assess

### Key points

* What is already known? Reporting of hypoglycaemia and its impact on individuals with diabetes is retrospective and subject to recall bias.

* What this study has found? High completion rates of ecological momentary assessment completed via the Hypo-METRICS app indicate the app is an acceptable research tool for collecting detailed data on hypoglycaemia frequency and impact in real-time in people with type 1 and insulin-treated type 2 diabetes.

* What are the implications of the study? This method of data collection should be considered for future research and clinical use when collecting data on the frequency and impact of hypoglycaemia.

how people with insulin-treated diabetes respond to EMA and to determine its utility in healthcare research.

The Hypo-METRICS study is a multinational 10-week observational study using EMA to determine the physical, psychological, social, and health economic impact of hypoglycaemia in people with T1D and T2D.¹⁷ We developed a bespoke smartphone app to collect real-time EMA data about the frequency and impact of hypoglycaemia on daily functioning.¹⁸,¹⁹ The Hypo-METRICS app was developed by a multidisciplinary research team, in conjunction with people with diabetes¹⁸ and has been validated in 100 participants of the Hypo-METRICS study.¹⁹

It is unknown if using an app to complete EMA multiple times per day for 10 weeks is a feasible method of data collection for people with insulin treated diabetes. The aim of the present study was therefore to determine
median(IQR) overall app completion rate was 91 (84–96)%, ranging from 90 (81–96)%, 89 (80–94)% and 94(87–97)% for morning, afternoon and evening checks, respectively. Older age, routine CGM use, greater time below 3.0 mmol/L, and active sensor time were positively associated with app completion.

**Discussion:** High app completion across all app domains and participant characteristics indicates the Hypo-METRICS app is an acceptable research tool for collecting detailed data on hypoglycaemia frequency and impact in real-time.

### KEYWORDS

continuous glucose monitoring, ecological momentary assessment, Hypoglycaemia, Mobile applications, type 1 diabetes, type 2 diabetes

# 1 | INTRODUCTION

Hypoglycaemia is a serious complication of insulin treatment for both type 1 diabetes (T1D) and insulin-treated type 2 diabetes (T2D), affecting both short and long-term health outcomes. It can be a key barrier to optimal diabetes self-management.¹⁻³ In addition to being a serious health risk, self-treated hypoglycaemia is a frequent burden on the daily lives of people with diabetes, impacting daily routines and behaviours and reducing quality of life.⁴,⁵

In most clinical studies, hypoglycaemia and its impact on daily life are typically reported retrospectively using self-report questionnaires and recording cumulative impacts of hypoglycaemia over weeks or months.⁶ This methodology is susceptible to recall bias and is unable to measure the impact of individual episodes of hypoglycaemia.⁷ Furthermore, where questionnaires are administered at hospital clinics or during clinical trials, these may not be representative of usual daily diabetes experience and thus lack ecological validity.⁸⁻¹⁰

Ecological momentary assessment (EMA) is a method of data collection that addresses these issues by using repeated sampling of participants' current experiences or behaviour, typically over a short duration (7–14 days), allowing real-time data collection about participant experiences in their daily lives and environments.¹¹ Prior research has used EMA to collect data about health-related behaviours and symptoms in various conditions, including T1D and T2D.¹²⁻¹⁵ A recent systematic review of health-related EMA studies in non-clinical populations found the highest EMA completion to occur in studies with student populations, event-contingent sampling, and a smartphone delivery system,¹² while other evidence suggests that the type of EMA sampling and study protocol has an effect on study completion in non-clinical, but not clinical populations.¹⁶ Investigations are needed to assess

### Key points

* What is already known? Reporting of hypoglycaemia and its impact on individuals with diabetes is retrospective and subject to recall bias.

* What this study has found? High completion rates of ecological momentary assessment completed via the Hypo-METRICS app indicate the app is an acceptable research tool for collecting detailed data on hypoglycaemia frequency and impact in real-time in people with type 1 and insulin-treated type 2 diabetes.

* What are the implications of the study? This method of data collection should be considered for future research and clinical use when collecting data on the frequency and impact of hypoglycaemia.

how people with insulin-treated diabetes respond to EMA and to determine its utility in healthcare research.

The Hypo-METRICS study is a multinational 10-week observational study using EMA to determine the physical, psychological, social, and health economic impact of hypoglycaemia in people with T1D and T2D.¹⁷ We developed a bespoke smartphone app to collect real-time EMA data about the frequency and impact of hypoglycaemia on daily functioning.¹⁸,¹⁹ The Hypo-METRICS app was developed by a multidisciplinary research team, in conjunction with people with diabetes¹⁸ and has been validated in 100 participants of the Hypo-METRICS study.¹⁹

It is unknown if using an app to complete EMA multiple times per day for 10 weeks is a feasible method of data collection for people with insulin treated diabetes. The aim of the present study was therefore to determine
the data collection completion rates and sociodemographic and clinical factors associated with app completion of participants using the Hypo-METRICS app over 10 weeks.

## 2 | METHODS

### 2.1 | Hypo-METRICS study protocol and data collection

The Hypo-METRICS study took place across nine sites in five countries (Austria, Denmark, France, The Netherlands, and United Kingdom) from October 2020 to August 2022. To be eligible for the Hypo-METRICS study, participants were required to have a clinical diagnosis of T1D or T2D treated with one or more daily injections of insulin, and to have experienced at least one hypoglycaemic episode of any degree of severity in the past 3 months. Potential participants were provided study information and gave written informed consent. The full study protocol including eligibility criteria, recruitment strategy, consent and baseline procedures has been previously published.¹⁷ Participants attended a baseline assessment including collection of blood

samples for HbA1c, medical and diabetes history, past severe hypoglycaemia episodes, and completion of electronic validated person-reported outcome measures (PROMs) using the Qualtrics survey platform (https://www.qualtrics.com/), and were instructed on how to use the study devices (a blinded freestyle Libre 2 and a Fitbit charge 4™ activity monitor) and the Hypo-METRICS app.¹⁷ The baseline assessment was done in-person or virtually via a videoconference according to local COVID-19 safety guidance and COVID-19 risk assessments. Data collection continued for 10 weeks, with an end of study visit collecting repeated baseline data (HbA1c, severe hypoglycaemia episodes and PROMs).

### 2.2 | Ecological momentary assessment

Participants were asked to complete three EMA questionnaires ('Check-ins') on the Hypo-METRICS app in the morning, afternoon, and evening, every day for 10 weeks (Figure 1). Check-ins consisted of questions on hypoglycaemia episodes since the last check-in, and aspects of daily functioning (Table S1). At the end of each evening check-in, participants were asked to complete the EQ-5D-5L, a brief validated questionnaire measuring health

<figure><img src="image_1.png" /><figcaption>**FIGURE 1** Screenshots of the Hypo-METRICS app on the uMotif Limited platform.</figcaption></figure>
the data collection completion rates and sociodemographic and clinical factors associated with app completion of participants using the Hypo-METRICS app over 10 weeks.

## 2 | METHODS

### 2.1 | Hypo-METRICS study protocol and data collection

The Hypo-METRICS study took place across nine sites in five countries (Austria, Denmark, France, The Netherlands, and United Kingdom) from October 2020 to August 2022. To be eligible for the Hypo-METRICS study, participants were required to have a clinical diagnosis of T1D or T2D treated with one or more daily injections of insulin, and to have experienced at least one hypoglycaemic episode of any degree of severity in the past 3 months. Potential participants were provided study information and gave written informed consent. The full study protocol including eligibility criteria, recruitment strategy, consent and baseline procedures has been previously published.¹⁷ Participants attended a baseline assessment including collection of blood

samples for HbA1c, medical and diabetes history, past severe hypoglycaemia episodes, and completion of electronic validated person-reported outcome measures (PROMs) using the Qualtrics survey platform (https://www.qualtrics.com/), and were instructed on how to use the study devices (a blinded freestyle Libre 2 and a Fitbit charge 4™ activity monitor) and the Hypo-METRICS app.¹⁷ The baseline assessment was done in-person or virtually via a videoconference according to local COVID-19 safety guidance and COVID-19 risk assessments. Data collection continued for 10 weeks, with an end of study visit collecting repeated baseline data (HbA1c, severe hypoglycaemia episodes and PROMs).

### 2.2 | Ecological momentary assessment

Participants were asked to complete three EMA questionnaires ('Check-ins') on the Hypo-METRICS app in the morning, afternoon, and evening, every day for 10 weeks (Figure 1). Check-ins consisted of questions on hypoglycaemia episodes since the last check-in, and aspects of daily functioning (Table S1). At the end of each evening check-in, participants were asked to complete the EQ-5D-5L, a brief validated questionnaire measuring health

<figure><img src="image_3.png" /><figcaption>**FIGURE 1** Screenshots of the Hypo-METRICS app on the uMotif Limited platform.</figcaption></figure>
status.²⁰ Participants recorded hypoglycaemia episodes and the symptoms they experienced in a symptom tracker ('motif', Figure 1) In real-time, after any hypoglycaemia episode had occurred and been treated. Once per week, participants were given two validated PROMs, the Patient-Reported Outcomes Measurement Information System (PROMIS) Sleep Disturbance scale to assess sleep quality²¹ and the Work Productivity and Activity Impairment Questionnaire to assess workplace productivity.²²

Check-ins were available at designated time periods to ensure responses were reflective of the associated time points.¹⁸ Participants received notifications on their phones to remind them to complete each check-in during the designated time period (morning: 6:00–12:00 with notification reminder at 7:00; afternoon 12:00–18:00 with notification reminder at 15:00; evening 18:00–24:00 with notification reminder at 21:00).

## 2.3 | Participants' characteristics

The following demographic and clinical baseline characteristics were considered: HbA1c, age, sex, type of diabetes, ethnicity, employment status, highest level of education, method of glucose monitoring, insulin therapy, and hypoglycaemia awareness status. These data were collected by research staff the baseline study visit and recorded electronically on REDCap.²³ Participants completed a questionnaire booklet with 14 questionnaires to assess different PROMs (Table 1). Scores were calculated based on the scoring methods for each questionnaire and subscale. Six glucose characteristics during the study were considered: Baseline HbA1c; percentage of time the study sensor was active; Time in range (TIR); time below range

(TBR); time above range (TAR). CGM metrics were derived from blinded CGM data collected during the study and calculated using the iglu package in R.

## 2.4 | App completion rates

Completion rates were calculated as the percentage of completed entries of check-ins and questionnaires out of the total maximum entries over 10 weeks. A maximum of 70 entries was possible for morning, afternoon, and evening check-ins, and EQ-5D-5L. A maximum of 10 entries was possible for weekly PROMIS and WPAI questionnaires.

## 2.5 | Statistical analysis

Median app completion rates (%) were calculated per participant for each element of data collection (morning, afternoon, and evening check-ins; EQ-5D-5L; weekly validated PROMs). We used linear regression with Bonferroni correction to determine if baseline demographic or clinical characteristics, PROMs, or glucose metrics were associated with overall app completion.

We used linear regression to examine the associations between baseline demographic, clinical, PROMs and glucose characteristics and overall app completion rates in three steps.

Firstly, we examined the relationships between baseline clinical and demographic characteristics (age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score) and app completion rates.

<table><caption>TABLE 1 Person-reported outcome measures and domains measured at Hypo-METRICS baseline appointment.</caption><thead><tr><td>Person-reported outcome measure</td><td>Domain assessed</td></tr></thead><tbody><tr><td>Dawn Impact of Diabetes Profile (DIDP)²⁴</td><td>Diabetes-specific quality of life</td></tr><tr><td>Generalised Anxiety Disorder scale (GAD-7)²⁵</td><td>Anxiety symptoms</td></tr><tr><td>Problem Areas In Diabetes (PAID)²⁶</td><td>Diabetes distress</td></tr><tr><td>Perceived Deficit Questionnaire (PDQ-20), Attention and concentration subscale, planning and organisation subscale, retroactive memory subscale and prospective memory subscale²⁷</td><td>Perceived cognitive difficulties</td></tr><tr><td>Patient Health Questionnaire (PHQ-9)²⁸</td><td>Depressive symptoms</td></tr><tr><td>Hypoglycaemia Fear Survey II (HFS-II), behaviour subscale, and worry subscale²⁹</td><td>Fear of hypoglycaemia</td></tr><tr><td>Hypoglycaemia Awareness Questionnaire (HypoA-Q)³⁰</td><td>Hypoglycaemia awareness</td></tr><tr><td>Short form questionnaire (SF-36) Vitality subscale³¹</td><td>Energy levels</td></tr><tr><td>Type-D personality scale (DS14), social inhibition subscale and negative affectivity subscale³²</td><td>Type D personality traits</td></tr></tbody></table>
status.²⁰ Participants recorded hypoglycaemia episodes and the symptoms they experienced in a symptom tracker ('motif', Figure 1) In real-time, after any hypoglycaemia episode had occurred and been treated. Once per week, participants were given two validated PROMs, the Patient-Reported Outcomes Measurement Information System (PROMIS) Sleep Disturbance scale to assess sleep quality²¹ and the Work Productivity and Activity Impairment Questionnaire to assess workplace productivity.²²

Check-ins were available at designated time periods to ensure responses were reflective of the associated time points.¹⁸ Participants received notifications on their phones to remind them to complete each check-in during the designated time period (morning: 6:00–12:00 with notification reminder at 7:00; afternoon 12:00–18:00 with notification reminder at 15:00; evening 18:00–24:00 with notification reminder at 21:00).

## 2.3 | Participants' characteristics

The following demographic and clinical baseline characteristics were considered: HbA1c, age, sex, type of diabetes, ethnicity, employment status, highest level of education, method of glucose monitoring, insulin therapy, and hypoglycaemia awareness status. These data were collected by research staff the baseline study visit and recorded electronically on REDCap.²³ Participants completed a questionnaire booklet with 14 questionnaires to assess different PROMs (Table 1). Scores were calculated based on the scoring methods for each questionnaire and subscale. Six glucose characteristics during the study were considered: Baseline HbA1c; percentage of time the study sensor was active; Time in range (TIR); time below range

(TBR); time above range (TAR). CGM metrics were derived from blinded CGM data collected during the study and calculated using the iglu package in R.

## 2.4 | App completion rates

Completion rates were calculated as the percentage of completed entries of check-ins and questionnaires out of the total maximum entries over 10 weeks. A maximum of 70 entries was possible for morning, afternoon, and evening check-ins, and EQ-5D-5L. A maximum of 10 entries was possible for weekly PROMIS and WPAI questionnaires.

## 2.5 | Statistical analysis

Median app completion rates (%) were calculated per participant for each element of data collection (morning, afternoon, and evening check-ins; EQ-5D-5L; weekly validated PROMs). We used linear regression with Bonferroni correction to determine if baseline demographic or clinical characteristics, PROMs, or glucose metrics were associated with overall app completion.

We used linear regression to examine the associations between baseline demographic, clinical, PROMs and glucose characteristics and overall app completion rates in three steps.

Firstly, we examined the relationships between baseline clinical and demographic characteristics (age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score) and app completion rates.

<table><caption>TABLE 1 Person-reported outcome measures and domains measured at Hypo-METRICS baseline appointment.</caption><thead><tr><td>Person-reported outcome measure</td><td>Domain assessed</td></tr></thead><tbody><tr><td>Dawn Impact of Diabetes Profile (DIDP)²⁴</td><td>Diabetes-specific quality of life</td></tr><tr><td>Generalised Anxiety Disorder scale (GAD-7)²⁵</td><td>Anxiety symptoms</td></tr><tr><td>Problem Areas In Diabetes (PAID)²⁶</td><td>Diabetes distress</td></tr><tr><td>Perceived Deficit Questionnaire (PDQ-20), Attention and concentration subscale, planning and organisation subscale, retroactive memory subscale and prospective memory subscale²⁷</td><td>Perceived cognitive difficulties</td></tr><tr><td>Patient Health Questionnaire (PHQ-9)²⁸</td><td>Depressive symptoms</td></tr><tr><td>Hypoglycaemia Fear Survey II (HFS-II), behaviour subscale, and worry subscale²⁹</td><td>Fear of hypoglycaemia</td></tr><tr><td>Hypoglycaemia Awareness Questionnaire (HypoA-Q)³⁰</td><td>Hypoglycaemia awareness</td></tr><tr><td>Short form questionnaire (SF-36) Vitality subscale³¹</td><td>Energy levels</td></tr><tr><td>Type-D personality scale (DS14), social inhibition subscale and negative affectivity subscale³²</td><td>Type D personality traits</td></tr></tbody></table>
Secondly, we examined the relationships between PROMs (Table 1) and app completion rates, adjusted for baseline clinical and demographic characteristics (age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score).

Lastly, we examined the relationships between glucose characteristics (baseline HbA1c; percent of blinded CGM active time; percent of time above 10.0 mmol/L (180 mg/dL); percent of time in clinical target range (4.0–10 mmol/L; 70–180 mg/dL); percent of time below 3.9 mmol/L (70 mg/dL); percent of time below 3.0 mmol/L (54 mg/dL)) and app completion rates, adjusted for baseline clinical and demographic characteristics (age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score).

Complete case analysis was used, participants with missing values were removed from the regression analysis.

All analyses were conducted using R studio version 2023.06.2 + 561.33

# 3 | RESULTS

## 3.1 | Baseline characteristics

A total of 602 participants completed the 10-week Hypo-METRICS study. Baseline demographic and clinical characteristics can be seen in Table 2.

## 3.2 | Hypo-METRICS app completion rates

Median (IQR) completion rate for morning check-ins was 90% (81–96%), afternoon check-ins was 89% (80–94%), and evening check-ins was 94% (87–97%). The median (IQR) completion rate for all three check-ins was 90% (83–95%).

Median completion rate for EQ-5D-5L was 93% (84–97%). Median completion rate for both weekly PROMIS and WPAI questionnaires was 100% (90–100%).

The overall median (IQR) completion rate for all questionnaires (Check-ins, EQ-5D-5L, and weekly validated questionnaires) was 91% (84–96%).

The overall study dropout rate was low with 17 participants withdrawing from the study during the 10-week data collection period, reporting study burden, problems using study devices, or personal circumstances such as illness as their reason for withdrawal. An additional 13 participants were lost to follow up during the 10-week study period.

## 3.3 | Factors associated with app completion

### 3.3.1 | Demographic and clinical characteristics

Age and routine use of CGM were positively associated with overall app completion rates of all questionnaires (Table 3), the association remained after correction for multiple comparisons. There was no association between completion rates and type of diabetes, education status, or employment status.

### 3.3.2 | Baseline person-reported outcome measures

After correction for multiple comparisons, no PROMs were significantly associated with app completion (Table 4).

### 3.3.3 | Glucose characteristics

The percent of time the study CGM was active, and study CGM time below 3.0 mmol/L were positively associated with app completion (Table 5).

# 4 | DISCUSSION

This study aimed to evaluate the usability and acceptability of the Hypo-METRICS app by examining app completion rates, and factors associated with app completion during the 10-week Hypo-METRICS study. The over-90% median completion rate across all questionnaires shows there was high engagement across study participants. Small effect sizes for factors associated with app completion (age, routine CGM use, active sensor time, and time below 3.0 mmol/L) suggest that the Hypo-METRICS app is an acceptable tool to collect EMA data about hypoglycaemia and its impact on daily functioning in people with T1D and T2D.

Across all questionnaires, weekly questionnaires had the highest completion rate, with a median of 100% completion. Of the daily check-ins, evening check-ins had the highest completion (94%), followed by EQ-5D-5L (92%) and morning check-ins (90%). The afternoon check-in was the questionnaire with the lowest median completion rate, although still high at 89%.

The Hypo-METRICS study, lasting 10 weeks for each participant, is one of the longest running EMA studies and has a large sample size (N = 602), with EMA studies in clinical populations generally having a median duration of
Secondly, we examined the relationships between PROMs (Table 1) and app completion rates, adjusted for baseline clinical and demographic characteristics (age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score).

Lastly, we examined the relationships between glucose characteristics (baseline HbA1c; percent of blinded CGM active time; percent of time above 10.0 mmol/L (180 mg/dL); percent of time in clinical target range (4.0–10 mmol/L; 70–180 mg/dL); percent of time below 3.9 mmol/L (70 mg/dL); percent of time below 3.0 mmol/L (54 mg/dL)) and app completion rates, adjusted for baseline clinical and demographic characteristics (age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score).

Complete case analysis was used, participants with missing values were removed from the regression analysis.

All analyses were conducted using R studio version 2023.06.2 + 561.33

# 3 | RESULTS

## 3.1 | Baseline characteristics

A total of 602 participants completed the 10-week Hypo-METRICS study. Baseline demographic and clinical characteristics can be seen in Table 2.

## 3.2 | Hypo-METRICS app completion rates

Median (IQR) completion rate for morning check-ins was 90% (81–96%), afternoon check-ins was 89% (80–94%), and evening check-ins was 94% (87–97%). The median (IQR) completion rate for all three check-ins was 90% (83–95%).

Median completion rate for EQ-5D-5L was 93% (84–97%). Median completion rate for both weekly PROMIS and WPAI questionnaires was 100% (90–100%).

The overall median (IQR) completion rate for all questionnaires (Check-ins, EQ-5D-5L, and weekly validated questionnaires) was 91% (84–96%).

The overall study dropout rate was low with 17 participants withdrawing from the study during the 10-week data collection period, reporting study burden, problems using study devices, or personal circumstances such as illness as their reason for withdrawal. An additional 13 participants were lost to follow up during the 10-week study period.

## 3.3 | Factors associated with app completion

### 3.3.1 | Demographic and clinical characteristics

Age and routine use of CGM were positively associated with overall app completion rates of all questionnaires (Table 3), the association remained after correction for multiple comparisons. There was no association between completion rates and type of diabetes, education status, or employment status.

### 3.3.2 | Baseline person-reported outcome measures

After correction for multiple comparisons, no PROMs were significantly associated with app completion (Table 4).

### 3.3.3 | Glucose characteristics

The percent of time the study CGM was active, and study CGM time below 3.0 mmol/L were positively associated with app completion (Table 5).

# 4 | DISCUSSION

This study aimed to evaluate the usability and acceptability of the Hypo-METRICS app by examining app completion rates, and factors associated with app completion during the 10-week Hypo-METRICS study. The over-90% median completion rate across all questionnaires shows there was high engagement across study participants. Small effect sizes for factors associated with app completion (age, routine CGM use, active sensor time, and time below 3.0 mmol/L) suggest that the Hypo-METRICS app is an acceptable tool to collect EMA data about hypoglycaemia and its impact on daily functioning in people with T1D and T2D.

Across all questionnaires, weekly questionnaires had the highest completion rate, with a median of 100% completion. Of the daily check-ins, evening check-ins had the highest completion (94%), followed by EQ-5D-5L (92%) and morning check-ins (90%). The afternoon check-in was the questionnaire with the lowest median completion rate, although still high at 89%.

The Hypo-METRICS study, lasting 10 weeks for each participant, is one of the longest running EMA studies and has a large sample size (N = 602), with EMA studies in clinical populations generally having a median duration of
<table><caption>TABLE 2 Baseline demographic and clinical characteristics of participants in the Hypo-METRICS study.</caption><thead><tr><th>Characteristic</th><th>Total (n = 602)</th><th>Type 1 diabetes (n = 277)</th><th>Type 2 diabetes (n = 325)</th></tr></thead><tbody><tr><td>Age (years), median (IQR)</td><td>56 (45–66)</td><td>47 (30–56)</td><td>63 (55–69)</td></tr><tr><td>Diabetes duration (years), median (IQR)</td><td>19 (11–27)</td><td>20 (9–34)</td><td>18 (13–24)</td></tr><tr><td>Sex, n (%)</td><td></td><td></td><td></td></tr><tr><td>Men</td><td>331 (55)</td><td>127 (46)</td><td>204 (63)</td></tr><tr><td>Women</td><td>269 (45)</td><td>148 (53)</td><td>121 (37)</td></tr><tr><td>Other</td><td>2 (0.4)</td><td>2 (0.7)</td><td>0</td></tr><tr><td>Ethnicity, n (%)</td><td></td><td></td><td></td></tr><tr><td>White</td><td>536 (89)</td><td>244 (88)</td><td>292 (90)</td></tr><tr><td>Black</td><td>14 (2.3)</td><td>4 (1.4)</td><td>10 (3.1)</td></tr><tr><td>Asian</td><td>19 (3.2)</td><td>3 (1.1)</td><td>16 (4.9)</td></tr><tr><td>Other</td><td>34 (5.6)</td><td>26 (9.4)</td><td>7 (2.2)</td></tr><tr><td>Employment, n (%)</td><td></td><td></td><td></td></tr><tr><td>Full-time employment</td><td>225 (37)</td><td>138 (50)</td><td>87 (27)</td></tr><tr><td>Retired</td><td>214 (36)</td><td>47 (17)</td><td>167 (51)</td></tr><tr><td>Part-time employment</td><td>84 (14)</td><td>54 (19)</td><td>30 (9.2)</td></tr><tr><td>Unemployed</td><td>59 (9.8)</td><td>23 (8.3)</td><td>36 (11)</td></tr><tr><td>Full-time education</td><td>20 (3.3)</td><td>15 (5.4)</td><td>5 (1.5)</td></tr><tr><td>Highest level of completed Education, n (%)</td><td></td><td></td><td></td></tr><tr><td>School</td><td>212 (35)</td><td>71 (26)</td><td>141 (43)</td></tr><tr><td>College/Undergraduate degree</td><td>233 (39)</td><td>121 (44)</td><td>112 (34)</td></tr><tr><td>Postgraduate degree</td><td>107 (18)</td><td>70 (25)</td><td>37 (11)</td></tr><tr><td>Other</td><td>50 (8.3)</td><td>15 (5.4)</td><td>35 (11)</td></tr><tr><td>HbA1c (mmol/mol), median (IQR)</td><td>57 (51–65)</td><td>56 (50–63)</td><td>59 (51–66)</td></tr><tr><td>HbA1c (%)</td><td>7.4 (6.8–8.1)</td><td>7.3 (6.7–7.9)</td><td>7.5 (6.8–8.2)</td></tr><tr><td>Insulin therapy, n (%)</td><td></td><td></td><td></td></tr><tr><td>Multiple daily injections (MDI)</td><td>331 (55)</td><td>169 (61)</td><td>162 (50)</td></tr><tr><td>Insulin pump</td><td>107 (18)</td><td>98 (35)</td><td>9 (2.8)</td></tr><tr><td>Bidaily mixed insulin (BD mix)</td><td>29 (4.8)</td><td>0</td><td>29 (8.9)</td></tr><tr><td>Basal plus</td><td>43 (7.1)</td><td>7 (2.5)</td><td>36 (11)</td></tr><tr><td>Three times daily mixed insulin (TDS)</td><td>7 (1.1)</td><td>0</td><td>7 (2.2)</td></tr><tr><td>Other</td><td>85 (14)</td><td>3 (1.1)</td><td>82 (25)</td></tr><tr><td>Method of glucose monitoring, n (%)</td><td></td><td></td><td></td></tr><tr><td>Continuous/Flash glucose monitoring</td><td>343 (57)</td><td>210 (76)</td><td>133 (41)</td></tr><tr><td>Capillary blood glucose</td><td>259 (43)</td><td>67 (24)</td><td>192 (59)</td></tr><tr><td>Awareness of hypoglycaemia (Gold score &lt;4), n (%)</td><td></td><td></td><td></td></tr><tr><td>Intact</td><td>456 (76)</td><td>219 (79)</td><td>237 (73)</td></tr><tr><td>Impaired</td><td>146 (24)</td><td>58 (21)</td><td>88 (27)</td></tr><tr><td>Country of participation, n (%)</td><td></td><td></td><td></td></tr><tr><td>United Kingdom</td><td>285 (47)</td><td>154 (56)</td><td>131 (40)</td></tr><tr><td>Netherlands</td><td>134 (22)</td><td>35 (13)</td><td>99 (30)</td></tr><tr><td>Austria</td><td>87 (14)</td><td>34 (12)</td><td>53 (16)</td></tr><tr><td>Denmark</td><td>68 (11)</td><td>30 (11)</td><td>38 (12)</td></tr><tr><td>France</td><td>28 (4.7)</td><td>24 (8.7)</td><td>4 (1.2)</td></tr></tbody></table>
<table><caption>TABLE 2 Baseline demographic and clinical characteristics of participants in the Hypo-METRICS study.</caption><thead><tr><th>Characteristic</th><th>Total (n = 602)</th><th>Type 1 diabetes (n = 277)</th><th>Type 2 diabetes (n = 325)</th></tr></thead><tbody><tr><td>Age (years), median (IQR)</td><td>56 (45–66)</td><td>47 (30–56)</td><td>63 (55–69)</td></tr><tr><td>Diabetes duration (years), median (IQR)</td><td>19 (11–27)</td><td>20 (9–34)</td><td>18 (13–24)</td></tr><tr><td>Sex, n (%)</td><td></td><td></td><td></td></tr><tr><td>Men</td><td>331 (55)</td><td>127 (46)</td><td>204 (63)</td></tr><tr><td>Women</td><td>269 (45)</td><td>148 (53)</td><td>121 (37)</td></tr><tr><td>Other</td><td>2 (0.4)</td><td>2 (0.7)</td><td>0</td></tr><tr><td>Ethnicity, n (%)</td><td></td><td></td><td></td></tr><tr><td>White</td><td>536 (89)</td><td>244 (88)</td><td>292 (90)</td></tr><tr><td>Black</td><td>14 (2.3)</td><td>4 (1.4)</td><td>10 (3.1)</td></tr><tr><td>Asian</td><td>19 (3.2)</td><td>3 (1.1)</td><td>16 (4.9)</td></tr><tr><td>Other</td><td>34 (5.6)</td><td>26 (9.4)</td><td>7 (2.2)</td></tr><tr><td>Employment, n (%)</td><td></td><td></td><td></td></tr><tr><td>Full-time employment</td><td>225 (37)</td><td>138 (50)</td><td>87 (27)</td></tr><tr><td>Retired</td><td>214 (36)</td><td>47 (17)</td><td>167 (51)</td></tr><tr><td>Part-time employment</td><td>84 (14)</td><td>54 (19)</td><td>30 (9.2)</td></tr><tr><td>Unemployed</td><td>59 (9.8)</td><td>23 (8.3)</td><td>36 (11)</td></tr><tr><td>Full-time education</td><td>20 (3.3)</td><td>15 (5.4)</td><td>5 (1.5)</td></tr><tr><td>Highest level of completed Education, n (%)</td><td></td><td></td><td></td></tr><tr><td>School</td><td>212 (35)</td><td>71 (26)</td><td>141 (43)</td></tr><tr><td>College/Undergraduate degree</td><td>233 (39)</td><td>121 (44)</td><td>112 (34)</td></tr><tr><td>Postgraduate degree</td><td>107 (18)</td><td>70 (25)</td><td>37 (11)</td></tr><tr><td>Other</td><td>50 (8.3)</td><td>15 (5.4)</td><td>35 (11)</td></tr><tr><td>HbA1c (mmol/mol), median (IQR)</td><td>57 (51–65)</td><td>56 (50–63)</td><td>59 (51–66)</td></tr><tr><td>HbA1c (%)</td><td>7.4 (6.8–8.1)</td><td>7.3 (6.7–7.9)</td><td>7.5 (6.8–8.2)</td></tr><tr><td>Insulin therapy, n (%)</td><td></td><td></td><td></td></tr><tr><td>Multiple daily injections (MDI)</td><td>331 (55)</td><td>169 (61)</td><td>162 (50)</td></tr><tr><td>Insulin pump</td><td>107 (18)</td><td>98 (35)</td><td>9 (2.8)</td></tr><tr><td>Bidaily mixed insulin (BD mix)</td><td>29 (4.8)</td><td>0</td><td>29 (8.9)</td></tr><tr><td>Basal plus</td><td>43 (7.1)</td><td>7 (2.5)</td><td>36 (11)</td></tr><tr><td>Three times daily mixed insulin (TDS)</td><td>7 (1.1)</td><td>0</td><td>7 (2.2)</td></tr><tr><td>Other</td><td>85 (14)</td><td>3 (1.1)</td><td>82 (25)</td></tr><tr><td>Method of glucose monitoring, n (%)</td><td></td><td></td><td></td></tr><tr><td>Continuous/Flash glucose monitoring</td><td>343 (57)</td><td>210 (76)</td><td>133 (41)</td></tr><tr><td>Capillary blood glucose</td><td>259 (43)</td><td>67 (24)</td><td>192 (59)</td></tr><tr><td>Awareness of hypoglycaemia (Gold score &lt;4), n (%)</td><td></td><td></td><td></td></tr><tr><td>Intact</td><td>456 (76)</td><td>219 (79)</td><td>237 (73)</td></tr><tr><td>Impaired</td><td>146 (24)</td><td>58 (21)</td><td>88 (27)</td></tr><tr><td>Country of participation, n (%)</td><td></td><td></td><td></td></tr><tr><td>United Kingdom</td><td>285 (47)</td><td>154 (56)</td><td>131 (40)</td></tr><tr><td>Netherlands</td><td>134 (22)</td><td>35 (13)</td><td>99 (30)</td></tr><tr><td>Austria</td><td>87 (14)</td><td>34 (12)</td><td>53 (16)</td></tr><tr><td>Denmark</td><td>68 (11)</td><td>30 (11)</td><td>38 (12)</td></tr><tr><td>France</td><td>28 (4.7)</td><td>24 (8.7)</td><td>4 (1.2)</td></tr></tbody></table>
<table><caption>TABLE 3 Associations of Hypo-METRICS app completion rates with demographic and clinical characteristics.</caption><thead><tr><th></th><th>Unadjusted estimates (β)<br>(95% CI)</th><th>p</th></tr></thead><tbody><tr><td><strong>Employment</strong></td><td></td><td></td></tr><tr><td>Full-time employment</td><td>Reference</td><td></td></tr><tr><td>Part-time employment</td><td>2.55 (−0.007, 5.11)</td><td>0.050</td></tr><tr><td>Full-time education</td><td>1.63 (−3.15, 6.40)</td><td>0.504</td></tr><tr><td>Unemployed</td><td>2.59 (−0.71, 4.32)</td><td>0.088</td></tr><tr><td>Retired</td><td>1.81 (−0.39, 5.70)</td><td>0.158</td></tr><tr><td><strong>Highest level of education</strong></td><td></td><td></td></tr><tr><td>School</td><td>Reference</td><td></td></tr><tr><td>Undergraduate degree</td><td>0.93 (−0.99, 2.86)</td><td>0.342</td></tr><tr><td>Postgraduate degree</td><td>1.04 (−1.39, 3.47)</td><td>0.400</td></tr><tr><td>Other</td><td>0.54 (−2.59, 3.77)</td><td>0.734</td></tr><tr><td>Age (years)</td><td>0.12 (0.04, 0.20)</td><td><strong>0.003*</strong></td></tr><tr><td><strong>Diabetes type</strong></td><td></td><td></td></tr><tr><td>Type 1</td><td>Reference</td><td></td></tr><tr><td>Type 2</td><td>−0.24 (−1.80, 2.29)</td><td>0.816</td></tr><tr><td><strong>Glucose monitoring method</strong></td><td></td><td></td></tr><tr><td>Capillary blood glucose monitoring</td><td>Reference</td><td></td></tr><tr><td>Continuous/Flash glucose monitoring</td><td>2.84 (1.06, 4.2)</td><td><strong>0.002*</strong></td></tr><tr><td>Gold score</td><td>−0.54 (−1.06, −0.03)</td><td>0.039</td></tr></tbody></table>

Note: Results of linear regression, unadjusted estimates with 95% confidence intervals and *p*-values. Number of complete cases for each variable *n* = 602.

*Significant with Bonferroni correction, *p* < 0.0083.

7 days and around 40 participants.¹⁶ Our completion rate is high, with previous health-related EMA studies reporting a median completion rate of 81% in both clinical and non-clinical populations.¹²,¹⁶ Even the afternoon check-in, which had the lowest completion rate of the daily questionnaires (likely due to potential time restriction in busy daily lives, as reported by participants³⁴), had a completion rate exceeding this. We attribute the high completion rate in this study, at least in part, to both ease of use of the app, and its perceived relevance, two factors driven by significant user involvement in the design process.¹⁸,³⁴ In addition, research teams at each site had frequent communication with study participants. At the baseline visit participants were briefed about the aims of the overall Hypo-METRICS study and the importance of completing the app for data collection. Research teams had biweekly phone calls with each participant to ensure the study protocol was being followed and to help participants with any technical issues throughout the study.¹⁷ While completion rates were high across all participant groups, the higher completion among older adults using a mobile-only data collection schedule is a novel finding, and important for the future of EMA research in diabetes.

We found participants who were older, using CGM, who wore the study CGM for longer and who had longer time below 3.0 mmol/L during the study had higher app

completion rates; however, effect sizes on all significant findings were small. Nevertheless, it is worth considering why these associations occurred. The age effect may relate to inclusion of older people with a large proportion (35.5%) of participants in retirement. These participants may have had more flexibility throughout the day to complete the EMA questionnaires when prompted by the app. Participants who use flash and CGM technology in their routine care may be more familiar with, or more willing to use, diabetes apps and logging information throughout the day, which might explain their greater completion rates of the app. Participants with more time below 3.0 mmol/L likely experience hypoglycaemia more frequently and may therefore have been reminded to complete the questionnaires within the app more often. We found that no PROMs had significant associations with app completion, indicating that app completion is not affected by differences in psychological outcomes.

A key limitation to this research is that our study population is not representative of all people with diabetes. Our participant sample is primarily white, highly educated (56.9% of participants having a college degree or higher), with a high proportion of our participants being retired, a common bias throughout EMA research.¹² Testing the app in populations that were largely missed in this study may be necessary for implementing the app in future research
<table><caption>TABLE 3 Associations of Hypo-METRICS app completion rates with demographic and clinical characteristics.</caption><thead><tr><th></th><th>Unadjusted estimates (β)<br>(95% CI)</th><th>p</th></tr></thead><tbody><tr><td><strong>Employment</strong></td><td></td><td></td></tr><tr><td>Full-time employment</td><td>Reference</td><td></td></tr><tr><td>Part-time employment</td><td>2.55 (−0.007, 5.11)</td><td>0.050</td></tr><tr><td>Full-time education</td><td>1.63 (−3.15, 6.40)</td><td>0.504</td></tr><tr><td>Unemployed</td><td>2.59 (−0.71, 4.32)</td><td>0.088</td></tr><tr><td>Retired</td><td>1.81 (−0.39, 5.70)</td><td>0.158</td></tr><tr><td><strong>Highest level of education</strong></td><td></td><td></td></tr><tr><td>School</td><td>Reference</td><td></td></tr><tr><td>Undergraduate degree</td><td>0.93 (−0.99, 2.86)</td><td>0.342</td></tr><tr><td>Postgraduate degree</td><td>1.04 (−1.39, 3.47)</td><td>0.400</td></tr><tr><td>Other</td><td>0.54 (−2.59, 3.77)</td><td>0.734</td></tr><tr><td>Age (years)</td><td>0.12 (0.04, 0.20)</td><td><strong>0.003*</strong></td></tr><tr><td><strong>Diabetes type</strong></td><td></td><td></td></tr><tr><td>Type 1</td><td>Reference</td><td></td></tr><tr><td>Type 2</td><td>−0.24 (−1.80, 2.29)</td><td>0.816</td></tr><tr><td><strong>Glucose monitoring method</strong></td><td></td><td></td></tr><tr><td>Capillary blood glucose monitoring</td><td>Reference</td><td></td></tr><tr><td>Continuous/Flash glucose monitoring</td><td>2.84 (1.06, 4.2)</td><td><strong>0.002*</strong></td></tr><tr><td>Gold score</td><td>−0.54 (−1.06, −0.03)</td><td>0.039</td></tr></tbody></table>

Note: Results of linear regression, unadjusted estimates with 95% confidence intervals and p-values. Number of complete cases for each variable *n* = 602.

*Significant with Bonferroni correction, *p* < 0.0083.

7 days and around 40 participants.¹⁶ Our completion rate is high, with previous health-related EMA studies reporting a median completion rate of 81% in both clinical and non-clinical populations.¹²,¹⁶ Even the afternoon check-in, which had the lowest completion rate of the daily questionnaires (likely due to potential time restriction in busy daily lives, as reported by participants³⁴), had a completion rate exceeding this. We attribute the high completion rate in this study, at least in part, to both ease of use of the app, and its perceived relevance, two factors driven by significant user involvement in the design process.¹⁸,³⁴ In addition, research teams at each site had frequent communication with study participants. At the baseline visit participants were briefed about the aims of the overall Hypo-METRICS study and the importance of completing the app for data collection. Research teams had biweekly phone calls with each participant to ensure the study protocol was being followed and to help participants with any technical issues throughout the study.¹⁷ While completion rates were high across all participant groups, the higher completion among older adults using a mobile-only data collection schedule is a novel finding, and important for the future of EMA research in diabetes.

We found participants who were older, using CGM, who wore the study CGM for longer and who had longer time below 3.0 mmol/L during the study had higher app

completion rates; however, effect sizes on all significant findings were small. Nevertheless, it is worth considering why these associations occurred. The age effect may relate to inclusion of older people with a large proportion (35.5%) of participants in retirement. These participants may have had more flexibility throughout the day to complete the EMA questionnaires when prompted by the app. Participants who use flash and CGM technology in their routine care may be more familiar with, or more willing to use, diabetes apps and logging information throughout the day, which might explain their greater completion rates of the app. Participants with more time below 3.0 mmol/L likely experience hypoglycaemia more frequently and may therefore have been reminded to complete the questionnaires within the app more often. We found that no PROMs had significant associations with app completion, indicating that app completion is not affected by differences in psychological outcomes.

A key limitation to this research is that our study population is not representative of all people with diabetes. Our participant sample is primarily white, highly educated (56.9% of participants having a college degree or higher), with a high proportion of our participants being retired, a common bias throughout EMA research.¹² Testing the app in populations that were largely missed in this study may be necessary for implementing the app in future research
<table><caption>TABLE 4 Associations of Hypo-METRICS app completion rates with person-reported outcome measures.</caption><thead><tr><th>Domain measured (PROM)</th><th>Adjusted estimates (β) (95% CI)</th><th>p</th></tr></thead><tbody><tr><td>Diabetes-specific quality of life (Dawn Impact of Diabetes Profile; DIDP)<sup>24</sup></td><td>-0.99 (-2.07, 0.09)</td><td>0.072</td></tr><tr><td>Generalised Anxiety Disorder (GAD-7)<sup>25</sup></td><td>-0.19 (-0.51, 0.12)</td><td>0.231</td></tr><tr><td>Diabetes distress (Problem Areas In Diabetes; PAID)<sup>26</sup></td><td>-0.01 (-0.05, 0.07)</td><td>0.765</td></tr><tr><td>Attention and concentration (Perceived Deficit Questionnaire; PDQ-20) attention and concentration subscale<sup>27</sup></td><td>0.05 (-0.36, 0.46)</td><td>0.800</td></tr><tr><td>PDQ-20 Planning and organisation subscale<sup>27</sup></td><td>-0.21 (-0.60, 0.18)</td><td>0.296</td></tr><tr><td>PDQ-20 Retroactive memory subscale<sup>27</sup></td><td>0.19 (-0.22, -0.59)</td><td>0.731</td></tr><tr><td>PDQ-20 Prospective memory subscale<sup>27</sup></td><td>-0.57 (-1.01, -0.13)</td><td>0.012</td></tr><tr><td>Depressive symptoms (Patient Health Questionnaire; PHQ-9)<sup>28</sup></td><td>0.15 (-0.16, 0.47)</td><td>0.332</td></tr><tr><td>Fear of hypoglycaemia (Hypoglycaemia Fear Survey II; HFS-II) Behaviour subscale<sup>29</sup></td><td>-0.07 (-0.17, 0.04)</td><td>0.222</td></tr><tr><td>HFS-II Worry subscale<sup>29</sup></td><td>-0.04 (-0.13, 0.05)</td><td>0.380</td></tr><tr><td>Hypoglycaemia awareness (Hypoglycaemia Awareness Questionnaire; HypoA-Q)<sup>30</sup></td><td>0.06 (-0.23, 0.05)</td><td>0.663</td></tr><tr><td>Short form questionnaire (SF-36) Vitality subscale<sup>31</sup></td><td>0.08 (-0.37, 0.53)</td><td>0.735</td></tr><tr><td>Type-D personality scale – Social inhibition subscale (DS14-SI)<sup>32</sup></td><td>0.11 (-0.01, 0.26)</td><td>0.183</td></tr><tr><td>Type-D personality scale – Negative affectivity subscale (DS14 NA)<sup>32</sup></td><td>0.21 (-0.001, 0.41)</td><td>0.048</td></tr></tbody></table>

Note: Results of linear regression, adjusted estimates with 95% confidence intervals and p-values. Results are adjusted for age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score. Number of complete cases for each variable *n* = 592. Significant with Bonferroni correction, *p* < 0.0025.

<table><caption>TABLE 5 Associations of Hypo-METRICS app completion rates with 10-week study glucose metrics.</caption><thead><tr><th></th><th>Adjusted estimates (β)<br>(95% CI)</th><th>p</th></tr></thead><tbody><tr><td>Baseline HbA1c</td><td>-0.001 (-0.03, 0.03)</td><td>0.938</td></tr><tr><td>% time blinded CGM is active</td><td>0.32 (0.24, 0.39)</td><td><strong>&lt;0.001*</strong></td></tr><tr><td>Time above range (&gt;10.0 mmol/L; 180 mg/dL)</td><td>7.74 (-6.63, 22.12)</td><td>0.291</td></tr><tr><td>Time in range (4.0–10 mmol/L; 70–180 mg/dL)</td><td>7.84 (-6.54, 22.12)</td><td>0.285</td></tr><tr><td>Time below range (&lt;3.9 mmol/L; 70 mg/dL)</td><td>7.29 (-7.08, 21.68)</td><td>0.319</td></tr><tr><td>Time below range (&lt;3.0 mmol/L; 54 mg/dL)</td><td>2.04 (0.84, 3.25)</td><td><strong>&lt;0.001*</strong></td></tr></tbody></table>

Note: Results of linear regression, adjusted estimates with 95% confidence intervals and p-values. Results are adjusted for age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score. Number of complete cases for each variable *n* = 594.

*Significant with Bonferroni correction, *p* < 0.0042.

(e.g. ethnic minorities, adolescents). The recruitment phase of the study began in October 2020 and ended in July 2022, throughout the COVID-19 pandemic. Many of our research participants were recruited remotely during lockdowns in each participating country; therefore, it is possible that lockdowns, changes to daily routines and responsibilities, and furlough from employment during the study may have increased, or decreased, some people's

availability to take part in the study during these time periods. Use of technology may have also been a factor in self-selected recruitment; potential participants who are familiar with using technology may be more willing to take part in a study using additional apps and sensors.

In summary, our data show that the Hypo-METRICS app is an acceptable tool to collect daily real-time data about hypoglycaemia frequency and impact on participant
<table><caption>TABLE 4 Associations of Hypo-METRICS app completion rates with person-reported outcome measures.</caption><thead><tr><th>Domain measured (PROM)</th><th>Adjusted estimates (β) (95% CI)</th><th>p</th></tr></thead><tbody><tr><td>Diabetes-specific quality of life (Dawn Impact of Diabetes Profile; DIDP)<sup>24</sup></td><td>-0.99 (-2.07, 0.09)</td><td>0.072</td></tr><tr><td>Generalised Anxiety Disorder (GAD-7)<sup>25</sup></td><td>-0.19 (-0.51, 0.12)</td><td>0.231</td></tr><tr><td>Diabetes distress (Problem Areas In Diabetes; PAID)<sup>26</sup></td><td>-0.01 (-0.05, 0.07)</td><td>0.765</td></tr><tr><td>Attention and concentration (Perceived Deficit Questionnaire; PDQ-20) attention and concentration subscale<sup>27</sup></td><td>0.05 (-0.36, 0.46)</td><td>0.800</td></tr><tr><td>PDQ-20 Planning and organisation subscale<sup>27</sup></td><td>-0.21 (-0.60, 0.18)</td><td>0.296</td></tr><tr><td>PDQ-20 Retroactive memory subscale<sup>27</sup></td><td>0.19 (-0.22, -0.59)</td><td>0.731</td></tr><tr><td>PDQ-20 Prospective memory subscale<sup>27</sup></td><td>-0.57 (-1.01, -0.13)</td><td>0.012</td></tr><tr><td>Depressive symptoms (Patient Health Questionnaire; PHQ-9)<sup>28</sup></td><td>0.15 (-0.16, 0.47)</td><td>0.332</td></tr><tr><td>Fear of hypoglycaemia (Hypoglycaemia Fear Survey II; HFS-II) Behaviour subscale<sup>29</sup></td><td>-0.07 (-0.17, 0.04)</td><td>0.222</td></tr><tr><td>HFS-II Worry subscale<sup>29</sup></td><td>-0.04 (-0.13, 0.05)</td><td>0.380</td></tr><tr><td>Hypoglycaemia awareness (Hypoglycaemia Awareness Questionnaire; HypoA-Q)<sup>30</sup></td><td>0.06 (-0.23, 0.05)</td><td>0.663</td></tr><tr><td>Short form questionnaire (SF-36) Vitality subscale<sup>31</sup></td><td>0.08 (-0.37, 0.53)</td><td>0.735</td></tr><tr><td>Type-D personality scale – Social inhibition subscale (DS14-SI)<sup>32</sup></td><td>0.11 (-0.01, 0.26)</td><td>0.183</td></tr><tr><td>Type-D personality scale – Negative affectivity subscale (DS14 NA)<sup>32</sup></td><td>0.21 (-0.001, 0.41)</td><td>0.048</td></tr></tbody></table>

Note: Results of linear regression, adjusted estimates with 95% confidence intervals and p-values. Results are adjusted for age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score. Number of complete cases for each variable *n* = 592.
Significant with Bonferroni correction, *p* < 0.0025.

<table><caption>TABLE 5 Associations of Hypo-METRICS app completion rates with 10-week study glucose metrics.</caption><thead><tr><th></th><th>Adjusted estimates (β)<br>(95% CI)</th><th>p</th></tr></thead><tbody><tr><td>Baseline HbA1c</td><td>-0.001 (-0.03, 0.03)</td><td>0.938</td></tr><tr><td>% time blinded CGM is active</td><td>0.32 (0.24, 0.39)</td><td><strong>&lt;0.001*</strong></td></tr><tr><td>Time above range (&gt;10.0 mmol/L; 180 mg/dL)</td><td>7.74 (-6.63, 22.12)</td><td>0.291</td></tr><tr><td>Time in range (4.0–10 mmol/L; 70–180 mg/dL)</td><td>7.84 (-6.54, 22.12)</td><td>0.285</td></tr><tr><td>Time below range (&lt;3.9 mmol/L; 70 mg/dL)</td><td>7.29 (-7.08, 21.68)</td><td>0.319</td></tr><tr><td>Time below range (&lt;3.0 mmol/L; 54 mg/dL)</td><td>2.04 (0.84, 3.25)</td><td><strong>&lt;0.001*</strong></td></tr></tbody></table>

Note: Results of linear regression, adjusted estimates with 95% confidence intervals and p-values. Results are adjusted for age, diabetes type, employment status, highest level of education, method of glucose monitoring, and Gold score. Number of complete cases for each variable *n* = 594.

*Significant with Bonferroni correction, *p* < 0.0042.

(e.g. ethnic minorities, adolescents). The recruitment phase of the study began in October 2020 and ended in July 2022, throughout the COVID-19 pandemic. Many of our research participants were recruited remotely during lockdowns in each participating country; therefore, it is possible that lockdowns, changes to daily routines and responsibilities, and furlough from employment during the study may have increased, or decreased, some people's

availability to take part in the study during these time periods. Use of technology may have also been a factor in self-selected recruitment; potential participants who are familiar with using technology may be more willing to take part in a study using additional apps and sensors.

In summary, our data show that the Hypo-METRICS app is an acceptable tool to collect daily real-time data about hypoglycaemia frequency and impact on participant
experiences across a range of ages and diabetes types across 5 European countries. In addition, our data show that with suitable tools, EMA is a feasible method to collect data that have previously been inaccessible and longer-duration EMA can be considered when conducting research in other conditions. The Hypo-METRICS app may be particularly useful for observational prospective studies, or when different treatments or interventions are being compared. The Hypo-METRICS app provides a comprehensive view of the impact of hypoglycaemia on people with diabetes, and should be considered when evaluating the frequency and impact of hypoglycaemia.

## AFFILIATIONS

## ACKNOWLEDGMENTS

We would like to thank all participants in the Hypo-METRICS study. We would like to thank the Hypo-RESOLVE Patient Advisory Committee for their support in the development of the Hypo-METRICS app. Research

nurses Charlotte Hansen, Pernille Banck-Petersen, Rikke Carstensen and Stine Kjøller are thanked for excellent handling of participants at NOH and SDCC, Denmark. Abbott Diabetes Care provided the continuous glucose monitors used in the study. This work was supported by the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement No 777460. The JU receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA and T1DExchange, JDRF, International Diabetes Federation (IDF) and The Leona M. and Harry B. Helmsley Charitable Trust (HCT). This paper reflects the author's view and the JU is not responsible for any use that may be made of the information it contains. This study represents independent research supported by the National Institute for Health and Care Research (NIHR) King's Clinical Research Facility and the NIHR Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. Work was supported by the NIHR Cambridge Biomedical Research Centre and carried out in the NIHR Cambridge Clinical Research Facility/ Translational Research Facility. The University of Cambridge has received salary support for MLE through the National Health Service in the East of England through the Clinical Academic Reserve.

## CONFLICT OF INTEREST STATEMENT

GME's position at King's College London is funded by a research grant from Novo Nordisk as part of their contribution to the Hypo-RESOLVE consortium. MLE has served on advisory boards and/or received lecture fees and/or research support from NovoNordisk, Eli Lilly, AstraZeneca, Medtronic, Dexcom, Ypsomed, Abbott Diabetes Care, Roche, Zucara, Pila Pharma. UPB has served on advisory boards and has received lecture fees from Sanofi and Novo Nordisk. ER has served as consultant/advisor for Abbott, Air Liquide SI, Astra-Zeneca, Boehringer-Ingelheim, Dexcom, Eli-Lilly, Hillo, Insulet, Medirio, Novo-Nordisk, Roche, Sanofi-Aventis, Tandem, and received research support from Dexcom and Tandem. JKM is a member of advisory boards of Abbott Diabetes Care, Becton-Dickinson, Boehringer Ingelheim, Eli Lilly, Embecta, Medtronic, NovoNordisk A/S, Roche Diabetes Care, Sanofi-Aventis, Viatris and received speaker honoria from A. Menarini Diagnostics, Abbott Diabetes Care, AstraZeneca, Boehringer Ingelheim, Dexcom, Eli Lilly, Medtrust, MSD, NovoNordisk A/S, Roche Diabetes Care, Sanofi, Servier, Viatris and Ypsomed. She is shareholder of decision Clinical Software GmbH and elyte Diagnostics and serves as CMO of elyte Diagnostics. JS has served on advisory boards for Janssen, Medtronic, Roche Diabetes

<footnote n="1">¹Department of Diabetes, School of Life Course Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK</footnote>

<footnote n="2">²Medical & Science, Patient Focused Drug Development, Novo Nordisk A/S, Søborg, Denmark</footnote>

<footnote n="3">³Department of Psychology, University of Southern Denmark, Odense, Denmark</footnote>

<footnote n="4">⁴Department of internal medicine, Radboud university medical centre, Nijmegen, The Netherlands</footnote>

<footnote n="5">⁵Department of internal medicine, division of Endocrinology, Maastricht University Medical Centre, Maastricht, The Netherlands</footnote>

<footnote n="6">⁶CARIM School for Cardiovascular Diseases, Maastricht University, Maastricht, The Netherlands</footnote>

<footnote n="7">⁷Division of Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, Graz, Austria</footnote>

<footnote n="8">⁸Division of Endocrinology and Diabetology, Medical University of Graz, Graz, Austria</footnote>

<footnote n="9">⁹Department of Endocrinology and Nephrology, Copenhagen University Hospital - North Zealand, Hillerød, Denmark</footnote>

<footnote n="10">¹⁰Institute of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark</footnote>

<footnote n="11">¹¹Steno Diabetes Center Copenhagen, Herlev, Denmark</footnote>

<footnote n="12">¹²Lund University Diabetes Center, Lund University, Lund, Sweden</footnote>

<footnote n="13">¹³Department of Endocrinology, Skåne University Hospital, Malmö, Sweden</footnote>

<footnote n="14">¹⁴Wellcome Trust-MRC Institute of Metabolic Science and Department of Medicine, University of Cambridge, Cambridge, UK</footnote>

<footnote n="15">¹⁵Department of Endocrinology, Diabetes, Nutrition, Montpellier University Hospital, Montpellier, France</footnote>

<footnote n="16">¹⁶Institute of Functional Genomics, University of Montpellier, CNRS, INSERM, Montpellier, France</footnote>

<footnote n="17">¹⁷Systems Medicine, School of Medicine, University of Dundee, Dundee, UK</footnote>

<footnote n="18">¹⁸University of Sheffield, Sheffield, UK</footnote>

<footnote n="19">¹⁹School of Psychology, Deakin University, Geelong, Victoria, Australia</footnote>

<footnote n="20">²⁰The Australian Centre for Behavioural Research in Diabetes, Diabetes Victoria, Melbourne, Victoria, Australia</footnote>

<footnote n="21">²¹Steno Diabetes Center Odense, Odense, Denmark</footnote>

<footnote n="22">²²Diabetes Research Centre, University of Leicester, Leicester, UK</footnote>
experiences across a range of ages and diabetes types across 5 European countries. In addition, our data show that with suitable tools, EMA is a feasible method to collect data that have previously been inaccessible and longer-duration EMA can be considered when conducting research in other conditions. The Hypo-METRICS app may be particularly useful for observational prospective studies, or when different treatments or interventions are being compared. The Hypo-METRICS app provides a comprehensive view of the impact of hypoglycaemia on people with diabetes, and should be considered when evaluating the frequency and impact of hypoglycaemia.

## AFFILIATIONS

## ACKNOWLEDGMENTS

We would like to thank all participants in the Hypo-METRICS study. We would like to thank the Hypo-RESOLVE Patient Advisory Committee for their support in the development of the Hypo-METRICS app. Research

nurses Charlotte Hansen, Pernille Banck-Petersen, Rikke Carstensen and Stine Kjøller are thanked for excellent handling of participants at NOH and SDCC, Denmark. Abbott Diabetes Care provided the continuous glucose monitors used in the study. This work was supported by the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement No 777460. The JU receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA and T1DExchange, JDRF, International Diabetes Federation (IDF) and The Leona M. and Harry B. Helmsley Charitable Trust (HCT). This paper reflects the author's view and the JU is not responsible for any use that may be made of the information it contains. This study represents independent research supported by the National Institute for Health and Care Research (NIHR) King's Clinical Research Facility and the NIHR Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. Work was supported by the NIHR Cambridge Biomedical Research Centre and carried out in the NIHR Cambridge Clinical Research Facility/ Translational Research Facility. The University of Cambridge has received salary support for MLE through the National Health Service in the East of England through the Clinical Academic Reserve.

## CONFLICT OF INTEREST STATEMENT

GME's position at King's College London is funded by a research grant from Novo Nordisk as part of their contribution to the Hypo-RESOLVE consortium. MLE has served on advisory boards and/or received lecture fees and/or research support from NovoNordisk, Eli Lilly, AstraZeneca, Medtronic, Dexcom, Ypsomed, Abbott Diabetes Care, Roche, Zucara, Pila Pharma. UPB has served on advisory boards and has received lecture fees from Sanofi and Novo Nordisk. ER has served as consultant/advisor for Abbott, Air Liquide SI, Astra-Zeneca, Boehringer-Ingelheim, Dexcom, Eli-Lilly, Hillo, Insulet, Medirio, Novo-Nordisk, Roche, Sanofi-Aventis, Tandem, and received research support from Dexcom and Tandem. JKM is a member of advisory boards of Abbott Diabetes Care, Becton-Dickinson, Boehringer Ingelheim, Eli Lilly, Embecta, Medtronic, NovoNordisk A/S, Roche Diabetes Care, Sanofi-Aventis, Viatris and received speaker honoria from A. Menarini Diagnostics, Abbott Diabetes Care, AstraZeneca, Boehringer Ingelheim, Dexcom, Eli Lilly, Medtrust, MSD, NovoNordisk A/S, Roche Diabetes Care, Sanofi, Servier, Viatris and Ypsomed. She is shareholder of decision Clinical Software GmbH and elyte Diagnostics and serves as CMO of elyte Diagnostics. JS has served on advisory boards for Janssen, Medtronic, Roche Diabetes

<footnote n="1">¹Department of Diabetes, School of Life Course Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK</footnote>

<footnote n="2">²Medical & Science, Patient Focused Drug Development, Novo Nordisk A/S, Søborg, Denmark</footnote>

<footnote n="3">³Department of Psychology, University of Southern Denmark, Odense, Denmark</footnote>

<footnote n="4">⁴Department of internal medicine, Radboud university medical centre, Nijmegen, The Netherlands</footnote>

<footnote n="5">⁵Department of internal medicine, division of Endocrinology, Maastricht University Medical Centre, Maastricht, The Netherlands</footnote>

<footnote n="6">⁶CARIM School for Cardiovascular Diseases, Maastricht University, Maastricht, The Netherlands</footnote>

<footnote n="7">⁷Division of Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, Graz, Austria</footnote>

<footnote n="8">⁸Division of Endocrinology and Diabetology, Medical University of Graz, Graz, Austria</footnote>

<footnote n="9">⁹Department of Endocrinology and Nephrology, Copenhagen University Hospital - North Zealand, Hillerød, Denmark</footnote>

<footnote n="10">¹⁰Institute of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark</footnote>

<footnote n="11">¹¹Steno Diabetes Center Copenhagen, Herlev, Denmark</footnote>

<footnote n="12">¹²Lund University Diabetes Center, Lund University, Lund, Sweden</footnote>

<footnote n="13">¹³Department of Endocrinology, Skåne University Hospital, Malmö, Sweden</footnote>

<footnote n="14">¹⁴Wellcome Trust-MRC Institute of Metabolic Science and Department of Medicine, University of Cambridge, Cambridge, UK</footnote>

<footnote n="15">¹⁵Department of Endocrinology, Diabetes, Nutrition, Montpellier University Hospital, Montpellier, France</footnote>

<footnote n="16">¹⁶Institute of Functional Genomics, University of Montpellier, CNRS, INSERM, Montpellier, France</footnote>

<footnote n="17">¹⁷Systems Medicine, School of Medicine, University of Dundee, Dundee, UK</footnote>

<footnote n="18">¹⁸University of Sheffield, Sheffield, UK</footnote>

<footnote n="19">¹⁹School of Psychology, Deakin University, Geelong, Victoria, Australia</footnote>

<footnote n="20">²⁰The Australian Centre for Behavioural Research in Diabetes, Diabetes Victoria, Melbourne, Victoria, Australia</footnote>

<footnote n="21">²¹Steno Diabetes Center Odense, Odense, Denmark</footnote>

<footnote n="22">²²Diabetes Research Centre, University of Leicester, Leicester, UK</footnote>
Care, and Sanofi Diabetes; her research group (Australian Centre for Behavioural Research in Diabetes [ACBRD]) has received honoraria for this advisory board participation and has also received unrestricted educational grants and in-kind support from Abbott Diabetes Care, AstraZeneca, Medtronic, Roche Diabetes Care, and Sanofi Diabetes. JS has also received sponsorship to attend educational meetings from Medtronic, Roche Diabetes Care, and Sanofi Diabetes, and consultancy income or speaker fees from Abbott Diabetes Care, AstraZeneca, Medtronic, Novo Nordisk, Roche Diabetes Care, and Sanofi Diabetes. FP has received unrestricted research funding support from Novo Nordisk, Sanofi and Eli Lilly. SRH has attended advisory boards for NovoNordisk, Eli Lilly, Zucara and Zealand Pharma for which his institution has received payment. He has received personal fees from NovoNordisk for serving on speaker panels. SAA has served on advisory boards for NovoNordisk and Medtronic and has spoken at an educational symposium sponsored by Sanofi. PC has received personal fees Abbott Diabetes Care, Insulet, Dexcom, Novo Nordisk, AstraZeneca, Medtronic, Roche Diabetes Care and Sanofi Diabetes. Research funding support from Abbott Diabetes Care, Medtronic and Novo Nordisk.

## ORCID

Natalie Zaremba https://orcid.org/0000-0002-1720-1621

Gilberte Martine-Edith https://orcid.

org/0000-0002-4592-3680

Patrick Divilly https://orcid.org/0000-0001-6916-3164

Uffe Soholm https://orcid.org/0000-0003-0848-9421

Melanie Broadley https://orcid.

org/0000-0003-4408-6304

Namam Ali https://orcid.org/0000-0003-4302-7820

Bastiaan de Galan https://orcid.

org/0000-0002-1255-7741

Julia K. Mader https://orcid.org/0000-0001-7854-4233

Julie Brosen https://orcid.org/0000-0001-7361-4407

Mark Evans https://orcid.org/0000-0001-8122-8987

Rory J. McCrimmon https://orcid.

org/0000-0002-3957-1981

Simon Heller https://orcid.org/0000-0002-2425-9565

Jane Speight https://orcid.org/0000-0002-1204-6896

Frans Pouwer https://orcid.org/0000-0002-8172-9818

Stephanie A. Amiel https://orcid.

org/0000-0003-2686-5531

Pratik Choudhary https://orcid.org/0000-0001-7635-4735

## REFERENCES

1. Pramming S, Thorsteinsson B, Theilgaard A, Pinner EM, Binder C. Cognitive function during hypoglycaemia in type I diabetes mellitus. *Br Med J (Clin Res Ed)*. 1986;292(6521):647-650. doi:10.1136/bmj.292.6521.647

2. Amiel SA, Aschner P, Childs B, et al. Hypoglycaemia, cardiovascular disease, and mortality in diabetes: epidemiology, pathogenesis, and management. *Lancet Diabetes Endocrinol*. 2019;7(5):385-396. doi:10.1016/S2213-8587(18)30315-2

3. Brod M, Christensen T, Bushnell DM. Impact of nocturnal hypoglycemic events on diabetes management, sleep quality, and next-day function: results from a four-country survey. *J Med Econ*. 2012;15(1):77-86. doi:10.3111/13696998.2011.624144

4. Nefs G, Pouwer F. The role of hypoglycemia in the burden of living with diabetes among adults with diabetes and family members: results from the DAWN2 study in The Netherlands. *BMC Public Health*. 2018;18(1):156. doi:10.1186/s12889-018-5064-y

5. Khunti K, Alsifri S, Aronson R, et al. Impact of hypoglycaemia on patient-reported outcomes from a global, 24-country study of 27,585 people with type 1 and insulin-treated type 2 diabetes. *Diabetes Res Clin Pract*. 2017;130:121-129. doi:10.1016/j.diabres.2017.05.004

6. Pedersen-Bjergaard U, Alsifri S, Aronson R, et al. Comparison of the HAT study, the largest global hypoglycaemia study to date, with similar large real-world studies. *Diabetes Obes Metab*. 2019;21(4):844-853. doi:10.1111/dom.13588

7. Pedersen-Bjergaard U, Pramming S, Thorsteinsson B. Recall of severe hypoglycaemia and self-estimated state of awareness in type 1 diabetes. *Diabetes Metab Res Rev*. 2003;19(3):232-240. doi:10.1002/dmrr.377

8. Warren RE, Frier BM. Hypoglycaemia and cognitive function. *Diabetes Obes Metab*. 2005;7(5):493-503. doi:10.1111/j.1463-1326.2004.00421.x

9. Little SA, Leelarathna L, Barendse SM, et al. Severe hypoglycaemia in type 1 diabetes mellitus: underlying drivers and potential strategies for successful prevention. *Diabetes Metab Res Rev*. 2014;30(3):175-190. doi:10.1002/dmrr.2492

10. Blome C, Augustin M. Measuring change in quality of life: bias in prospective and retrospective evaluation. *Value Health J Int Soc Pharmacoeconomics Outcomes Res*. 2015;18(1):110-115. doi:10.1016/j.jval.2014.10.007

11. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. *Annu Rev Clin Psychol*. 2008;4(1):1-32. doi:10.1146/annurev.clinpsy.3.022806.091415

12. Perski O, Keller J, Kale D, et al. Understanding health behaviours in context: A systematic review and meta-analysis of ecological momentary assessment studies of five key health behaviours. *Health Psychol Rev*. 2022;16(4):576-601. doi:10.1080/17437199.2022.2112258

13. Kivelä L, van der Does WAJ, Riese H, Antypa N. Don't miss the moment: A systematic review of ecological momentary assessment in suicide research. *Front Digit Health*. 2022;4:876595. doi:10.3389/fdgth.2022.876595

14. Nam S, Griggs S, Ash GI, et al. Ecological momentary assessment for health behaviors and contextual factors in persons with diabetes: A systematic review. *Diabetes Res Clin Pract*. 2021;174:108745. doi:10.1016/j.diabres.2021.108745

15. Maher JP, Rebar AL, Dunton GF. Ecological momentary assessment is a feasible and valid methodological tool to measure older Adults' physical activity and sedentary behavior. *Front Psychol*. 2018;9:1485. doi:10.3389/fpsyg.2018.01485

16. Williams MT, Lewthwaite H, Fraysse F, Gajewska A, Ignatavicius J, Ferrar K. Compliance with Mobile ecological momentary assessment of self-reported health-related behaviors and psychological constructs in adults: systematic review
Care, and Sanofi Diabetes; her research group (Australian Centre for Behavioural Research in Diabetes [ACBRD]) has received honoraria for this advisory board participation and has also received unrestricted educational grants and in-kind support from Abbott Diabetes Care, AstraZeneca, Medtronic, Roche Diabetes Care, and Sanofi Diabetes. JS has also received sponsorship to attend educational meetings from Medtronic, Roche Diabetes Care, and Sanofi Diabetes, and consultancy income or speaker fees from Abbott Diabetes Care, AstraZeneca, Medtronic, Novo Nordisk, Roche Diabetes Care, and Sanofi Diabetes. FP has received unrestricted research funding support from Novo Nordisk, Sanofi and Eli Lilly. SRH has attended advisory boards for NovoNordisk, Eli Lilly, Zucara and Zealand Pharma for which his institution has received payment. He has received personal fees from NovoNordisk for serving on speaker panels. SAA has served on advisory boards for NovoNordisk and Medtronic and has spoken at an educational symposium sponsored by Sanofi. PC has received personal fees Abbott Diabetes Care, Insulet, Dexcom, Novo Nordisk, AstraZeneca, Medtronic, Roche Diabetes Care and Sanofi Diabetes. Research funding support from Abbott Diabetes Care, Medtronic and Novo Nordisk.

## ORCID

Natalie Zaremba https://orcid.org/0000-0002-1720-1621

Gilberte Martine-Edith https://orcid.

org/0000-0002-4592-3680

Patrick Divilly https://orcid.org/0000-0001-6916-3164

Uffe Soholm https://orcid.org/0000-0003-0848-9421

Melanie Broadley https://orcid.

org/0000-0003-4408-6304

Namam Ali https://orcid.org/0000-0003-4302-7820

Bastiaan de Galan https://orcid.

org/0000-0002-1255-7741

Julia K. Mader https://orcid.org/0000-0001-7854-4233

Julie Brosen https://orcid.org/0000-0001-7361-4407

Mark Evans https://orcid.org/0000-0001-8122-8987

Rory J. McCrimmon https://orcid.

org/0000-0002-3957-1981

Simon Heller https://orcid.org/0000-0002-2425-9565

Jane Speight https://orcid.org/0000-0002-1204-6896

Frans Pouwer https://orcid.org/0000-0002-8172-9818

Stephanie A. Amiel https://orcid.

org/0000-0003-2686-5531

Pratik Choudhary https://orcid.org/0000-0001-7635-4735

## REFERENCES

1. Pramming S, Thorsteinsson B, Theilgaard A, Pinner EM, Binder C. Cognitive function during hypoglycaemia in type I diabetes mellitus. *Br Med J (Clin Res Ed)*. 1986;292(6521):647-650. doi:10.1136/bmj.292.6521.647

2. Amiel SA, Aschner P, Childs B, et al. Hypoglycaemia, cardiovascular disease, and mortality in diabetes: epidemiology, pathogenesis, and management. *Lancet Diabetes Endocrinol*. 2019;7(5):385-396. doi:10.1016/S2213-8587(18)30315-2

3. Brod M, Christensen T, Bushnell DM. Impact of nocturnal hypoglycemic events on diabetes management, sleep quality, and next-day function: results from a four-country survey. *J Med Econ*. 2012;15(1):77-86. doi:10.3111/13696998.2011.624144

4. Nefs G, Pouwer F. The role of hypoglycemia in the burden of living with diabetes among adults with diabetes and family members: results from the DAWN2 study in The Netherlands. *BMC Public Health*. 2018;18(1):156. doi:10.1186/s12889-018-5064-y

5. Khunti K, Alsifri S, Aronson R, et al. Impact of hypoglycaemia on patient-reported outcomes from a global, 24-country study of 27,585 people with type 1 and insulin-treated type 2 diabetes. *Diabetes Res Clin Pract*. 2017;130:121-129. doi:10.1016/j.diabres.2017.05.004

6. Pedersen-Bjergaard U, Alsifri S, Aronson R, et al. Comparison of the HAT study, the largest global hypoglycaemia study to date, with similar large real-world studies. *Diabetes Obes Metab*. 2019;21(4):844-853. doi:10.1111/dom.13588

7. Pedersen-Bjergaard U, Pramming S, Thorsteinsson B. Recall of severe hypoglycaemia and self-estimated state of awareness in type 1 diabetes. *Diabetes Metab Res Rev*. 2003;19(3):232-240. doi:10.1002/dmrr.377

8. Warren RE, Frier BM. Hypoglycaemia and cognitive function. *Diabetes Obes Metab*. 2005;7(5):493-503. doi:10.1111/j.1463-1326.2004.00421.x

9. Little SA, Leelarathna L, Barendse SM, et al. Severe hypoglycaemia in type 1 diabetes mellitus: underlying drivers and potential strategies for successful prevention. *Diabetes Metab Res Rev*. 2014;30(3):175-190. doi:10.1002/dmrr.2492

10. Blome C, Augustin M. Measuring change in quality of life: bias in prospective and retrospective evaluation. *Value Health J Int Soc Pharmacoeconomics Outcomes Res*. 2015;18(1):110-115. doi:10.1016/j.jval.2014.10.007

11. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. *Annu Rev Clin Psychol*. 2008;4(1):1-32. doi:10.1146/annurev.clinpsy.3.022806.091415

12. Perski O, Keller J, Kale D, et al. Understanding health behaviours in context: A systematic review and meta-analysis of ecological momentary assessment studies of five key health behaviours. *Health Psychol Rev*. 2022;16(4):576-601. doi:10.1080/17437199.2022.2112258

13. Kivelä L, van der Does WAJ, Riese H, Antypa N. Don't miss the moment: A systematic review of ecological momentary assessment in suicide research. *Front Digit Health*. 2022;4:876595. doi:10.3389/fdgth.2022.876595

14. Nam S, Griggs S, Ash GI, et al. Ecological momentary assessment for health behaviors and contextual factors in persons with diabetes: A systematic review. *Diabetes Res Clin Pract*. 2021;174:108745. doi:10.1016/j.diabres.2021.108745

15. Maher JP, Rebar AL, Dunton GF. Ecological momentary assessment is a feasible and valid methodological tool to measure older Adults' physical activity and sedentary behavior. *Front Psychol*. 2018;9:1485. doi:10.3389/fpsyg.2018.01485

16. Williams MT, Lewthwaite H, Fraysse F, Gajewska A, Ignatavicius J, Ferrar K. Compliance with Mobile ecological momentary assessment of self-reported health-related behaviors and psychological constructs in adults: systematic review
and meta-analysis. *J Med Internet Res.* 2021;23(3):e17023.
doi:10.2196/17023

17. Divilly P, Zaremba N, Mahmoudi Z, et al. Hypo-METRICS: Hypoglycaemia—MEasurement, ThResholds and ImpaCtS—A multi-country clinical study to define the optimal threshold and duration of sensor-detected hypoglycaemia that impact the experience of hypoglycaemia, quality of life and health economic outcomes: the study protocol. *Diabet Med.* 2022;39(9):e14892.
doi:10.1111/dme.14892

18. Søholm U, Broadley M, Zaremba N, et al. Investigating the day-to-day impact of hypoglycaemia in adults with type 1 or type 2 diabetes: design and validation protocol of the hypo-METRICS application. *BMJ Open*. 2022;12(2):e051651. doi:10.1136/bmjopen-2021-051651

19. Søholm U, Broadley M, Zaremba N, et al. Psychometric properties of an innovative smartphone application to investigate the daily impact of hypoglycemia in people with type 1 or type 2 diabetes: the hypo-METRICS app. *PLoS One*. 2023;18(3):e0283148.
doi:10.1371/journal.pone.0283148

20. Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). *Qual Life Res Int J Qual Life Asp Treat Care Rehab.* 2011;20(10):1727-1736. doi:10.1007/s11136-011-9903-x

21. Yu L, Buysse DJ, Germain A, et al. Development of short forms from the PROMIS sleep disturbance and sleep-related impairment item banks. *Behav Sleep Med.* 2011;10(1):6-24. doi:10.1080/015402002.2012.636266

22. Reilly MC, Zbrozek AS, Dukes EM. The validity and reproducibility of a work productivity and activity impairment instrument. *PharmacoEconomics.* 1993;4(5):353-365.
doi:10.2165/00019053-199304050-00006

23. Harris PA, Taylor R, Minor B, et al. The REDCap consortium: building an international community of software platform partners. *J Biomed Inform.* 2019;95:103208. doi:10.1016/j.jbi.2019.103208

24. Holmes-Truscott E, Skovlund SE, Hendrieckx C, Pouwer F, Peyrot M, Speight J. Assessing the perceived impact of diabetes on quality of life: psychometric validation of the DAWN2 impact of diabetes profile in the second diabetes MILES - Australia (MILES-2) survey. *Diabetes Res Clin Pract.* 2019;150:253-263.
doi:10.1016/j.diabres.2019.03.020

25. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. *Arch Intern Med.* 2006;166(10):1092. doi:10.1001/archinte.166.10.1092

26. Polonsky WH, Anderson BJ, Lohrer PA, et al. Assessment of diabetes-related distress. *Diabetes Care.* 1995;18(6):754-760.
doi:10.2337/diacare.18.6.754

27. Sullivan MJ, Edgley K, Dehoux E. A survey of multiple sclerosis: I. Perceived cognitive problems and compensatory strategy use. *Can. J Rehabil.* 1990;4(2):99-105.

28. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. *J Gen Intern Med.* 2001;16(9):606-613. doi:10.1046/j.1525-1497.2001.016009606.x

29. Gonder-Frederick L, Schmidt K, Vajda K, et al. Psychometric properties of the hypoglycemia fear survey-ii for adults with type 1 diabetes. *Diabetes Care.* 2011;34(4):801-806. doi:10.2337/dc10-1343

30. Speight J, Barendse SM, Singh H, et al. Characterizing problematic hypoglycaemia: iterative design and preliminary psychometric validation of the Hypoglycaemia awareness questionnaire (HypoA-Q). *Diabet Med.* 2016;33(3):376-385.
doi:10.1111/dme.12824

31. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. *Med Care.* 1992;30(6):473-483.

32. Denollet J. DS14: standard assessment of negative affectivity, social inhibition, and type D personality. *Psychosom Med.* 2005;67(1):89-97. doi:10.1097/01.psy.0000149256.81953.49

33. Team R. RStudio: Integrated Development Environment for R. 2023. Version 2023.06.2+561.

34. Søholm U, Zaremba N, Broadley M, et al. Assessing the content validity, acceptability, and feasibility of the hypo-METRICS app: survey and interview study. *JMIR Diabetes.* 2023;8(1):e42100.
doi:10.2196/42100

## SUPPORTING INFORMATION

Additional supporting information can be found online in the Supporting Information section at the end of this article.

**How to cite this article:** Zaremba N, Martine-Edith G, Divilly P, et al. Associations of clinical, psychological, and sociodemographic characteristics and ecological momentary assessment completion in the 10-week Hypo-METRICS study: Hypoglycaemia MEasurements ThResholds and ImpaCtS. *Diabet Med.* 2024;00:e15345. doi:10.1111/dme.15345
and meta-analysis. *J Med Internet Res.* 2021;23(3):e17023.
doi:10.2196/17023

17. Divilly P, Zaremba N, Mahmoudi Z, et al. Hypo-METRICS: Hypoglycaemia—MEasurement, ThResholds and ImpaCtS—A multi-country clinical study to define the optimal threshold and duration of sensor-detected hypoglycaemia that impact the experience of hypoglycaemia, quality of life and health economic outcomes: the study protocol. *Diabet Med.* 2022;39(9):e14892.
doi:10.1111/dme.14892

18. Søholm U, Broadley M, Zaremba N, et al. Investigating the day-to-day impact of hypoglycaemia in adults with type 1 or type 2 diabetes: design and validation protocol of the hypo-METRICS application. *BMJ Open*. 2022;12(2):e051651. doi:10.1136/bmjopen-2021-051651

19. Søholm U, Broadley M, Zaremba N, et al. Psychometric properties of an innovative smartphone application to investigate the daily impact of hypoglycemia in people with type 1 or type 2 diabetes: the hypo-METRICS app. *PLoS One*. 2023;18(3):e0283148.
doi:10.1371/journal.pone.0283148

20. Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). *Qual Life Res Int J Qual Life Asp Treat Care Rehab.* 2011;20(10):1727-1736. doi:10.1007/s11136-011-9903-x

21. Yu L, Buysse DJ, Germain A, et al. Development of short forms from the PROMIS sleep disturbance and sleep-related impairment item banks. *Behav Sleep Med.* 2011;10(1):6-24. doi:10.1080/15402002.2012.636266

22. Reilly MC, Zbrozek AS, Dukes EM. The validity and reproducibility of a work productivity and activity impairment instrument. *PharmacoEconomics.* 1993;4(5):353-365.
doi:10.2165/00019053-199304050-00006

23. Harris PA, Taylor R, Minor B, et al. The REDCap consortium: building an international community of software platform partners. *J Biomed Inform.* 2019;95:103208. doi:10.1016/j.jbi.2019.103208

24. Holmes-Truscott E, Skovlund SE, Hendrieckx C, Pouwer F, Peyrot M, Speight J. Assessing the perceived impact of diabetes on quality of life: psychometric validation of the DAWN2 impact of diabetes profile in the second diabetes MILES - Australia (MILES-2) survey. *Diabetes Res Clin Pract.* 2019;150:253-263.
doi:10.1016/j.diabres.2019.03.020

25. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. *Arch Intern Med.* 2006;166(10):1092. doi:10.1001/archinte.166.10.1092

26. Polonsky WH, Anderson BJ, Lohrer PA, et al. Assessment of diabetes-related distress. *Diabetes Care.* 1995;18(6):754-760.
doi:10.2337/diacare.18.6.754

27. Sullivan MJ, Edgley K, Dehoux E. A survey of multiple sclerosis: I. Perceived cognitive problems and compensatory strategy use. *Can. J Rehabil.* 1990;4(2):99-105.

28. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. *J Gen Intern Med.* 2001;16(9):606-613. doi:10.1046/j.1525-1497.2001.016009606.x

29. Gonder-Frederick L, Schmidt K, Vajda K, et al. Psychometric properties of the hypoglycemia fear survey-ii for adults with type 1 diabetes. *Diabetes Care.* 2011;34(4):801-806. doi:10.2337/dc10-1343

30. Speight J, Barendse SM, Singh H, et al. Characterizing problematic hypoglycaemia: iterative design and preliminary psychometric validation of the Hypoglycaemia awareness questionnaire (HypoA-Q). *Diabet Med.* 2016;33(3):376-385.
doi:10.1111/dme.12824

31. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. *Med Care.* 1992;30(6):473-483.

32. Denollet J. DS14: standard assessment of negative affectivity, social inhibition, and type D personality. *Psychosom Med.* 2005;67(1):89-97. doi:10.1097/01.psy.0000149256.81953.49

33. Team R. RStudio: Integrated Development Environment for R. 2023. Version 2023.06.2+561.

34. Søholm U, Zaremba N, Broadley M, et al. Assessing the content validity, acceptability, and feasibility of the hypo-METRICS app: survey and interview study. *JMIR Diabetes.* 2023;8(1):e42100.
doi:10.2196/42100

## SUPPORTING INFORMATION

Additional supporting information can be found online in the Supporting Information section at the end of this article.

**How to cite this article:** Zaremba N, Martine-Edith G, Divilly P, et al. Associations of clinical, psychological, and sociodemographic characteristics and ecological momentary assessment completion in the 10-week Hypo-METRICS study: Hypoglycaemia MEasurements ThResholds and ImpaCtS. *Diabet Med.* 2024;00:e15345. doi:10.1111/dme.15345