# Standardization of the assessment process within telerehabilitation in chronic diseases: a scoping meta-review

Blandine Chapel, François Alexandre, Nelly Heraud, Roxana Ologeanu-Taddei, Anne-Sophie Cases, François Bughin, Maurice Hayot

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Blandine Chapel, François Alexandre, Nelly Heraud, Roxana Ologeanu-Taddei, Anne-Sophie Cases, et al.. Standardization of the assessment process within telerehabilitation in chronic diseases: a scoping meta-review. BMC Health Services Research, 2022, 22 (1), pp.984. 10.1186/s12913-022-08370-y. hal-03750633

HAL Id: hal-03750633

https://hal.science/hal-03750633

Submitted on 14 Aug 2022

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Standardization of the assessment process within telerehabilitation in chronic diseases: a scoping meta-review

Blandine Chapel<fnref n="1" />*, François Alexandre², Nelly Heraud², Roxana Ologeanu-Taddei³, Anne-Sophie Cases<fnref n="1" />, François Bughin⁴ and Maurice Hayot⁴

**Abstract**

**Background:** Telerehabilitation (TR) interventions are receiving increasing attention. They have been evaluated in various scientific areas through systematic reviews. However, there is a lack of data on how to standardize assessment and report on their domains to guide researchers across studies and bring together the best evidence to assess TR for chronic diseases.

**Aims and objectives:** The aim of this study was to identify domains of assessment in TR and to qualitatively and quantitatively analyze how and when they are examined to gain an overview of assessment in chronic disease.

**Methods:** A scoping meta-review was carried out on 9 databases and gray literature from 2009 to 2019. The keyword search strategy was based on "telerehabilitation", "evaluation", "chronic disease" and their synonyms. All articles were subjected to qualitative analysis using the Health Technology Assessment (HTA) Core Model prior to further analysis and narrative synthesis.

**Results:** Among the 7412 identified articles, 80 studies met the inclusion criteria and addressed at least one of the noncommunicable diseases (NCD) categories of cardiovascular disease (cardiovascular accidents), cancer, chronic respiratory disease, diabetes, and obesity. Regarding the domains of assessment, the most frequently occurring were "social aspect" (n = 63, 79%) (e.g., effects on behavioral changes) and "clinical efficacy" (n = 53, 66%), and the least frequently occurring was "safety aspects" (n = 2, 3%). We also identified the phases of TR in which the assessment was conducted and found that it most commonly occurred in the *pilot study* and *randomized trial* phases and least commonly occurred in the *design, pretest, and post-implementation* phases.

**Conclusions:** Through the HTA model, this scoping meta-review highlighted 10 assessment domains which have not been studied with the same degree of interest in the recent literature. We showed that each of these assessment domains could appear at different phases of TR development and proposed a new cross-disciplinary and comprehensive method for assessing TR interventions. Future studies will benefit from approaches that leverage the best evidence regarding the assessment of TR, and it will be interesting to extend this assessment framework to other chronic diseases.

**Keywords:** Telerehabilitation, Health technology assessment, Evaluation, Chronic disease, Scoping meta-review

**Background**

At a time when life expectancy is increasing, chronic or noncommunicable diseases (NCDs) are also on the rise [1]. The latest World Health Organization (WHO, 2015)

<footnote>*Correspondence: blandine.chapel@hotmail.fr</footnote>

<footnote n="1">¹ University of Montpellier, Montpellier Research of Management, Montpellier, France</footnote>
report indicates that NCDs, like mainly heart and lung disease, obesity, cancer, and diabetes, are responsible for 16 million premature deaths (before the age of 70) each year. As the leading cause of morbidity, disability and mortality in industrialized countries, NCDs constitute a real public health problem, and their prevention, treatment and risk factors have become major international issues [2]. Along with other well-established treatments such as pharmacotherapy, supplemental oxygen, or noninvasive ventilation, nondrug interventions, such as rehabilitation, have become the standard of care for these diseases. Rehabilitation is a comprehensive intervention focused on the patient's needs [3]. There are two major objectives in rehabilitation. The first objective is to restore or optimize functional ability for a chronic patient. This goal is achieved by the implementation of exercise training to improve the symptomatology of the disease. The second objective is to encourage health-promoting behaviors. For this purpose, therapeutic patient education, which supports self-management of the disease, is used in several activities like smoking cessation, nutrition and physical activity [3].

At present, due to the use of information and communication technologies (ICT), the use of telemedicine and the potential effectiveness of mobile applications [4, 5], telerehabilitation (TR) is increasingly attracting the attention of policymakers, payers, healthcare professionals, patients and the scientific community. TR is a subcategory of telehealth that uses different types of communication modalities, which can be divided into synchronous such as live video, and asynchronous, such as e-mail or remote monitoring [6], to provide clinical rehabilitation services from a distance [7]. TR is emerging as an innovative approach to providing remote care and to deploying rehabilitation [8–12], to improve accessibility and continuity of care and to educate patients about adherence and long-term maintenance of the beneficial effects of rehabilitation [13, 14]. TR is an intervention that emerged at the end of the 1990s [15], and it represents a powerful tool for improving the management of daily practice and the creation of networks between health structures such as hospitals or clinics, and services.

However, despite undeniable evidence of the contribution of TR, many TR interventions still fail and rarely achieve technology adoption [16, 17], user engagement and intervention effectiveness [18–20]. Considering the several elements that could explain this failure, it is clear that TR suffers from a lack of exhaustive development in its many dimensions, and that a gap may exist between the design of TR and the moment when it will be used [21]. Indeed, assessing TR is a complex matter for several reasons: TR is a broad concept that requires

multidisciplinary collaborations with many stakeholders [22], who must simultaneously consider the dimensions that characterize TR intervention: technological, clinical and others like ethical, cost-effectiveness, social [23–26]. In addition, health information systems should be assessed with the same rigor as a new drug or treatment program to prevent decisions about future deployments of ICT in the health sector from being determined by social, economic, and/or political circumstances rather than by robust scientific evidence [27]. Thus, we must be able to evaluate TR interventions while they are being designed, developed and deployed [27]. Hence, in assessing TR, an extensive appraisal including these different dimensions is needed in each phase of the technology's life cycle [27–29], as was reported in the e-health evaluation model by Enam et al., (2018) [30].

The aim of this scoping meta-review [31] was to systematically map recent research to contribute to the standardization of the assessment process within TR applications. First, to identify the domains of TR assessment for chronic diseases, we used a comprehensive evaluation framework with a multidisciplinary approach called the Health Technology Assessment (HTA) Core Model [32]. The HTA includes ten domains and appears to be the most complete framework. In addition, we identified the phases of TR interventions in which assessment occurs by mapping out the content of the reviews and grouping the phases of intervention with similar objectives, activities, or results. On this basis, our two research questions were as follows:

1. What domains of evaluation have been identified in the literature on TR for chronic disease?

2. What are the assessment domains of the different phases of TR?

Finally, the findings allow us to present a novel way of examining the assessment of TR interventions and could provide a reference and information for policymakers, clinicians and researchers regarding the development of assessment guidelines.

## Methods

This study is based on a new method, the scoping meta-review [31]. This method combines the scoping review and meta-review methods. Scoping reviews entail reviewing the emerging literature to provide an initial indication of the size and nature of the available literature on a particular topic [33, 34]. Meta-reviews involve synthesizing evidence from a set of systematic reviews [35, 36]. We first considered performing a simple scoping review, considering that the field is diverse and complex [37]. However, during the initial extraction of the data, we noticed the existence of
numerous systematic reviews on various aspects of TR assessment. Thus, it seemed relevant and feasible to undertake a scoping review of the systematic reviews related to TR assessment approaches [31]. The advantage of relying on systematic reviews is that they can provide a solid and reliable synthesis of work in the field [37].

## Protocol

We followed the guidelines of Levac et al. (2010), updated from the initial work of Arksey and O'Malley (2005). In addition, we wrote the protocol using the "Preferred Reporting Items for Systematic reviews and Meta-Analyses Extension for Scoping Reviews" (PRISMA-ScR) [38]. Levac et al. (2010) suggest that the protocol should not be designed as a rigid tool and strictly applied because it is likely that as familiarity with the literature is increased, researchers will be able to redefine search terms and undertake more sensitive literature searches. The process is not linear but iterative, requiring us to engage with each stage in a reflexive way and, where necessary, to repeat the steps to ensure that the literature is covered comprehensively. Hence, in our study, the protocol was used as a guide, and we followed it when necessary [39]. The final protocol is not publicly available; however, it can be made available upon request to the corresponding author.

## Eligibility criteria

The methodological approach of the scoping review allowed for the identification and alteration of the inclusion and exclusion criteria as articles were selected. Inclusion and exclusion were performed first on the basis of article selection through the "title" and "abstract" filters and then by reading the articles in full.

Studies that were in line with the *inclusion criteria* (1) were in the systematic literature review format; (2) addressed at least one of the NCD categories of cardiovascular disease (cardiovascular accidents), cancer, chronic respiratory disease, diabetes, or obesity identified on the basis of the prevalence and importance of the common behavioral risk factors such as smoking, poor diet, sedentary lifestyle, and harmful use of alcohol (according to the figures provided by the WHO); (3) addressed TR in the sense of the definition given in the rationale (see the introduction); (4) included features of the definition of rehabilitation presented in the rationale; (5) contained at least one intervention offering physical activity as part of a multidisciplinary approach; (6) dated from 2009 to 2019 because the field of TR must take into account the increasing evolution of ICT in health such as connected objects and mobile devices; (7) reported in French or English (criteria adopted for practical reasons: material from other languages was excluded due to the

cost and time involved in translation); and (8) used an adult population 18+ years old.

Studies that were in line with the *exclusion criteria* (1) presented interventions without technology or limited to a telephone follow-up approach; (2) presented a single study or were opinion papers, draft syntheses, abstract/conference proceedings (oral presentations and posters), chapters, discussions, letters, books available electronically, and theses; (3) were those where it was impossible to identify the type of intervention performed; (4) addressed an intervention with technology limited to the physical activity dimension alone (without multidisciplinary approach) or did not contain a physical activity dimension; or (5) addressed methods/requirements without real evaluation related to TR or for which it was impossible to locate the full text.

## Information sources

The field of TR must take into account the growing ICT evolution in health such as connected objects and mobile devices; thus, we included studies published only between January 2009 and October 2019. We conducted extensive literature searches in the electronic bibliographic databases most likely to contain the type of study we are looking for. The databases are multidisciplinary, covering fields from computer science to health science: MEDLINE (PubMed), Web of Science, Cochrane Library, ABI, Business Source Premier, PsycINFO, Science Direct, Academic Search Premier, and SPORTDiscus. We conducted additional searches in the gray literature (a) by consulting the reference lists of the included studies and (b) by searching repositories of gray literature: CADTH, Directory of Open Access Journals (DOAJ), Occupational Therapy Systematic Evaluation of Evidence (Otseeker), International Prospective Register of Systematic Reviews (PROSPERO), OpenSIGLE (OpenGrey), and the New York Academy of Medicine Library's Grey Literature Report. The research strategy was planned and carried out through structured team discussions and in consultation with a university librarian so that the strategy could be refined in light of the initial results. We exported the final search results to Zotero (5.0.95.1, Roy Rosenzweig Center for History and New Media, Fairfax, Virginia), a bibliographic database. This software facilitated the management of the research, particularly the identification and removal of duplicates.

## Search—identification of relevant studies

We identified the keywords through, on the one hand, the medical subject headings (MesH) providing the controlled vocabulary for MEDLINE/PubMed and, on the other hand, other keywords, which we call free vocabulary. Free vocabulary was added based on the expertise
of the different team members but also by reading the abstracts in first intention (the first reading performed on the abstracts) if it seemed relevant and necessary. The search strategy was based on the terms "telerehabilitation" AND "evaluation" AND "chronic disease" and all their synonyms (see Additional file 1). Each database was searched individually. The keyword search strategy, based on the use of the Boolean operators AND / OR as well as ti(title) and ab(abstract), is described below.

## Selection of sources of evidence

To increase consistency among the 5 reviewers, pairs were created to independently evaluate article titles and abstracts for inclusion in the study. Evaluators met at the beginning, midpoint and end of the process to discuss issues and uncertainties related to the selection of potentially relevant studies and to re-evaluate and refine the research strategy if necessary. We then independently reviewed the full papers for inclusion. Disagreements over study selection and data extraction were resolved by consensus and discussion with other reviewers to make a final decision on inclusion if necessary.

## Data charting process

To begin, the first author developed a data table. We used a conceptual framework to guide the data extraction (Additional file 2). Subsequently, the other coauthors discussed and validated this table to determine the variables to be extracted. Once this first version was finalized, three members of the evaluation team tested the table by independently collecting data on three articles to share their perspectives concerning the dimensions to be collected. Next, two of these reviewers conducted data collection on 10% of the corpus of selected articles and discussed the results, continuously updating the data charting table in an iterative process. We then carried out a second calibration exercise testing the % agreement, with a predetermined level of agreement (70% to 80%) [38]. The concordance determined by the Kendall concordance coefficient (W) in SPSS software was greater than 80% (mean: 87%; Kendall's W = 0.8697). The first researcher thus finalized the coding alone, and any disagreements and questions were resolved through discussion between the two reviewers. Moreover, by charting the content of the reviews and grouping those with similar objectives, activities, and/or outcomes, thematic analysis could be used to determine whether the reviews focused on certain phases of the technology life cycle rather than others.

## Data items

The data extracted from the articles are as follows: author(s), year of publication, location of study, design

of review (narrative review, descriptive review, scoping review, meta-analysis, systematic review, theoretical review, etc.), title, type of pathology, fields of technology (m-health, e-health, etc.), definition of technology, tools associated with technology (SMS, apps, web, etc.), end-users, number of studies included (study design and participants), whether type of evaluation allows for consideration of and focus on phases of the technology life cycle (design, pretest, pilot study, randomized trial, post-implementation), key findings, and critical appraisal of researcher (if applicable). For each review, we extracted the presence and number of evaluation domains based on the HTA model (see the following paragraph). In accordance with the PRISMA-ScR, we did not perform a quality assessment or quality evaluation, as this is not essential for scoping review methodologies. Thus, the methodological rigor of the published articles was not a criterion for inclusion or exclusion for two reasons. First, scales measuring this quality, such as Assessing the Methodological Quality of Systematic Reviews (AMSTAR), tend to focus on experimental studies (interventions). Given the other possible aspects of TR (e.g., ethical or economical), we did not want to discriminate against those studies. Second, these scales are relevant if we need to compare the effectiveness of the interventions studies. This is not the aim of our study; rather, we were interested in the various aspects of the TR assessment beyond medical effectiveness.

## Critical appraisal of individual sources of evidence: the hta (healthcare technology assessment) core model framework [32]

We needed an analytical framework, a multidisciplinary approach that could include all domains of TR assessment, which encompasses medical as well as non-medical domains, to know what was being evaluated and to break down the silos of the assessment dimensions. This early assessment can be important because it can influence the TR project before the implementation. For this purpose, we chose the Health Technology Assessment (HTA) Core Model [32] from among the evaluation frameworks available in the literature [40–44] because it suggests what kinds of information one can find in an HTA report, and its definition encompasses the dimensions of multidisciplinarity and comprehensiveness. The definition of a *health technology assessment* is as follows: “*a multidisciplinary process that summarises information about the medical, social, economic and ethical issues related to the use of a health technology in a systematic, transparent, unbiased, robust manner*” (p13) [32]. The structure of the information collected is as follows: the domains of assessment (the broad framework within which the technology is considered), the topics of assessment (more
specific considerations in one of the domains), and the issues raised (even more specific considerations on one of the topics that may be similar to research questions in scientific studies). The structure of the model is based on the combination of these three points to define the different assessment elements and facilitate a shared understanding of what belongs to HTA. Additional file 2 below provides an overview of the ten domains described in the HTA core model: health problem and current use of technology, description and technical characteristics of technology, safety, accuracy, clinical effectiveness, costs and economic evaluation, ethical analysis, organizational aspects, social aspects, and legal aspects.

**Methods of handling and summarizing the data**

A qualitative synthesis of the included studies is conducted to chart the literature on the domains of TR assessment. The data are summarized using descriptive tables of the categories developed from the HTA framework. Additionally, a qualitative inductive and content analysis approach allowed us to bring out other elements of TR assessment (completing the existing framework). Finally, thematic analysis was applied by mapping out the content of the papers and grouping the phases of intervention with similar objectives, activities, or results. The objective of the analysis was to understand whether the researchers emphasize some phases over others during evaluation and, if so, what phases are most frequently evaluated during an intervention.

**Results**

**Selection of sources of evidence**

The search of the 9 databases generated 7412 results (Fig. 1). After elimination of duplicates, 5306 publications remained. The review of the titles and abstracts led to the exclusion of 5174 publications, leaving 132 publications requiring screening on the basis of the full text. Full text screening helped to remove an additional 54 publications, leaving 78 articles. The most common reason for excluding citations during full-text screening was that the studies did not include physical activity in their interventions (n=24). Subsequently, 2 articles were added from the gray literature and manual research. A total of 80 publications remained (Fig. 1), all of which focused on one or more domains of TR assessment and required further analysis.

**Characteristics of the reviews included and reports according to the eunethta template**

*Year of publication and geographical distribution*—Table 1 shows the number of reviews included by year bracket between 2009 and October 2019. Few reviews were published between 2009 and 2013 (n=14, 18%). Most of the

articles were published from 2014 onwards, and more than one-third of the articles (n=26, 33%) were published from 2018—2019, the last two years studied. The majority of the reviews (n=31, 39%) were from Europe, followed by North America (n=25, 31%), Oceania (Australia and New Zealand) (n=13, 16%), Asia (mainly China) (n=9, 11%) and South America (n=2, 3%).

*Pathologies concerned*—The main categories of NCDs were cardiovascular diseases (heart attacks), cancers, chronic respiratory diseases (such as chronic obstructive pulmonary disease or asthma), obesity and diabetes. Fifty-four articles (68%) focused on a single pathology, and 26 (32%) focused on > = 2 pathologies. More than half of the articles dealt with diabetes (n=45, 56%), followed by cardiovascular disease (n=38, 48%). One-third of the reviews were on chronic respiratory diseases (n=25, 31%). A quarter (n=20, 25%) of the reviews addressed cancer, closely followed by obesity (n=19, 24%) (see Table 1).

*Types of systematic literature reviews*—As shown in Table 1, more than 80% of the reviews were additionally derived from qualitative systematic reviews (n=52, 65%), meta-analyses (n=3, 4%) or the performance of both at the same time (n=14, 18%). For the other reviews, we found scoping reviews (n=4, 5%), meta-reviews (umbrella) (n=4, 5%), a meta-ethnography and a descriptive review.

**Areas of intervention and definitions**

In our research, the term "telerehabilitation", according to the definition given in the rationale, was found under different names in each of the included reviews. The most-used terms were rather generic: "m-health" (n=33, 41%), "e-health" (n=16, 20%), and "telehealth" (n=16, 20%) or, more rarely, "web-based intervention/rehabilitation" (n=6, 8%), "e-health/m-health" (n=3, 4%), "digital health intervention" (n=2, 3%), and "telemedicine" (n=2, 3%). The term "telerehabilitation" appeared only twice (3%). Systematic reviews used different definitions corresponding to the field of intervention stated in the research. Despite the sometimes disparate definitions, the numerous reviews nevertheless provided an evaluation of different studies (with different interventions) that may correspond to the definition of TR given in the rationale. In this way, some authors raised the issue of the difficulty of determining the element of effectiveness of the TR interventions evaluated [45].

**Types of associated technological tools**

We classified the technological tools mobilized into several categories (see Additional file 3). In most studies, a combination of several tools was identified to enable the interventions to be carried out [41, 46–48].
<figure><img src="image_3.png" /></figure>

Mobile and internet/website applications were most commonly used, with 51 articles (17% for each). Short message system tools followed, with 47 articles (14%). Often, in addition to these first three tools, phone calls (n=39, 11%), digital devices (e.g., connected objects)

(n=36, 11%), and emails (n=29, 9%) were added. Less frequently, it was also possible to identify the following tools: videos/images (n=22, 6%), videoconferences (n=14, 4%), and social networks (n=14, 4%). More rarely, we found that studies used personal health
<table><caption>**Table 1** Characteristics of the included documents</caption><thead><tr><th>Characteristics</th><th>No. of reviews (n=80)</th><th>Percentage (%)</th></tr></thead><tbody><tr><td><strong>Year of publication</strong></td><td></td><td></td></tr><tr><td>2009—2011</td><td>7</td><td>9</td></tr><tr><td>2012—2013</td><td>7</td><td>9</td></tr><tr><td>2014—2015</td><td>19</td><td>24</td></tr><tr><td>2016—2017</td><td>21</td><td>26</td></tr><tr><td>2018—2019</td><td>26</td><td>33</td></tr><tr><td><strong>Location of study</strong></td><td></td><td></td></tr><tr><td>Europe (including UK)</td><td>31</td><td>39</td></tr><tr><td>North America</td><td>25</td><td>31</td></tr><tr><td>Oceania</td><td>13</td><td>16</td></tr><tr><td>Asia</td><td>9</td><td>11</td></tr><tr><td>South America</td><td>2</td><td>3</td></tr><tr><td><strong>Pathologies concerned</strong></td><td></td><td></td></tr><tr><td>Diabetes</td><td>45</td><td>56</td></tr><tr><td>Cardiovascular diseases</td><td>38</td><td>48</td></tr><tr><td>Chronic respiratory diseases</td><td>25</td><td>31</td></tr><tr><td>Cancers</td><td>20</td><td>25</td></tr><tr><td>Obesity</td><td>19</td><td>24</td></tr><tr><td><strong>Design of the review</strong></td><td></td><td></td></tr><tr><td>QSR</td><td>52</td><td>65</td></tr><tr><td>QSR and MA</td><td>14</td><td>18</td></tr><tr><td>Umbrella (meta-review)</td><td>4</td><td>5</td></tr><tr><td>Scoping review</td><td>4</td><td>5</td></tr><tr><td>MA</td><td>3</td><td>4</td></tr><tr><td>QSR and meta-synthesis (meta-ethnography)</td><td>1</td><td>1</td></tr><tr><td>Realist review</td><td>1</td><td>1</td></tr><tr><td>Descriptive</td><td>1</td><td>1</td></tr><tr><td><strong>Field of technology (as stated by the authors)</strong></td><td></td><td></td></tr><tr><td>m-health</td><td>33</td><td>41</td></tr><tr><td>e-health</td><td>16</td><td>20</td></tr><tr><td>Telehealth</td><td>16</td><td>20</td></tr><tr><td>Web (internet)-based intervention</td><td>6</td><td>8</td></tr><tr><td>m-health and e-health</td><td>3</td><td>4</td></tr><tr><td>Telemedicine</td><td>2</td><td>3</td></tr><tr><td>Telerehabilitation</td><td>2</td><td>3</td></tr><tr><td>Digital health intervention</td><td>2</td><td>3</td></tr></tbody></table>

MA Meta-analysis, QSR Qualitative systematic review

reports (n=4, 1%) or other technological tools, such as logbooks, virtual reality, or digital libraries.

## Completeness of the reports according to the EUnetHTA template

Given the number of sources included in the scoping meta-review, the relevant data from each source are provided in Additional File 4. On average, we found a total

<table><caption>**Table 2** EUnetHTA HTA Core Model Domains included in the reviews</caption><thead><tr><th>Domains evaluated</th><th>No. of domain appearances</th><th>Percentage (%)</th></tr></thead><tbody><tr><td>HTA Core Model domains</td><td></td><td></td></tr><tr><td>1. Social</td><td>63</td><td>79</td></tr><tr><td>2. Clinical effectiveness</td><td>53</td><td>66</td></tr><tr><td>3. Health problem</td><td>46</td><td>58</td></tr><tr><td>4. Description and technical characteristics of technology</td><td>37</td><td>46</td></tr><tr><td>5. Cost and economic evaluation</td><td>14</td><td>18</td></tr><tr><td>6. Organizational</td><td>12</td><td>15</td></tr><tr><td>7. Legal</td><td>12</td><td>15</td></tr><tr><td>8. Ethical</td><td>2</td><td>3</td></tr><tr><td>9. Safety</td><td>2</td><td>3</td></tr><tr><td>10. Accuracy</td><td>-</td><td>-</td></tr></tbody></table>

of 3 HTA domains evaluated per review. Table 2 shows the number of HTA domains appearing in the reviews. Briefly, the most represented domains were *social aspect*, in 79% (n=63) of the reviews, and *clinical efficacy*, in 66% (n=53). *Ethical analysis* and *safety aspects* were both evaluated only in 3% (n=2) of the reviews studied, and *accuracy* was not represented in our data.

## Telerehabilitation assessment domains identified

In this section, we present the TR domains that review authors evaluated. To understand these domains, we mapped them using the principles derived from the EUnetHTA HTA framework (see Methodology). After extracting the domains during the qualitative analysis, we classified them into nine categories: *social aspects*, *health problem and current use of technology*, *description and technical characteristics of technology*, *costs and economic evaluation*, *organizational aspects*, *legal aspects*, *ethical analysis*, and *safety*. Table 3 shows these domains and the key aspects of their measurement.

## Additional non-hta TR domain

The HTA framework was not developed specifically for TR, and during the inductive and thematic analysis and by comparing results between reviewers, we found that 95% of the reviews (n=76/80) evaluated the "interventional aspect". This interventional aspect combines the characteristics of the interventions or their functionalities and the application of recommendations and theoretical foundations to construct these interventions. Therefore, we decided to add this assessment domain to complement the HTA framework.
<table><caption>**Table 3** Description of identified telerehabilitation assessment domains</caption><thead><tr><td>Domain assessed</td><td>Key subjects of measurement</td></tr></thead><tbody><tr><td>Social aspects</td><td>Major life areas: Technologies are seen as opportunities to identify similar user communities and share experiences among peers [49]. Effect on psychological distress [50], stress management, fatigue [50, 51], knowledge of treatment and chronic disease [52–55]. Encouraging but varied effects on self-efficacy [53, 56, 57]. Effects on behavioral changes in PA, diet, medication adherence, and smoking [53, 57–62]. Individual: Ten reviews examine the acceptability of the intervention [63, 64] mainly to the patient as an end user [58, 65, 66], and 15 reviews study satisfaction [58, 67, 68]. Facilitators and individual barriers are also studied [45, 69]. Communication: Evaluation of the usability of technologies during the development process [25, 46, 58, 63, 70]. Need for targeted technology [65] and stimulation of user engagement, motivation and involvement over time [68, 71, 72], and the quality of patient-caregiver interaction [64, 65, 68, 73]</td></tr><tr><td>Clinical effectiveness</td><td>Health outcomes: Benefits and unanticipated negative effects of telerehabilitation interventions compared to standard interventions (usual care). Outcomes include postintervention mortality [74, 75], clinical results (blood lipids, blood pressure, hemoglobin A1c, weight and BMI) [20, 47, 53, 58, 76], quality of life [51, 56, 66, 68], anxiety, depression [51, 58, 75, 77–80], and physical functions (exercise capacity, exercise tolerance); presented in the short [47, 54, 81], medium and/or long term [82, 83]. Patient satisfaction: Willingness to reuse or recommend the technology [51, 55]. Comparative accuracy of a replacement technology: More specific or safer technological intervention than an older or comparable technological intervention (with more features, feedback, educational messages, or combinations of technological tools) [20, 65, 81]</td></tr><tr><td>Health problems and current use of technology</td><td>Target condition: Differences in the effectiveness of the intervention according to the various targeted pathologies, possible differences from one pathology to another [50, 65]. Example of type 1 diabetes and type 2 diabetes [20, 76]. Utilization: differences in use between countries or a lack of education in low-income countries [66, 84]. Identification of the applicability and acceptability of telerehabilitation in primary care, general practice and hospital settings [85]. Other: Only one study focused on evaluating the actors involved in the design of the technology (i.e., a team of IT developers) [86]</td></tr><tr><td>Description and technical characteristics of technology</td><td>Features of the technology: General information (name, type of device, language, etc.) [48, 67, 86], purpose of using the technology (e.g., to promote behavioral change) [45, 58, 79, 87], technical characteristics (ergonomics, functionalities, interoperability) [88–91]. Investments and tools required to use the technology: Type of operating system and its availability [49, 71, 84, 86, 92]; the brand and relevance of technological tools [23, 25, 59]. Training and information needed to utilize the technology: Protocols, educational materials, recommendations, and documents developed to make the intervention appropriate for the target population [48, 65, 67, 70, 92–94]. Need to establish feasibility, accessibility and usability studies [89]</td></tr><tr><td>Costs and economic evaluation</td><td>Unit costs: Related to the unit costs of the resources used (e.g., technology acquisition costs or the cost of specific actions) [49, 65, 70, 84, 86, 95, 96]. Outcomes: Health cost outcomes by type of telerehabilitation intervention [97], compared to a control group [68], to prevent, predict or minimize exacerbation [98]. Cost-effectiveness: Intervention that can be cost-effective under certain conditions [56, 95]. Despite being minimally studied, "urgent" need to be performed controlled and homogeneous trials [99]</td></tr><tr><td>Organizational aspects</td><td>Process: Monitor care outcome processes, such as maintenance of the behavioral effects of the intervention [65], clinic attendance, the effectiveness of the chronic disease surveillance system or the compliance of tools used to improve clinic attendance (e.g., SMS reminders) [95]. Interest in having a multidisciplinary team trained in motivational feedback [100]; fund technology-oriented studies and encourage proposals from interdisciplinary groups of researchers [65]. Structure: The effects of the implementation of interventions on hospital admissions, the use of health resources [72, 101, 102], clinical workload and workflow, and dependence on technology for work [103]. Management: Interest in proposing multiple models of patient management (e.g., integrating alternative models) based on evidence, responding to the needs and profile of patients [68, 77]. Take into account the intentions of future caregivers to integrate technological tools into their practice [65, 89]</td></tr><tr><td>Legal aspects</td><td>End-user: Identify the various target populations [53, 69, 77, 104] and those that are poorly studied [48, 53] to carefully examine the possibility of generalizing new modalities of intervention and their potential dissemination. Privacy of the patient and authorization and safety: Describe procedures to ensure the security and storage of private data [64] and identify whether problems occur in the private sphere [67]. Legal regulation of novel/experimental techniques: Adapt the development of new mobile applications to regulations (e.g., medical devices) [89]. Regulation of the market: Identify whether reimbursement of intervention systems is possible and by whom [65]</td></tr><tr><td>Ethical analysis</td><td>Principal questions about the ethical aspects of technology: Spreading the use of technologies (e.g., ethical challenges of privacy and data security) [25]</td></tr></tbody></table>
<table><caption>**Table 3** (continued)</caption><thead><tr><td>Domain assessed</td><td>Key subjects of measurement</td></tr></thead><tbody><tr><td>Safety</td><td>Technology-dependent safety risks: Identify potential problems with the reliability and validity of information entered into the technology by the patient or caregiver; identify the number of adverse effects of interventions in patients [63]. Use- or user-dependent safety risks: Identify potential complications that may arise due to certain functionalities (e.g., misinterpretation of information sent) [65]</td></tr></tbody></table>

## Characteristics of the intervention

The main characteristics of the intervention can be classified according to the strategies used. An average of 5 strategies was identified per review: educational information (n=61; 76%), communication with others (n=53; 66%), self-management (physical activity, diet, medication adherence, smoking) (n=64; 80%), feedback and self-monitoring (n=48; 60%), use of prompts/cues (reminders and alerts) (n=38; 48%), exercise training (n=32; 40%), psychosocial support (n=12; 15%), stress management (n=10; 13%), patient assessment (n=8; 10%) and others. Additional file 5 gives a sample of the intervention characteristics identified in ten reviews.

## Application of recommendations and theoretical foundations for behavior change

Clinical recommendations suggest that ongoing behavioral support is necessary for lifestyle changes to be sustainable [58]. Many reviews (n=31; 39%) present intervention characteristics based on specific theories/conceptual frameworks for designing and optimizing TR interventions [53, 84, 94]. The behavior change theory (BCT) developed by Abraham & Michie (2008) is the most widely used behavior change theory in technological applications (n=13/31) [41, 60]. The most used BCTs include “goal setting”, “self-monitoring of behavior”, “information about health consequences”, “social support”, and “feedback and monitoring”. The transtheoretical model [105] and social cognitive theory [106] are the next most applied (n=10/31). Many other theories are mentioned more sporadically, such as self-efficacy theory [58], the theory of planned behavior—reasoned action [94], social ecological theory [47], social support theory [53], the self-management model [72], self-determination theory [84], and cognitive behavior theory [64].

## Phases of the telerehabilitation assessment process

To answer our second research question, we focused on how the assessment was conducted in the distinct development phases of TR: *design*, *pretest*, *pilot study*, *randomized trial* and *post-implementation*. This led us to develop the *telerehabilitation assessment process* (Fig. 2), which illustrates the accumulation of evidence by crossing the assessment domains with the distinct development phases of TR. The domains of assessment

(i.e., *health problem and current use of technology*, *description and technical characteristics of technology*, *safety*, *clinical effectiveness*, *costs and economic evaluation*, *ethical analysis*, *organizational aspects*, *social aspects*, and *legal aspects*) vary in each phase. The results show that assessment is mainly carried out in the *pilot study* and *randomized trial* phases. For example, during the *pilot study* phase, the focus of assessment shifts primarily to the *social aspect*, followed mainly by *clinical effectiveness*. On the other hand, assessment is rarely carried out in the *design*, *pretest*, and *post-implementation* phases. When a TR intervention initiates with the *design* phase, the decisions are made based on the evaluation of the *description and technical characteristics of technology*, *social aspects*, *costs and economic evaluation*, *organizational aspects*,

<figure><img src="image_4.png" /><figcaption>**Fig. 2** Telerehabilitation assessment process</figcaption></figure>
*legal aspects* and *ethical analysis*. The *health problem and current use of technology* and *safety* domains appear in the *pretest* phase. Although reviews of the post-implementation phase are limited, this comprehensive evaluation process can be used to gradually accumulate evidence that could be used to make future decisions.

Discussion

This scoping meta-review was conducted to identify the
different domains of TR assessment for chronic diseases
and provide a comprehensive view of TR assessment
through the analytic framework of HTA. The results indi-
cate that many systematic reviews are generally focused
on a limited number of assessment dimensions.

In identifying and summarizing the main domains of
assessment, we highlighted the multidisciplinarity and
comprehensiveness of the assessment of TR. Our study
shows that nine out of the ten domains composing the
HTA framework have been explored by TR reviews
(social aspects, clinical effectiveness, description and tech-
nical characteristics of technology, health problem and
current use of technology, costs and economic evaluation,
legal aspects, organizational aspects, safety, and ethical
analysis). This result reflects the relevance of this frame-
work for our specific analysis.

Much of the focus centered on the domains of *social aspect* and *clinical effectiveness*. Together, they represent 48% of occurrences, though they constitute only 20% of the HTA domains (n=2/9 domains). This reflects an imbalance in the assessment of the different domains. To date, research in these two domains has relied primarily on randomized controlled trials (RCTs) to assess TR. The performance of an RCT is considered the “gold standard” in research [107], and the RCT is a unique approach of achieving lifestyle changes in patients with chronic diseases [65]. Despite this major interest, some researchers have questioned their usefulness given the complexity of assessing TR interventions [107]. Our results also show that the least frequently occurring domains were *safety* and *ethical analysis*. Although our review highlights some promising emerging results that may help commissioners, developers, and users manage risks and improve patient safety [63, 65], several studies have shown that mobile medical applications (apps) could compromise patient safety [108, 109]. Future research could develop a risk framework that users, developers, and other stakeholders can use to assess the likely risks posed by specific apps in a specific context [110]. Finally, concerning *ethical analysis*, despite the fast-paced growth of TR, only a few articles propose suggestions to practitioners for addressing ethical challenges such as acquiring compliant

software, receiving training, creating informed con-
sent procedures, and using an ethical decision-making
model [111].

Regarding our choice to mobilize the HTA Core Model, we evaluated its relevance and operationality with respect to TR. This framework had not yet been applied in the field of TR, so we sometimes had difficulty classifying some of the data using the HTA domains. Indeed, there was ambiguity regarding some items that could be classified under multiple domains at the same time. For example, in this framework, the description of the *social aspects* domain includes the effects on behavioral changes in physical activity and diet. These elements could also be classified under *clinical effectiveness* for a health or rehabilitation expert. We can also take a critical view of the results with respect to the frequency of the appearance of certain domains (e.g., the *social aspects* domain is present at a greater frequency than the *clinical effectiveness* domain). Furthermore, with regard to the model, it would be interesting to consider its supplementation or comparison with other validated frameworks. For example, the literature proposes many (more or less comprehensive) approaches to e-health [40], m-health and even telemedicine [38, 105–107] assessment to assist decision makers who want to introduce and use this technology.

Moreover, we identified an additional non-HTA
domain, the *interventional aspect*, which defined several
attributes of TR intervention assessment. The majority
of published reviews (n=76, 95%) examine the different
characteristics of interventions that engage the patient
and foster the success of TR to promote behavior change
and positive health outcomes. A number of reviews pro-
vide encouraging evidence about BCTs and their benefits
for the improvement of physical activity outcomes [60].
In contrast to this literature, a recent meta-review high-
lighted the need for better implementation tools that
support patient engagement and identified the neces-
sity of optimizing the design of the self-management
resources included in or with guidelines [112]. Thus, a
variety of theories offer insight into how patients’ percep-
tions influence their behavior and can be used to design
and then evaluate self-management guideline tools.

In all the existing models, we found no framework
that contained all the dimensions [40–44]. The HTA
framework was not specifically related to the TR inter-
vention, but it allowed us to position the review at a
broader level of assessment. If we had focused only on
the intervention (the domain alone being restrictive), we
would have excluded the other dimensions. We therefore
took a broader perspective of the assessment by focus-
ing on encompassing more domains rather than on the
intervention.
For our secondary objective, we aimed to identify the phases of TR interventions in which assessment occurs. Our results showed a marked interaction between assessment domains and the distinct development phases of TR. This allowed us to highlight which domain was assessed at which phase, suggesting that it is inappropriate to assess all domains in a single phase. This TR assessment process can capture comprehensive, dynamic and complex evidence, crossing the various domains of assessment with the development phases of TR. While many TR assessments are still quite disconnected from each other and therefore fail to create a synergic effect in TR research efforts, one suggestion for interpretation would be that there is confusion or misunderstanding on the part of different stakeholders of the various assessment possibilities to be performed in the early stages of TR development.

Furthermore, through comparison with existing literature, we observed that this TR assessment process is comparable to the e-health evaluation model of Enam et al. (2018) [30]. For example, when e-health intervention is initiated at the *design* phase, the decisions are made solely based on the assessment of the *technological* and *cost* domains of technology development, whereas in this scoping meta-review, they also include *social aspects*, *organizational aspects*, *legal aspects* and *ethical analysis*. Therefore, our process proposes an additional specification not present in the evaluation model of e-health interventions in general.

This TR assessment process could become cumbersome because of high resource consumption, but it is not a prescription, just a way to show the progression of evidence in TR applications in a reliable manner. In that respect, the true value of this review is the fact that it suggests that more work is needed to make sure that other relevant domains in each phase of development are incorporated over time. As such the HTA model may be useful in identifying what domains may be necessary to strengthen as they are not currently sufficiently evaluated. In addition, since the HTA model did not integrate interventional aspects, a combination of two frameworks could have been a solution. From a multidisciplinary point of view, one could also address the mediators, moderators and mechanisms of change, in order to replicate one's findings [113].

Finally, only two reviews actually use the word *telerehabilitation*; thus, the field may be divided in terms of terminology. The impact of this term usage in relation to identifying a common evaluation agenda is a most relevant issue, again relating to the maturity of the field [114]. TR is intended to be multidisciplinary, and thus to serve different professions, which partly contributes to a division of knowledge at present. There is now a need

for harmonization to understand the heterogeneity of TR. Each society has its own definition of the types of rehabilitation [115], and it might be relevant to consider a federation or a transversal definition that encompasses the multiple disciplines. At present, one possible interpretation would be that it is either a problem of maturity of the discipline or an inadequacy of a global and specific definition for each type of pathology (example: rehabilitation in musculoskeletal diseases is not the same as diabetes or obesity).

## Limitations

The main strengths of this review are the use of the scoping review methodology, which enabled coverage of a very broad range of topics; the comprehensive search strategy developed; and the rigorous quality assessment of each review by two independent researchers. However, there are a number of limitations that must be highlighted. First, a scoping meta-review can only report on literature that has been included in published reviews. Thus, some recently published primary research might not be included. Another limitation was that our electronic database searches may have missed relevant citations. This limitation is potentially due (1) to restriction of the search to English and French language publications and (2) to certain documents that may have been omitted, unknowingly and unintentionally. However, we have included and analyzed many journals in this scoping meta-review. Additionally, these TR reviews included only the five major groups of known chronic diseases that represent the highest rate of premature mortality, limiting the generalizability of the results. It would be interesting to determine whether this TR assessment process could be extended and applied to other chronic pathologies that require TR, such as osteoarthritis or stroke [116, 117].

Finally, this scoping meta-review shows that semantically, the remote delivery of rehabilitation is not homogeneous: the terms used include "m-health", "e-health", "telehealth", "web-based intervention/rehabilitation", "digital health intervention", "telemedicine" and "telerehabilitation". There is a use of multiple definitions and an apparent lack of solidarity in defining TR. How, then, do we – collectively – define TR? The confusion extends to other aspects of the TR domain [114]. According to Scott et al. (2013), it seems important to resolve the semantic issues around "e-health strategies" and identify barriers to TR, such as profession-centric nomenclature. Further discussion can then be pursued to ensure that the diversity of TR is understood and that the appropriate mix of specific solutions is brought to bear in response to defined health needs.
## Conclusions

This scoping meta-review reported on a large number of reviews that focused on assessing TR intervention for chronic diseases. By proposing and using a comprehensive assessment framework for TR, our results highlighted ten assessment domains and a list of the main related aspects. The different domains mobilized for assessment are not all studied with the same degree of interest. Furthermore, we showed that each of these assessment domains could appear at different phases of TR development, whereas current research generally focuses on one or a few assessment dimensions. These main contributions allow us to enrich this literature on the assessment of TR and propose new cross-disciplinary and complete method for the assessment of TR interventions.

Due to the challenge of integrating TR into the management of patients with chronic diseases, this framework could guide future studies in developing a research agenda on TR assessment with the aim of obtaining a comprehensive view of the assessment of TR. Thus, improved validation of evaluation methods could facilitate the transferability of results among similar studies and bring together the best evidence to assess TR interventions across a broad range of domains.

## Abbreviations

AMSTAR: Assessing the Methodological Quality of Systematic Review; CADTH: Canadian Agency for Drugs and Technologies in Health; HTA: Healthcare Technology Assessment; ICT: Information and communication technologies; NCD: Noncommunication diseases; PRISMA-ScR: Preferred Reporting Items for Systematic reviews and Meta-Analysis Extension for Scoping Review; RCT: Randomizes Controlled Trials; WHO: World Health Organization.

## Acknowledgements

The authors would like to thank the following people or institutions for their contributions to the paper: Montpellier University for a PhD thesis grant and for providing the librarian who helped us navigate the databases used in this systematic scoping meta-review. Dr. Guy Pare from the Department of Information Technology, Montreal, for clarifying the appropriate methodology for a scoping review during a meeting in Montpellier. Finally, the collaboration with the health professionals from the university hospital of Montpellier and the researchers from private health companies provided an interesting data source and expertise for the research objective.

## Authors' contributions

BC conceived the study. BC, FA, NH, FB and MH participated in the design of the study and the initial syntheses of the results. BC designed the search

strategy with input from FA, NH, FB, MH, RO, and ASC and drafted this section of the methods. BC, FA, NH, MH, and FB conducted the title and abstract screening. BC, FA and NH extracted the data and conducted the coding and analyses. All authors were involved in drafting and critically revising the manuscript. All authors read and approved the final manuscript prior to its submission.

## Funding

This study is the first of three studies that constitutes a PhD thesis. For the duration of the scoping meta-review, the tuition fees were funded by the University of Montpellier. The funders had no role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript; or decision to submit the manuscript for publication.

## Availability of data and materials

All data generated or analyzed during this study are included in this published article and its supplementary information files.

## Declarations

### Ethics approval and consent to participate

Not applicable.

### Consent for publication

Not applicable.

### Competing interests

The authors declare that they have no competing interests.

### Author details

¹University of Montpellier, Montpellier Research of Management, Montpellier, France. ²Direction de La Recherche Clinique Et de L'Innovation en Santé, Korian ; GCS CIPS, 800 Avenue Joseph Vallot, Lodève, France. ³Toulouse Business School, Toulouse, France. ⁴PhyMedExp, University of Montpellier, INSERM, CNRS, CHRU Montpellier, Montpellier, France.

## References

1. Murray CJ, Lopez AD. Evidence-based health policy-lessons from the Global Burden of Disease Study. Science. 1996;274(5288):740-3.

2. Mazzucca S, Arredondo EM, Hoelscher DM, Haire-Joshu D, Tabak RG, Kumanyika SK, et al. Expanding implementation research to prevent chronic diseases in community settings. Annu Rev Public Health. 2021;42:135-58.

3. Celli BR, MacNee W, Agusti A, Anzueto A, Berg B, Buist AS, et al. Standards for the diagnosis and treatment of patients with COPD: a summary of the ATS/ERS position paper. Eur Respir J. 2004;23(6):932-46.

4. Korpershoek YJ, Vervoort SC, Trappenburg JC, Schuurmans MJ. Perceptions of patients with chronic obstructive pulmonary disease and their health care providers towards using mHealth for self-management of exacerbations: a qualitative study. BMC Health Serv Res. 2018;18(1):757.

5. Latif S, Rana R, Qadir J, Ali A, Imran MA, Younis MS. Mobile health in the developing world: Review of literature and lessons from a case study. IEEE Access. 2017;5:11540-56.

6. Latifi R, et al. Clinical TelemedicinePractice: From Ad hoc Medicine to Modus Operandi. Telemedicine, telehealth and Telepresence: Principles, strategies, applications, and new directions. Cham, Switzerland: Springer Nature; 2020. p. 43-9.

7. Kairy D, Lehoux P, Vincent C, Visintin M. A systematic review of clinical outcomes, clinical process, healthcare utilization and costs associated with telerehabilitation. Disabil Rehabil. 2009;31(6):427-47.

8. Cranen K, Groothuis-Oudshoorn CG, Vollenbroek-Hutten MM, IJzerman MJ. Toward patient-centered telerehabilitation design: understanding chronic pain patients' preferences for web-based exercise
telerehabilitation using a discrete choice experiment. J Med Internet Res. 2017;19:e26 JMIR Publications Inc., Toronto, Canada.

9. Chumbler NR, Quigley P, Sanford J, Griffiths P, Rose D, Morey M, et al. Implementing telerehabilitation research for stroke rehabilitation with community dwelling veterans: Lessons learned. Int J Telerehabil. 2010;2(1):15 University Library Systems, University of Pittsburgh.

10. Lai JC, Woo J, Hui E, Chan W. Telerehabilitation—a new model for community-based stroke rehabilitation. J Telemed Telecare. 2004;10(4):199–205 SAGE Publications Sage UK: London, England.

11. Russell TG. Physical rehabilitation using telemedicine. J Telemed Telecare. 2007;13(5):217–20.

12. Schwamm LH, Holloway RG, Amarenco P, Audebert HJ, Bakas T, Chumbler NR, et al. A review of the evidence for the use of telemedicine within stroke systems of care: a scientific statement from the American Heart Association/American Stroke Association. Stroke. 2009;40(7):2616–34.

13. Jafni TI, Bahari M, Ismail W, Radman A. Understanding the Implementation of Telerehabilitation at Pre-Implementation Stage: A Systematic Literature Review. Procedia Computer Science Elsevier. 2017;124:452–60.

14. Hoas H, Andreassen HK, Lien LA, Hjalmarsen A, Zanaboni P. Adherence and factors affecting satisfaction in long-term telerehabilitation for patients with chronic obstructive pulmonary disease: a mixed methods study. BMC Med Inform Decis Mak. 2016;16(1):26.

15. Friedman RH, Stollerman J, Rozenblyum L, Belfer D, Selim A, Mahoney D, et al. A telecommunications system to manage patients with chronic disease. Studies in health technology and informatics. 1998;52:1330–4.

16. Dutot V, Bergeron F, Rozhkova K, Moreau N. Factors Affecting the Adoption of Connected Objects in e-Health: A Mixed Methods Approach. Systèmes d'Information et Management. 2019;23(4):3.

17. Scott Kruse C, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: A systematic review. J telemed telecare. 2018;24(1):4–12 SAGE Publications Sage UK: London, England.

18. Thies K, Anderson D, Cramer B. Lack of adoption of a mobile app to support patient self-management of diabetes and hypertension in a federally qualified health center: interview analysis of staff and patients in a failed randomized trial. JMIR Hum Factors. 2017;4(4): e24.

19. Barberan-Garcia A, Vogiatzis I, Solberg H, Vilaró J, Rodríguez D, Garåsen H, et al. Effects and barriers to deployment of telehealth wellness programs for chronic patients across 3 European countries. Respiratory medicine. 2014;108(4):628–37.

20. Kitsiou S, Paré G, Jaana M, Gerber B. Effectiveness of mHealth interventions for patients with diabetes: an overview of systematic reviews. PloS one. 2017;12(3):e0173160.

21. Heeks R. Health information systems: Failure, success and improvisation. Int J Med Informatics. 2006;75(2):125–37.

22. Pagliari C. Design and evaluation in eHealth: challenges and implications for an interdisciplinary field. J Med Internet Res. 2007;9(2): e15.

23. Hailey D, Roine R, Ohinmaa A, Dennett L. Evidence of benefit from telerehabilitation in routine care: a systematic review. J Telemed Telecare. 2011;17(6):281–7.

24. Brennan DM, Mawson S, Brownsell S. Telerehabilitation: enabling the remote delivery of healthcare, rehabilitation, and self management. Stud Health Technol Inform. 2009;145:231–48.

25. Fu H, McMahon SK, Gross CR, Adam TJ, Wyman JF. Usability and clinical efficacy of diabetes mobile applications for adults with type 2 diabetes: a systematic review. Diabetes Res Clin Pract. 2017;131:70–81.

26. Heynsbergh N, Heckel L, Botti M, Livingston PM. Feasibility, useability and acceptability of technology-based interventions for informal cancer carers: a systematic review. BMC Cancer. 2018;18(1):244.

27. Catwell L, Sheikh A. Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS Med. 2009;6(8): e1000126.

28. Vitacca M, Mazzu M, Scalvini S. Socio-technical and organizational challenges to wider e-Health implementation. Chron Respir Dis. 2009;6(2):91–7.

29. Bates DW, Wright A. Evaluating eHealth: undertaking robust international cross-cultural eHealth research. PLoS Med. 2009;6(9): e1000105.

30. Enam A, Torres-Bonilla J, Eriksson H, others. Evidence-based evaluation of eHealth interventions: systematic literature review. Journal of medical Internet research. 2018;20(11):e10971.

31. Sarrami-Foroushani P, Travaglia J, Debono D, Clay-Williams R, Braithwaite J. Scoping meta-review: introducing a new methodology. Clin Transl Sci. 2015;8(1):77–81 Wiley Online Library.

32. Lampe K, Pasternack I, others. HTA Core Model for Diagnostic Technologies v 1.0. Work Package 4: The HTA Core Model. 2008.

33. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32 Taylor & Francis.

34. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement sci. 2010;5(1):69 BioMed Central.

35. Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions. John Wiley & Sons; 2019.

36. Paré G, Trudel MC, Jaana M, Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015;52(2):183–99 Elsevier.

37. Sarrami-Foroushani P, Travaglia J, Debono D, Braithwaite J. Key concepts in consumer and community engagement: a scoping meta-review. BMC health serv res. 2014;14(1):250 BioMed Central.

38. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

39. Li Y, Marier-Bienvenue T, Perron-Brault A, Wang X, Paré G. Blockchain technology in business organizations: A scoping review. Proceedings of the 51st Hawaii International Conference on System Sciences. 2018. p. 4474–83. http://hdl.handle.net/10125/50454.

40. Khoja S, Durrani H, Scott RE, Sajwani A, Piryani U. Conceptual framework for development of comprehensive e-health evaluation tool. Telemed J E Health. 2013;19(1):48–53 Mary Ann Liebert, Inc. 140 Huguenot Street, 3rd Floor New Chellel, NY 10801 USA.

41. Beatty AL, Fukuoka Y, Whooley MA. Using mobile technology for cardiac rehabilitation: a review and framework for development and evaluation. J Am Heart Assoc. 2013;2(6):e000568.

42. Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehlg G, Sinha C, et al. Guidelines for reporting ofhealth interventions using mobile phones: mobile health (mHealth) evidencereporting and assessment (mERA) checklist. BMJ. 2016;352:74.

43. Broadway M, Carrion C, Vallespin B, Saadatford O, Puigdomènech E, Espallargues M, et al. mHealth assessment: conceptualization of a global framework. JMIR Mhealth Uhealth. 2017;5(5): e60.

44. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. 2015;3(1): e27.

45. Brunton L, Bower P, Sanders C. The contradictions of telehealth user experience in chronic obstructive pulmonary disease (COPD): a qualitative meta-synthesis. PloS one. 2015;10(10):e0139561.

46. Alessa T, Abdi S, Hawley MS, de Witte L, others. Mobile apps to support the self-management of hypertension: systematic review of effectiveness, usability, and user satisfaction. JMIR mHealth and uHealth. 2018;6(7):e10723.

47. Bennett G, Steinberg D, Stoute C, Lanpher M, Lane I, Askew S, et al. Electronic health (e Health) interventions for weight management among racial/ethnic minority adults: a systematic review. Obes Rev. 2014;15:146–58.

48. Isaacs T, Hunt D, Ward D, Rooshenas L, Edwards L. The inclusion of ethnic minority patients and the role of language in telehealth trials for type 2 diabetes: a systematic review. J Med Internet Res. 2016;18(9):256.

49. Chomutare T, Fernandez-Luque L, Årsand E, Hartvigsen G. Features of mobile diabetes applications: review of the literature and analysis of current applications compared against evidence-based guidelines. J Med Internet Res. 2011;13(3): e65.

50. Silva EH, Lawler S, Langbecker D. The effectiveness of mHealth for self-management in improving pain, psychological distress, fatigue, and sleep in cancer survivors: a systematic review. J Cancer Surviv. 2019;13(1):97–107.

51. Seiler A, Klaas V, Tröster G, Fagundes CP. eHealth and mHealth interventions in the treatment of fatigued cancer survivors: A systematic review and meta-analysis. Psychooncology. 2017;26(9):1239–53.

52. Triberti S, Savioni L, Sebri V, Pravettoni G. eHealth for improving quality of life in breast cancer patients: a systematic review. Cancer Treat Rev. 2019;74:1–14.
53. Cotter AP, Durant N, Agne AA, Cherrington AL. Internet interventions to support lifestyle modification for diabetes management: a systematic review of the evidence. J Diabetes Complications. 2014;28(2):243-51.

54. Yasmin F, Banu B, Zakir SM, Sauerborn R, Ali L, Souares A. Positive influence of short message service and voice call interventions on adherence and health outcomes in case of chronic disease care: a systematic review. BMC Med Inform Decis Mak. 2016;16(1):46.

55. Unni E, Gabriel S, Ariely R. A review of the use and effectiveness of digital health technologies in patients with asthma. Ann Allergy Asthma Immunol. 2018;121(6):680-91.

56. Pfaeffli Dale L, Dobson R, Whittaker R, Maddison R. The effectiveness of mobile-health behaviour change interventions for cardiovascular disease self-management: a systematic review. Eur J Prev Cardiol. 2016;23(8):801-17.

57. Hall AK, Cole-Lewis H, Bernhardt JM. Mobile text messaging for health: a systematic review of reviews. Annu Rev Public Health. 2015;36:393-415.

58. Dounavi K, Tsoumani O. Mobile health applications in weight management: a systematic literature review. Am J Prev Med. 2019;56(6):894-903.

59. Goode AD, Lawler SP, Brakenridge CL, Reeves MM, Eakin EG. Telephone, print, and Web-based interventions for physical activity, diet, and weight control among cancer survivors: a systematic review. J Cancer Surviv. 2015;9(4):660-82.

60. Duff OM, Walsh DM, Furlong BA, O'Connor NE, Moran KA, Woods CB. Behavior change techniques in physical activity eHealth interventions for people with cardiovascular disease: systematic review. J Med Internet Res. 2017;19(8):e281.

61. Cassimatis M, Kavanagh DJ. Effects of type 2 diabetes behavioural telehealth interventions on glycaemic control and adherence: a systematic review. J Telemed Telecare. 2012;18(8):447-50.

62. Lundell S, Holmner Å, Rehn B, Nyberg A, Wadell K. Telehealthcare in COPD: a systematic review and meta-analysis on physical outcomes and dyspnea. Respir med. 2015;109(1):11-26.

63. Puig EP, Robles N, Saigi-Rubió F, Zamora A, Moharra M, Paluzie G, et al. Assessment of the Efficacy, Safety, and Effectiveness of Weight Control and Obesity Management Mobile Health Interventions: Systematic Review. JMIR Mhealth Uhealth. 2019;7(10): e12612.

64. Qudah B, Luetsch K. The influence of mobile health applications on patient-healthcare provider relationships: a systematic, narrative review. Patient Educ Couns. 2019;102(6):1080-9.

65. Jones KR, Lekhak N, Kaewluang N. Using mobile phones and short message service to deliver self-management interventions for chronic conditions: A meta-review. Worldviews on Evidence-Based Nursing. 2014;11(2):81-8.

66. Hamilton SJ, Mills B, Birch EM, Thompson SC. Smartphones in the secondary prevention of cardiovascular disease: a systematic review. BMC Cardiovasc Disord. 2018;18(1):25.

67. Banbury A, Nancarrow S, Dart J, Gray L, Parkinson L. Telehealth interventions delivering home-based support group videoconferencing: systematic review. J Med Internet Res. 2018;20(2): e25.

68. Gaikwad R, Warren J. The role of home-based information and communications technology interventions in chronic disease management: a systematic literature review. Health Informatics J. 2009;15(2):122-46.

69. Gorst SL, Armitage CJ, Brownell S, Hawley MS. Home telehealth uptake and continued use among heart failure and chronic obstructive pulmonary disease patients: a systematic review. Ann Behav Med. 2014;48(3):323-36.

70. Rodriguez AQ, Wägner AM. Mobile phone applications for diabetes management: A systematic review. Endocrinologia, diabetes y nutricion. 2019;66(5):330-7.

71. Bellei EA, Biduski D, Cechetti NP, De Marchi ACB. Diabetes mellitus m-Health applications: a systematic review of features and fundamentals. Telemedicine and e-Health. 2018;24(11):839-52.

72. Borries TM, Dunbar A, Bhukhen A, Rismany J, Kilham J, Feinn R, et al. The impact of telemedicine on patient self-management processes and clinical outcomes for patients with types I or II diabetes mellitus in the United States: a scoping review. Diabetes Metab Syndr. 2019;13(2):1353-7.

73. Vassilev I, Rowsell A, Pope C, Kennedy A, O'Cathain A, Salisbury C, et al. Assessing the implementability of telehealth interventions for self-management support: a realist review. Implement Sci. 2015;10(1):59.

74. Huang K, Liu W, He D, Huang B, Xiao D, Peng Y, et al. Telehealth interventions versus center-based cardiac rehabilitation of coronary artery disease: a systematic review and meta-analysis. Eur J Prev Cardiol. 2015;22(8):959-71.

75. Neubeck L, Redfern JU, Fernandez R, Briffa T, Bauman A, Freedman SB. Telehealth interventions for the secondary prevention of coronary heart disease: a systematic review. Eur J Cardiovasc Prev Rehabil. 2009;16(3):281-9.

76. Hou C, Carter B, Hewitt J, Francisa T, Mayor S. Do mobile phone applications improve glycemic control (HbA1c) in the self-management of diabetes? A systematic review, meta-analysis, and GRADE of 14 randomized trials. Diabetes Care. 2016;39(11):2089-95.

77. Clark RA, Conway A, Poulsen V, Keech W, Tirimacco R, Tideman P. Alternative models of cardiac rehabilitation: a systematic review. Eur J Prev Cardiol. 2015;22(1):35-74.

78. Rincon E, Monteiro-Guerra F, Rivera-Romero O, Dorronzoro-Zubiete E, Sanchez-Bocanegra CL, Gabarron E. Mobile phone apps for quality of life and well-being assessment in breast and prostate cancer patients: systematic review. JMIR Mhealth Uhealth. 2017;5(12): e187.

79. Bardus M, Smith JR, Samaha L, Abraham C. Mobile phone and web 2.0 technologies for weight management: a systematic scoping review. J Med Internet Res. 2015;17(11).

80. Kebede MM, Liedtke TP, Möllers T, Pischke CR. Characterizing active ingredients of eHealth interventions targeting persons with poorly controlled type 2 diabetes mellitus using the behavior change techniques taxonomy: scoping review. J Med Internet Res. 2017;19(10): e348.

81. Hutchesson MJ, Rollo ME, Krukowski R, Ells L, Harvey J, Morgan PJ, et al. eHealth interventions for the prevention and treatment of overweight and obesity in adults: a systematic review with meta-analysis. Obes Rev. 2015;16(5):376-92.

82. Wang Y, Xue H, Huang Y, Huang L, Zhang D. A systematic review of application and effectiveness of mHealth interventions for obesity and diabetes treatment and self-management. Adv Nutr. 2017;8(3):449-62.

83. Rawstorn JC, Gant N, Direito A, Beckmann C, Maddison R. Telehealth exercise-based cardiac rehabilitation: a systematic review and meta-analysis. Heart. 2016;102(15):1183-92.

84. Xiong S, Berkhouse H, Schooler M, Pu W, Sun A, Gong E, et al. Effectiveness of mHealth interventions in improving medication adherence among people with hypertension: a systematic review. Curr Hypertens Rep. 2018;20(10):86.

85. Liang X, Wang Q, Yang X, Cao J, Chen J, Mo X, et al. Effect of mobile phone intervention for diabetes on glycaemic control: a meta-analysis. Diabet Med. 2011;28(4):455-63.

86. Arnhold M, Quade M, Kirch W. Mobile applications for diabetics: a systematic review and expert-based usability evaluation considering the special requirements of diabetes patients age 50 years or older. J Med Internet Res. 2014;16(4):104.

87. Wu C, Wu Z, Yang L, Zhu W, Zhang M, Zhu Q, et al. Evaluation of the clinical outcomes of telehealth for managing diabetes: a PRISMA-compliant meta-analysis. Medicine. 2018;97(43).

88. Arambepola C, Ricci-Cabello I, Manikavasagam P, Roberts N, French DP, Farmer A. The impact of automated brief messages promoting lifestyle changes delivered via mobile devices to people with type 2 diabetes: a systematic literature review and meta-analysis of controlled trials. J Med Internet Res. 2016;18(4):86.

89. Nussbaum R, Kelly C, Quinby E, Mac A, Parmanto B, Dicianno BE. Systematic review of mobile health applications in rehabilitation. Arch Phys Med Rehabil. 2019;100(1):115-27.

90. Palmer M, Sutherland J, Barnard S, Wynne A, Rezel E, Doel A, et al. The effectiveness of smoking cessation, physical activity/diet and alcohol reduction interventions delivered by mobile phones for the prevention of non-communicable diseases: a systematic review of randomised controlled trials. PLoS ONE. 2018;13(1): e0189801.

91. Ramadas A, Quek KF, Chan C, Oldenburg B. Web-based interventions for the management of type 2 diabetes mellitus: a systematic review of recent evidence. Int J Med Informatics. 2011;80(6):389-405.

92. Ramsey RR, Caromody JK, Voorhees SE, Warning A, Cushing CC, Guilbert TW, et al. A systematic evaluation of asthma management apps examining behavior change techniques. J Allergy Clin Immunol Pract. 2019;7(8):2583-91.
93. James DC, Harville C, Sears C, Efunbumi O, Bondoc I. Participation of African Americans in e-Health and m-Health studies: a systematic review. Telemedicine and e-Health. 2017;23(5):351-64.

94. Winter SJ, Sheats JL, King AC. The use of behavior change techniques and theory in technologies for cardiovascular disease prevention and treatment in adults: a comprehensive review. Prog Cardiovasc Dis. 2016;58(6):605-12.

95. Beratarrechea A, Lee AG, Willner JM, Jahangir E, Ciapponi A, Rubinstein A. The impact of mobile health interventions on chronic disease outcomes in developing countries: a systematic review. Telemed J E Health. 2014;20(1):75-82 Mary Ann Liebert, Inc. 140 Huguenot Street, 3rd Floor New Rochelle, NY 10801 USA.

96. Sun C, Malcolm JC, Wong B, Shorr R, Doyle M-A. Improving glycemic control in adults and children with type 1 diabetes with the use of smartphone-based mobile applications: a systematic review. Can J Diabetes. 2019;43(1):51-8.

97. Iribarren SJ, Cato K, Falzon L, Stone PW. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS ONE. 2017;12(2): e0170581.

98. Gaveikaite V, Grundstrom C, Winter S, Chouvarda I, Maglaveras N, Priori R. A systematic map and in-depth review of European telehealth interventions efficacy for chronic obstructive pulmonary disease. Respir Med. 2019;158:78-88.

99. Tchero H, Kangambega P, Briatte C, Brunet-Houdard S, Retali G-R, Rusch E. Clinical effectiveness of telemedicine in diabetes mellitus: a meta-analysis of 42 randomized controlled trials. Telemedicine and e-Health. 2019;25(7):569-83.

100. Bassi N, Karagodin I, Wang S, Vassallo P, Priyanath A, Massaro E, et al. Lifestyle modification for metabolic syndrome: a systematic review. Am J Med. 2014;127(12):1242-e1.

101. Ruiz-Perez I, Bastos A, Serrano-Ripoll MJ, Ricci-Cabello I. Effectiveness of interventions to improve cardiovascular healthcare in rural areas: A systematic literature review of clinical trials. Prev Med. 2019;119:132-44.

102. Lee J-A, Choi M, Lee SA, Jiang N. Effective behavioral intervention strategies using mobile health applications for chronic disease management: a systematic review. BMC Med Inform Decis Mak. 2018;18(1):12.

103. Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth chronic disease management on treatment adherence and patient outcomes: a systematic review. J Med Internet Res. 2015;17(2): e52.

104. Enwald HPK, Huotari MLA. Preventing the obesity epidemic by second generation tailored health communication: an interdisciplinary review. J Med Internet Res. 2010;12(2):e24.

105. Prochaska JO, Marcus BH. The transtheoretical model: Applications to exercise. 1994.

106. Bandura A. Human agency in social cognitive theory. Am Psychol. 1989;44(9):1175.

107. May C. Mobilising modern facts: health technology assessment and the politics of evidence. Sociol Health Illn. 2006;28(5):513-32.

108. Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. 2012;10(1):1-11.

109. Van Velsen L, Beaujean DJ, van Gemert-Pijnen JE. Why mobile health app overload drives us crazy, and how to restore the sanity. BMC Med Inform Decis Mak. 2013;13(1):1-5.

110. Lewis TL, Wyatt JC. mHealth and mobile medical apps: a framework to assess risk and promote safer use. J Med Internet Res. 2014;16(9): e210.

111. Holmes CM, Reid CA. Ethics in telerehabilitation: looking ahead. J Appl Rehabil Couns. 2018;49(2):14-23.

112. Vernooij RW, Willson M, Gagliardi AR. Characterizing patient-oriented tools that could be packaged with guidelines to promote self-management and guideline adoption: a meta-review. Implement Sci. 2015;11(1):1-13.

113. Kazdin AE. Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. Am Psychol. 2008;63(3):146-59.

114. Scott RE, Mars M, Jordanova M. Would a rose by any other name-cause such confusion? Journal of the International Society for Telemedicine and eHealth. 2013;1(2):52-3.

115. Bonnevie T, Lebret M, McKeough Z. Urgent need to define telerehabilitation for respiratory disease. Respirology. 2021;26(7):713-4.

116. Shukla H, Nair S, Thakker D. Role of telerehabilitation in patients following total knee arthroplasty: Evidence from a systematic literature review and meta-analysis. J Telemed Telecare. 2017;23(2):339-46.

117. Knepley KD, Mao JZ, Wieczorek P, Okoye FO, Jain AP, Harel NY. Impact of telerehabilitation for stroke-related deficits. Telemedicine and e-Health. 2021;27(3):239-46.