# Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature Review

Line Farah, Isabelle Borget, Nicolas Martelli, Alexandre Vallee

► To cite this version:

Line Farah, Isabelle Borget, Nicolas Martelli, Alexandre Vallee. Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature Review. Journal of Medical Internet Research, 2024, 26, pp.e51514. 10.2196/51514. inserm-04707078

HAL Id: inserm-04707078

https://inserm.hal.science/inserm-04707078v1

Submitted on 24 Sep 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Review

# Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature Review

Line Farah¹,²*, PharmD; Isabelle Borget²,³,⁴, PharmB, PhD; Nicolas Martelli²,⁵, PharmD, PhD; Alexandre Vallee⁶*, MD, PhD

¹Innovation Center for Medical Devices Department, Foch Hospital, Suresnes, France

²Groupe de Recherche et d'accueil en Droit et Economie de la Santé Department, University Paris-Saclay, Orsay, France

³Department of Biostatistics and Epidemiology, Gustave Roussy, University Paris-Saclay, Villejuif, France

⁴Oncostat U1018, Inserm, Équipe Labellisée Ligue Contre le Cancer, University Paris-Saclay, Villejuif, France

⁵Pharmacy Department, Georges Pompidou European Hospital, Paris, France

⁶Department of Epidemiology and Public Health, Foch Hospital, Suresnes, France

*these authors contributed equally

**Corresponding Author:**

Line Farah, PharmD
Innovation Center for Medical Devices Department
Foch Hospital
40 rue Worth
Suresnes, 92150
France
Phone: 33 952329655
Email: line.farah1@gmail.com

## Abstract

**Background:** Artificial intelligence (AI)-based medical devices have garnered attention due to their ability to revolutionize medicine. Their health technology assessment framework is lacking.

**Objective:** This study aims to analyze the suitability of each health technology assessment (HTA) domain for the assessment of AI-based medical devices.

**Methods:** We conducted a scoping literature review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We searched databases (PubMed, Embase, and Cochrane Library), gray literature, and HTA agency websites.

**Results:** A total of 10.1% (78/775) of the references were included. Data quality and integration are vital aspects to consider when describing and assessing the technical characteristics of AI-based medical devices during an HTA process. When it comes to implementing specialized HTA for AI-based medical devices, several practical challenges and potential barriers could be highlighted and should be taken into account (AI technological evolution timeline, data requirements, complexity and transparency, clinical validation and safety requirements, regulatory and ethical considerations, and economic evaluation).

**Conclusions:** The adaptation of the HTA process through a methodological framework for AI-based medical devices enhances the comparability of results across different evaluations and jurisdictions. By defining the necessary expertise, the framework supports the development of a skilled workforce capable of conducting robust and reliable HTAs of AI-based medical devices. A comprehensive adapted HTA framework for AI-based medical devices can provide valuable insights into the effectiveness, cost-effectiveness, and societal impact of AI-based medical devices, guiding their responsible implementation and maximizing their benefits for patients and health care systems.

(J Med Internet Res 2024;26:e51514) doi: 10.2196/51514

## KEYWORDS

artificial intelligence; machine learning; health technology assessment; medical devices; evaluation
# Introduction

## Background

Artificial intelligence (AI) has emerged as a transformative technology with vast potential across various sectors, including health care [1,2]. In this field, AI-based medical devices have garnered significant attention due to their ability to revolutionize diagnosis, treatment, and patient monitoring [3]. These devices use advanced algorithms and machine learning techniques to analyze complex medical data sets, thereby providing valuable

insights and support to health care professionals in their decision-making. To meet the ever-increasing demand for integration of AI-based medical devices into clinical practice, their efficient evaluation through adapted health technology assessment (HTA) is now crucial [4]. Several frameworks have been published showing an adaptation of items for reporting clinical trials related to AI-based medical devices (Table 1). However, there is still a lack of a much needed adaptation of the standard HTA framework to better suit the assessment of AI-based health care technologies.

<table><caption>**Table 1.** Summary of different frameworks on artificial intelligence (AI)-based health technologies.</caption><thead><tr><td>Framework</td><td>Function</td><td>Definition and aim</td><td>Domains and items assessed</td><td>Reference</td></tr></thead><tbody><tr><td>HTA<sup>a</sup> Core Model</td><td>Medical device assessment</td><td>A systematic evaluation of a medical technology's clinical, economic, ethical, and social implications to determine its overall value and impact on health care delivery; a methodological framework for production and sharing of HTA information</td><td>It evaluates nine domains: (1) health problem and current use of technology, (2) description and technical characteristics of the technology, (3) safety, (4) clinical effectiveness, (5) costs and economic evaluation, (6) ethical analysis, (7) organizational aspects, (8) patients and social aspects, and (9) legal aspects</td><td>European Network for Health Technology Assessment [5]</td></tr><tr><td>CONSORT-AI<sup>b</sup></td><td>Trial reporting</td><td>A set of recommendations for clinical trial reports evaluating interventions with an AI component</td><td>It evaluates the 25 CONSORT<sup>c</sup> 2010 items+14 AI-specific extension items</td><td>Liu et al [6]</td></tr><tr><td>SPIRIT-AI<sup>d</sup></td><td>Trial reporting</td><td>New reporting guidelines for clinical trial protocols evaluating interventions with an AI component</td><td>It evaluates the 33 SPIRIT<sup>e</sup> 2013 items+ 15 AI-specific extension items</td><td>Rivera et al [7]</td></tr><tr><td>TRIPOD-AI<sup>f</sup></td><td>Research reporting</td><td>Reports of research or endeavors in which a multivariable prediction model is being developed (or updated) or validated (tested) using any (supervised) machine learning technique</td><td>It evaluates the 22 TRIPOD<sup>g</sup> 2013 items and the AI extension item definitions</td><td>Collins et al [8]</td></tr></tbody></table>

An HTA involves a systematic evaluation of a medical technology's clinical, economic, ethical, and social implications to determine its overall value and impact on health care delivery [9]. The European Network for Health Technology Assessment has designed an HTA Core Model that provides a methodological framework for production and sharing of HTA information [5]. It evaluates the following nine domains: (1) health problem and current use of technology, (2) description and technical characteristics of the technology, (3) safety, (4) clinical effectiveness, (5) costs and economic evaluation, (6) ethical analysis, (7) organizational aspects, (8) patients and social aspects, and (9) legal aspects (Multimedia Appendix 1).

A full understanding of the capabilities, limitations, and specificities of AI-based medical devices is paramount to

complete these HTA domain assessments and, thereby, inform evidence-based decision-making and allow for policy development and the responsible integration of these technologies into health care systems [10,11]. Much uncertainty remains with regard to the reliability of AI-based medical devices, data issues, and regulatory processes, resulting in multiple challenges faced by HTA agencies assessing new technologies and delivering their approval [4]. Nevertheless, AI-based medical devices require strict regulations and specific legislations [12-14]. Over the last few decades, the evaluation of AI-based medical devices through an HTA process has received growing interest, as shown by the increase from 1 published article in 1990 to 94 in December 2023, with 484 articles in total during the period 1990-2023 (Table 2).

<footnote><sup>a</sup>HTA: health technology assessment.</footnote>

<footnote><sup>b</sup>CONSORT-AI: Consolidated Standards of Reporting Trials–Artificial Intelligence.</footnote>

<footnote><sup>c</sup>CONSORT: Consolidated Standards of Reporting Trials.</footnote>

<footnote><sup>d</sup>SPIRIT-AI: Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence.</footnote>

<footnote><sup>e</sup>SPIRIT: Standard Protocol Items: Recommendations for Interventional Trials.</footnote>

<footnote><sup>f</sup>TRIPOD-AI: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis-Artificial Intelligence.</footnote>

<footnote><sup>g</sup>TRIPOD: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis.</footnote>
<table><caption>**Table 2.** Occurrences of “technology assessment” AND “medical device” AND “artificial intelligence” from inception to December 2023 in the PubMed literature.</caption><thead><tr><td>Year</td><td>Articles (n=484), n (%)</td></tr></thead><tbody><tr><td>1990-1999</td><td>5 (1%)</td></tr><tr><td>2000-2009</td><td>44 (9%)</td></tr><tr><td>2010-2019</td><td>151 (32%)</td></tr><tr><td>2020-2023</td><td>273 (58%)</td></tr></tbody></table>

## Objectives

The objective of this review was to critically assess the comprehensive suitability of the current HTA process for AI-based medical devices. By evaluating the performance and capabilities of AI-based medical devices across multiple dimensions, this review aimed to inform health care professionals, policy makers, and researchers about the challenges and opportunities associated with these technologies. Ultimately, this review sought to facilitate evidence-based decision-making; promote responsible implementation; and maximize the potential benefits of AI-based medical devices in improving health care quality, accessibility, and outcomes.

To this end, we analyzed the suitability of each HTA domain for the assessment of AI-based medical devices and proposed an adapted HTA framework.

## Methods

### Search Strategies

A scoping literature review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology [15,16]. All articles related to HTA methods for AI-based medical devices were selected. Data extraction focused on assessment criteria, methodological evaluations, and results.

The search terms and strategy outlined in Textbox 1 were used for the scoping review.

<table><caption>**Textbox 1.** Search strategies and Medical Subject Heading (MeSH) terms used for the scoping literature review.</caption><thead><tr><th>Database and search strategy</th></tr></thead><tbody><tr><td><ul><li>PubMed</li><li>Health technology assessment<ul><li>“technology assessment, biomedical” [MeSH term] OR (“technology” [all fields] AND “assessment” [all fields] AND “biomedical” [all fields]) OR “biomedical technology assessment” [all fields] OR (“health” [all fields] AND “technology” [all fields] AND “assessment” [all fields]) OR “health technology assessment” [all fields]</li></ul></li><li>Medical device<ul><li>“equipment and supplies” [MeSH term] OR (“equipment” [all fields] AND “supplies” [all fields]) OR “equipment and supplies” [all fields] OR (“medical” [all fields] AND “device” [all fields]) OR “medical device” [all fields]</li></ul></li><li>Artificial intelligence<ul><li>“artificial intelligence” [MeSH term] OR (“artificial” [all fields] AND “intelligence” [all fields]) OR “artificial intelligence” [all fields]</li></ul></li><li>Embase<ul><li>(“artificial intelligence” OR “artificial intelligence-based medical device”) AND “technology assessment”</li></ul></li><li>Cochrane Library<ul><li>(technology assessment) and (artificial intelligence)</li></ul></li></ul></td></tr></tbody></table>

## Databases

We searched multiple databases: PubMed, Embase, and the Cochrane Library. Additional articles were retrieved manually from the gray literature and from HTA agency websites (Bundesinstitut für Arzneimittel und Medizinprodukte [Federal Institute for Drugs and Medical Devices in Germany], Haute Autorité de Santé [French Health Authority], National Health Service, National Institute for Health and Care Excellence,

National Institute of Public Health, International Network of Agencies for Health Technology Assessment, and the European Network for Health Technology Assessment).

## Study Selection Process

The studies were selected by 2 reviewers (LF and AV). After removing duplicates, both reviewers independently screened the abstracts to select eligible articles and then analyzed full-text reports for eligibility. A third party (NM) resolved the possible
discrepancies highlighted during the selection process should
a consensus not be reached. An extraction database was used
to list the selected articles meeting the inclusion criteria to
ensure that all eligible articles were included.

The inclusion and exclusion criteria were related to (1) language (this review was limited to the English and French languages), (2) article type (reviews and primary research were included, whereas other article types were excluded, such as abstracts, commentaries, editorials, letters to the editor, case reports, case series, animal studies, phase-1 and phase-2 studies, pilot studies, duplicate studies, irrelevant studies, studies with the wrong aim, and studies available in abstract form only), (3) type of technology (only AI-based medical devices were eligible), and (4) type of evaluation (HTA articles were included).

Data Extraction

In total, 2 analysts (LF and AV) independently extracted data
items from the selected articles. A third party was involved to
resolve any discrepancies highlighted during the selection
process. The following data were extracted from each article:
(1) general characteristics of the studies (authors, journal, and
publication date), (2) study objective, and (3) HTA assessment
domain for AI-based medical devices related to the article.

Methodological Quality Appraisal

Neither the methodological quality nor the risk of bias of the
included articles were assessed, consistent with scoping review
guidance [15].

Application Example of the HTA of an AI-Based
Medical Device

Concerning each HTA domain, we added a case study of the
HTA of several AI-based medical devices in diabetes to illustrate
our recommendations.

Results

Scoping Review Results

The literature search resulted in 775 citations summarized in the PRISMA flowchart (Figure 1). After removal of 32.6% (253/775) of duplicates, we carried out an initial screening of the remaining 522 publications, resulting in 129 (24.7%) potentially relevant full-text papers. After further screening, we assessed 77 reports for eligibility, based on which we excluded 11 (14%) for not referring to AI-based medical devices and 2 (3%) for not being HTAs. We then included a further 14 records from 59 potential citations available from HTA agency websites and gray literature.

<figure><img src="image_2.png" /><figcaption>**Figure 1.** PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart of the study selection process. AI: artificial intelligence; HTA: health technology assessment; MD: medical device.</figcaption></figure>

This gave a total of 78 included articles covering one or multiple
HTA domains, the distribution of which is provided at the
bottom of Figure 1.

We then summarized the data collection for each selected article
in Multimedia Appendix 2 [1,2,4,9,10,17-89].

Suitability of Each HTA Domain for the Assessment
of AI-Based Medical Devices

As domain 1, health problem and current use of technology, is
suitable for any type of medical technology and systematically
addressed these technologies, we focused on the suitability of
the HTA domains (from 1 to 9).
## Domains 1 and 2 (Health Problem, Current Use, Description, and Technical Characteristics of the Technology)

Data quality and integration are vital aspects to consider when describing and assessing the technical characteristics of AI-based medical devices during an HTA process [17]. These devices often rely on accessing and analyzing diverse health care data sources, including electronic health records, medical images, genetic data, and wearable device data. Therefore, it is essential to evaluate their ability to seamlessly integrate and exchange data with existing health care systems [18].

First, interoperability refers to the ability of AI-based medical devices to interact and communicate with other health care technologies and systems [10]. This includes the ability to access and use data from different sources, such as laboratory systems, imaging archives, and patient health records. The assessment should consider whether the devices adhere to relevant data standards and protocols, ensuring efficient data exchange and compatibility with existing health care infrastructures [19].

Second, data integration involves the ability of AI-based medical devices to aggregate and analyze data from multiple sources to provide comprehensive and accurate insights [20]. The HTA should assess whether the devices can handle different data types, formats, and resolutions and whether they can effectively integrate and harmonize data from disparate sources.

In addition, data privacy and security considerations are crucial when evaluating AI-based medical devices [21]. The assessment should examine whether the devices comply with relevant data protection regulations, use appropriate data anonymization and encryption techniques, and have robust security measures in place to protect patient information [22].

Therefore, a comprehensive HTA should address both the interoperability and data integration capabilities of AI-based medical devices, ensuring that they can seamlessly interact with existing health care systems and integrate data from multiple sources and that they adhere to data privacy and security standards. By evaluating all these aspects, the HTA could determine the devices' feasibility, scalability, and potential impact on health care delivery.

## Domain 3 (Safety) and Domain 4 (Clinical Effectiveness)

The assessment of clinical effectiveness and the impact on patient outcomes is a crucial aspect when evaluating AI-based medical devices through a comprehensive HTA [4]. While accuracy and performance metrics are important, it is essential to determine how these devices translate into tangible benefits for patients and health care delivery [23].

Clinical utility refers to the extent to which the AI-based medical device improves clinical decision-making, patient outcomes, and health care processes [24]. The HTA should examine whether the device provides actionable and reliable information that potentially helps health care professionals make more accurate diagnoses or improve treatment plans or monitoring strategies [25,26]. It should also evaluate the potential impact of the device on patient outcomes by assessing, for example,

improved survival rates, reduced complications, or enhanced quality of life [27].

To assess clinical utility, the HTA should consider the device's performance in relevant clinical scenarios and its ability to address specific clinical questions or challenges. This may involve evaluating the device's performance against established clinical guidelines or expert opinions as well as considering the device's potential to fill gaps in clinical practice or enhance existing diagnostic or treatment methods.

Furthermore, the HTA should examine the broader impact of AI-based medical devices on health care systems and resource allocation [21]. This includes evaluating the devices' potential to optimize resource use, reduce health care costs, or improve workflow efficiency. Economic evaluations such as cost-effectiveness analyses can provide insights into the value for money and long-term cost savings associated with the adoption of these devices [28,29].

A comprehensive HTA should thoroughly assess the clinical effectiveness and impact on patient outcomes of AI-based medical devices, examining their performance in relevant clinical scenarios, their alignment with clinical guidelines, and their potential to improve health care processes and resource allocation. By evaluating these aspects, the HTA can provide a holistic understanding of the AI devices' effectiveness and their potential to positively transform health care delivery [21].

AI-based medical devices must undergo rigorous clinical validation to assess their performance and reliability in real-world health care settings [30,31]. Clinical validation involves evaluating the device's accuracy, sensitivity, specificity, and overall diagnostic or prognostic performance. In addition, the devices should be tested across diverse patient populations and compared against gold-standard reference methods or expert opinions [32].

Furthermore, the use of real-world evidence (RWE) is crucial for a comprehensive HTA [33]. RWE involves gathering data from routine clinical practice, electronic health records, and other sources to evaluate the device's effectiveness and safety in real-world settings [34]. These data can help assess the device's performance in a broader patient population and identify any potential limitations or biases that may arise in specific clinical scenarios.

Robust clinical validation studies and the integration of RWE provide critical evidence for the evaluation of AI-based medical devices in an HTA process [4]. These studies should include a sufficient sample size, appropriate study design, and statistical analysis to ensure the validity and generalizability of the results [35-38]. By considering clinical validation and RWE adapted to AI-based medical devices, HTA can provide valuable insights into their clinical utility and impact in real-world health care settings, facilitating evidence-based decision-making for their adoption and use.

A lack of safety evaluation was highlighted in a systematic review, showing that only 9% of AI-based medical device studies evaluated safety criteria [4]. However, safety is crucial for the confidence in and adoption of AI-based technologies for both patients and health care professionals [39]. While these
devices have the potential to improve diagnosis and treatment outcomes, their integration into clinical practice must be accompanied by robust safety evaluations [2,40]. Identifying and mitigating potential risks, such as algorithmic bias, data privacy breaches, and algorithm failures, is essential to protect patient well-being and maintain trust in these innovative technologies [41].

To ensure the safety of AI-based medical devices, it is crucial to establish standardized evaluation frameworks and guidelines [39]. These frameworks should encompass rigorous testing methodologies, validation procedures, and continuous monitoring of AI performance. Collaboration among stakeholders, including manufacturers, regulatory agencies, health care providers, and researchers, is essential to develop and implement comprehensive safety evaluation protocols. Addressing the safety gap in AI-based medical devices not only ensures patient welfare but also instills confidence in health care professionals to embrace and use these technologies effectively [40]. By prioritizing safety evaluation, we can unleash the full potential of AI-based medical devices, leading to transformative advancements in health care delivery [1].

## Domain 5 (Costs and Economic Evaluation)

Costs and economic evaluation play a crucial role in the comprehensive assessment of health technology adoption, particularly in the context of the HTA of AI-based medical devices [4,9,42]. While AI technologies show huge potential to improve health care delivery, reduce time to diagnosis, and save money, their economic impact requires careful consideration to ensure their successful integration and sustainability [23].

When assessing AI-based medical devices through the HTA process, economic evaluations involve analyzing the costs and benefits associated with the adoption and use of the technology [4]. These evaluations go beyond the upfront costs of acquiring the AI-based medical device; they encompass various factors, such as training, infrastructure modifications, maintenance, and ongoing operational costs [43]. Cost analysis also includes improved patient outcomes, reduced hospital readmissions, and shortened hospital stays. By conducting economic evaluations, decision makers can gauge the cost-effectiveness and cost utility to weigh the affordability of AI-based medical devices for their particular use against the potential mid- and long-term cost savings considered [44].

HTA evaluates the suitability of AI-based medical devices by considering their potential economic benefits, risks, and ethical implications [45,46]. By incorporating economic evaluation into the HTA process, decision makers can make evidence-based choices about the allocation of limited health care resources and prioritize interventions that provide the greatest value for money [4,47].

Finally, considering cost savings and conducting economic evaluations as part of the HTA process helps facilitate the adoption of AI-based medical devices [37]. It provides decision makers with the necessary information to determine the financial feasibility and potential return on investment associated with implementing these technologies. HTA ensures that AI-based

medical devices are suitable for integration into the health care system, fostering confidence in their effectiveness, efficiency, and cost utility [4,43,48].

## Domain 6 (Ethical Analysis) and Domain 8 (Patients and Social Aspects)

Ethical and societal implications are critical aspects that must be considered when evaluating the suitability of AI-based medical devices for a comprehensive HTA [45]. The integration of AI technologies into health care raises important ethical concerns that need to be addressed to ensure responsible and equitable use [49].

Data privacy and patient consent are primary ethical considerations [50]. The HTA should evaluate whether AI-based medical devices adhere to strict data protection regulations, maintain patient confidentiality, and obtain appropriate informed consent for data use. It is essential to assess whether the devices have mechanisms in place to handle sensitive patient information securely and protect it against unauthorized access or data breaches.

Algorithmic fairness and bias are additional ethical concerns [51]. AI algorithms can inadvertently perpetuate biases present in the data used for training, resulting in unequal treatment or access to health care resources [52]. The HTA should assess whether the devices have been evaluated for fairness and bias and consider the steps taken to mitigate any identified biases [53].

Moreover, the HTA should examine the impact of AI-based medical devices on health care disparities and access to care [54]. It is crucial to assess whether the devices have the potential to exacerbate existing inequalities or whether they can contribute to reducing disparities by improving health care access, particularly for underserved populations.

Accountability and transparency in AI decision-making processes are also important ethical considerations [55]. The HTA should evaluate whether the AI devices provide clear explanations for their outputs and ensure that health care professionals and patients can understand and challenge the device's recommendations [21].

A comprehensive HTA should thoroughly examine the ethical and societal implications of AI-based medical devices, ensuring that they prioritize patient privacy, fairness, and equitable access to care; fostering trust; and ensuring that these devices align with societal values and goals.

To ensure the adoption of AI-based medical devices through a truthfully AI concept, explainability, interpretability, and transparency are important considerations for their comprehensive HTA [56]. Indeed, these devices often use complex algorithms and machine and deep learning techniques, resulting in their operating as “black boxes” that reach decisions and recommendations that are challenging to understand [55]. However, explainability and transparency are essential to ensure trust, accountability, and acceptance of AI-based technologies in health care. First, Explainability refers to the ability to understand and interpret the reasoning behind the device’s outputs [56]. It involves providing clinicians and users with
transparent explanations of how the AI algorithm processes input data and generates results. This enables health care professionals to trust the device's recommendations and make informed decisions based on the provided information. The HTA should evaluate the extent to which AI-based medical devices can provide interpretable and understandable explanations of their decision-making processes [17,46]. Second, Transparency involves disclosing important information about the AI-based medical devices, including the data used for training, algorithmic methodologies used, and potential limitations or biases [57]. Transparency promotes trust and allows stakeholders to assess the device's reliability and potential risks. The HTA should assess whether the device manufacturers provide clear documentation and information to health care professionals and patients about the device's capabilities, limitations, and potential errors [21,58,59]. Transparency is closely linked to regulatory considerations [60,61]. The HTA should consider whether the device complies with relevant regulatory standards and whether the manufacturers have provided the necessary documentation and evidence to support their ethical claims [21,62,63].

A comprehensive HTA process should address concerns regarding the explainability and transparency of AI-based medical devices, trust, accountability, and ethical implications related to the use of AI in health care [45,46].

## Domain 7 (Organizational Aspects)

Organizational aspects play a crucial role in the effective integration of AI-based medical devices into health care systems [64]. To ensure consistency, transparency, and comparability in the evaluation process, there is a growing need for a robust methodological framework that provides standardized guidelines for assessing the organizational impact of the implementation of AI-based medical devices [65]. According to a descriptive analysis led in German hospitals, the main barriers to AI-based medical device adoption were lack of resources (staff, knowledge, and financial) [66].

Clear indicators are needed to measure the organizational readiness for and impact of AI-based medical devices [4,10,67]. Several criteria have been highlighted, such as (1) health care workplace readiness and stakeholder acceptance, (2) AI-based medical device organization alignment assessment, and (3) business plan (financing and investments) [64].

## Domain 9 (Legal Aspects)

The roles and responsibilities of health care professionals are also impacted by these AI solutions [68]. It is necessary to evaluate whether they complement or replace health care professionals' expertise and whether additional training, supervision, or support is required for their optimal use [69,70]. As AI technologies could evolve and become more prevalent in health care, it is crucial to ensure that these devices comply with existing legal frameworks and regulations [49,71]. HTA plays a pivotal role in evaluating the legal implications of

AI-based medical devices by assessing factors such as data privacy, security, liability, and regulatory compliance [72].

One key legal aspect to consider is data privacy and protection [73,74]. AI-based medical devices often rely on vast amounts of patient data for training and decision-making. Therefore, it is essential to evaluate whether these devices adhere to relevant data protection laws, such as the General Data Protection Regulation in the European Union [75,76]. HTA examines the measures taken by AI-based medical device manufacturers to safeguard patient privacy, including data anonymization, encryption, and secure data storage practices [37]. These issues are highlighted in the Artificial Intelligence Act published in June 2023 by the European Commission [77,78].

Cybersecurity is another critical consideration [21,79]. As AI-based medical devices handle sensitive patient data and make critical health care decisions, it is crucial to assess the security measures implemented to prevent unauthorized access, data breaches, or tampering [80]. HTA evaluates the robustness of the security protocols implemented by device manufacturers and their compliance with industry standards and regulations [21,81].

Liability is also a significant legal aspect to be addressed in the context of AI-based medical devices [82,83]. When errors or adverse events occur due to the use of these devices, determining liability can be complex [84]. HTA examines the legal frameworks and liability guidelines pertaining to AI technologies, including whether clear guidelines exist regarding the responsibility of manufacturers, health care providers, and users in case of device malfunctions or errors [85]. Assessing liability aspects within the HTA process helps establish accountability and ensures that legal frameworks adequately address potential risks [86].

Regulatory compliance is a crucial consideration when assessing the suitability of AI-based medical devices for HTA [87]. Depending on the jurisdiction, AI devices need to undergo regulatory approval processes before being introduced into the market [88]. HTA examines whether AI-based medical device manufacturers have obtained the necessary regulatory approvals, such as clearance from the relevant health authorities or certification from regulatory bodies such as the Food and Drug Administration (FDA) in the United States or notified bodies in Europe [89]. This evaluation ensures that AI-based medical devices comply with existing regulations and are fit for clinical use [61].

# Discussion

## Recommendations to Adapt the 9 HTA Domains for AI-Based Medical Devices

Taking into account the previous considerations, some recommendations could be proposed to adapt and personalize these standard HTA domains to AI-based medical devices' specificities. Therefore, we suggest 4 main recommendations by domain in Table 3.
<table><caption>**Table 3.** Recommendations to adapt and personalize the health technology assessment (HTA) of standard HTA domains to artificial intelligence (AI)-based medical device specificities.</caption><thead><tr><th>Domains</th><th>4 recommendations per domain</th><th>Application examples</th></tr></thead><tbody><tr><td>1 and 2—health problem, description, and technical characteristics of the technology</td><td><ul><li>Assess the technical performance and reliability of the AI-based medical device, especially for ML<sup>a</sup> and DI<sup>b</sup>-based medical devices</li><li>Evaluate the usability and user interface of the AI-based medical device</li><li>Assess the interoperability and compatibility of the AI-based medical device with existing systems</li><li>Regularly update the assessment to reflect AI technological advancements</li><li>Update the assessment periodically to incorporate new information and advancements related to the AI-based medical device</li></ul></td><td>Consider the example of an AI-based DMS<sup>c</sup> that assists patients with diabetes in managing their condition by providing blood glucose predictions based on their planned actions, thus contributing to better decision-making. The description and technical features involve defining patient data collection, data preprocessing, selecting relevant features, constructing an AI model, training the model & validating it using separate data, deploying it to provide blood glucose predictions to patients, and continuously improving it through real-time data collection.</td></tr><tr><td>3—safety</td><td><ul><li>Evaluate AI compliance with safety regulations and standards and AI risk management processes</li><li>Evaluate the AI's robustness and resilience to errors or failures</li><li>Assess the AI's impact on patient safety, user safety, and long-term safety</li><li>Consider the durability of the AI technology; regularly monitor and update safety information; and regularly monitor and reassess the AI's safety profile incorporating new information, evidence, and user experiences</li></ul></td><td>The potential risk of patient injury of an insulin delivery AI-based medical device system should be taken into account in the risk management process. In the case of an evolutive DL-based medical device without continuous ongoing safety assessment, a wrong dosage administration due to an AI error could provoke a serious adverse event (ie acid ketosis coma for a patient with diabetes). A risk management plan should be available and regularly updated with AI changes impact on safety plan.</td></tr><tr><td>4—clinical effectiveness</td><td><ul><li>Compare the AI technology to existing alternatives or standard of care</li><li>Assess the generalizability of the clinical evidence and analyze the clinical outcomes in subpopulations</li><li>Consider patient-reported outcomes, quality-of-life measures, and impact on clinical workflow</li><li>Regularly update the clinical evidence base of the AI-based medical devices and monitor consistency during the AI lifetime</li></ul></td><td>The FDA<sup>d</sup> in the United States has proposed a regulatory framework for modifications to evolutive AI- and ML-based software as a medical device with modification guidance focused on the risk to users and patients resulting from the AI changes. They have asked for an Algorithm Change Protocol with specific methods in place to achieve and control the risks of the anticipated types of modifications related to performance, use, or inputs. A continuous clinical effectiveness assessment could be an interesting approach for an AI-based medical device diabetes foot ulcer detection system for patients needing adaptive treatment modifications to prevent ulcer development.</td></tr><tr><td>5—costs and economic evaluation</td><td><ul><li>Assess the budget impact of adopting the AI-based medical device within the health care system</li><li>If needed, conduct a comprehensive cost-effectiveness analysis of the AI-based medical device compared to standard care and assess long-term impact on health care resource use (hospital admissions, length of stay, and time to diagnosis), health care disparities, and access to care in case of evolution of the AI's performance</li><li>Explore potential reimbursement strategies and work with payers to develop reimbursement models that align with the value and impact of the AI device (on clinical effectiveness, cost-effectiveness, and long-term sustainability)</li><li>Establish mechanisms for ongoing monitoring and reassessment of the AI device's economic value to ensure that it remains aligned with evolving health care priorities and resource allocation strategies</li></ul></td><td>To evaluate a diabetic retinopathy screening AI-based medical device, clinical effectiveness is not sufficient. The cost-effectiveness of different types of AI diabetic retinopathy screening should be compared with no screening and ophthalmologist screening. A recent study demonstrated that AI-based screening was the most cost-effective, not only saving costs but also improving the quality of life of patients with diabetes. In this case, the long-term assessment of the economic impact of AI introduction in diabetic retinopathy screening highlighted the added value of this technology.</td></tr></tbody></table>
<table><thead><tr><th>Domains</th><th>4 recommendations per domain</th><th>Application examples</th></tr></thead><tbody><tr><td>7—organizational aspects</td><td><ul><li>Assess the organizational readiness and capacity for implementing AI-based medical devices in health care settings</li><li>Conduct a thorough analysis of the impact of implementing AI-based medical devices on health care workflows and processes</li><li>Establish clear guidelines and protocols for the appropriate use and integration of AI-based medical devices</li><li>Monitor and evaluate the impact of AI-based medical device implementation on patient outcomes, quality of care, and patient satisfaction</li><li>Regularly review and update implementation strategies based on the findings from monitoring AI</li></ul></td><td>Decision makers and technology advocates need to address the complexities of AI more comprehensively and understand the systemic challenges that its adoption poses to health care organizations and systems. As an example, consider an AI tool used for diagnosing diabetic retinopathy in a primary care setting, such as by a family physician or nurse. In theory, this could lead to shorter waiting times for patients. However, if the health care organization faces challenges such as a shortage of specialized staff (eg, ophthalmologists), insufficient organization of care pathways, and lack of specialized facilities for proper management and follow-up after diagnosis, the introduction of AI might adversely affect the quality of care and patient experience.</td></tr><tr><td>Domains 6, 8, and 9—ethical analysis or patients and social aspects or legal aspects</td><td><ul><li>Conduct an ethical analysis and evaluate the social implications of implementing the AI-based medical devices</li><li>Assess the legal and regulatory aspects related to the AI-based medical device</li><li>Promote transparency, interpretability, and explainability of the AI-based medical devices</li><li>Monitor and address emerging ethical, social, and legal issues related to the AI-based medical devices</li><li>Establish mechanisms for ongoing surveillance and evaluation of the AI device's ethical, social, and legal implications</li></ul></td><td>AI-based continuous glucose monitoring and insulin pumps should also be assessed on ethical and legal aspects. The risks of data theft and privacy breaches necessitate careful consideration of ethical and legal issues for patients. Although AI can aid in decision-making, it cannot wholly substitute a physician's expertise. Effective regulations and systems designed to ensure safety, reduce bias, and enhance transparency are essential.</td></tr></tbody></table>

The adaptation of the HTA process through a methodological framework for AI-based medical devices enhances the comparability of results across different evaluations and jurisdictions [90-92]. It promotes consistency in the assessment methodologies, reporting formats, and presentation of findings, enabling decision makers to make informed choices based on reliable and comparable evidence. This standardization contributes to the overall credibility and acceptance of HTA outcomes related to AI-based medical devices [93]. In addition,

a methodological framework would address the need for appropriate expertise and skills to conduct the HTA of AI-based medical devices [94-99]. It would outline the qualifications and competencies required for the individuals involved in the assessment, including knowledge of AI technologies [46]. By defining the necessary expertise, the framework supports the development of a skilled workforce capable of conducting robust and reliable HTAs of AI-based medical devices (Figure 2).

<footnote><sup>a</sup>ML: machine learning.</footnote>

<footnote><sup>b</sup>DL: deep learning.</footnote>

<footnote><sup>c</sup>DMS: diabetes management system.</footnote>

<footnote><sup>d</sup>FDA: Food and Drug Administration.</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 2.** Suggested recommendations to adapt each standard health technology assessment (HTA) domain for the evaluation of artificial intelligence (AI)-based medical devices (MDs). RWE: real-world evidence.</figcaption></figure>

Evaluating the trade-offs and weighing different features of AI-based medical devices is indeed a complex and important task, especially in domains such as health care where the impact on human lives is significant. The acceptability of AI-based medical devices should be assessed on a case-by-case basis considering various factors, including, for instance, performance, accuracy, cost, explainability, and the specific context in which they are being used.

Trade-offs between accuracy and cost are common in AI. It may be acceptable for AI to increase follow-up care costs if it significantly improves accuracy and patient outcomes. For

example, if an AI system can detect diseases at an earlier stage, it might lead to more effective treatment and, ultimately, lower overall health care costs in the long run.

The balance between explainability and performance is a critical consideration. While explainable algorithms are preferred for safety and transparency reasons, there may be cases in which a highly complex, unexplainable algorithm outperforms explainable ones. In such cases, the trade-off between transparency and performance should be carefully evaluated based on the specific use case and potential risks.
The minimum performance for adding this technology to the tools available to human clinicians depends on the specific task and the level of trust that patients, health care providers, and regulators have in the technology. Some key factors to consider include the following: (1) The complexity of the task—AI-based medical devices should excel in tasks that are well defined and data driven, but they may not replace human clinicians in tasks requiring complex decision-making, empathy, or ethical considerations. (2) Safety and reliability—AI-based medical devices should demonstrate a high level of safety and reliability, ideally surpassing the performance of human clinicians in terms of avoiding errors. (3) Ethical considerations—AI-based medical devices should adhere to ethical standards, including patient privacy, informed consent, and unbiased decision-making, which are often considered even more important than performance metrics. (4) Regulatory approval—regulatory bodies often establish performance thresholds for AI-based medical devices. Compliance with these thresholds is essential for market acceptance.

In general, AI should aim to complement and enhance the capabilities of human clinicians rather than completely replacing them. The specific threshold for acceptable performance will vary across applications and contexts, and it should be determined through a combination of rigorous testing, peer-reviewed studies, and input from health care professionals and patients.

It is important to note that ethical considerations, patient safety, and the potential for bias should always be at the forefront of these discussions, and AI-based medical devices should not be adopted solely for the sake of automation or cost reduction if they compromise these critical aspects of health care.

## Use Case of the Application of the Aforementioned HTA Recommendations for AI: AI-Based Medical Devices in Pathways for Patients With Diabetes

Concerning the HTA domain 3 on the safety of an AI-based medical device, the potential risk of patient injury of an insulin delivery AI-based medical device system should be taken into account in the risk management process of algorithm development [96]. In the case of an evolutive deep learning-based medical device without continuous ongoing safety assessment, a wrong dosage administration due to an AI error, for instance, could provoke a serious adverse event such as an acid ketosis coma for a patient with diabetes.

A risk management plan should be available for users and regularly updated with the impact of the AI changes on the safety plan.

In relation to the HTA domain 4, which focuses on the effectiveness of an AI-based medical device, the FDA in the United States has proposed a regulatory framework for modifications to evolutive AI- and machine learning-based software as a medical device with modification guidance focused on the risk to users or patients resulting from the AI changes [100]. They have asked for an Algorithm Change Protocol with specific methods in place to achieve and control the risks of the anticipated types of modifications related to performance, use, or inputs. A continuous clinical effectiveness assessment could

be an interesting approach for an AI-based medical device detection system for diabetes foot ulcer for patients needing adaptative treatment modifications to prevent ulcer development [101].

Concerning the HTA domain 5, to evaluate a diabetic retinopathy screening AI-based medical device, clinical effectiveness is not sufficient. The cost-effectiveness of different types of AI diabetic retinopathy screening should be compared with no screening and ophthalmologist screening. A recent study demonstrated that AI-based screening was the most cost-effective, not only saving costs but also improving the quality of life of patients with diabetes [44]. In this case, the long-term assessment of the economic impact of AI introduction in diabetic retinopathy screening highlighted the added value of this technology.

The HTA domain 7 on organizational impact has to assess how the AI-based medical device can be effectively integrated into the health care pathway and prevent wasteful spending. More thorough attention must be paid to the following aspects: (1) evaluating needs and determining the added value of the implementation of the AI-based medical device; (2) assessing workplace preparedness, including stakeholder acceptance of the introduction of the AI-based medical device and involvement; and (3) analyzing the alignment between AI technology and organizational structure [64]. Decision makers and technology advocates need to address the complexities of AI more comprehensively and understand the systemic challenges that its adoption poses to health care organizations and systems. As an example, consider an AI tool used for diagnosing diabetic retinopathy in a primary care setting, such as by a family physician or nurse [102]. In theory, this could lead to shorter waiting times for patients. However, if the health care organization faces challenges such as a shortage of specialized staff (eg, ophthalmologists), insufficient organization of care pathways, and lack of specialized facilities for proper management and follow-up after diagnosis, the introduction of AI might adversely affect the quality of care and patient experience. In such a scenario, the AI application might merely transfer the delay from primary to secondary care, failing to address the fundamental issue.

Finally, concerning the last domains (6, 8, and 9) about ethical, patient, social, and legal aspects, AI-based continuous glucose monitoring and insulin pumps should also be assessed on ethical and legal aspects [103]. While citizen juries have generally shown support for AI in research and treatment, concerns remain. The risks of data theft and privacy breaches necessitate careful consideration of ethical and legal issues for patients. Although AI can aid in decision-making, it cannot wholly substitute a physician's expertise. Effective regulations and systems designed to ensure safety, reduce bias, and enhance transparency are essential.

By implementing these recommendations, stakeholders can foster the responsible integration, regulation, and evaluation of AI-based medical devices. These measures can enhance the evidence base, address ethical concerns, and maximize the potential benefits of these AI technologies in improving health
care outcomes while protecting patient safety, privacy, and equity.

## Practical Challenges and Potential Barriers to Implementing HTAs Specific for AI-Based Medical Devices

When it comes to implementing specialized HTAs for AI-based medical devices, several practical challenges and potential barriers could be encountered and should be taken into account: First, AI technological evolution timeline—AI technologies evolve at a much faster pace compared to traditional medical devices, making it challenging for HTA frameworks to keep up with the latest developments and assess their long-term impact effectively. AI-based medical devices have a short product lifetime, between 12 and 18 months, in contrast to drug products. This shorter life cycle highlights the need for evolutive and fast-track HTA processes for AI-based medical devices. Second, data requirements and quality—AI systems rely heavily on large data sets for training and validation. Ensuring the availability of high-quality, representative data is a significant challenge. There is also the issue of data privacy and security, which must be addressed. The availability and quality of data and evidence required for conducting HTAs on AI-based medical devices present a complex and evolving landscape. Assessing the feasibility and challenges associated with gathering such data is crucial for robust evaluations. First, the availability of data can vary significantly depending on the AI-based medical device. While some devices may have access to vast amounts of high-quality real-world patient data, others might face limitations due to the novelty of the technology or issues related to data privacy. Second, the quality of data is paramount as inaccurate or biased data can lead to flawed assessments. Ensuring data accuracy, representativeness, and relevance is a constant challenge in AI-based medical device evaluations [104]. The rapid pace of AI development can result in limited long-term data, making it difficult to assess the device's sustained performance and safety [105]. Balancing the need for robust evidence with the dynamic nature of AI technologies is a significant challenge that HTA organizations must address to provide valuable insights for informed decision-making in health care. Third, complexity and transparency—the complex algorithms used in AI-based medical devices can be difficult to understand and assess, leading to issues with transparency and explainability [17]. This complexity can pose a challenge for regulators and assessors in HTA processes. Fourth, clinical validation and safety requirements—generating robust clinical evidence to demonstrate the safety, efficacy, and effectiveness of AI-based medical devices can be challenging. This includes proving that these devices perform consistently across diverse patient populations. Moreover, to account for a life cycle that

would include updates that may improve performance, one solution to investigate could be the proposition of the FDA in the United States in 2019 to experiment with a "dynamic certification" [100]. It allows for the re-evaluation of the AI-based medical device in case of a substantial modification of the indication of the medical device or the way to deliver the diagnosis, for instance. Fifth, regulatory and ethical considerations—adapting existing regulatory frameworks to accommodate AI-based medical devices, addressing ethical concerns such as bias, and ensuring equitable access are critical challenges [106]. Sixth, economic evaluation—determining the cost-effectiveness of AI-based medical devices, especially when benefits might be indirect or long term, poses a unique challenge for HTA [93]. Seventh, stakeholder engagement and trust—building trust among health care providers, patients, and policy makers regarding the reliability, trustworthiness, and usefulness of AI-based medical devices is crucial but challenging [107]. Eighth, integration into health care systems and interoperability—the integration of AI-based medical devices into existing health care workflows and systems can be complex and resource intensive [18]. Ninth, global and local applicability—ensuring that AI-based medical devices are effective and appropriate for use in different global and local contexts, considering varying health care systems and population needs, is another significant barrier.

Practically, one suggestion to implement such frameworks could be to implement these recommendations in the future European clinical joint assessment guidelines. As they are being currently discussed in a European project on a common HTA process for connected medical devices that includes AI-based medical devices, it could be the opportunity to tackle these challenges at the European level [108].

Addressing these challenges requires a concerted effort from regulators, health care providers, technology developers, and other stakeholders in the health care ecosystem.

## Conclusions

AI-based medical devices have the potential to transform health care delivery, but the suitability of the current comprehensive HTA requires careful adaptation of the evaluation across the 9 dimensions. While these AI devices show promise in improving accuracy, safety, and efficiency, there is a need for robust clinical validation, integration into workflows, economic evaluation, and addressing of ethical and legal implications. A comprehensive adapted HTA framework for AI-based medical devices can provide valuable insights into their effectiveness, cost-effectiveness, and societal impact, guiding their responsible implementation and maximizing their benefits for patients and health care systems.

## Data Availability

The data sets generated during and analyzed during this study are available from the corresponding author on reasonable request.

## Authors' Contributions

LF and AV contributed to the conceptualization. LF and AV contributed to the methodology. LF, AV, NM, and IB contributed to the article selection. LF and AV contributed to the writing and original draft preparation. NM and IB contributed to the writing
and review and editing. LF and AV contributed to the validation. The authors have read and agreed to the published version of the manuscript.

## Conflicts of Interest

None declared.

## Multimedia Appendix 1

The 9 health technology assessment (HTA) domains for the assessment of artificial intelligence-based medical devices adapted from the HTA Core Model proposed by the European Network for Health Technology Assessment.

[DOCX File, 13 KB-Multimedia Appendix 1]

## Multimedia Appendix 2

Summary results of the scoping review and general characteristics of the studies included in the scoping review.

[DOCX File, 27 KB-Multimedia Appendix 2]

## References

1. Bajwa J, Munir U, Nori A, Williams B. Artificial intelligence in healthcare: transforming the practice of medicine. Future Healthc J. Jul 2021;8(2):e188-e194. [doi: 10.7861/fhj.2021-0095] [Medline: 34286183]

2. Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. In: Artificial Intelligence in Healthcare. Cambridge, MA. Academic Press; 2020.

3. Park CW, Seo SW, Kang N, Ko B, Choi BW, Park CM, et al. Artificial intelligence in health care: current applications and issues. J Korean Med Sci. Nov 02, 2020;35(42):e379. [doi: 10.3346/jkms.2020.35.e379] [Medline: 33140591]

4. Farah L, Davaze-Schneider J, Martin T, Nguyen P, Borget I, Martelli N. Are current clinical studies on artificial intelligence-based medical devices comprehensive enough to support a full health technology assessment? A systematic review. Artif Intell Med. Jun 2023;140:102547. [doi: 10.1016/j.artmed.2023.102547] [Medline: 37210155]

5. HTA core model. European Network for Health Technology Assessment. URL: https://www.eunethta.eu/hta-core-model/ [accessed 2021-10-20]

6. Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK, SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. Sep 2020;26(9):1364-1374. [doi: 10.1038/s41591-020-1034-x] [Medline: 32908283]

7. Rivera SC, Liu X, Chan AW, Denniston AK, Calvert MJ, SPIRIT-AICONSORT-AI Working Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension. BMJ. Sep 09, 2020;370:m3210. [doi: 10.1136/bmj.m3210] [Medline: 32907797]

8. Collins GS, Dhiman P, Andaur Navarro CL, Ma J, Hooft L, Reitsma JB, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. Jul 09, 2021;11(7):e048008. [doi: 10.1136/bmjopen-2020-048008] [Medline: 34244270]

9. Chen Y. Health technology assessment and economic evaluation: is it applicable for the traditional medicine? Integr Med Res. Mar 2022;11(1):100756. [doi: 10.1016/j.imr.2021.100756] [Medline: 34401322]

10. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med. Jan 2019;25(1):30-36. [doi: 10.1038/s41591-018-0307-0] [Medline: 30617336]

11. Giordano C, Brennan M, Mohamed B, Rashidi P, Modave F, Tighe P. Accessing artificial intelligence for clinical decision-making. Front Digit Health. Jun 25, 2021;3:645232. [doi: 10.3389/fdghs.2021.645232] [Medline: 34713115]

12. Vayena E, Blasimme A, Cohen IG. Machine learning in medicine: addressing ethical challenges. PLoS Med. Nov 6, 2018;15(11):e1002689. [doi: 10.1371/journal.pmed.1002689] [Medline: 30399149]

13. Dzobo K, Adotey S, Thomford NE, Dzobo W. Integrating artificial and human intelligence: a partnership for responsible innovation in biomedical engineering and medicine. OMICS. May 2020;24(5):247-263. [doi: 10.1089/omi.2019.0038] [Medline: 31313972]

14. Zawati M, Lang M. What's in the box?: uncertain accountability of machine learning applications in healthcare. Am J Bioeth. Nov 2020;20(11):37-40. [doi: 10.1080/15265161.2020.1820105] [Medline: 33103971]

15. Peters MD, Godfrey C, McInerney P, Khalil H, Larsen P, Marnie C, et al. Best practice guidance and reporting items for the development of scoping review protocols. JBI Evid Synth. Apr 01, 2022;20(4):953-968. [doi: 10.11124/JBIES-21-00242] [Medline: 35102103]

16. Pollock D, Peters MD, Khalil H, McInerney P, Alexander L, Tricco AC, et al. Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evid Synth. Mar 01, 2023;21(3):520-532. [doi: 10.11124/JBIES-22-00123] [Medline: 36081365]
17. Farah L, Murris JM, Borget I, Guilloux A, Martelli NM, Katsahian SI. Assessment of performance, interpretability, and explainability in artificial intelligence-based health technologies: what healthcare stakeholders need to know. Mayo Clin Proc Digit Health. Jun 2023;1(2):120-138. [doi: 10.1016/j.mcpdig.2023.02.004]

18. Lehne M, Sass J, Essenwanger A, Schepers J, Thun S. Why digital medicine depends on interoperability. NPJ Digit Med. Aug 20, 2019;2:79. [doi: 10.1038/s41746-019-0158-1] [Medline: 31453374]

19. Esmaeilzadeh P. Use of AI-based tools for healthcare purposes: a survey study from consumers' perspectives. BMC Med Inform Decis Mak. Jul 22, 2020;20(1):170. [doi: 10.1186/s12911-020-01191-1] [Medline: 32698869]

20. Zia A, Aziz M, Popa I, Khan SA, Hamedani AF, Asif AR. Artificial intelligence-based medical data mining. J Pers Med. Aug 24, 2022;12(9):1359. [doi: 10.3390/jpm12091359] [Medline: 36143144]

21. Ming J, He Y, Yang Y, Hu M, Zhao X, Liu J, et al. Health technology assessment of medical devices: current landscape, challenges, and a way forward. Cost Eff Resour Alloc. Oct 05, 2022;20(1):54. [doi: 10.1186/s12962-022-00389-6] [Medline: 36199144]

22. Filkins BL, Kim JY, Roberts B, Armstrong W, Miller MA, Hultner ML, et al. Privacy and security in the era of digital health: what should translational researchers know and do about it? Am J Transl Res. Mar 15, 2016;8(3):1560-1580. [Medline: 27186282]

23. Johnson KB, Wei WQ, Weeraratne D, Frisse ME, Misulis K, Rhee K, et al. Precision medicine, AI, and the future of personalized health care. Clin Transl Sci. Jan 2021;14(1):86-93. [doi: 10.1111/cts.12884] [Medline: 32961010]

24. Sloane EJ, Silva RJ. Artificial intelligence in medical devices and clinical decision support systems. In: Clinical Engineering Handbook (Second Edition). Cambridge, MA. Academic Press; 2020.

25. Steuten LM. Early stage health technology assessment for precision biomarkers in oral health and systems medicine. OMICS. Jan 2016;20(1):30-35. [doi: 10.1089/omi.2015.0174] [Medline: 26760958]

26. Javaid M, Haleem A, Pratap Singh R, Suman R, Rab S. Significance of machine learning in healthcare: features, pillars and applications. Int J Intell Networks. 2022;3:58-73. [doi: 10.1016/j.ijin.2022.05.002]

27. Rowland SP, Fitzgerald JE, Holme T, Powell J, McGregor A. What is the clinical value of mHealth for patients? NPJ Digit Med. Jan 13, 2020;3:4. [doi: 10.1038/s41746-019-0206-x] [Medline: 31970289]

28. Voets MM, Veltman J, Slump CH, Siesling S, Koffijberg H. Systematic review of health economic evaluations focused on artificial intelligence in healthcare: the tortoise and the cheetah. Value Health. Mar 2022;25(3):340-349. [doi: 10.1016/j.jval.2021.11.1362] [Medline: 35227444]

29. Kiritsis A, Redekop WK. The economic evaluation of medical devices: challenges ahead. Appl Health Econ Health Policy. Feb 2013;11(1):15-26. [doi: 10.1007/s40258-012-0006-9] [Medline: 23329383]

30. Park SH, Choi J, Byeon JS. Key principles of clinical validation, device approval, and insurance coverage decisions of artificial intelligence. Korean J Radiol. Mar 2021;22(3):442-453. [doi: 10.3348/kjr.2021.0048] [Medline: 33629545]

31. Tsopra R, Fernandez X, Luchinat C, Alberghina L, Lehrach H, Vanoni M, et al. A framework for validating AI in precision medicine: considerations from the European ITFoC consortium. BMC Med Inform Decis Mak. Oct 02, 2021;21(1):274. [doi: 10.1186/s12911-021-01634-3] [Medline: 34600518]

32. Bolboacă SD. Medical diagnostic tests: a review of test anatomy, phases, and statistical treatment of data. Comput Math Methods Med. May 28, 2019;2019:1891569. [doi: 10.1155/2019/1891569] [Medline: 31275427]

33. Hogervorst MA, Pontén J, Vreman RA, Mantel-Teeuwisse AK, Goettsch WG. Real world data in health technology assessment of complex health technologies. Front Pharmacol. Feb 10, 2022;13:837302. [doi: 10.3389/fphar.2022.837302] [Medline: 35222045]

34. Simon GE, Bindman AB, Dreyer NA, Platt R, Watanabe JH, Horberg M, et al. When can we trust real-world data to evaluate new medical treatments? Clin Pharmacol Ther. Jan 2022;111(1):24-29. [doi: 10.1002/cpt.2252] [Medline: 33932030]

35. Pongiglione B, Torbica A, Blommestein H, de Groot S, Ciani O, Walker S, et al. Do existing real-world data sources generate suitable evidence for the HTA of medical devices in Europe? Mapping and critical appraisal. Int J Technol Assess Health Care. Apr 26, 2021;37(1):e62. [doi: 10.1017/S0266462321000301] [Medline: 33896433]

36. Pongiglione B, Torbica A. How real can we get in generating real world evidence? Exploring the opportunities of routinely collected administrative data for evaluation of medical devices. Health Econ. Sep 2022;31 Suppl 1(Suppl 1):25-43. [doi: 10.1002/hec.4562] [Medline: 35762465]

37. Zemplényi A, Tachkov K, Balkanyi L, Németh B, Petykó ZI, Petrova G, et al. Recommendations to overcome barriers to the use of artificial intelligence-driven evidence in health technology assessment. Front Public Health. Apr 26, 2023;11:1088121. [doi: 10.3389/fpubh.2023.1088121] [Medline: 37181704]

38. Daubner-Bendes R, Kovács S, Niewada M, Huic M, Drummond M, Ciani O, et al. Quo Vadis HTA for medical devices in central and eastern Europe? Recommendations to address methodological challenges. Front Public Health. Jan 08, 2021;8:612410. [doi: 10.3389/fpubh.2020.612410] [Medline: 33490024]

39. Larson DB, Harvey H, Rubin DL, Irani N, Tse JR, Langlotz CP. Regulatory frameworks for development and evaluation of artificial intelligence-based diagnostic imaging algorithms: summary and recommendations. J Am Coll Radiol. Mar 2021;18(3 Pt A):413-424. [doi: 10.1016/j.jacr.2020.09.060] [Medline: 33096088]

40. Choudhury A, Asan O. Role of artificial intelligence in patient safety outcomes: systematic literature review. JMIR Med Inform. Jul 24, 2020;8(7):e18599. [doi: 10.2196/18599] [Medline: 32706688]
41. Belenguer L. AI bias: exploring discriminatory algorithmic decision-making models and the application of possible machine-centric solutions adapted from the pharmaceutical industry. AI Ethics. 2022;2(4):771-787. [doi: 10.1007/s43681-022-00138-8] [Medline: 35194591]

42. Binder L, Ghadban M, Sit C, Barnard K. Health technology assessment process for oncology drugs: impact of CADTH changes on public payer reimbursement recommendations. Curr Oncol. Mar 01, 2022;29(3):1514-1526. [doi: 10.3390/curroncol29030127] [Medline: 35323327]

43. Wolff J, Pauling J, Keck A, Baumbach J. The economic impact of artificial intelligence in health care: systematic review. J Med Internet Res. Feb 20, 2020;22(2):e16866. [doi: 10.2196/16866] [Medline: 32130134]

44. Abràmoff MD, Roehrenbeck C, Trujillo S, Goldstein J, Graves AS, Repka MX, et al. A reimbursement framework for artificial intelligence in healthcare. NPJ Digit Med. Jun 09, 2022;5(1):72. [doi: 10.1038/s41746-022-00621-w] [Medline: 35681002]

45. Bélisle-Pipon JC, Couture V, Roy MC, Ganache I, Goetghebeur M, Cohen IG. What makes artificial intelligence exceptional in health technology assessment? Front Artif Intell. Nov 02, 2021;4:736697. [doi: 10.3389/frai.2021.736697] [Medline: 34796318]

46. Alami H, Lehoux P, Auclair Y, de Guise M, Gagnon MP, Shaw J, et al. Artificial intelligence and health technology assessment: anticipating a new level of complexity. J Med Internet Res. Jul 07, 2020;22(7):e17707. [doi: 10.2196/17707] [Medline: 32406850]

47. Love-Koh J, Peel A, Rejon-Parrilla JC, Ennis K, Lovett R, Manca A, et al. The future of precision medicine: potential impacts for health technology assessment. Pharmacoeconomics. Dec 2018;36(12):1439-1451. [doi: 10.1007/s40273-018-0686-6] [Medline: 30003435]

48. Gomez Rossi J, Rojas-Perilla N, Krois J, Schwendicke F. Cost-effectiveness of artificial intelligence as a decision-support system applied to the detection and grading of melanoma, dental caries, and diabetic retinopathy. JAMA Netw Open. Mar 01, 2022;5(3):e220269. [doi: 10.1001/jamanetworkopen.2022.0269] [Medline: 35289862]

49. Naik N, Hameed BM, Shetty DK, Swain D, Shah M, Paul R, et al. Legal and ethical consideration in artificial intelligence in healthcare: who takes responsibility? Front Surg. Mar 14, 2022;9:862322. [doi: 10.3389/fsurg.2022.862322] [Medline: 35360424]

50. Haynes CL, Cook GA, Jones MA. Legal and ethical considerations in processing patient-identifiable data without patient consent: lessons learnt from developing a disease register. J Med Ethics. May 2007;33(5):302-307. [doi: 10.1136/jme.2006.016907] [Medline: 17470509]

51. McCradden MD, Joshi S, Mazwi M, Anderson JA. Ethical limitations of algorithmic fairness solutions in health care machine learning. Lancet Digit Health. May 2020;2(5):e221-e223. [doi: 10.1016/S2589-7500(20)30065-0] [Medline: 33328054]

52. Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential biases in machine learning algorithms using electronic health record data. JAMA Intern Med. Nov 01, 2018;178(11):1544-1547. [doi: 10.1001/jamainternmed.2018.3763] [Medline: 30128552]

53. Fletcher RR, Nakeshimana A, Olubeko O. Addressing fairness, bias, and appropriate use of artificial intelligence and machine learning in global health. Front Artif Intell. Apr 15, 2021;3:561802. [doi: 10.3389/frai.2020.561802] [Medline: 33981989]

54. Tachkov K, Zemplenyi A, Kamusheva M, Dimitrova M, Siirtola P, Pontén J, et al. Barriers to use artificial intelligence methodologies in health technology assessment in central and east European countries. Front Public Health. Jul 14, 2022;10:921226. [doi: 10.3389/fpubh.2022.921226] [Medline: 35910914]

55. Durán JM, Jongsma KR. Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. J Med Ethics. Mar 18, 2021. (forthcoming). [doi: 10.1136/medethics-2020-106820] [Medline: 33737318]

56. Amann J, Blasimme A, Vayena E, Frey D, Madai VI, Precise4Q consortium. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. Nov 30, 2020;20(1):310. [doi: 10.1186/s12911-020-01332-6] [Medline: 33256715]

57. Kiseleva A, Kotzinos D, De Hert P. Transparency of AI in healthcare as a multilayered system of accountabilities: between legal requirements and technical limitations. Front Artif Intell. May 30, 2022;5:879603. [doi: 10.3389/frai.2022.879603] [Medline: 35707765]

58. Baltaxe E, Hsieh HW, Roca J, Cano I. The assessment of medical device software supporting health care services for chronic patients in a tertiary hospital: overarching study. J Med Internet Res. Jan 04, 2023;25:e40976. [doi: 10.2196/40976] [Medline: 36598817]

59. Garfield S, Polisena J, S Spinner D, Postulka A, Y Lu C, Tiwana SK, et al. Health technology assessment for molecular diagnostics: practices, challenges, and recommendations from the medical devices and diagnostics special interest group. Value Health. 2016;19(5):577-587. [doi: 10.1016/j.jval.2016.02.012] [Medline: 27565275]

60. Fraser AG, Biasin E, Bijnens B, Bruining N, Caiani EG, Cobbaert K, et al. Artificial intelligence in medical device software and high-risk medical devices - a review of definitions, expert recommendations and regulatory initiatives. Expert Rev Med Devices. Jun 2023;20(6):467-491. [doi: 10.1080/17434440.2023.2184685] [Medline: 37157833]
61. Beckers R, Kwade Z, Zanca F. The EU medical device regulation: implications for artificial intelligence-based medical device software in medical physics. Phys Med. Mar 2021;83:1-8. [doi: 10.1016/j.ejmp.2021.02.011] [Medline: 33657513]

62. Melvin T, Torre M. New medical device regulations: the regulator's view. EFORT Open Rev. Jun 03, 2019;4(6):351-356. [doi: 10.1302/2058-5241.4.180061] [Medline: 31312522]

63. Fleetcroft C, McCulloch P, Campbell B. IDEAL as a guide to designing clinical device studies consistent with the new European Medical Device Regulation. BMJ Surg Interv Health Technol. Mar 04, 2021;3(1):e000066. [doi: 10.1136/bmjsit-2020-000066] [Medline: 35047800]

64. Alami H, Lehoux P, Denis JL, Motulsky A, Petitgand C, Savoldelli M, et al. Organizational readiness for artificial intelligence in health care: insights for decision-making and practice. J Health Organ Manag. Dec 03, 2020. (forthcoming). [doi: 10.1108/JHOM-03-2020-0074] [Medline: 33258359]

65. Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo RM. Methodological frameworks and dimensions to be taken into consideration in digital health technology assessment: protocol for a scoping review. JMIR Res Protoc. Oct 11, 2022;11(10):e39905. [doi: 10.2196/39905] [Medline: 36222788]

66. Weinert L, Müller J, Svensson L, Heinze O. Perspective of information technology decision makers on factors influencing adoption and implementation of artificial intelligence technologies in 40 German hospitals: descriptive analysis. JMIR Med Inform. Jun 15, 2022;10(6):e34678. [doi: 10.2196/34678] [Medline: 35704378]

67. de Hond AA, Leeuwenberg AM, Hooft L, Kant IM, Nijman SW, van Os HJ, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. NPJ Digit Med. Jan 10, 2022;5(1):2. [doi: 10.1038/s41746-021-00549-7] [Medline: 35013569]

68. Ahuja AS. The impact of artificial intelligence in medicine on the future role of the physician. PeerJ. 2019;7:e7702. [doi: 10.7717/peerj.7702] [Medline: 31592346]

69. Widrig D, Tag B. HTA and its legal issues: a framework for identifying legal issues in health technology assessment. Int J Technol Assess Health Care. Dec 2014;30(6):587-594. [doi: 10.1017/S0266462314000683] [Medline: 25816824]

70. Vella Bonanno P, Bucsics A, Simoens S, Martin AP, Oortwijn W, Gulbinovič J, et al. Proposal for a regulation on health technology assessment in Europe - opinions of policy makers, payers and academics from the field of HTA. Expert Rev Pharmacoecon Outcomes Res. Jun 2019;19(3):251-261. [doi: 10.1080/14737167.2019.1575730] [Medline: 30696372]

71. McKee M, Wouters OJ. The challenges of regulating artificial intelligence in healthcare comment on "clinical decision support and new regulatory frameworks for medical devices: are we ready for it? - a viewpoint paper". Int J Health Policy Manag. 2023;12:7261. [doi: 10.34172/ijhpm.2022.7261] [Medline: 36243948]

72. Hordern V. Data protection compliance in the age of digital health. Eur J Health Law. Jun 2016;23(3):248-264. [doi: 10.1163/15718093-12341393] [Medline: 27491248]

73. Stanberry B. The legal and ethical aspects of telemedicine. 2: data protection, security and European law. J Telemed Telecare. 1998;4(1):18-24. [doi: 10.1258/1357633981931236] [Medline: 9640706]

74. Poullet Y. Legal aspects of data protection in medical informatics. Stud Health Technol Inform. 1991;1:138-160. [Medline: 10164065]

75. Dove ES, Chen J. To what extent does the EU general data protection regulation (GDPR) apply to citizen scientist-led health research with mobile devices? J Law Med Ethics. Mar 2020;48(1_suppl):187-195. [doi: 10.1177/1073110520917046] [Medline: 32342746]

76. Marovic B, Curcin V. Impact of the European general data protection regulation (GDPR) on health data management in a European Union candidate country: a case study of Serbia. JMIR Med Inform. Apr 17, 2020;8(4):e14604. [doi: 10.2196/14604] [Medline: 32301736]

77. EU Artificial Intelligence Act. URL: https://artificialintelligenceact.eu/ [accessed 2023-07-24]

78. Proposal for a regulation of the European parliament and of the council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts. European Union. 2021. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206 [accessed 2023-07-24]

79. Camara C, Peris-Lopez P, Tapiador JE. Security and privacy issues in implantable medical devices: a comprehensive survey. J Biomed Inform. Jun 2015;55:272-289. [doi: 10.1016/j.jbi.2015.04.007] [Medline: 25917056]

80. Migliore A, Ratti M, Cerbo M, Jefferson T. Health Technology Assessment: managing the introduction and use of medical devices in clinical practice in Italy. Expert Rev Med Devices. May 2009;6(3):251-257. [doi: 10.1586/erd.09.2] [Medline: 19419283]

81. Pisapia A, Banfi G, Tomaiuolo R. The novelties of the regulation on health technology assessment, a key achievement for the European Union health policies. Clin Chem Lab Med. Jul 26, 2022;60(8):1160-1163. [doi: 10.1515/cclm-2022-0228] [Medline: 35624072]

82. Maliha G, Gerke S, Cohen IG, Parikh RB. Artificial intelligence and liability in medicine: balancing safety and innovation. Milbank Q. Sep 2021;99(3):629-647. [doi: 10.1111/1468-0009.12504] [Medline: 33822422]

83. Jassar S, Adams SJ, Zarzeczny A, Burbridge BE. The future of artificial intelligence in medicine: medical-legal considerations for health leaders. Healthc Manage Forum. May 2022;35(3):185-189. [doi: 10.1177/08404704221082069] [Medline: 35354409]
84. Samore MH, Evans RS, Lassen A, Gould P, Lloyd J, Gardner RM, et al. Surveillance of medical device-related hazards and adverse events in hospitalized patients. JAMA. Jan 21, 2004;291(3):325-334. [doi: 10.1001/jama.291.3.325] [Medline: 14734595]

85. Bleher H, Braun M. Diffused responsibility: attributions of responsibility in the use of AI-driven clinical decision support systems. AI Ethics. 2022;2(4):747-761. [doi: 10.1007/s43681-022-00135-x] [Medline: 35098247]

86. Street J, Stafinski T, Lopes E, Menon D. Defining the role of the public in Health Technology Assessment (HTA) and HTA-informed decision-making processes. Int J Technol Assess Health Care. Apr 2020;36(2):87-95. [doi: 10.1017/S0266462320000094] [Medline: 32151290]

87. Massella M, Dri DA, Gramaglia D. Regulatory considerations on the use of machine learning based tools in clinical trials. Health Technol (Berl). 2022;12(6):1085-1096. [doi: 10.1007/s12553-022-00708-0] [Medline: 36373014]

88. Marcus HJ, Payne CJ, Hughes-Hallett A, Marcus AP, Yang GZ, Darzi A, et al. Regulatory approval of new medical devices: cross sectional study. BMJ. May 20, 2016;353:i2587. [doi: 10.1136/bmj.i2587] [Medline: 27207165]

89. Milam ME, Koo CW. The current status and future of FDA-approved artificial intelligence tools in chest radiology in the United States. Clin Radiol. Feb 2023;78(2):115-122. [doi: 10.1016/j.crad.2022.08.135] [Medline: 36180271]

90. Vreman RA, Mantel-Teeuwisse AK, Hövels AM, Leufkens HG, Goettsch WG. Differences in health technology assessment recommendations among European jurisdictions: the role of practice variations. Value Health. Jan 2020;23(1):10-16. [doi: 10.1016/j.jval.2019.07.017] [Medline: 31952664]

91. Allen N, Walker SR, Liberti L, Salek S. Health technology assessment (HTA) case studies: factors influencing divergent HTA reimbursement recommendations in Australia, Canada, England, and Scotland. Value Health. Mar 2017;20(3):320-328. [doi: 10.1016/j.jval.2016.10.014] [Medline: 28292476]

92. Agboola F, Wright AC, Herron-Smith S, Mathur D, Rind D. Evaluation of diversity of clinical trials informing health technology assessments in the United States: a 5-year analysis of institute for clinical and economic review assessments. Value Health. Sep 2023;26(9):1345-1352. [doi: 10.1016/j.jval.2023.05.014] [Medline: 37244417]

93. Hendrix N, Veenstra DL, Cheng M, Anderson NC, Verguet S. Assessing the economic value of clinical artificial intelligence: challenges and opportunities. Value Health. Mar 2022;25(3):331-339. [doi: 10.1016/j.jval.2021.08.015] [Medline: 35227443]

94. Nielsen CP, Lauritsen SW, Kristensen FB, Bistrup ML, Cecchetti A, Turk E. Involving stakeholders and developing a policy for stakeholder involvement in the European network for health technology assessment, EUnetHTA. Int J Technol Assess Health Care. Dec 2009;25 Suppl 2:84-91. [doi: 10.1017/S0266462309990729] [Medline: 20030895]

95. Scott AM, Wale JL, HTAi PatientCitizen Involvement in HTA Interest Group, Patient InvolvementEducation Working Group. Patient advocate perspectives on involvement in HTA: an international snapshot. Res Involv Engagem. Jan 10, 2017;3:2. [doi: 10.1186/s40900-016-0052-9] [Medline: 29062527]

96. Gagnon MP, Desmartis M, Gagnon J, St-Pierre M, Gauvin FP, Rhainds M, et al. Introducing the patient's perspective in hospital health technology assessment (HTA): the views of HTA producers, hospital managers and patients. Health Expect. Dec 2014;17(6):888-900. [doi: 10.1111/hex.12010] [Medline: 23046439]

97. Gagnon M, Desmartis M, Gagnon J, St-Pierre M, Rhainds M, Coulombe M, et al. Framework for user involvement in health technology assessment at the local level: views of health managers, user representatives, and clinicians. Int J Technol Assess Health Care. Jan 2015;31(1-2):68-77. [doi: 10.1017/S0266462315000070] [Medline: 25952585]

98. Angelis A, Kanavos P. Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment. Pharmacoeconomics. May 2016;34(5):435-446. [doi: 10.1007/s40273-015-0370-z] [Medline: 26739955]

99. Wale JL, Thomas S, Hamerlijnck D, Hollander R. Patients and public are important stakeholders in health technology assessment but the level of involvement is low - a call to action. Res Involv Engagem. Jan 05, 2021;7(1):1. [doi: 10.1186/s40900-020-00248-9] [Medline: 33402216]

100. Artificial intelligence and machine learning (AI/ML)-enabled medical devices. U.S. Food & Drug Administration. Oct 19, 2023. URL: https://tinyurl.com/m8ba5aw4 [accessed 2023-11-25]

101. Cassidy B, Hoon Yap M, Pappachan JM, Ahmad N, Haycocks S, O'Shea C, et al. Artificial intelligence for automated detection of diabetic foot ulcers: a real-world proof-of-concept clinical evaluation. Diabetes Res Clin Pract. Nov 2023;205:110951. [doi: 10.1016/j.diabres.2023.110951] [Medline: 37848163]

102. Mackenzie SC, Sainsbury CA, Wake DJ. Diabetes and artificial intelligence beyond the closed loop: a review of the landscape, promise and challenges. Diabetologia. Feb 2024;67(2):223-235. [doi: 10.1007/s00125-023-06038-8] [Medline: 37979006]

103. Sridhar GR, Lakshmi G. Ethical issues of artificial intelligence in diabetes mellitus. Med Res Arch. Aug 2023;11(8):1-8. [doi: 10.18103/mra.v11i8.4287]

104. Zhang L, Zhang L. Artificial intelligence for remote sensing data analysis: a review of challenges and opportunities. IEEE Geosci Remote Sens Mag. Apr 13, 2022;10(2):270-294. [FREE Full text] [doi: 10.1109/MGRS.2022.3145854]

105. Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias and clinical safety. BMJ Qual Saf. Mar 2019;28(3):231-237. [doi: 10.1136/bmjqs-2018-008370] [Medline: 30636200]
106. Beil M, Proft I, van Heerden D, Sviri S, van Heerden PV. Ethical considerations about artificial intelligence for prognostication in intensive care. Intensive Care Med Exp. Dec 10, 2019;7(1):70. [doi: 10.1186/s40635-019-0286-6] [Medline: 31823128]

107. Bærøe K, Miyata-Sturm A, Henden E. How to achieve trustworthy artificial intelligence for health. Bull World Health Organ. Apr 01, 2020;98(4):257-262. [doi: 10.2471/BLT.19.237289] [Medline: 32284649]

108. Julian E, Gianfrate F, Sola-Morales O, Mol P, Bergmann JF, Salmonson T, et al. How can a joint European health technology assessment provide an 'additional benefit' over the current standard of national assessments? : insights generated from a multi-stakeholder survey in hematology/oncology. Health Econ Rev. Jun 02, 2022;12(1):30. [doi: 10.1186/s13561-022-00379-7] [Medline: 35652987]

Abbreviations

AI: artificial intelligence

FDA: Food and Drug Administration

HTA: health technology assessment

PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RWE: real-world evidence

Edited by Y Zhuang; submitted 02.08.23; peer-reviewed by A de Hond, E Vashishtha, I Cano, N Hendrix; comments to author 28.10.23; revised version received 17.12.23; accepted 28.12.23; published 13.05.24

Please cite as:

Farah L, Borget I, Martelli N, Vallee A

Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature

Review

J Med Internet Res 2024;26:e51514

URL: https://www.jmir.org/2024/1/e51514

doi: 10.2196/51514

PMID:

©Line Farah, Isabelle Borget, Nicolas Martelli, Alexandre Vallee. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.05.2024. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.