# Systematic literature review on implicit factors influencing the HTA deliberative processes in Europe

Clara Monleón, Hans-Martin Späth, Carlos Crespo, Claude Dussart, Mondher Toumi

► To cite this version:

Clara Monleón, Hans-Martin Späth, Carlos Crespo, Claude Dussart, Mondher Toumi. Systematic literature review on implicit factors influencing the HTA deliberative processes in Europe. Journal of Market Access & Health Policy, 2022, 10 (1), 10.1080/20016689.2022.2094047. [hal-04754324](https://hal.science/hal-04754324v1)

HAL Id: hal-04754324

https://hal.science/hal-04754324v1

Submitted on 25 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Systematic literature review on the implicit factors influencing the HTA deliberative process in Europe

Clara Monleón, Hans-Martin Späth, Carlos Crespo, Claude Dussart & Mondher Toumi

To cite this article: Clara Monleón, Hans-Martin Späth, Carlos Crespo, Claude Dussart & Mondher Toumi (2022) Systematic literature review on the implicit factors influencing the HTA deliberative process in Europe, Journal of Market Access & Health Policy, 10:1, 2094047, DOI: 10.1080/20016689.2022.2094047

To link to this article: https://doi.org/10.1080/20016689.2022.2094047

<figure><img src="image_2.png" /></figure>

© 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.

<figure><img src="image_3.png" /></figure>

Published online: 28 Jun 2022.

<figure><img src="image_4.png" /></figure>

Submit your article to this journal

<figure><img src="image_5.png" /></figure>

View related articles

<figure><img src="image_6.png" /></figure>

View Crossmark data

CrossMark
REVIEW ARTICLE

<figure><img src="image_7.png" /></figure>

OPEN ACCESS

<figure><img src="image_8.png" /></figure>

Check for updates

# Systematic literature review on the implicit factors influencing the HTA deliberative process in Europe

Clara Monleón IDa, Hans-Martin Späth IDa, Carlos Crespo IDb, Claude Dussart Id, and Mondher Toumi Idc

## ABSTRACT

**Objectives:** Deliberative processes in Health Technologies Assessment (HTA) result in recommendations that determine the reimbursement of medicines, diagnostics or devices. These processes are governed by explicit criteria, but are also influenced by implicit factors. The objective of this work was to identify the implicit factors influencing HTA deliberative processes in five European countries (France, Germany, Italy, Spain and the UK).

**Methods:** A systematic review of literature published between 2009 and 2019 was conducted. The search was performed in Pubmed, The Cochrane Database of Systematic Reviews, Google Scholar and Center for Reviews and Dissemination. The ISPOR database was searched manually.

**Results:** Out of 100 eligible publications, eight articles were selected for data extraction and analysis. The implicit factors in the HTA deliberative process most frequently mentioned in the identified literature are value judgments, biases, preferences and subjectivity. Five out of the eight articles highlight the need to further improve the transparency of the process, and three provide recommendations on how to address the influence of implicit factors on the HTA deliberative process through a framework.

**Conclusion:** Even in countries with a long HTA history, evidence on implicit factors is scarce. Some methods have been recommended for addressing these factors. Further research is required to characterize the implicit factors in the HTA deliberative process at a country level and explore potential ways to mitigate the influence of these factors on the HTA deliberative process.

## ARTICLE HISTORY

Received 02 December 2021
Revised 13 June 2022
Accepted 20 June 2022

## KEYWORDS

Health technology
assessment; deliberative
process; decision making;
healthcare systems

## Introduction

In the last decades, substantial efforts have been made to improve decision-making and promote equitable, efficient, and high-quality healthcare systems. As part of such efforts, the process of Health Technology Assessment (HTA) has arisen as one that uses explicit methods and criteria to determine the value of a health technology at different points in its lifecycle to inform decision-making and promote an equitable and high-quality system [1].

With global healthcare expenditure rising from 478.81 US $ per capita in 2000 to 1,015.87 US$ per capita in 2016 [2] the HTA process is becoming increasingly relevant as a tool for assessing the value of new drug therapies in the healthcare system and subsequent decisions on coverage and reimbursement [3].

In an era where healthcare systems strive to deliver innovation and healthcare coverage, the HTA process

has come to play a critical role in deliberative decision-making processes, as a means of demonstrating added value of new health interventions (i.e., drugs, medical devices, surgical procedures [4]) beyond efficacy and safety [5] to inform reimbursement decisions [6].

One of the key strategic objectives of the European Network for HTA (EUnetHTA) has been to promote more effective use of resources and increase HTA input into decision-making processes in Europe [7]. The EUnetHTA Core model comprises nine dimensions of value, with safety, efficacy and cost-effectiveness being the most commonly assessed factors in Europe [5] and country-specific guidelines have been described elsewhere [8–13].

Beyond the evidence supporting the technology under review, HTA recommendations may be influenced by implicit factors inherent in the decision-making itself [14]. There is evidence showing that discrepancies may occur between

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<footnote>ªEA 4129 (Parcours Santé Systémique) Département de Santé Publique, Claude Bernard University, University of Lyon, Lyon, France;</footnote>

<footnote>ᵇStatistics Department, University of Barcelona, Barcelona Spain; ᵈPublic Health Department, Aix Marseille University, France</footnote>
<figure><img src="image_11.png" /></figure>

recommendations by HTA advisory bodies and their related reimbursement decisions, which could potentially compromise the transparency and fairness of healthcare resource allocation [15]. Such discrepancies may to some extent be explained by variations in decision criteria, and result from contextual or implicit factors [16].

For the purpose of this systematic literature review (SLR), implicit factors can be any factors not explicitly stated in the publication of respective HTA body's guidelines.

The geographic scope of this research covers Germany, France, UK and Spain. These five countries represent 73% of the European Union population in 2020 and according to Eurostat the health care expenditure of all providers in these countries represented 72% of the EU-28 in 2019 [17]; they all have established HTA agencies and recognised HTA processes [18,19].

According to the HTAi Global Policy Forum 2019 [15] the term deliberation refers to critical assessment and discussion of an issue, and involves the weighing of arguments for and against a measure. Deliberation follows the assessment phase during which evidence is gathered and synthesised, and focuses on the integration of this information through an appraisal process, after which recommendations are made [15,16].

The objective of this systematic literature review (SLR) was to identify and categorise implicit factors involved in the HTA deliberative process of medicines in Germany, France, Italy, UK and Spain.

# Materials and methods

## Eligibility criteria

A systematic review was conducted using the reporting items for systematic reviews and meta-analyses (PRISMA) guidelines (20,21). Two reviewers (CM and CC) screened the citations (titles, abstracts, and full text from eligible articles), and performed data extraction and analysis. Each methodological step was performed independently by both reviewers. Each reviewer's results were compared and discussed, and in case of divergence, a consensus was reached. Where consensus could not be reached, a third reviewer (MT) performed the assessment to reach a final decision.

## Information sources and search strategy

The literature review was performed in the following selected databases: PubMed, Google Scholar, The Cochrane Database of Systematic Reviews (CDSR), and Center for Reviews and Dissemination (CRD).

ISPOR presentations database was searched manually. The reference sections of all identified articles were screened for additional articles that may be relevant for this SLR.

Searches were limited to the period 2009–2019, and to the following languages in the above databases: English, French, German, Italian or Spanish.

The search strategy is detailed in the Supplementary file.

Results were imported into an Excel file, and duplicated articles were removed.

In addition, grey literature searches were conducted using Google to identify documents not indexed in scientific journals, i.e., reports or textbooks, which were added to the bank of references as applicable.

## Study selection

Citations were screened following a stepwise approach. In the first step, all titles and abstracts identified in the search strategy were independently reviewed by two reviewers against pre-defined eligibility criteria from the study protocol. Studies were categorised as 'included', 'unsure' or 'excluded'.

The second step included a full-text review of all studies categorised as 'included' or 'unsure' during the first phase. The full-text review was continued until all articles had been categorised as either 'included' or 'excluded'.

## Citations' and full-text screening by eligibility criteria

### Inclusion criteria

* Articles containing information regarding implicit factors influencing HTA deliberative processes on medicines.

* Articles containing information about any of the five target countries together with information about other countries, were eligible for inclusion. However, only information pertaining to the target countries was synthesised.

### Exclusion criteria

* Languages other than English, French, German, Italian and Spanish

* Year of publication: articles published before 2009 and after 2019

* Articles describing HTA decision-making processes related to medical devices, diagnostic tests or medical procedures, specific therapeutic areas, and regional or local HTA decision-making processes.
<figure><img src="image_12.png" /></figure>

* Articles describing shared decision making between patients and healthcare providers

## Data extraction and analysis

Data extraction from eligible articles was performed in two steps. The first step was to perform an extraction of implicit factors affecting HTA assessments from each eligible article. This was followed by analysis of the data collected from the first extraction, and a second extraction was performed using a method known as the general inductive method [19]. This method allows the development of general conclusions based on particular facts. Thus, the collected information was classified into categories of implicit factors.

## Results

A total of 1,034 references were obtained from the literature search strategy (PubMed: 961; Google Scholar: 45; Cochrane: 5 and CRD: 10) and from the ISPOR Presentations database manual search [13]. Removal of 459 duplicates resulted in 575 references, 100 of which were selected for full text assessment (PubMed: 98; CRD: 2). From this selection, 92 articles were excluded for the following reasons: lack of relevant information on the decision-making process (n = 76); methodology design for HTA (n = 12); HTA decision-making in excluded countries (n = 4). Thus, eight articles were selected for data extraction and analysis (Figure 1).

A description of the eight articles with the main findings is listed in Tables 1,2. Among the eight articles found in the literature, there were five articles that recognise and explain the occurrence of implicit factors in the HTA deliberative process, highlight the need to further improve the deliberative process, and outline a conceptual approach to address the problem [6,14,22-24] (Table 1). In addition, there were three articles that recognise and elaborate on these implicit factors, and provide recommendations on how to address implicit factors and improve the transparency of the HTA decision-making through a framework [25-27] (Table 2).

The HTA deliberative process is influenced by a number of implicit factors related to the behaviour and personal values of the individuals involved as well as to the context in which this process is performed. These factors have been grouped under categories based on the frequency they have been mentioned in the literature. The categories are ethics,

psychology, qualification and experience, politics and society, culture, functional role and disease perception (Table 3).

The implicit factors under 'Ethics' are the most frequently mentioned across the identified articles [6,14,22-24,26,27] (143 times), and this category includes: value judgments, biases, intrinsic values, values of knowledge, ethical issues, ethical implications, moral values, equity, equality, fairness, disparities, vested interests, interests and implicit assumed desire. Value judgments are the most referenced under this category and are constitutive of HTA decision-making. However they may differ from appraisal to appraisal, and they may be considered differently by the stakeholders involved in the appraisal committees since values, principles and considerations differ from person to person. Many of these value judgments are implicit and the objectivity is compromised making the HTA decision-making process biased [14].

Psychology may entail all factors related to the personality and subjective factors such as "preferences, "personal values, personal considerations 'personality', 'opinion', 'gut feeling', 'overconfidence on own judgment' [22,25].

Qualification and experience refer to the qualification and previous experience in similar situations of the members providing the recommendations [14,25].

Political influences and societal values or perceptions may also impact the HTA process. This category can include 'political processes', 'political influences', 'political pressures', 'social values' and 'social perceptions' [22,25-27].

The cultural values of individual HTA members or the organisational culture are also factors that may impact the recommendations. They are referred in the literature as 'cultural values' and 'organizational culture' [22,24].

As per the functional role, this is related to the roles and responsibilities of the HTA members on the committee such as 'individual responsibility' and 'power of status' [14,22].

Finally, the disease perception such as the burden of disease and unmet need may differ from person to person and may thus interfere with the recommendations on the technology [14,22].

Regarding the impact these implicit factors may have on the HTA deliberative process, it is argued that factors that fall into the categories of ethics (value judgments, bias and fairness) functional role (individual responsibility),and disease perception (disease severity) may have consequences for resource allocation. As an example, personal biases of HTA members may result in
<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /><figcaption>Figure 1. PRISMA flow diagram.</figcaption></figure>

individual interests being favoured over public interests, which has consequences for resource allocation compromising the fairness and legitimacy of these decisions [27] (Table 3).

If deliberative processes are informal, unstructured, and unstandardised, these key issues may be overlooked, making it difficult to address the personal preferences or subjectivity of decision-makers [6,22,24].

As part of the recommendations, there are three articles that propose a framework for addressing the implicit factors, and five articles that provide recommendations without a framework.

Those proposing a framework and guidelines on how to formalise the decision-making process call for a more integrative perspective on HTA, aligning it with evidence-informed deliberative and legitimate processes. Frameworks for improved HTA conduct

and processes have been developed aiming for an explicit, transparent, and inclusive process [25–27].

The ‘evidence-informed deliberative processes’ is proposed as a hybrid framework that addresses social values and ethical issues with the goal to enhance legitimacy. This framework entails multicriteria decision analysis and accountability for reasonableness and supports the HTA agencies in their efforts to organise their processes, leveraging the key elements of the framework which are stakeholder involvement in the assessment and deliberation phases and the implications for the HTA process [27].

Another proposed framework captures the integration of elements present in the decision-making which emerged from research into the four key domains of structure, bias, culture, and impact [25].

Following the recommendations, it is suggested that HTA agencies subject their decision-making criteria to
<figure><img src="image_15.png" /><figcaption>Table 1. Articles proposing a concept as a way to address the research question.</figcaption></figure>

<table><thead><tr><th>AUTHORS</th><th>OBJECTIVE</th><th>ISSUES FOUND IN THE HTA DECISION-MAKING PROCESS</th><th>RECOMMENDATIONS</th></tr></thead><tbody><tr><td>Bujar et al. 2019</td><td>To identify and investigate decision-making framework utilisation within 24 pharmaceutical al companies and 16 HTA agencies and assess the use of different methodologies for the quality of decision-making process. The study also investigated the different cognitive biases that occur in decision-making</td><td>Unstructured process<br>Presence of biases<br>Decision-making is influenced by biases and emotional judgements.<br>Need to improve the quality of the decision-making process by making it more formal and predictable<br>Divergence in HTA and reimbursement decision-making<br>The majority of organisations do not measure the decision-making even if there is the belief that this assessment is key</td><td>There is a need of having more consistent, structured and predictable decision-making processes during the life cycle of medicines. This could be achieved through systematically evaluating the quality of decision making and encouraging utilisation of an international formal decision-making framework within companies and agencies.</td></tr><tr><td>Fischer et al. 2013</td><td>To analyse influences of transparency, stakeholder participation and the scientific assessment on decision outcomes in the deliberative process.</td><td>Impact of transparency on the reasonableness of decisions is contested</td><td>Decision outcomes of coverage decisions appeared to be influenced mostly by the use of evidence rather than the degree of transparency or participation. The inclusiveness of the HTA process (the number of stakeholders involved) increases the likelihood of a positive reimbursement decision</td></tr><tr><td>Phelps et al. 2018</td><td>Identify and analyse the value judgements present in the HTA process</td><td>Deliberative processes today are often informal and unstructured and often lack transparency. It is unclear what factors have been considered and how the decisions have been reached.</td><td>MCDA models may provide the best opportunity for improvement, but they have not yet been perfected. Greater testing and use of MCDA models are recommended</td></tr><tr><td>Kristensen et al. 2019</td><td>To synthesize HTA good practices to support decision-making to create a basis for education and improved consistency in approaches to HTA informed decision making.</td><td>Lack of research on structured approach</td><td>Need to focus on developing good practices in using evidence to support decision-making through monitoring of HTA implementation and its inputs to various types of decision-making</td></tr><tr><td>Hofmann et al. 2014</td><td>Review of existing methods solving aggregation issues through a health economics approach.</td><td>Value judgments are not explicit in the HTA appraisals and decision-making processes. Acknowledging and explicitly addressing value judgments may improve the accountability of HTA.</td><td>Making explicit the value judgments implicit in HTA as well as in the appraisal phase can promote the robustness of the decision-making. This can be supported by guidelines on the HTA appraisal and decision-making process. More research on identifying and onhow to highlight and integrate value judgments in HTA is needed.</td></tr></tbody></table>

public scrutiny and communicate the criteria behind their reimbursement recommendations. Additionally, to make the process more democratic, the importance of HTA agencies having permanent lay members on their appraisal committees, as representatives of the public interest, is highlighted [27].

When it comes to the other group of articles that do not propose a framework, the focus of their recommendations is on assessing and understanding the process and quality of HTA decision-making, as a mean of providing insights into the factors that influence the recommendations and decisions derived from it. Interventions like increasing the degree of participation from stakeholders or getting formal feedback from internal and external stakeholders regarding the decision making could help to improve the quality of the decision-making process itself, and ultimately transparency and consistency in key decisions [14,23,25]. Another approach to formalising the HTA

decision-making process and making it more transparent is the Multicriteria Decision Analysis (MCDA) method. This method expands upon implicit factors not captured in the cost-effectiveness analysis such as fairness, equity or disparities [27]. Making explicit the implicit factors in the HTA process could contribute to improve transparency and legitimacy of the decision-making process [22].

## Discussion

Like other deliberative decision-making processes, HTA deliberative process is a complex process that is affected by factors that are generally recognised and agreed upon (such as safety and cost-effectiveness), as well as other factors that are not explicitly stated and vary between countries, systems and individuals. The latter factors account for variability in the decision-making criteria [7,27].
<figure><img src="image_16.png" /><figcaption>Table 2. Articles discussing or proposing a framework as a way to address the research question.</figcaption></figure>

<table><thead><tr><th>AUTHORS</th><th>OBJECTIVE</th><th>ISSUES FOUND IN THE DECISION-MAKING PROCESS</th><th>FRAMEWORK</th><th>RECOMMENDATIONS</th></tr></thead><tbody><tr><td>Donelan<br>et al.<br>2015</td><td>To investigate and identify the issues that Influence Quality decision- Making through semi-structured interviews to decision-makers</td><td>Clear evidence of the presence of personal biases of decision-makers, other factors impacting the subjective decision-making approach of individuals, the culture and behaviour of the organisations in decision-making.</td><td>Integrated approach to quality decision-making capturing four stepwise domains: the structure, bias, impact and culture. The next phase following the design of a 'framework for decision-making' would be the development of a quantitative tool its psychometric robustness.</td><td>Quality decision-making needs a 'structured approach', education and training in decision-making techniques, evaluating the importance of the options an appreciation of the impact of the decision made. An integrated approach to quality decision-making requires a systematic framework that considers four essential domains, (structure, bias, culture and impact)<br>-Broad and multidisciplinary stakeholder involvement will be necessary throughout the process.<br>-The HTA should include standardised methods to identify and appraise evidence for clinically important moderators or predictors of treatment effects.<br>-It is important to monitor and review the processes and results to assess their efficiency, consistency, and sustainability over time.<br>-Transparency and robustness of the process will increase accountability and predictability</td></tr><tr><td>Oortwijn<br>et al.<br>2017</td><td>To assess the level of comprehensiveness of HTA practices around the globe and formulate recommendations to enhance legitimacy and fairness of the decision-making process</td><td>Assessment and appraisal processes does not seem aligned in some systems. Regarding the appraisal phase, the monitoring and assessment of the process is not well established across HTA systems.</td><td>INTEGRATE-HTA model and the Accountability for Reasonableness framework</td><td>-HTA agencies should take responsibility of organising stakeholder involvement.<br>-Agencies are advised to integrate their assessment and appraisal phases<br>-HTA agencies should subject their decision-making criteria to public scrutiny.<br>-HTA agencies are advised to use a checklist of potentially relevant criteria and to provide argumentation for how each criterion affected the recommendation.<br>-HTA agencies must publish their argumentation and install options for appeal.</td></tr><tr><td>Baltussen<br>R. et al<br>2017</td><td>Describe the key elements of the framework Presents the implications for the conduct of HTA</td><td>Presence of 'vested interests' in the formal HTA processes</td><td>Adopting evidence-informed deliberative processes as a value assessment framework could be an important step forward for HTA agencies in optimising the legitimacy of their priority- setting decisions.</td><td></td></tr></tbody></table>
<figure><img src="image_17.png" /><figcaption>Table 3. Implicit factors identified in the literature: frequency of mentions and impact in the HTA deliberative process.</figcaption></figure>

<table><thead><tr><th>CATEGORY</th><th>IMPLICIT FACTORS</th><th>FREQUENCY OF MENTIONS</th><th>IMPACT ON THE HTA DECISION-MAKING</th></tr></thead><tbody><tr><td rowspan="10">ETHICS</td><td>Value judgments [6,14]</td><td>76</td><td rowspan="10">Fairness of resources allocation to health technologies may be compromised. May capture particular interests to the detriment of public interests, comoprimising equal access to good quality healthcare</td></tr><tr><td>Bias [24]</td><td>34</td></tr><tr><td>Equity [14]</td><td>12</td></tr><tr><td>Equality [14]</td><td>4</td></tr><tr><td>Intrinsic values [14]</td><td>2</td></tr><tr><td>Moral values [14]</td><td>2</td></tr><tr><td>Interests [23]</td><td>2</td></tr><tr><td>Implicit assumed desire [14]</td><td>2</td></tr><tr><td>Vested interests [27]</td><td>2</td></tr><tr><td>Values of knowledge [14]</td><td>1</td></tr><tr><td>Ethical issues [6]</td><td>1</td></tr><tr><td>Ethical implications [6]</td><td>1</td></tr><tr><td>Fairness [14]</td><td>1</td></tr><tr><td rowspan="10">PSYCHOLOGY</td><td>Preferences [14]</td><td>16</td><td rowspan="10">Not mentioned</td></tr><tr><td>Subjective [25]</td><td>13</td></tr><tr><td>Training [25]</td><td>9</td></tr><tr><td>Gut feeling [25]</td><td>6</td></tr><tr><td>Opinion [25]</td><td>4</td></tr><tr><td>Personal considerations [25]</td><td>2</td></tr><tr><td>Personal values [22]</td><td>1</td></tr><tr><td>Personality [22]</td><td>1</td></tr><tr><td>Overconfidence in own judgment [25]</td><td>1</td></tr><tr><td rowspan="10">QUALIFICATION AND EXPERIENCE</td><td>Qualification [25]</td><td>5</td><td rowspan="10">Not mentioned</td></tr><tr><td>Precedents for similar previous decisions [25]</td><td>1</td></tr><tr><td>Previous decision-making mistakes [25]</td><td>1</td></tr><tr><td>Experience in previous decision-making [25]</td><td>1</td></tr><tr><td>Experience [14]</td><td>1</td></tr><tr><td>Precedents for similar previous decisions [25]</td><td>1</td></tr><tr><td>Experience in previous decision-making [25]</td><td>1</td></tr><tr><td>Experience [14]</td><td>1</td></tr><tr><td>Precedents for similar previous decisions [25]</td><td>1</td></tr><tr><td>Experience in previous decision-making [25]</td><td>1</td></tr><tr><td rowspan="6">POLITICS & SOCIETY</td><td>Social values [22,27]</td><td>4</td><td rowspan="6">Not mentioned</td></tr><tr><td>Political processes of the country [35]</td><td>1</td></tr><tr><td>Political influences [25]</td><td>1</td></tr><tr><td>Political pressures [24]</td><td>1</td></tr><tr><td>Societal perceptions [27]</td><td>1</td></tr><tr><td>Cultural values [22]</td><td>1</td></tr><tr><td rowspan="2">CULTURE</td><td>Organizational culture [24]</td><td>1</td><td rowspan="2">Not mentioned</td></tr><tr><td>Individual responsibility [14]</td><td>1</td></tr><tr><td rowspan="2">FUNCTIONAL ROLE</td><td>Power of status [22]</td><td>1</td><td rowspan="2">May impact the healthcare resource allocation</td></tr><tr><td>Disease severity [22]</td><td>1</td></tr><tr><td>DISEASE PERCEPTION</td><td>Burden of disease [14]</td><td>1</td><td></td></tr></tbody></table>

This systematic literature review found that implicit factors, being constitutive of the HTA deliberative process, may interfere with the HTA recommendations. It also found a lack of a standardised framework for addressing these factors across the studied countries. For this reason, there is a need to unmask and clarify these factors to increase the transparency, fairness, impartiality and formality of the process [23,26,28].

Some HTA agencies have used frameworks for specific deliberative processes. However, these frameworks do not account for subjective elements such as behavioural and cognitive aspects that may influence the final decisions [29].

Inconsistencies, variability and lack of predictability have been reported in current HTA value frameworks; to address these challenges, methodologies such as MCDA for healthcare decision making have been frequently debated over the last years [30]. This method seems appropriate as a concept for integrating diverse attributes as part of an explicit approach [28]. However, due to practical limitations with respect to its implementation, it is mainly used for experimental or academic purposes [31].

EUnetHA is an example of an HTA framework that is used to assess and appraise new technologies. EUnetHTA proposed its HTA Core Model as a way to harmonise the HTA assessment process across countries
<figure><img src="image_18.png" /></figure>

[32–34]. However, the HTA core model does not address the implicit factors, except for marginally alluding to them in a section called ‘value judgements’. Nor does this framework assign a dimension to these value judgments and their influence, to allow their impact on the deliberative and HTA processes to be monitored and quantified [34].

Other initiatives that have been developed to guide the HTA deliberative process were highlighted in the Background Paper from the HTAi Global Policy Forum 2020, which proposed a range of questions, principles and actions that could be used to guide the development of deliberative processes within HTA. This includes guidance on the selection of committee members and allocation of roles, as well as recommendations regarding stakeholder involvement and the cognitive aspects present in the process of deliberation. The HTAi Global Policy Forum also recommends using decision-making frameworks to make the process more structured, explicit and formal and minimise the influence of individual stakeholder interests on the decision-making process to safeguard its fairness [15].

Other elements that could be integrated into formal HTA assessments or frameworks include broader dimensions of value, such as cultural and social values that are not commonly addressed in HTA assessments [15,23,31].

Additionally, the patient perspective is an important feature to be considered in these formal assessments [16]. In some areas with high unmet medical needs, such as rare diseases, the role of patients is being increasingly recognised at all stages of evidence development, resulting in active patient involvement in the HTA process. Also, in cases where the evidence is uncertain, increased participation of patient organisations may increase the likelihood of a positive reimbursement decision [18]. From a societal standpoint, decision criteria need to be informed by society's values and preferences, which involves seeking to incorporate the priorities and perspectives of citizens into the decision-making process [33].

The results of this SLR highlight the need for a standardised framework that addresses implicit factors in the HTA deliberative process. Further research is advised to better understand the impact that the identified implicit factors may have on the deliberative process in each country and explore potential ways to mitigate the influence of these factors on the HTA deliberative process.

## Strengths and limitations

To the best of our knowledge, this is the first SLR focused on the implicit factors in the HTA deliberative

process, which is the main strength of this study. To facilitate the analysis, the identified implicit factors were categorised by type. Acknowledging the scarcity of information on this matter, the research group aimed to have a broad linguistic scope and for this reason, the SLR included publications in English, Spanish, French, Italian and German.

As for the limitations of this study, there is scarce literature published on the research topic with limited authors and HTA agencies involvement. On this line, the evidence on the impact of those implicit factors is scarce.

This SLR comprehends articles published during the period 2009–2019 in a limited number of countries in Europe.

Lastly, as any SLR, in the identified literature it is not always evident to find the information needed to answer all the relevant questions to perform the required analysis. In our case, there was scarce of information regarding the description of many implicit factors and their impact on the HTA decision-making from a general and country level perspective.

## Acknowledgments

We thank Alba Villacampa and David Carcedo for their assistance in identifying and reviewing some of the articles included in this SLR.

## Disclosure statement

At the time of the submission of this manuscript Clara Monleón was a full-time employee of Takeda Pharmaceuticals International AG. However, this research has been performed independently from her prior affiliation and it is part of her PhD-Thesis.

At the time of the submission of this manuscript, Mondher Toumi was a consultant of Creativ-Ceutical, a consulting firm specialised in health economics and market access which offered a broad range of services for private manufacturers and governmental bodies.

Carlos Crespo is the CEO of Axentiva Solutions, a consulting firm specialised in healtheconomics and market access.

Conflict of interest for the rest of authors: None.

## ORCID

Clara Monleón http://orcid.org/0000-0002-7764-6112

Hans-Martin Späth http://orcid.org/0000-0001-8534-4555

Carlos Crespo http://orcid.org/0000-0001-6066-0818

Claude Dussart http://orcid.org/0000-0002-1091-9139

Mondher Toumi http://orcid.org/0000-0001-7939-7204

## Funding

This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
<figure><img src="image_19.png" /></figure>

## References

[1] O'Rourke B, Oortwijn W, and Schuller T. The new definition of health technology assessment: A milestone in international collaboration. In: International Journal of Technology Assessment in Health Care; 2020. doi:10.1017/S0266462320000215.

[2] World Health Organization Global Health Expenditure database (apps.who.int/nha/database) [Internet]. November 2020. Available from: https://data.worldbank.org/indicator/SH.XPD.CHEX.PC.CD

[3] Brookes VJ, Del Rio Vilas VJ, and Ward MP. Disease prioritization: what is the state of the art?. Epidemiol. Infect. 2015. doi:10.1017/S0950268815000801.

[4] Oortwijn W, Jansen M, and Baltussen R. Use of evidence-informed deliberative processes by health technology assessment agencies around the globe. Int J Health Policy Manag. 2020 Jan 1;9(1):27-33. doi:10.15171/ijhpm.2019.72.

[5] Daniels N, and van der Wilt GJ. Health technology assessment, deliberative process, and ethically contested issues. Int J Technol Assess Health Care. 2016;32(1-2):10-15. doi:10.1017/S0266462316000155.

[6] Kristensen FB, Husereau D, and Huić M, et al. Identifying the need for good practices in health technology assessment: summary of the ISPOR HTA council working group report on good practices in HTA. Value Health. 2019; Jan 1.22(1):13-20. doi:10.1016/j.jval.2018.08.010.

[7] Kristensen FB, Lampe K, and Wild C, et al. The HTA core model®—10 years of developing an international framework to share multidimensional value assessment. Value Health Internet. 2017;20(2):244-250. doi:10.1016/j.jval.2016.12.010.

[8] Guide to the methods of technology appraisal 2013 process and methods. NICE 2013. [Internet]. 2013 cited 2020 Oct 4. Available from 2020 Oct 4: nice.org.uk/process/pmg9

[9] HAS. Doctrine de la commission de la transparence. 2020; Jan 2021. Available from: www.has-sante.fr

[10] Gemeinsamer Bundesausschus (GBA). Basis of assessment - federal joint committee [Internet]. cited 2020 Oct 3]. 2020 Oct 3 Available from: https://www.g-ba.de/ueberden-gba/aufgabe-arbeitsweise/bewertungsgrundlagen/

[11] Epstein D, and Espín J. Evaluation of new medicines in Spain and comparison with other European countries. Gac Sanit. 2020 Mar 1;34(2):133-140. doi:10.1016/j.gaceta.2019.02.009.

[12] Medicinale DU. Linee Guida Per La Compilazione Del Dossier A Supporto Della Domanda Di Rimborsabilità E Prezzo.

[13] Fricke FU, and Dauben HP. Health technology assessment: a perspective from Germany. Value Health. 2009;12(SUPPL. 2):S20-S27. doi:10.1111/j.1524-4733.2009.00555.x.

[14] Hofmann B, Cleemput I, and Bond K, et al. Revealing and acknowledging value judgments in health technology assessment. Int J Technol Assess Health Care. 2014 Mar 30;30(6):579-586. doi:10.1017/S0266462314000671.

[15] Bond K. Deliberative Processes in Health Technology Assessment: Prospects, Problems, and Policy Proposals. HTAi Global Policy Forum. 2020.

[16] Hofmann B, Bond K, and Sandman L. Evaluating facts and facting evaluations: on the fact-value relationship in HTA. J Eval Clin Pract. 2018; doi:10.1111/jep.12920J-Eval-Clin-Pract.2018;24:957-965.wileyonlinelibrary.com/journal/jep957.

[17] Eurostat - Data Explorer [Internet]. cited 2022 Jun 13]. 2022 Jun 13 Available from: https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=hlth_sha11 hp&lang=en

[18] Angelis A, Lange A, Kanavos P. Using health technology assessment to assess the value of new medicines: results of a systematic review and expert consultation across eight European countries. Eur J Health Econ. 2018 Jan 1;19 (1):123-152.

[19] The World Bank. Population, total - Spain, France, Germany UK Italy, The World Bank [Internet]. 2022; Available from: https://data.worldbank.org/indicator/SP.POP.TOTL?locations=ES-FR-DE-GB-IT-EU

[20] Moher D, Liberati A, and Tetzlaff J. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement, et al. PLoS Medicine. 2009; doi:10.1371/journal.pmed.1000097.

[21] Braun V, and Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006; doi:10.1191/1478088706qp063oa.

[22] Phelps CE, Lakdawalla DN, and Basu A, et al. Approaches to aggregation and decision making—a health economics approach: an ISPOR special task force report [5]. Value Health. 2018 Feb 1;21(2):146-154. doi:10.1016/j.jval.2017.12.010.

[23] Fischer KE, Rogowski WH, and Leidl R, et al. Transparency vs. closed-door policy: do process characteristics have an impact on the outcomes of coverage decisions? A statistical analysis. Health Policy. 2013; doi:10.1016/j.healthpol.2013.04.011.

[24] Bujar M, McAuslane N, Walker SR, et al. Quality decision making in health technology assessment: issues facing companies and agencies. Ther Innov Regul Sci. 2019; DOI:10.1177/2168479019833660

[25] Donelan R, Walker S, and Salek S. Factors influencing quality decision-making: regulatory and pharmaceutical industry perspectives. Pharmacoepidemiol Drug Saf. 2015 Mar 1;24 (3):319-328. doi:10.1002/pds.3752.

[26] Oortwijn W, Determann D, and Schiffers K, et al. Towards integrated health technology assessment for improving decision making in selected countries. Value Health. 2017 Sep 1;20(8):1121-1130. doi:10.1016/j.jval.2017.03.011.

[27] Baltussen R, Paul Maria Jansen M, and Bijlmakers L, et al. Value assessment frameworks for HTA agencies: the organization of evidence-informed deliberative processes. Value Health. 2017 Feb 1;20(2):256-260. doi:10.1016/j.jval.2016.11.019.

[28] Marsh K, Ijzerman M, and Thokala P, et al. Multiple criteria decision analysis for health care decision making - emerging good practices: report 2 of the ISPOR MCDa emerging good practices task force. Value Health. 2016 Mar 1;19(2):125-137. doi:10.1016/j.jval.2015.12.016.

[29] Barratt A. Evidence based medicine and shared decision making: the challenge of getting both evidence and
<figure><img src="image_20.png" /></figure>

preferences into health care. Patient Educ Couns. 2008 Dec;73(3):407-412. doi:10.1016/j.pec.2008.07.054.

[30] Thokala P, Devlin N, and Marsh K, et al. Multiple criteria decision analysis for health care decision making - an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016 Jan 1;19 (1):1-13. doi:10.1016/j.jval.2015.12.003.

[31] Bujar M, McAuslane N, and Walker SR. Evaluating Quality of Decision-Making Processes in Medicines' Development, Regulatory Review, and Health Technology Assessment: A Systematic Review of the Literature. , et al. In: Frontiers in Pharmacology. 2017; doi:10.3389/fphar.2017.00189.

[32] EUnetHTA joint action. HTA core model handbook. The EUnetHTA JA [Internet]. 2016;2:1-14. Available from: https://www.eunethta.eu/wp-content/uploads/

2018/03/HTACoreModel3.0-1.pdf%0Ahttps://www.cor ehta.info

[33] Krahn M, Miller F, and Bayoumi A, et al. Development of the Ontario decision framework: a values based framework for health technology assessment. Int J Technol Assess Health Care. 2018;34(3):290-299. doi:10.1017/S0266462318000235.

[34] Drummond MF, Schwartz JS, and Jönsson B, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24:244-258. doi:10.1017/S0266462308080343.

[35] Oortwijn W, and Klein P. Addressing health system values in health technology assessment: the use of evidence-informed deliberative processes. International journal of technology assessment in health care. 2019; doi:10.1017/S0266462319000187.