<figure><img src="image_1.png" /></figure>

HAL
open science

# Towards an improved acceptability assessment model of innovative solutions in the design phase

Linda Stefany Acosta Salgado, Auguste Rakotondranaivo, Eric Bonjour

► To cite this version:

Linda Stefany Acosta Salgado, Auguste Rakotondranaivo, Eric Bonjour. Towards an improved acceptability assessment model of innovative solutions in the design phase. International Journal of Business Information Systems, 2022, 41 (1), 10.1504/IJBIS.2020.10029802. hal-02845177

HAL Id: hal-02845177

https://hal.science/hal-02845177

Submitted on 3 Feb 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Linda Acosta-Salgado* Auguste Rakotondranaivo
and Eric Bonjour

ERPI (Research team on innovation process)

University of Lorraine

Nancy, France

E-mail: linda.acosta-salgado@univ-lorraine.fr

E-mail: auguste.rakotondranaivo@univ-lorraine.fr

E-mail: eric.bonjour@univ-lorraine.fr

*Corresponding author

**Abstract:** The development of innovative solutions that satisfy stakeholder needs and have successful selling is one of the main challenges of design processes. To estimate this success, the assessment of acceptability or acceptance of innovations has been the subject of multiple research studies and has led to different models that mainly concern the acceptance of the new technology in the usage phase. However, with regard to evolution in design, these models do not seem to be adapted. This paper proposes to characterize existing evaluation models by a literature review, reveals unexplored characteristics of acceptability evaluation and provides guidance for future research.

**Keywords:** Innovation, user-centred design, evaluation, acceptability, Technology Acceptance Model

**Reference** to this paper should be made as follows: Acosta-Salgado, L., Rakotondranaivo A. and Bonjour E. (2020) ‘Towards an improved acceptability assessment model of innovative solutions in the design phase’, *Int. J. Business Information Systems*, Vol. X, No. Y4, pp.000–000.

**Biographical notes:** Acosta-Salgado L. is a PhD student in Industrial Systems Engineering in the Laboratory ERPI (Innovative Process Research Team) at the University of Lorraine. She completed master's studies in Innovation Management and Industrial Design. Her current research interest focuses on acceptability assessment of innovations in design processes.

Rakotondranaivo A. is an assistant professor at the University of Lorraine, teaching at the Graduate School of Agronomy and Food Industries (ENSAIA) and doing his research in the Laboratory ERPI (Innovative Process Research Team). His current research interests are systems modelling for design, decision support, optimization and production of products and services.

Bonjour E. is a full professor specialized in systems engineering, at the University of Lorraine, teaching at the Graduate School of Innovation and Systems Engineering (ENSGSI) and doing his research in the Laboratory ERPI (Innovative Process Research Team). His main research interests touch upon the fields of innovation, model-based systems engineering, product architecture, knowledge management, project management and organization and competency management.
# 1. Introduction

Innovation is a complex concept, which has been widely defined in the literature in various disciplines (Baregheh, Rowley and Sambrook, 2009). There is no one universal definition. Its meaning changes according to disciplines, organizational structures (Akenroye, 2012) and domain (Weberg, 2009). Innovation can be the process or outcome of creating, developing and introducing new ideas, products, services or methods that meet needs and create compelling value for users (Boly, Camargo and Morel, 2016). Innovation changes the systems of use and knowledge of which they are a part, it encourages the emergence of new solutions and the evolution of our lifestyles (Skiba, 2014; Taylor, 2017). However, for a new solution to be considered as an innovation, it must be inserted and adopted by society (OECD and Eurostat, 2018).

The design and development processes for innovative solutions have evolved over time. They first focused on the technical and functional characteristics. Changes in the market and consumption trends have led to the integration of users into these processes. Considering their needs allows a greater differentiation and competitiveness of companies. Various user-centred design approaches have been developed in recent years, including user innovation, user experience, emotional design, design thinking, open innovation (Skiba, 2014). These trends are characterized by collaborative and iterative work with future users at different stages of the design process (project definition, ideation and prototyping). These steps aim to provide an innovative solution that meets needs. However, to ensure the relevance of the solution and increase its acceptance, it is necessary to integrate an evaluation step with future users and other stakeholders (Lacombe and Esselimani, 2017).

The assessment activity is a critical step in the design process. It allows the solution concept to be tested by future users in an iterative way. The objective is to assess the acceptability of the proposal in order to gradually improve it until an innovative solution is consolidated. By carrying out this step, the risk of rejection of a finished product can be limited, which increases the probability of success of the innovation project (Bobillier-Chaumon and Dubois, 2009).

The notions of acceptability and acceptance are often confused. Studies have been carried out to differentiate between these two concepts (Bobillier-Chaumon and Dubois, 2009; Alexandre et al., 2018; Busse and
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

Siebert, 2018). These definitions are summarized in Figure 1. The main
difference between the two concepts is based on the stage at which they
are evaluated in the new product development process and the purpose for
which the evaluation is being conducted. Acceptance is the demonstrated
willingness (within a group of users) to start or continue using an
innovative solution, for the tasks it is intended to support (Morris and
Turner, 2001; Salavaara et al., 2009). Acceptance assessment aims to
estimate the perception of benefits and risks of solutions based on the
users' experience and feelings in their use, taking into account various
factors. It is carried out in the marketing and use stage and mainly
concerns finished products. Acceptability is defined as the subjective
representation (attitudes, opinions, etc.) of the use of an innovative
solution during the design phase (Tricot et al., 2003; Arbelaez Garces,
Bonjour and Rakotondranaivo, 2016). The acceptability assessment aims
to predict the behaviour of future users, in order to limit the risks of
rejection and ensure the success of innovation projects. It evaluates some
of the factors in Intermediate Design Objects (IDO) (Février, 2011). IDO
is a partial result achieved during the project, such as a plan, a model, a
prototype (Galvez Manriquez, 2015).

<figure><img src="image_3.png" /><figcaption>Figure 1. Acceptability and acceptance concepts</figcaption></figure>

A great deal of research has been conducted on the acceptability or
acceptance assessment of innovations and has proposed several theoretical
models. These models have evolved over time by integrating new factors
and new relationships between these factors (Taherdoost, 2018). The main
motivation for this evolution has been the constant development of new
technologies and the diversification of fields of applications (Sharma and
Mishra, 2014). The first models published were based on the intention of
individuals to predict their behaviour. The pioneer concerns the theory of reasoned action (Theory of Reasoned Action – TRA) (Fishbein and Ajzen, 1975). A series of technology acceptance behaviour models followed, including the Theory of Planned Behaviour (TPB) (Ajzen, 1991) and the Technology Acceptance Model (TAM) (Davis, 1985). The latter is the most widely used model to predict the acceptance of technology by individuals, thanks to its simplicity and generality (Hornbæk and Hertzum, 2017). TAM explains the attitude and intent of a person to use a technology by using two factors: perceived usefulness and perceived ease of use. This model has been modified several times to assess different solutions and obtain a better acceptance prediction. New factors and relationships between factors were proposed. In this way, many models have been created, including TAM 2 (Venkatesh and Davis, 2000), UTAUT (Unified Theory of Acceptance and Use of Technology) (Venkatesh et al., 2003), TAM 3 (Venkatesh and Bala, 2008). In addition, other models have been created from the extension of the previous ones, such as (Ghazizadeh, Lee and Boyle, 2012; Munir, 2014; Alharbi, 2017).

Recent literature reviews have identified and analysed various publications on existing assessment models (Sharma and Mishra, 2014; Marangunić and Granić, 2015; Momani and Jamous, 2017; Taherdoost, 2018). These studies focused on existing acceptance assessment models, their components (factors, groupings, relationships between them) and application domains. Few studies have examined acceptability assessment models and no review of these models associated with design processes has been found. Moreover, they did not characterize the models taking into account other aspects related to the assessment activity, such as the purposes of the evaluation, the actors involved, the evaluation sites, the time when the evaluation was carried out, the resources used.

In addition, the characteristics of new design methods (dynamic and collaborative processes, where multiple actors participate in the different stages of solution development) must be taken into account. In this context, the improvement of these acceptability assessment models has become important. Before claiming to propose improved models, it is necessary to have a good knowledge of the current models in order to identify the necessary adaptations. This paper presents a review of the scientific literature to characterize the assessment models used and the promising research directions.
The following section presents how the 4W1H (What, Who, Where, When and How method, adapted from the well-known Five W’s approach (Who, What, Where, When, Why?) was applied to conduct the literature review and which corpus of publications was considered. The third section describes the results obtained with discussions. A fourth section proposes ways to improve the assessment activity. Finally, the last section concludes this study by highlighting its implications and limitations.

## 2. Literature Review Process

The literature review applies a 4W1H method to characterize acceptability or acceptance assessment and to identify the information needed for the study. This method is applied to publications in the SCOPUS database. The following subsection presents the implementation of the 4W1H method while the second subsection gives details about the document selection process.

### 2.1. Method of Identifying Information

For the analysis of the publications, the 4W1H methodology was used. Unlike other techniques, it makes it possible to identify all the characteristics of a situation, a problem, an activity, by asking and answering the appropriate questions (Lannoy, 2009; Dubuisson-Quellier et al., 2013). Other research has used this method to understand a knowledge domain (Bajaj et al., 2018), collect and categorize information (Palafox and Hashimoto, 2010), characterize challenges in a structured way (Zhang et al., 2014) and guide projects (Dautelle, 2013).

As part of our research, this method was used to characterize models for the acceptability and acceptance assessment of innovative solutions presented in the literature, by systematically answering the following questions:

- What: What acceptability or acceptance assessment about? This question aims at identifying the types of innovative solutions that are the objects of the evaluation. These innovative solutions can be: computer technology products (software, websites, or digital applications), non-computer technology products (communication technologies, robots, or cars), non-technology products (food products,
handicrafts), services (home help, deliveries), organizations (product-service organizations, stakeholder organizations) and practices or procedures (programs, protocols, management system).

- Who: Who are the actors involved in the assessment of acceptability or acceptance? This question has been divided into two parts: how many types of actors? And which profile of actors? The answers to the first question concern the number of types of actors who participated in a study. For example, if a group of students has participated in the evaluation of a self-learning application, then they are considered as one type of actor. However, if a group of teachers also participated in the evaluation of this application, they are considered as two types. In the second question, the stakeholder profiles were divided between users and experts.

- Where: Where is the assessment of acceptability or acceptance carried out? This question is intended to determine where the assessments were conducted. Possible answers are: in the actual context of use (real context), in locations different from the actual context of use (experimental context), or in a non-established context, since the evaluation was done by electronic means (virtual or digital context).

- When: When is the acceptability assessment or acceptance carried out? This question is intended to determine the phase of the design process during which the evaluation was conducted. Three response options were identified: during the design phase, before the use phase or during the use phase.

- How: How is the assessment of acceptability or acceptance carried out? This question aims to identify how the evaluation activity was conducted. For this reason, the question is divided into four parts: which model was used? What method was chosen to collect data? What technological tool was used in the collection? And what kind of analysis was done? The answers to the first question concern the models set out in the introduction, such as TRA, TAM, and UTAUT. In the second question, we refer to the use of methods such as interviews, questionnaires, observations. In the third question, we seek to identify the type of tools, for example the paper or digital questionnaire, or virtual reality. Finally, the fourth question asks whether the methods used to analyse the information collected are quantitative or qualitative.
The information identified by applying these questions was coded and
listed in an Excel document.

## 2.2. Publication Selection

The process followed for the selection of publications is graphically
represented in Figure 2. The Scopus database was used because it contains

<figure><img src="image_4.png" /><figcaption>Figure 2. Publication selection process</figcaption></figure>

the largest number of documents compared to other databases
(Gudanowska, 2017). The journals and conferences referred to in this
database cover multiple disciplines (Engineering, Computer Science,
Decision Sciences, Social Sciences, “Business, Management and
Accounting”, Psychology) that may be concerned with innovation
acceptance topics. Their papers have been reviewed by international peers
from various scientific fields (Regolini and Jannés-Ober, 2013). The
research we present below was conducted in March 2019. We identified
journal articles and conference proceedings in English from 1961 to 2018,
where the heading "title" contained the word "accepta*", plus one of the
following terms: ("model* or method*") and ("predict* or evaluat* or
assess* or valid* or valid* or estimat*"). The asterisk was used to search
for all variations of the word used, this encouraged the inclusion of
publications dealing with acceptability and acceptance.

The search found 334 publications. The representation of results by year
showed an increase in the number of results between 2006 and 2018. Documents published on these dates have been included in the analysis. The following inclusion criteria were used: (1) publications cited at least once (except for the last three years of the sample, as it is likely that these documents have not yet been cited because of their novelty); (2) publications that assess the acceptability or acceptance of an innovative solution (products, services, organizations); (3) publications that conducted the evaluation with one or more people.

As a result, several publications were not selected: 38 documents published between 1961 and 2005; 7 documents without abstracts; 39 publications without any citations; 12 documents that did not assess the acceptability or acceptance of an innovative solution, such as publications that dealt with the acceptance of an individual, by himself or by a group of people in a particular life context; 87 documents that did not assess acceptability or acceptance by people, for example publications that addressed acceptance in accordance with standards. In addition, 25 additional documents were excluded because the full text was not found. Finally, 126 publications were selected for the study.

**3. Characterization of Models for Assessing Acceptability or Acceptance**

The 126 selected publications were published mainly in the thematic areas of computer sciences, social sciences, engineering and business (Figure 3). These publications included articles and conference proceedings with titles containing the words acceptance and acceptability. The presence of one of these words in the title has been quantified. The

<figure><img src="image_5.png" /><figcaption>Figure 3. Publications by thematic domain (Source: Scopus)</figcaption></figure>
results obtained show that a large proportion of the publications use the
word acceptance (87.3%), while a smaller percentage use the word
acceptability (8.7%). Other publications used both words. The analysis is
carried out on the totality of articles, independently of the content of the
word acceptability or acceptance. Appendix 1 lists the publications
analysed with the coded results and Appendix 2 shows the responses for
both the publications dealing with acceptance and acceptability and for all
publications. For each item of the questionnaire, the results obtained are
presented and discussed.

**3.1. Technologies are the first object of the assessment (what)**

A high percentage of publications (66.7%) assessed the acceptability or acceptance of computer products. These are mainly mobile websites, programs or applications that aim to encourage, for example, e-learning (Dai *et al.*, 2011; Alsofyani and Eynon, 2012), e-shopping (Halbach and Gong, 2011; Shukla and Sharma, 2018), financial resource management (Lai and Pires, 2010; Kumar and Shenbagaraman, 2017). In addition, 24.6% of the publications evaluated non-computer technologies, including mainly health technologies. For example: a device for threading a compression stocking (Arbelaez Garces, Bonjour and Rakotondranaivo, 2016), a remote monitoring system (Gagnon *et al.*, 2012). The remaining documents assessed practices (such as medical procedures or engineering processes) (4.8%), non-technological products (2.4%), services (0.8%) and organizations (0.8%).

Innovation can involve a wide variety of artefacts. However, the results
obtained show that most acceptability or acceptance assessments are done
on technological products.

**3.2. Innovative solutions are evaluated by a single type of actor, the users (who)**

According to the results obtained, in 86.5% of the publications, the acceptability or acceptance assessment involved only one type of actor. In 10.3%, two types of actors were involved. The remaining percentage corresponds to documents that included three (2.4%) or four actors (0.8%).

The main profile of these actors is that of users or future users (89.7%),
such as employees who evaluated a production system (Jokonda, 2015), students who evaluated new online learning systems (Yi-cheng et al., 2007). In addition, some documents included experts as evaluators of the solution (1.6%) and a very small percentage included both users and experts (0.8%). Finally, 7.9% of publications did not make an assessment with some type of actor, but instead reviewed literature on acceptability or acceptance models, generally to propose new models (Turner et al., 2010).

The publications analysed show a low integration of the various types of actors in the acceptability or acceptance estimation, mainly users or future users. Assessments carried out with different actors analyse the results as a whole. This means that the preferences of different types of actors are not differentiated and no conflict between preferences is identified. For example, in Randell's paper (Randell, Backhouse and Nelson, 2015), the use of videoconferencing in a clinical study centre was evaluated with paediatricians, diabetologists, trial coordinators and nurses. Although different actors participated in the assessment, their preferences were analysed as a single group of "technology users" actors.

The treatment of preferences according to multiple types of actors can generate different conflicts, such as contradictions and paradoxes (Amason, 1996; Ballard-Reisch and Turner, 2017). The failure to identify and manage these conflicts can be decisive for the future acceptance of an innovative solution.

### 3.3. The real context of use is the place most often used (where)

Thirty-two publications do not specify the context in which the assessments were conducted (25.4%). Of the documents that provided this information, 59.5% conducted the evaluation in the real context of use. For example, a new water supply system has been evaluated among individuals (Nancarrow, Porter and Leviston, 2010), or a shared bicycle system is being assessed in the city (Hazen, Overstreet and Wang, 2015). In addition, 8.7% of publications assess acceptability or acceptance in a virtual or online context, such as virtual robots (Weistroffer et al., 2013). Less significantly, 6.3% of the documents evaluated innovations in an experimental context, in other words, a place different from the context of use, for example a home help robot was evaluated in a public place (Weiss et al., 2008).
Each context has its own characteristics, including rules (formal and
informal), individual forms of interaction between people, or a culture
(Bobillier-Chaumon and Dubois, 2009). The context in which an
assessment is conducted can influence how individuals perceive
innovation. Places for assessing acceptability or acceptance must be
chosen while knowing the possible biases.

**3.4. The assessment is carried out mainly in the use phase (when)**

Twenty-four of the publications reviewed do not specify the phase at which innovations were assessed (19%). Of the documents that provided this information, 65.9% conducted the evaluation during the use of the innovative solution. For example, Wu (Wu *et al.*, 2008) evaluated medical devices that have been in use for some time. Among other results, 8.7% of the publications assessed an innovative solution before use, such as the evaluation of mobile communication devices by a future and potential user (Sek *et al.*, 2010). Only 6.3% of publications conducted the assessment in the design phase. In these studies, no finished product was evaluated. They concerned acceptability evaluation of concepts, models or prototypes of a proposed solution (Osswald *et al.*, 2012; Weistroffer *et al.*, 2013).

The high percentage of assessments during the use phase means that
most publications are focused on acceptance. Solutions evaluated
correspond, in most cases, to finished products. That is, solutions that have
completed the design process and on which modifications or
improvements deemed necessary cannot be incorporated. It is easier and
less costly to modify the proposed solution if the evaluation is carried out
as early as possible in the design phase.

**3.5. TAM model and quantitative data analysis methods are most commonly
used for the assessment (how)**

*Model used:* Eleven publications do not specify the method used to assess acceptability or acceptance (8.7%). Most documents that include the word acceptability in the title do not refer to a specific model. Among the publications that provided this information, a high percentage use known models. The two most important percentages are the original version of TAM (36.5%) and the adaptations it has undergone (36.5%). These adaptations include the addition of new factors, different groupings and
redefinition of the relationships between them. A smaller percentage used the UTAUT method (6.3%). Other publications, used an alternative model (7.9%), either adopted by another author or proposed by the study. Generally, they are developed from known models. Finally, 4% of the documents integrated between three and four models, where TAM is included in most proposals.

***Data collection methods:*** Publications used only one method (86.5%) or 2 to 3 methods (13.5%) for data collection. Among them, questionnaires are predominant (82.5%) followed by individual interviews (6.3%), observation sessions (4%), focus groups (2.4%) and physiological measurements (0.8%). In addition, twenty-four publications compiled information from other research, reviewing the literature (19%).

***Technological tools:*** In 81% of the publications, it was not possible to identify the tools used. These publications largely correspond to those that used questionnaires and interviews. Among the tools identified, e-mail is the most widely used (15.9%), followed by telephone (1.6%), virtual reality (0.8%) and other audio-visual tools (0.8%).

***Data analysis method:*** the analysis methods used were mostly quantitative (81.1%). The most commonly used methods are regressions and factor analysis. However, very few publications have used other quantitative methods such as multi-criteria analyses (Lee et al., 2011) or artificial intelligence methods such as Bayesian networks (Luo, Su and Lee, 2011; Arbelaez Garces, Rakotondranaivo and Bonjour, 2016a). A smaller percentage used qualitative methods (4.8%), mainly content analysis (Durodolu, 2016). The other publications did not specify the method used (13.5%).

## 3.6. Additional Elements from Other Literature Reviews

Other literature reviews have not addressed many of the model characteristics, as in this study. However, in addition to our results, this research has examined other model characteristics, which describe their evolution. For example, Momani and Jamous (Momani and Jamous, 2017) studied the evolution of assessment models and found that the structure is very similar. The main reason, they explained, is because they were developed from previous models and most of them come from behavioural studies. The difference between these models is how acceptability or
acceptance is explained, that is, with the factors and the various ways of
linking them.

The degree of acceptability of a product varies depending on the type of
factors taken into account to explain it (Barcenilla and Bastien, 2009).
Alexandre, Reynaud, Osiurak and Navarro (Alexandre et al., 2018) found
that the factors are generally the same in several models, but named
differently. These factors are generally associated with cognitive abilities,
emotional aspects such as anxiety or fear should also be taken into account
(Marangunić and Granić, 2015). In addition, Liu and Cheng (Liu and
Cheng, 2015) considered that most factors are mainly profit-oriented, in
other words, they do not take into account negative aspects such as risks or
disadvantages. Assessing benefits and risks for each stakeholder is
necessary to ensure that the satisfactory decisions (or trade-offs) are made
about an innovation.

4. Future Research Avenues

The characterization of existing assessment models using the 4W1H
methodology clearly confirms the under-representation of acceptability
assessment models and the evaluation is mainly applied to finished
products. On this basis and considering the results of other research, we
propose some elements of reflection to improve these models. The
proposed aspects aim to better adapt acceptability assessment models to
new innovative design processes (user-centred, collaborative and
iterative). The points proposed here should be studied and further
developed in future research, in order to integrate them into acceptability
assessment activities.

- Innovation is not limited to technology development. Other types of innovations such as new services, product-service systems, practices, methods, layouts of workplaces, processes and responsibilities could be further assessed. The identification of factors related to other innovation types and the assessment of their relationships should be the subject of future research.

- The current innovation design processes are collaborative. They involve multiple actors in the different phases. Innovations have multiple stakeholders. The integration of the latter into the design phase will increase the probability of future acceptance. An
acceptability assessment model should incorporate the opinions of
various types of users and profiles.

- Assessing the acceptability of an innovative solution with several stakeholders can lead to disagreements and conflicts. The identification and management of these problems could contribute to improving the acceptability of the innovative solution.

- The evaluation context is a crucial factor in shaping user's assessments and perceptions about innovation (Valacich, Wang and Jessup, 2018). Real contexts are relevant for the assessment of innovative solutions. However, in the face of collaborative design processes, it is not always possible to evaluate the product at different stages of development and in different contexts of use. In response to these difficulties, various innovation approaches, such as living labs (Hossain, Leminen and Westerlund, 2019), have been developed, allowing users and multiple stakeholders to be involved in designing a solution. The living lab uses neutral evaluation spaces, where real conditions of use can be simulated physically (scale model, prototype) or virtually (virtual reality, augmented reality) in the same space.

- Evaluating an innovative solution in the use phase makes it difficult to improve it, as it is more expensive to make changes to the product. It is necessary to develop a new acceptability assessment model for solutions under development. The results can lead to product improvements at a lower cost and increase the probability of market success. In addition, they contribute to the work of the design team, allowing the sharing of common cognitive representations of the solution being developed (Tricot et al., 2003).

- The design processes are also iterative, which involves the development of progressive representations of the innovative solution (concept, idea, plan, scale model, prototype) in the different design stages. The assessment of these representations, not only lead to the fidelity of the proposal, but also allow a better understanding of the problem and consequently, a better definition of the design specifications. The models of acceptability assessment must be adapted to the progressive evolution of the design specifications and to the different representations of the proposed solutions, for which, the dynamic models are the most convenient.

- To obtain information from a user, technological advances allow us to have more advanced tools than online or paper questionnaires. These
technologies also make it possible to understand psychological and
physiological aspects that are not easily expressed by people. It is
possible to use technologies such as physiological sensors, eye-
tracking systems, videos.

- The factors for acceptability or acceptance assessment used in existing models are principally oriented towards the benefits of using the solution. However, negative aspects, such as risks, are decisive in rejecting an innovative solution. In the design process, the assessment of acceptability should include joint modelling of benefits and risks.

- The analytical methods used to assess acceptability must be capable of processing a large amount of information. This data can come from different sources of information such as questionnaires, observations, or physiological measurements. In addition, the models should also be able to evaluate causality between factors. The statistical methods commonly used in the literature are not sufficient to process this type of information. The use of artificial intelligence methods may be relevant. For example, Arbelaez, Bonjour and Rakotondranaivo (Arbelaez Garces, Rakotondranaivo and Bonjour, 2016b) and Luo, Su and Lee (Luo, Su and Lee, 2011) tested Bayesian networks for acceptability or acceptance assessment.

Conclusions

This research makes two contributions to the body of knowledge on the
acceptability assessment. The characterization of existing models for
assessing acceptability or acceptance through criteria, on the one hand,
and the proposal of research and improvement axes for future models for
assessing acceptability in the design phase, on the other hand, have not
been considered by other literature reviews.

The 4W1H method applied to characterize assessment model confirmed
and identified the differences between acceptability and acceptance. The
differences between these two types of assessments have not been
examined using this method in previous studies. With regard to the first
two questions of the method "what" and "who" there is no significant
difference. Both models could be used to evaluate several types of
innovations and with different users and stakeholders. With regard to the
third question "where", both types of evaluations could be conducted in a
real context. However, other contexts could be used to assess the
acceptability of different product representations in the design phase. As for the question of the "when", these two types of assessment differ in the stage at which they are carried out in the design process. Acceptability is assessed from the early stages of design and acceptance in the use phase, when the product is finished.

With regard to the question "how", the acceptability and acceptance assessment has several common aspects. Such as the use of questionnaires and other methods of collecting information from users, the use of different technologies to collect information and data analysis using quantitative analysis methods. The differences between these two types of assessments relate to the theoretical models and techniques used to collect the information. Known theoretical models include factors associated with the experience of using a finished product and in a real context, which is relevant for acceptance assessment. However, these factors must be adapted to the intermediate design object being evaluated and the context in which the evaluation is being conducted.

The method used to conduct this study has some limitations that should be addressed in future studies. On the one hand, publications from a single database were selected. Although these documents have been published in thematic areas relevant to the research topic, it is possible that other databases may contain other relevant publications that have not been incorporated into this research. On the other hand, several studies were excluded because they did not contain a summary and the full text could not be found. Some of these documents may have been relevant for analysis purposes. Finally, another limitation of this study is that some publications did not provide sufficient data for our analysis.

## References

Abu-dalbouh, H. M. (2016) ‘Using a Modified Technology Acceptance Model to Evaluate Designing Eight Queens Chess Puzzle Game’, *Journal of Computer Sciences*, 12(5), pp. 232–240. doi: 10.3844/jcssp.2016.232.240.

Ahmad, T. B. T. et al. (2010) ‘Faculty’s acceptance of computer based technology: Cross-validation of an extended model’, *Australasian Journal of Educational Technology*, 26(2), pp. 268–279. doi: 10.14742/ajet.1095.
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

Aizstrauta, D. et al. (2013) ‘Validation of Integrated Acceptance and Sustainability Assessment Methodology’, *Procedia Computer Science*. Elsevier Masson SAS, 26, pp. 33–40. doi: 10.1016/j.procs.2013.12.005.

Aizstrauta, D. and Ginters, E. (2015) ‘Integrated acceptance and sustainability assessment model transformations into executable system dynamics model’, in *Procedia Computer Science*. Valmiera, Latvia: Elsevier, pp. 92–97. doi: 10.1016/j.procs.2015.12.364.

Aizstrauta, D. and Ginters, E. (2017a) ‘Sociotechnical systems acceptance and sustainability assessment using dynamic simulation model’, in Affenzeller et al. (eds) *Proceedings of the European Modeling and Simulation Symposium*. Barcelona, pp. 519–526.

Aizstrauta, D. and Ginters, E. (2017b) ‘Using Market Data of Technologies to Build a Dynamic Integrated Acceptance and Sustainability Assessment Model’, in *Procedia computer Science*. Riga, Latvia: Elsevier, pp. 501–508. doi: 10.1016/j.procs.2017.01.165.

Ajzen, I. (1991) ‘The Theory of Planned Behavior’, *Organizational behavior and human decision processes*, 50(2), pp. 179–211. doi: 10.1016/0749-5978(91)90020-T.

Akenroye, T. O. (2012) ‘Factors influencing innovation in healthcare: A conceptual synthesis’, *Innovation Journal*, 17(2), pp. 1–21. Available at: https://www.innovation.cc/scholarly-style/akenroye_concept_framework_force_drive_innovate_healthSector_v17i2a3.pdf.

Al-khateeb, F. B. (2007) ‘Predicting Internet Usage in Two Emerging Economies Using an Extended Technology Acceptance Model (TAM)’, in *Proceedings of the 2007 International Symposium on Collaborative Technologies and Systems, CTS*. Orlando, Florida, pp. 143–149. doi: 10.1109/CTS.2007.4621750.

Alexandre, B. et al. (2018) ‘Acceptance and acceptability criteria: a literature review’, *Cognition, Technology & Work*. Springer London, 20(2), pp. 165–177. doi: 10.1007/s10111-018-0459-1.

Alharbi, S. (2017) ‘An extended UTAUT model for understanding of the effect of trust on users’ acceptance of cloud computing’, *International Journal of Computer Applications in Technology*, 56(1), p. 65. doi: 10.1504/ijcat.2017.086562.

Alkis, N. and Özcan, S. (2010) ‘Work in progress - A modified Technology Acceptance Model for e-assessment: Intentions of engineering students to use web-based assessment tools’, in *40th ASEE/IEEE Frontiers in Education Conference*. Washington, DC, pp. 26–28.

Alotaibi, S. J. and Wald, M. (2013) ‘Evaluation of the UTAUT Model for Acceptable User Experiences in Identity Access Management Systems’,
Acosta-Salgado, L., Rakotondranaivo A. and Bonjour E

in *The 8th International Conference for Internet Technology and Secured Transactions (ICITST-2013)*. London, England, pp. 232–237.

Alper, S. J. et al. (2007) ‘Using the Technology Acceptance Model to Predict Violations in the Medication Use Process’, in *Proceedings of the human factors and ergonomics society 51st annual meeting*. Pennsylvania, pp. 745–749.

Alryalat, M. A. A. (2017) ‘Measuring Citizens’ Adoption of Electronic Complaint Service (ECS) in Jordan: Validation of the Extended Technology’, *International Journal of Electronic Government Research*, 13(2), pp. 47–65. doi: 10.4018/IJEGR.2017040103.

Alshibly, H. H. (2011) ‘An Extended Tam Model to Evaluate User’s Acceptance of Electronic Cheque Clearing Systems at Jordanian Commercial Banks’, *Australian Journal of Basic and Applied Sciences*, 5(5), pp. 147–156.

Alsofyani, M. M. and Eynon, R. (2012) ‘A preliminary evaluation of short blended online training workshop for TPACK development using technology acceptance model’, *TOJET: The Turkish Online Journal of Educational Technology*, 11(3), pp. 20–32.

Amason, A. C. (1996) ‘Distinguising the effects of functional and dysfunctional conflict on strategic decision making: Resolving a paradox for top management teams.’, *Academy of Management Journal*, 39(1), pp. 123–148. doi: 10.2307/256788.

Ambak, K. (2013) ‘Application of Technology Acceptance Model in Predicting Behavioral Intention to Use Safety Helmet Reminder System’, *Research Journal of Applied Sciences, Engineering and Technology*, 5(3), pp. 881–888.

An, J.-Y. et al. (2007) ‘Theory Development in Nursing and Healthcare Informatics, A Model Explaining and Predicting Information and Communication Technology Acceptance by Healthcare Consumers’, *Advances in Nursing Science*, 30(3), pp. E37–E49.

Arbelaez Garces, G., Bonjour, E. and Rakotondranaivo, A. (2016) ‘New product acceptability evaluation and improvement model with knowledge reuse’, *IFAC-PapersOnLine*. Elsevier B.V., 49(12), pp. 1104–1109. doi: 10.1016/j.ifacol.2016.07.639.

Arbelaez Garces, G., Rakotondranaivo, A. and Bonjour, E. (2016a) ‘An acceptability estimation and analysis methodology based on Bayesian networks’, *International Journal of Industrial Ergonomics*. Elsevier Ltd, 53, pp. 245–256. doi: 10.1016/j.ergon.2016.02.005.

Arbelaez Garces, G., Rakotondranaivo, A. and Bonjour, E. (2016b) ‘Improving users’ product acceptability: an approach based on Bayesian networks and a simulated annealing algorithm’, *International Journal of Production Research*, 54(17), pp. 5151–5168. doi: 10.1080/00207543.2016.1156183.
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

Bahry, F. D. S., Anwar, N. and Amran, N. (2012) ‘Predicting intended to use of Web portal Using Extended Technology Acceptance Model ( TAM ) Some perspective on Information Management students .’, in *IEEE Symposium on Business, Engineering and Industrial Applications*. Bandung, pp. 229–234. doi: 10.1109/ISBEIA.2012.6422875.

Bajaj, G. et al. (2018) ‘4W1H in IoT Semantics’, *IEEE Access*, 6(1), pp. 65488–65506. doi: 10.1109/ACCESS.2018.2878100.

Ballard-Reisch, D. and Turner, P. K. (2017) *Contradictions, Tensions, Paradoxes, and Dialectics*. John Wiley, The International Encyclopedia of Organizational Communication. John Wiley. Edited by C. R. Scott and L. Lewis. New York, United States: Wiley-Blackwell. doi: 10.1002/9781118955567.wbieoc043.

Barcenilla, J. and Bastien, J. M. C. (2009) ‘L’acceptabilité des nouvelles technologies : quelles relations avec l’ergonomie, l’utilisabilité et l’expérience utilisateur ?’, *Le travail humain*, 72(4), pp. 311–331. doi: 10.3917/th.724.0311.

Baregheh, A., Rowley, J. and Sambrook, S. (2009) ‘Towards a multidisciplinary definition of innovation’, *Management Decision*, 47(8), pp. 1323–1339. doi: 10.1108/00251740910984578.

Beldad, A. D. and Hegner, S. M. (2017) ‘Expanding the Technology Acceptance Model with the Inclusion of Trust, Social Influence, and Health Valuation to Determine the Predictors of German Users ’ Willingness to Continue using a Fitness App: A Structural Equation Modeling Approach’, *International Journal of Human-Computer Interaction*. Taylor & Francis, 34(9), pp. 882–893. doi: 10.1080/10447318.2017.1403220.

Bennani, A. and Oumlil, R. (2010) ‘Do constructs of technology acceptance model predict the ICT appropriation by physicians and nurses in healthcare public centres in Agadir, South of Morocco?’, in *Proceedings of the Third International Conference on Health Informatics*. Valencia, Spain, pp. 241–249. doi: 10.5220/0002714502410249.

Bervell, B. and Umar, I. N. (2017) ‘Validation of the UTAUT Model : Re-Considering Non-Linear Relationships of Exogeneous Variables in Higher Education Technology Acceptance Research’, *EURASIA Journal of Mathematics Science and Technology Education*, 13(10), pp. 6471–6490. doi: 10.12973/ejmste/78076.

Besbes, A. et al. (2016) ‘A Cross-Cultural Validation of the Tourism Web Acceptance Model (T-WAM) in Different Cultural Contexts’, *Journal of International Consumer Marketing*, 28(3), pp. 211–226. doi: 10.1080/08961530.2016.1152524.

Bobillier-Chaumon, M.-E. and Dubois, M. (2009) ‘L’adoption des technologies
en situation professionnelle : quelle articulation possible entre
acceptabilité et acceptation ?', *Le travail humain, Presses Universitaires
de France*, 72(4), pp. 355-382. Available at: https://halshs.archives-
ouvertes.fr/halshs-01562034.

Boly, V., Camargo, M. and Morel, L. (2016) *Ingénierie de l'innovation*. 3rd edn.
Paris: Lavoisier.

Briz-ponde, L. and García-peñalvo, F. J. (2015) ‘An Empirical Assessment of a
Technology Acceptance Model for Apps in Medical Education’, *Journal
of medical systems*, 39(176), pp. 1–5. doi: 10.1007/s10916-015-0352-x.

Busse, M. and Siebert, R. (2018) ‘Acceptance studies in the field of land use—A
critical and systematic review to advance the conceptualization of
acceptance and acceptability’, *Land Use Policy*. Elsevier, 76, pp. 235–
245. doi: 10.1016/j.landusepol.2018.05.016.

Calisir, F. et al. (2014) ‘Predicting the Intention to Use a Web-Based Learning System: Perceived Content Quality, Anxiety, Perceived System Quality, Image, and the Technology Acceptance Model’, *Human Factors and Ergonomics in Manufacturing & Service Industries*, 24(5), pp. 515–531. doi: 10.1002/hfm.

Calisir, F., Altin Gumussoy, C. and Bayram, A. (2009) ‘Predicting the behavioral intention to use enterprise resource planning systems’, *Management Research News*, 32(7), pp. 597–613. doi: 10.1108/01409170910965215.

Chan, H. C. and Teo, H. (2007) ‘Evaluating the Boundary Conditions of the Technology Acceptance Model: An Exploratory Investigation’, *ACM Transactions on Computer-Human Interaction*, 14(2), pp. 1–22. doi: 10.1145/1275511.1275515.

Chang, K. C. et al. (2012) ‘Integrating loss aversion into a technology acceptance model to assess the relationship between website quality and website user’s behavioural intentions’, *Total Quality Management and Business Excellence*, 23(7–8), pp. 913–930. doi: 10.1080/14783363.2011.637793.

Chen, C.-D., Fan, Y.-M. and Farn, C.-K. (2007) ‘Predicting electronic toll collection service adoption: An integration of the technology acceptance model and the theory of planned behavior’, *Transportation Research Part C: Emerging Technologies*, 15(5), pp. 300–311. doi: 10.1016/j.trc.2007.04.004.

Chen, H. et al. (2012) ‘Applying the technology acceptance model to evaluate the learning companion recommendation system on Facebook’, in *2012 IEEE Fourth International Conference on Technology for Education Applying*. Hyderabad, pp. 160–163. doi: 10.1109/T4E.2012.36.

Chen, M.-F. and Lin, N.-P. (2017) ‘Incorporation of health consciousness into the technology readiness and acceptance model to predict app download and usage intentions’, *Internet Research*, 28(2), pp. 351–373. doi: 10.1108/IntR-03-2017-0099.
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

Cheung, R. and Vogel, D. (2013) ‘Predicting user acceptance of collaborative technologies: An extension of the technology acceptance model for e-learning’, *Computers and Education*. Elsevier Ltd, 63, pp. 160–175. doi: 10.1016/j.compedu.2012.12.003.

Chooprayoon, V., Fung, C. C. and Depickere, A. A. (2007) ‘TECTAM, a modified technology acceptance model to assess E-commerce technologies adoption by Thai SME’, in *IEEE Region 10 Annual International Conference, Proceedings/TENCON*. Taipei, pp. 1–4. doi: 10.1109/TENCON.2007.4428977.

Dai, C. Y. et al. (2011) ‘The research on user satisfaction of easy teaching Web of Taipei assessed via information quality, system quality, and technology acceptance model’, in *ICCSE 2011 - 6th International Conference on Computer Science and Education, Final Program and Proceedings*. Virgo, Singapore, pp. 758–762. doi: 10.1109/ICCSE.2011.6028748.

Dasgupta, A. et al. (2009) ‘Application of the Extended Technology Acceptance Model in predicting pharmacists’ intention to use personal digital assistants’, *Journal of the American Pharmacists Association*. Elsevier Masson SAS, 49(6), pp. 792–796. doi: 10.1331/JAPhA.2009.08136.

Dautelle, R. (2013) ‘Qualité et Recherche dans un contexte d’obligation de résultat’, in *QUALITA2013*. Compiègne, France, pp. 1–9.

Davis, F. D. (1985) *A technology acceptance model for empirically testing new end-user information systems: Theory and results*. Massachusetts Institute of Technology. doi: oclc/56932490.

Dawson, C. H., Mackrill, J. B. and Cain, R. (2017) ‘Assessing user acceptance towards automated and conventional sink use for hand decontamination using the technology acceptance model’, *Ergonomics*. Taylor & Francis, 60(12), pp. 2–14. doi: 10.1080/00140139.2017.1316018.

Dubuisson-Quellier, S. et al. (2013) ‘Annexe 2 : Une méthodologie pour la constitution d’une bibliographie’, *Sociologie*, 4(4), pp. 1–7. Available at: https://journals.openedition.org/sociologie/2040 (Accessed: 31 March 2019).

Ducey, A. J. and Cooverd, M. D. (2016) ‘Predicting tablet computer use : An extended Technology Acceptance Model for physicians’, *Health Policy and Technology*. Elsevier, 5(3), pp. 268–284. doi: 10.1016/j.hlpt.2016.03.010.

Durodolu, O. O. (2016) *Technology Acceptance Model as a predictor of using information system* ' to acquire information literacy skills, *Library Philosophy and Practice* (e-journal). Available at: http://digitalcommons.unl.edu/libphilprac/1450 (Accessed: 4 April 2019).
Faqih, K. M. S. (2011) ‘Integrating Perceived Risk and Trust with Technology Acceptance Model: An Empirical Assessment of Customers’ Acceptance of Online Shopping in Jordan’, in *International Conference on Research and Innovation in Information Systems*. Kuala Lumpur, pp. 1–5. doi: 10.1109/ICRIIS.2011.6125686.

Fares, S. et al. (2008) ‘Measuring Energy Expenditure in Community- Dwelling Older Adults: Are Portable Methods Valid and Acceptable?’, *Journal of the American Dietetic Association*, 108(3), pp. 544–548. doi: 10.1016/j.jada.2007.12.012.

Février, F. (2011) *Vers un modèle intégrateur ‘expérience-acceptation’ : rôle des affects et de caractéristiques personnelles et contextuelles dans la détermination des intentions d’usage d’un environnement numérique erique de travail*. Université de Rennes 2.

Fishbein, M. and Ajzen, I. (1975) *Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research*. Addison-we. Philippines. doi: 10.1017/CBO9781107415324.004.

Füssl, E. et al. (2015) ‘Methodological development of a specific tool for assessing acceptability of assistive systems of powered two-wheeler-riders’, *IET Intelligent Transport Systems*, 9(1), pp. 12–21. doi: 10.1049/iet-its.2014.0026.

Gagnon, M. P. et al. (2012) ‘Using a Modified Technology Acceptance Model to Evaluate Healthcare Professionals’ Adoption of a New Telemonitoring System’, *Telemedicine and e-Health*, 18(1), pp. 54–59. doi: 10.1089/tmj.2011.0066.

Galvez Manriquez, D. (2015) *Evaluation de la capacité à innover: Une approche par auto évaluation et suivi supporté par des analyses multicritères dynamiques*. Université de Lorraine.

Gamal Aboelmaged, M. (2010) ‘Predicting e-procurement adoption in a developing country’, *Industrial Management & Data Systems*, 110(3), pp. 392–414. doi: 10.1108/02635571011030042.

Ghazizadeh, M., Lee, J. D. and Boyle, L. N. (2012) ‘Extending the Technology Acceptance Model to assess automation’, *Cognition, Technology and Work*, 14, pp. 39–49. doi: 10.1007/s10111-011-0194-3.

Girgis, A. et al. (2017) ‘eHealth System for Collecting and Utilizing Patient Reported Outcome Measures for Personalized Treatment and Care (PROMPT-Care) Among Cancer Patients: Mixed Methods Approach to Evaluate Feasibility and Acceptability’, *Journal of medical internet research*, 19(10), pp. 1–13. doi: 10.2196/jmir.8360.

Guarino, H. et al. (2016) ‘A Mixed-Methods Evaluation of the Feasibility, Acceptability, and Preliminary Efficacy of a Mobile Intervention for Methadone Maintenance Clients’, *Psychology of Addictive Behaviors*, 30(1), pp. 1–11. doi: 10.1037/adb0000128.
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

Gudanowska, A. E. (2017) ‘A Map of Current Research Trends within Technology Management in the Light of Selected Literature’, *Management and Production Engineering Review*, 8(1), pp. 78–88. doi: 10.1515/mper-2017-0009.

Hakami, Y. A., Hussei, A. R. C. Bin and Adenuga, K. I. (2016) ‘Preliminary model for computer based assessment acceptance in developing countries’, *Journal of Theoretical and Applied Information Technology*, 85(2), pp. 192–206.

Halbach, M. and Gong, T. (2011) ‘What Predicts Commercial Bank Leaders’ Intention to Use Mobile Commerce? The Roles of Leadership Behaviors, Resistance to Change, and Technology Acceptance Model’, *Journal of Electronic Commerce in Organizations*, 9(3), pp. 1–19. doi: 10.4018/jeco.2011070101.

Ham, S., Kim, W. G. and Forsythe, H. W. (2008) ‘Restaurant Employees’ Technology Use Intention: Validating Technology Acceptance Model with External Factors’, *Journal of Hospitality & Leisure marketing*, 17(1–2), pp. 78–98. doi: 10.1080/10507050801978422.

Hazen, B. T., Overstreet, R. E. and Wang, Y. (2015) ‘Predicting Public Bicycle Adoption Using the Technology Acceptance Model’, *Sustainability* 2015, 7, pp. 14558–14573. doi: 10.3390/su71114558.

Heerink, M. et al. (2010) ‘Assessing acceptance of assistive social agent technology by older adults: The almere model’, *International Journal of Social Robotics*, 2(4), pp. 361–375. doi: 10.1007/s12369-010-0068-5.

Helia, V. N. et al. (2017) ‘Modified technology acceptance model for hospital information system evaluation – a case study’, in *MATEC Web of Conferences*, pp. 1–5. doi: 10.1051/matecconf/201815401101.

Hoeft, K. S. et al. (2015) ‘Using community participation to assess acceptability of “Contra Caries”, a theory-based, promotora-led oral health education program for rural Latino parents: a mixed methods study’, *BMC Oral Health*, 15(103), pp. 1–11. doi: 10.1186/s12903-015-0089-4.

Hong, S. H. and Yu, J. H. (2018) ‘Identification of external variables for the Technology Acceptance Model (TAM) in the assessment of BIM application for mobile devices’, in *IOP Conference Series: Materials Science and Engineering*. Penang, Malaysia, pp. 1–5. doi: 10.1088/1757-899X/401/1/012027.

Hornbæk, K. and Hertzum, M. (2017) ‘Technology Acceptance and User Experience: A Review of the Experiential Component in HCI’, *ACM Transactions on Computer-Human Interaction*, 24(5), pp. 1–30. doi: 10.1145/3127358.

Hossain, M., Leminen, S. and Westerlund, M. (2019) ‘A systematic review of living lab literature’, *Journal of Cleaner Production*. Elsevier Ltd, 213,
Acosta-Salgado, L., Rakotondranaivo A. and Bonjour E

pp. 1-25. doi: 10.1016/j.jclepro.2018.12.257.

Jamil, L. S. (2017) ‘Assessing the behavioural intention of students towards learning management system, through Technology Acceptance Model - case of iraqi universities’, *Journal of Theoretical and Applied Information Technology*, 95(16), pp. 3825-3840.

Jokonda, O. (2015) ‘Validating technology Acceptance Model (TAM) during IT adoption in organizations’, in *IEEE 7th International Conference on Cloud Computing Technology and Science*. Vancouver, British Columbia, Canada, pp. 509–516. doi: 10.1109/CloudCom.2015.56.

Jongchul, O. and Sung-joon, Y. (2014) ‘Telematics and Informatics Validation of Haptic Enabling Technology Acceptance Model (HE-TAM): Integration of IDT and TAM’, *Telematics and Informatics*. Elsevier Ltd, 31, pp. 585–596. doi: 10.1016/j.tele.2014.01.002.

Khan, I. U. et al. (2018) ‘Assessing the Physicians’ Acceptance of E-Prescribing in a Developing Country: An Extension of the UTAUT Model With Moderating Effect of Perceived Organizational Support’, *Journal of Global Information Management*, 26(3), pp. 121–142. doi: 10.4018/JGIM.2018070109.

Al Khasawneh, M. H. and Shuhaiber, A. (2018) ‘Developing and validating a comprehensive model of factors influencing consumer acceptance of SMS advertising: empirical evidence using SEM-PLS’, *International journal Business Information Systems*, 27(3), pp. 298–330. doi: 10.1504/IJBIS.2018.089859.

Kim, H. et al. (2016) ‘Consumer adoption of smart in-store technology: assessing the predictive value of attitude versus beliefs in the technology acceptance model’, *International journal of fashion design, technology and education*, 10(1), pp. 26–36. doi: 10.1080/17543266.2016.1177737.

Kloeckner, F. et al. (2014) ‘Development of a prediction model on the acceptance of electronic laboratory notebooks in academic environments’, *Biomedizinische Technik*, 59(2), pp. 95–102. doi: 10.1515/bmt-2013-0023.

Kowitlawakul, Y. (2011) ‘The technology acceptance model: Predicting nurses’ intention to use telemedicine technology (eICU)’, *CIN - Computers Informatics Nursing*, 29(7), pp. 411–418. doi: 10.1097/NCN.0b013e3181f9dd4a.

Kumar, G. and Shenbagaraman, V. M. (2017) ‘The customers’ perception of mobile banking adoption in Chennai City. An empirical assessment of an extended technology acceptance model’, *International Journal of Business Information Systems*, 26(1), pp. 46–65.

Kung-Teck, W. et al. (2013) ‘Understanding student teachers’ behavioural intention to use technology: Technology Acceptance Model (TAM) validation and testing’, *International Journal of Instruction*, 6(1), pp. 89–
Kwee-meier, S. T., Bützler, J. E. and Schlick, C. (2016) ‘Development and validation of a technology acceptance model for safety-enhancing , wearable locating systems enhancing , wearable locating systems’, *Behaviour & Information Technology*, 35(5), pp. 394–409. doi: 10.1080/0144929X.2016.1141986.

Lacombe, A. and Esselimani, A. (2017) *Livre blanc : Tester, du premier concept au produit fini*. Paris, France. Available at: https://www.leslivresblancs.fr/livre/entreprise/qualite/tester-du-premier-concept-au-produit-fini (Accessed: 16 September 2018).

Lai, A. C. K. et al. (2009) ‘An evaluation model for indoor environmental quality (IEQ) acceptance in residential buildings’, *Energy and Buildings*, 41(9), pp. 930–936. doi: 10.1016/j.enbuild.2009.03.016.

Lai, C. S. K. and Pires, G. (2010) ‘Testing of a Model Evaluating e-Government Portal Acceptance and Satisfaction’, *The Electronic Journal Information Systems Evaluation*, 13(1), pp. 35–46.

Lannoy, A. (2009) ‘L’amélioration continue’, *Option/Bio*, 20(427), pp. 24–26. doi: 10.1016/s0992-5945(09)70299-5.

Lee, Y. et al. (2011) ‘Analysis of fuzzy Decision Making Trial and Evaluation Laboratory on technology acceptance model’, *Expert Systems With Applications*. Elsevier Ltd, 38(12), pp. 14407–14416. doi: 10.1016/j.eswa.2011.04.088.

Lee, Y. C. et al. (2010) ‘Analysis of adopting an integrated decision making trial and evaluation laboratory on a technology acceptance model’, *Expert Systems with Applications*. Elsevier Ltd, 37(2), pp. 1745–1754. doi: 10.1016/j.eswa.2009.07.034.

Lin, C. A. and Kim, T. (2016) ‘Predicting user response to sponsored advertising on social media via the technology acceptance model’, *Computers in Human Behavior*. Elsevier Ltd, 64, pp. 710–718. doi: 10.1016/j.chb.2016.07.027.

Lin, F., Fofanah, S. S. and Liang, D. (2011) ‘Assessing citizen adoption of e-Government initiatives in Gambia: A validation of the technology acceptance model in information systems success’, *Government Information Quarterly*. Elsevier Inc., 28(2), pp. 271–279. doi: 10.1016/j.giq.2010.09.004.

Lindsay, R., Jackson, T. W. and Cooke, L. (2013) ‘Empirical evaluation of a technology acceptance model for mobile policing’, *Police Practice and Research*, 15(5), pp. 419–436. doi: 10.1080/15614263.2013.829602.

Liu, C. F. and Cheng, T. J. (2015) ‘Exploring critical factors influencing physicians’ acceptance of mobile electronic medical records based on the
Acosta-Salgado, L., Rakotondranaivo A. and Bonjour E

dual-factor model: A validation in Taiwan', *BMC Medical Informatics and Decision Making*, 15(1), pp. 1-12. doi: 10.1186/s12911-014-0125-3.

López-Nicolás, C., Molina-Castillo, F. J. and Bouwman, H. (2008) ‘An assessment of advanced mobile services acceptance: Contributions from TAM and diffusion theory models’, *Information and Management*, 45(6), pp. 359–364. doi: 10.1016/j.im.2008.05.001.

Luo, S. T., Su, C. T. and Lee, W. C. (2011) ‘Constructing intelligent model for acceptability evaluation of a product’, *Expert Systems with Applications*. Elsevier Ltd, 38(11), pp. 13702–13710. doi: 10.1016/j.eswa.2011.04.162.

Manimaran, S. and Lakshmi, K. B. (2013) ‘Development of model for assessing the acceptance level of users in rural healthcare system of Tamilnadu, India’, *Technology and Health Care*, 21(5), pp. 479–492. doi: 10.3233/THC-130751.

Marangunić, N. and Granić, A. (2015) ‘Technology acceptance model: a literature review from 1986 to 2013’, *Universal Access in the Information Society*, 14(1), pp. 81–95. doi: 10.1007/s10209-014-0348-1.

Martignoni, R., Stanoevska-Slabeva, K. and Mueller, D. (2008) ‘Evaluation of Future Mobile Services Based on the Technology Acceptance Model’, in *ECIS 2008 Proceedings*. Galway, Ireland, pp. 2–15. Available at: http://aisel.aisnet.org/ecis2008.

Mcpherson, K. E. et al. (2016) ‘Acceptability and Feasibility of Peer Assisted Supervision and Support for Intervention Practitioners: A Q-methodology Evaluation’, *Journal of Child and Family Studies*. Springer US, 25(3), pp. 720–732. doi: 10.1007/s10826-015-0281-9.

Mohamadali, N. A. and Garibaldi, J. (2010) ‘A Novel Evaluation Model of user Acceptance of Software Technology in healthcare sector’, in *Proceedings of 3rd International Conference on Health Informatics*. Valencia, Spain, pp. 392–397.

Momani, A. M. and Jamous, M. M. (2017) ‘The Evolution of Technology Acceptance Theories’, *International Journal of Contemporary Computer Research (IJCCR)*, 1(1), pp. 51–58.

Morris, M. G. and Turner, J. M. (2001) ‘Assessing users’ subjective quality of experience with the world wide web: An exploratory examination of temporal changes in technology acceptance’, *International Journal of Human Computer Studies*, 54(6), pp. 877–901. doi: 10.1006/ijhc.2001.0460.

Munir, A. R. (2014) ‘Extending the Technology Acceptance Model to predict the acceptance of customer toward mobile banking service in Sulawesi selatan’, *International Journal of Economic Research*, 14(4), pp. 1–11.

Nagy, J. T. (2018) ‘Evaluation of Online Video Usage and Learning Satisfaction: An Extension of the Technology Acceptance Model’,
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

*International Review of Research in Open and Distributed Learning,*
19(1), pp. 1–26. doi: 10.19173/irrodl.v19i1.2886.

Nancarrow, B. E., Porter, N. B. and Leviston, Z. (2010) ‘Predicting community acceptability of alternative urban water supply systems: A decision making model’, *Urban Water Journal*, 7(3), pp. 197–210. doi: 10.1080/1573062X.2010.484500.

Napitupulu, D., Kadar, J. A. and Jati, R. K. (2017) ‘Validity Testing of Technology Acceptance Model Based on Factor Analysis Approach’, *Indonesian Journal of Electrical Engineering and Computer Science*, 5(3), pp. 697–704. doi: 10.11591/ijeecs.v5.i3.pp697-704.

Nikou, S. A. and Economides, A. A. (2017) ‘Mobile-Based Assessment: Integrating acceptance and motivational factors into a combined model of Self-Determination Theory and Technology Acceptance’, *Computers in Human Behavior*, 68, pp. 83–95. doi: 10.1016/j.chb.2016.11.020.

OECD and Eurostat (2018) *Oslo Manual 2018: Guidelines for collecting, reporting and using data on innovation*. The Measurement of Scientific, Technological and Innovation Activities. 4th edn. Paris/Eurostat, Luxembourg. doi: 10.1787/9789264304604-en.

Orruño, E. et al. (2011) ‘Evaluation health-care professionals using a modified Technology Acceptance Model’, *Journal of Telemedicine and Telecare*, 17, pp. 303–307. doi: 10.1258/jtt.2011.101101.

Osswald, S. et al. (2012) ‘Predicting Information Technology Usage in the Car: Towards a Car Technology Acceptance Model’, in *Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications*. Portsmouth, NH, USA, pp. 51–58. doi: http://dx.doi.org/10.1145/2390256.2390264.

Palafox, L. and Hashimoto, H. (2010) ‘Human action recognition using 4W1H and particle swarm optimization clustering’, in *Conference Proceeding HSI’2010, 3rd International Conference on Human System Interaction*. Rzeszow, Poland, pp. 369–373. doi: 10.1109/HSI.2010.5514542.

Pardamean, B. and Susanto, M. (2012) ‘Assessing user acceptance toward blog technology using the UTAUT model’, *International Journal of Mathematics and Computers in Simulation*, 6(1), pp. 203–212. doi: 10.1145/2390256.2390264.

Park, S. Y., Nam, M. W. and Cha, S. B. (2012) ‘University students’ behavioral intention to use mobile learning: Evaluating the technology acceptance model’, *British Journal of Educational Technology*, 43(4), pp. 592–605. doi: 10.1111/j.1467-8535.2011.01229.x.

Pattansheti, M. et al. (2016) ‘Development, measurement and validation of an integrated technology readiness acceptance and planned behaviour model for Indian mobile banking industry’, *International Journal of Business*
Acosta-Salgado, L., Rakotondranaivo A. and Bonjour E

*Information Systems*, 22(3), pp. 316-342. doi: 10.1504/IJBIS.2016.076875.

Persico, D., Manca, S. and Pozzi, F. (2014) ‘Adapting the technology acceptance model to evaluate the innovative potential of e-learning systems’, *Computers in Human Behavior*. Elsevier Ltd, 30, pp. 614–622. doi: 10.1016/j.chb.2013.07.045.

Poniatowski, L. H. et al. (2014) ‘Validity and Acceptability of a High-Fidelity Physical Simulation Model for Training of Laparoscopic Pyeloplasty’, *Journal of endourology*, 28(4), pp. 393–398. doi: 10.1089/end.2013.0678.

Rahi, S., Ghanib, M. A. and Alnaser, F. M. (2017) ‘Predicting customer’s intentions to use internet banking: the role of technology acceptance model (TAM) in e-banking’, *Management Science Letters*, 7(11), pp. 513–524. doi: 10.5267/j.msl.2017.8.004.

Ramayasa, I. P. (2015) ‘Evaluation Model of Success and Acceptance of E-Learning’, *Journal of Theoretical and Applied Information Technology*, 82(3), pp. 462–469.

Rana, N. P. et al. (2014) ‘Examining Adoption of Electronic District (e- District) System in Indian Context: A Validation of Extended Technology Acceptance Model’, in *Twentieth Americas Conference on Information Systems, Savannah*. Georgia, USA, pp. 1–11.

Randell, R., Backhouse, M. R. and Nelson, E. A. (2015) ‘Videoconferencing for site initiations in clinical studies: Mixed methods evaluation of usability, acceptability, and impact on recruitment’, *Informatics for Health and Social Care*, 41(4), pp. 1–11. doi: 10.3109/17538157.2015.1064424.

Razmak, J. and Bélanger, C. (2017) ‘Using the technology acceptance model to predict patient attitude towards personal health records in regional communities’, *Information Technology & People*, 31(2), pp. 306–326. doi: 10.1108/ITP-07-2016-0160 Permanent.

Regolini, A. and Jannés-Ober, E. (2013) ‘A Bibliometric Study of Informing Science: The International Journal of an Emerging Transdiscipline’, *Informing Science*, 16(1), pp. 117–130.

Rezaei, M. et al. (2008) ‘Predicting e-learning application in agricultural higher education using technology acceptance model’, *Turkish Online Journal of Distance Education*, 98(1), pp. 85–95.

Rigopoulou, I. D., Chaniotakis, I. E. and Kehagias, J. D. (2017) ‘An extended technology acceptance model for predicting smartphone adoption among young consumers in Greece’, *International Journal of Mobile Communications*, 15(4), pp. 372–387. doi: 10.1504/IJMC.2017.084860.

Rouibah, K. and Abbas, H. (2006) ‘A Modified Technology Acceptance Model for Camera Mobile Phone Adoption: Development and validation’, in
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

17th Australasian Conference on Information Systems. Adelaide, pp. 1-
11.

Salavaara, A. et al. (2009) ‘Acceptance or Appropriation? A Design-Oriented Critique of Technology Acceptance Models’, in Saariluoma, P. and Isomäki, H. (eds) *Future interaction design II*. London, England: Springer, pp. 157–173. doi: 10.1007/978-1-84800-385-9.

Sánchez-Prieto, J. C., Olmos-Migueláñez, S. and García-Peñalvo, F. J. (2016) ‘Informal tools in formal contexts: Development of a model to assess the acceptance of mobile technologies among teachers’, *Computers in Human Behavior*, 55(A), pp. 519–528. doi: 10.1016/j.chb.2015.07.002.

Schneberger, S., Amoroso, D. L. and Durfee, A. (2007) ‘Factors that influence the performance of computer-based assessments: an extension of the technology acceptance model’, *The Journal of Computer Information Systems*, 48(2), pp. 74–90. doi: 10.1080/08874417.2008.11646011.

Schrier, T., Erdem, M. and Brewer, P. (2010) ‘Merging task-technology fit and technology acceptance models to assess guest empowerment technology usage in hotels’, *Journal of Hospitality and Tourism Technology*, 1(3), pp. 201–217. doi: 10.1108/17579881011078340.

Seen, M., Rouse, A. C. and Beaumont, N. (2007) ‘Explaining and predicting information systems acceptance and success: An integrative model’, in *Proceedings of the Fifteenth European Conference on Information Systems, ECIS*. St. Gallen, Switzerland: University of St. Gallen, pp. 1356–1367.

Sek, Y. et al. (2010) ‘Prediction of User Acceptance and Adoption of Smart Phone for Learning with Technology Acceptance Model’, *Journal of Applied Sciences*, 10(20), pp. 2395–2402.

Shah, S. A. M. et al. (2013) ‘Employee Behavior Towards Adoption of E-learning Courses: Validating Technology Acceptance Model’, *Mediterranean Journal of Social Sciences*, 4(14), pp. 765–774. doi: 10.5901/mjss.2013.v4n14p765.

Sharif, M. et al. (2017) ‘Predicting adoption of biological control among Iranian rice farmers: An application of the extended technology acceptance model (TAM2)’, *Crop Protection*. Elsevier Ltd, 96, pp. 88–96. doi: 10.1016/j.cropro.2017.01.014.

Sharma, R. and Mishra, R. (2014) ‘A Review of Evolution of Theories and Models of Technology Adoption’, *Indore Management Journal*, 6(2), p. 44. doi: 10.9790/487X-1810063748.

Shukla, A. and Sharma, S. K. (2018) ‘Evaluating Consumers’ Adoption of Mobile Technology for Grocery Shopping: An Application of Technology Acceptance Model’, *Vision*, 22(2), pp. 185–198. doi: 10.1177/0972262918766136.
Skiba, N. (2014) *Processus d'innovation centré sur l'utilisateur: identification des besoins et interprétation de données issues de l'intégration de l'utilisateur dans le processus de co-conception*. Université de Lorraine.

Sol, R. and Baras, K. (2016) ‘Assessment of Activity Trackers: Toward an Acceptance Model’, in *Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct*. Heidelberg, Germany, pp. 570–575.

Strudwick, G. (2015) ‘Predicting Nurses’ Use of Healthcare Technology Using the Technology Acceptance Model’, *CIN: Computers, Informatics, Nursing*, 33(5), pp. 189–198. doi: 10.1097/CIN.0000000000000142.

Suroso, J. S., Retnowardhani, A. and Fernando, A. (2017) ‘Evaluation Of Knowledge Management System Using Technology Acceptance Model’, in *Proceeding EECSI 2017*. Yogyakarta, Indonesia, pp. 19–21.

Taherdoost, H. (2018) ‘Development of an adoption model to assess user acceptance of e-service technology: E-Service Technology Acceptance Model’, *Behaviour & Information Technology*. Taylor & Francis, 37(2), pp. 173–197. doi: 10.1080/0144929X.2018.1427793.

Tanaka, Y. et al. (2015) ‘Major psychological factors affecting consumer’s acceptance of food additives: Validity of a new psychological model’, *British Food Journal*, 117(11), pp. 2788–2800. doi: 10.1108/BFJ-02-2015-0062.

Tarlani, A. et al. (2015) ‘Towards the Acceptance of RSS to Support Learning: An empirical study to validate the Technology Acceptance Model in Lebanon’, *Electronic Journal of e-Learning*, 13(1), pp. 30–41.

Taylor, S. P. (2017) ‘What Is Innovation? A Study of the Definitions, Academic Models and Applicability of Innovation to an Example of Social Housing in England’, *Open Journal of Social Sciences*, 5, pp. 128–146. doi: 10.4236/jss.2017.511010.

Teo, T. et al. (2009) ‘Assessing the intention to use technology among pre-service teachers in Singapore and Malaysia: A multigroup invariance analysis of the Technology Acceptance Model (TAM)’, *Computers and Education*. Elsevier Ltd, 53(3), pp. 1000–1009. doi: 10.1016/j.compedu.2009.05.017.

Teo, T. (2010) ‘An Empirical Study to Validate the technology Acceptance Model (TAM) in Explaining the Intention to use technology among Educational users’, *International Journal of Information and Communication Technology Education*, 6(4), pp. 1–12. doi: 10.4018/jicte.2010100101.

Tricot, A. et al. (2003) ‘Utilité, utilisabilité, acceptabilité: interpréter les relations entre trois dimensions de l’évaluation des EIAH’, *Environnements Informatiques pour l’Apprentissage Humain*, 53(9), pp. 391–400. doi: 10.1017/CBO9781107415324.004.
*Towards an improved acceptability assessment model of innovative solutions in the
design phase*

Turner, M. et al. (2010) ‘Does the technology acceptance model predict actual use? A systematic literature review’, *Information and Software Technology*. Elsevier B.V., 52(5), pp. 463–479. doi: 10.1016/j.infsof.2009.11.005.

Valacich, J. S., Wang, X. and Jessup, L. N. (2018) ‘Did I Buy the Wrong Gadget? How the Evaluability of and Subsequent Product Choice’, *MIS Quarterly*, 42(2), pp. 633–644.

Venkatesh, V. et al. (2003) ‘User Acceptance of Information Technology: Toward a Unified View’, *MIS Quarterly*, 27(3), pp. 425–478. doi: 10.2307/30036540.

Venkatesh, V. and Bala, H. (2008) ‘Technology acceptance model 3 and a research agenda on interventions’, *Decision Sciences*, 39(2), pp. 273–315. doi: 10.1111/j.1540-5915.2008.00192.x.

Venkatesh, V. and Davis, F. D. (2000) ‘A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies’, *Management Science*, 46(2), pp. 186–204. doi: 10.1287/mnsc.46.2.186.11926.

Wasitarini, D. E. and Tritawirasta, W. (2015) ‘Assessing Users’ Acceptance toward a Closed Access Library Service System Using the UTAUT Model: A Case Study at the National Library of Indonesia’, in *International Conference on Information Technology Systems and Innovation (ICITSI)*. Bandung, Bali, pp. 1–4.

Weberg, D. (2009) ‘Innovation in Healthcare. A concept analysis’, *Nurs admin Q*, 33(3), pp. 227–237. doi: 10.1097/NAQ.0b013e3181accaf5.

Weiss, A. et al. (2008) ‘A methodological variation for acceptance evaluation of human-robot interaction in public places’, in *Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN*. United States, pp. 713–718. doi: 10.1109/ROMAN.2008.4600751.

Weistroffer, V. et al. (2013) ‘A Methodology to Assess the Acceptability of Human-Robot Collaboration Using Virtual Reality’, in *19th ACM Symposium on Virtual Reality Software and Technology*. Singapore, pp. 39–48. doi: 10.1145/2503713.2503726.

Wolk, R. M. (2007) ‘Using the Technology Acceptance Model for Outcomes Assessment in Higher Education’, in *Proceedings of the Information Systems Education Conference, ISECON*. Pittsburgh, pp. 1–16.

Wu, J. et al. (2008) ‘Testing the technology acceptance model for evaluating healthcare professionals ’ intention to use an adverse event reporting system’, *International Journal for Quality in Health Care*, 20(2), pp. 123–129. doi: 10.1093/intqhc/mzm074.
Wu, J. H., Chen, Y. C. and Lin, L. M. (2007) ‘Empirical evaluation of the revised end user computing acceptance model’, *Computers in Human Behavior*, 23(1), pp. 162–174. doi: 10.1016/j.chb.2004.04.003.

Wu, Y. et al. (2016) ‘An Integrated BIM and cost estimating blended learning model – acceptance differences between experts and novice’, *Eurasia Journal of Mathematics, Science & Technology Education*, 12(5), pp. 1347–1363. doi: 10.12973/eurasia.2016.1517a.

Yi-cheng, C. et al. (2007) ‘Predicting College Student ’ Use of E-Learning Systems: an Attempt to Extend Technology Acceptance Model’, in *PACIS 2007 Proceedings*, pp. 172–183.

Yusoff, A., Crowder, R. and Gilbert, L. (2010) ‘Validation of serious games attributes using the technology acceptance model’, in *2nd International Conference on Games and Virtual Worlds for Serious Applications, VS-GAMES 2010*. Braga, Portugal, pp. 45–51. doi: 10.1109/VS-GAMES.2010.7.

Zahid, J. A., Ashraf, M. and Malik, B. T. (2013) ‘An impact assessment of information and communication technology for disabled people through technology acceptance model in Bangladesh’, *International Journal of Services, Technology and Management*, 19(1/2/3), pp. 43–62. doi: 10.1504/IJSTM.2013.054197.

Zhang, D. et al. (2014) ‘4W1H in mobile crowd sensing’, *IEEE Communications Magazine*, 52(8), pp. 42–48. doi: 10.1109/MCOM.2014.6871668.

Zhu, D., Lin, T. C. and Hsu, Y. (2012) ‘Total Quality Management & Business Excellence Using the technology acceptance model to evaluate user attitude and intention of use for online games’, *Total Quality Management & Business Excellence*, 23(7–8), pp. 965–980. doi: 10.1080/14783363.2012.704269.
<table><caption>Appendix 1. List of publications</caption><thead><tr><th rowspan="2">Author, year</th><th rowspan="2">What</th><th colspan="2">Who</th><th rowspan="2">Where</th><th rowspan="2">When</th><th colspan="4">How</th></tr><tr><th>Profile</th><th>#</th><th>Model</th><th>Data collection</th><th>Tool</th><th>Analysis method</th></tr></thead><tbody><tr><td>(Taherdoost, 2018)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Razmak and Bélanger, 2017)</td><td>CTP</td><td>Us</td><td>2</td><td>EC</td><td>BU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Khan et al., 2018)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>UTAUT</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Chen and Lin, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>VC</td><td>BU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Beldad and Hegner, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>VC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Hong and Yu, 2018)</td><td>CTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Shukla and Sharma, 2018)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Helia et al., 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Al Khasawneh and Shuhaiber, 2018)</td><td>CTP</td><td>Us</td><td>1</td><td>VC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Nagy, 2018)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Nikou and Economides, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Kim et al., 2016)</td><td>NCTP</td><td>Us</td><td>1</td><td>VC</td><td>BU</td><td>TAM</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Napitupulu, Kadar and Jati, 2017)</td><td>NCTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Rahi, Ghanib and Alnaser, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Munir, 2014)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Dawson, Mackrill and Cain, 2017)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>ID</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Girgis et al., 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>NS</td><td>Qn/IE</td><td>Nn</td><td>Ql</td></tr><tr><td>(Sharif et al., 2017)</td><td>Pt</td><td>Us</td><td>1</td><td>RC</td><td>BU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Bervell and Umar, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>UTAUT</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Aizstrauta and Ginters, 2017a)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>AM</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Suroso, Retnowardhani and Fernando, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Jamil, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Rigopoulou, Chaniotakis and Kehagias, 2017)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>2M+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Alryalat, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>EC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Kumar and Shenbagaraman, 2017)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>2M+</td><td>Lt/Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Sánchez-Prieto, Olmos-Migueláñez and García-Peñalvo, 2016)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>NS</td></tr><tr><td>(Lin and Kim, 2016)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Kwee-meier, Bützler and Schlick, 2016)</td><td>CTP</td><td>Us</td><td>1</td><td>VC</td><td>DU</td><td>TAM</td><td>Qn/IE</td><td>Em</td><td>Qt</td></tr><tr><td>(Ducey and Coover, 2016)</td><td>NCTP</td><td>Us</td><td>2</td><td>VC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Guarino et al., 2016)</td><td>CTP</td><td>Us</td><td>1</td><td>EC</td><td>DU</td><td>NS</td><td>Qn/IE</td><td>Nn</td><td>Qt</td></tr><tr><td>(Wu et al., 2016)</td><td>CTP</td><td>Us</td><td>2</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Durodolu, 2016)</td><td>CTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>Ql</td></tr><tr><td>(Mcpherson et al., 2016)</td><td>Pt</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>NS</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Jokonda, 2015)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Sol and Baras, 2016)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Besbes et al., 2016)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Arbelaez Garces, Rakotondranaivo and Bonjour, 2016a)</td><td>NCTP</td><td>Us</td><td>3</td><td>NS</td><td>ID</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Pattansheti et al., 2016)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>2M+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Aizstrauta and Ginters, 2017b)</td><td>NCTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>AM</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Randell, Backhouse and Nelson, 2015)</td><td>CTP</td><td>Us</td><td>4</td><td>RC</td><td>DU</td><td>NS</td><td>Qn/IE</td><td>TP</td><td>Ql</td></tr></tbody></table>
Acosta-Salgado, L., Rakotondranaivo A. and Bonjour E

<table><tr><td>(Abu-dalbouh, 2016)</td><td>Pd</td><td>Us</td><td>1</td><td>NS</td><td>DU</td><td>TAM+</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Wasitarini and Tritawirasta, 2015)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>DU</td><td>UTAUT</td><td>Qn/IE/Ob</td><td>Nn</td><td>Qt</td></tr><tr><td>(Hakami, Hussei and Adenuga, 2016)</td><td>CTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Arbelaez Garces, Bonjour and Rakotondranaivo, 2016)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>ID</td><td>UTAUT</td><td>Ob/Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Briz-ponde and García-peñalvo, 2015)</td><td>CTP</td><td>Us</td><td>2</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Strudwick, 2015)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Tahini et al., 2015)</td><td>CTP</td><td>EU</td><td>2</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Hoeft et al., 2015)</td><td>Pt</td><td>Us</td><td>1</td><td>EC</td><td>DU</td><td>NS</td><td>Qn/GE</td><td>Nn</td><td>Ql</td></tr><tr><td>(Liu and Cheng, 2015)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>AM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Hazen, Overstreet and Wang, 2015)</td><td>OG</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Tanaka et al., 2015)</td><td>Pd</td><td>Us</td><td>1</td><td>RC</td><td>BU</td><td>NS</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Aizstrauta and Ginters, 2015)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>AM</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Füssl et al., 2015)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>AM</td><td>Lt/Qn/GE</td><td>Nn</td><td>NS</td></tr><tr><td>(Ramayasa, 2015)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>2M+</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Persico, Manca and Pozzi, 2014)</td><td>CTP</td><td>Us</td><td>3</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>NS</td></tr><tr><td>(Calisir et al., 2014)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Jongchul and Sung-joon, 2014)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>TAM+</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Poniatowski et al., 2014)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>NS</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Lindsay, Jackson and Cooke, 2013)</td><td>CTP</td><td>Ex</td><td>1</td><td>NS</td><td>NS</td><td>TAM+</td><td>IE</td><td>Nn</td><td>Ql</td></tr><tr><td>(Kloeckner et al., 2014)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>NS</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Rana et al., 2014)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Cheung and Vogel, 2013)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Kung-Teck et al., 2013)</td><td>CTP</td><td>Us</td><td>2</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Weistroffer et al., 2013)</td><td>NCTP</td><td>Us</td><td>1</td><td>VC</td><td>ID</td><td>AM</td><td>Qn/PM</td><td>VR</td><td>Qt</td></tr><tr><td>(Ambak, 2013)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn/IE</td><td>Nn</td><td>Qt</td></tr><tr><td>(Aizstrauta et al., 2013)</td><td>NCTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>UTAUT</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Shah et al., 2013)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Manimaran and Lakshmi, 2013)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Zahid, Ashraf and Malik, 2013)</td><td>CTP</td><td>Us</td><td>2</td><td>EC</td><td>DU</td><td>TAM</td><td>Qn/GE</td><td>Nn</td><td>Qt</td></tr><tr><td>(Alotaibi and Wald, 2013)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>ID</td><td>UTAUT</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Park, Nam and Cha, 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Ghazizadeh, Lee and Boyle, 2012)</td><td>NCTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>TAM+</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Gagnon et al., 2012)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Osswald et al., 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>EC</td><td>ID</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Pardamean and Susanto, 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>UTAUT</td><td>Ob/Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Chang et al., 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Zhu, Lin and Hsu, 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Alsofyani and Eynon, 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Chen et al., 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Bahry, Anwar and Amran, 2012)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Lin, Fofanah and Liang, 2011)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Em</td><td>Ql</td></tr><tr><td>(Kowitlawakul, 2011)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>BU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Orruño et al., 2011)</td><td>NCTP</td><td>Us</td><td>3</td><td>VC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Lee et al., 2011)</td><td>Pt</td><td>Ex</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Faqih, 2011)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Luo, Su and Lee, 2011)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>AM</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Dai et al., 2011)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Halbach and Gong, 2011)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr></table>
Towards an improved acceptability assessment model of innovative solutions in the design phase

<table><tr><td>(Alshibly, 2011)</td><td>CTP</td><td>Us</td><td>2</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Turner et al., 2010)</td><td>CTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>TAM</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Heerink et al., 2010)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>AM</td><td>Ob/Qn</td><td>Av</td><td>Qt</td></tr><tr><td>(Gamal Aboelmaged, 2010)</td><td>CTP</td><td>Us</td><td>2</td><td>RC</td><td>BU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Lee et al., 2010)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>BU</td><td>TAM+</td><td>Qn/IE</td><td>Nn</td><td>Qt</td></tr><tr><td>(Yusoff, Crowder and Gilbert, 2010)</td><td>Pt</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Ahmad et al., 2010)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Nancarrow, Porter and Leviston, 2010)</td><td>Sv</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>AM</td><td>Qn</td><td>TP</td><td>Qt</td></tr><tr><td>(Schrier, Erdem and Brewer, 2010)</td><td>CTP</td><td>Us</td><td>1</td><td>VC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Sek et al., 2010)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>BU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Mohamadali and Garibaldi, 2010)</td><td>CTP</td><td>Us</td><td>2</td><td>NS</td><td>NS</td><td>NS</td><td>Qn</td><td>Nn</td><td>NS</td></tr><tr><td>(Teo, 2010)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Alkis and Özcan, 2010)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>NS</td></tr><tr><td>(Bennani and Oumlil, 2010)</td><td>CTP</td><td>Us</td><td>2</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Teo et al., 2009)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>BU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Lai et al., 2009)</td><td>Pd</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>NS</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Calisir, Altin Gumussoy and Bayram, 2009)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>BU</td><td>2M+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Dasgupta et al., 2009)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Lai and Pires, 2010)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(López-Nicolás, Molina-Castillo and Bouwman, 2008)</td><td>CTP</td><td>Us</td><td>2</td><td>VC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Wu et al., 2008)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Weiss et al., 2008)</td><td>NCTP</td><td>Us</td><td>1</td><td>EC</td><td>ID</td><td>UTAUT</td><td>Ob/Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Rezaei et al., 2008)</td><td>CTP</td><td>Us</td><td>2</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Fares et al., 2008)</td><td>NCTP</td><td>Us</td><td>1</td><td>EC</td><td>DU</td><td>NS</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Ham, Kim and Forsythe, 2008)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Martignoni, Stanoevska-Slabeva and Mueller, 2008)</td><td>CTP</td><td>Us</td><td>1</td><td>VC</td><td>ID</td><td>TAM</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(Chen, Fan and Farn, 2007)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Wu, Chen and Lin, 2007)</td><td>NCTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>TAM+</td><td>Lt</td><td>Nn</td><td>Qt</td></tr><tr><td>(Schneberger, Amoroso and Durfee, 2007)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Em</td><td>Qt</td></tr><tr><td>(An et al., 2007)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>NS</td><td>TAM+</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Chan and Teo, 2007)</td><td>CTP</td><td>Us</td><td>1</td><td>NS</td><td>DU</td><td>TAM</td><td>Lt/Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Yi-cheng et al., 2007)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Chooprayoon, Fung and Depickere, 2007)</td><td>CTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>TAM+</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Seen, Rouse and Beaumont, 2007)</td><td>CTP</td><td>Lt</td><td>1</td><td>NS</td><td>NS</td><td>AM</td><td>Lt</td><td>Nn</td><td>NS</td></tr><tr><td>(Wolk, 2007)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Alper et al., 2007)</td><td>Pt</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Al-khateeb, 2007)</td><td>CTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr><tr><td>(Rouibah and Abbas, 2006)</td><td>NCTP</td><td>Us</td><td>1</td><td>RC</td><td>DU</td><td>TAM+</td><td>Qn</td><td>Nn</td><td>Qt</td></tr></table>

**What:** Computer technology products (CTP), Non-computer technology products (NCTP), Practices (Pt), Service (Sv), Organizational (Og), Product (Pd),

**Who:** Users and Future Users (Us), Experts (Ex), Experts/Users (EU), Literature (Lt),

**Where:** Real Context (RC), Virtual/Digital Context (VC), Experimentation Context (EC), Not specified (NS)

**When:** During use (DU), Before use (BU), In design (ID),

**How:** TAM (TAM), Adapted TAM (TAM+), UTAUT (UTAUT), Another model (AM), More than two models (2M+), Questionnaire (Qn), Individual interview (IE), Observation (Ob), Group interview (GE), Physiological measurement (PM), None (Nn), E-mail (Em), Telephone (Tp), Virtual reality (VR), Audiovisual (Av), Quantitative (Qt), Qualitative (Ql),
Acosta-Salgado, L., Rakotondranaivo A. and Bonjour E

## Appendix 2. Results

<table><thead><tr><th></th><th>Acceptability<br>11 (8.7%)</th><th>Acceptance<br>110 (87.3%)</th><th>Acceptance<br>/acceptability<br>5 (4%)</th><th>General<br>126 (100%)</th></tr></thead><tbody><tr><td><strong>What</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Computer technology products (CTP)</td><td>4 (36.4%)</td><td>77 (70.0%)</td><td>3 (60.0%)</td><td>84 (66.7%)</td></tr><tr><td>Non-computer technology products (NCTP)</td><td>5 (45.5%)</td><td>25 (22.7%)</td><td>1 (20.0%)</td><td>31 (24.6%)</td></tr><tr><td>Practices (Pt)</td><td>2 (18.2%)</td><td>4 (3.6%)</td><td>0</td><td>6 (4.8%)</td></tr><tr><td>Service (Sv)</td><td>0</td><td>0</td><td>1 (20.0%)</td><td>1 (0.8%)</td></tr><tr><td>Organizational (Og)</td><td>0</td><td>1 (0.9%)</td><td>0</td><td>1 (0.8%)</td></tr><tr><td>Product (Pd)</td><td>0</td><td>3 (2.7%)</td><td>0</td><td>3 (2.4%)</td></tr><tr><td><strong>Who</strong></td><td></td><td></td><td></td><td></td></tr><tr><td><strong>Profile of actor</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Users and future users (Us)</td><td>11 (100%)</td><td>97 (88.2%)</td><td>5 (100.0%)</td><td>113 (89.7%)</td></tr><tr><td>Literature (Lt)</td><td>0</td><td>10 (9.1%)</td><td>0</td><td>10 (7.9%)</td></tr><tr><td>Experts (Ex)</td><td>0</td><td>2 (1.8%)</td><td>0</td><td>2 (1.6%)</td></tr><tr><td>Experts/Users (EU)</td><td>0</td><td>1 (0.9%)</td><td>0</td><td>1 (0.8%)</td></tr><tr><td><strong>How many types of actors</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>1</td><td>10 (90.9%)</td><td>95 (86.4%)</td><td>4 (80.0%)</td><td>109 (86.5%)</td></tr><tr><td>2</td><td>0</td><td>13 (11.8%)</td><td>0</td><td>13 (10.3%)</td></tr><tr><td>3</td><td>0</td><td>2 (1.8%)</td><td>1 (20.0%)</td><td>3 (2.4%)</td></tr><tr><td>4</td><td>1 (9.1%)</td><td>0</td><td>0</td><td>1 (0.8%)</td></tr><tr><td><strong>Where</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Real Context (RC)</td><td>5 (45.5%)</td><td>67 (60.9%)</td><td>3 (60.0%)</td><td>75 (59.5%)</td></tr><tr><td>Virtual/digital context (VC)</td><td>1 (9.1%)</td><td>10 (9.1%)</td><td>0</td><td>11 (8.7%)</td></tr><tr><td>Experimentation context (EC)</td><td>3 (27.3%)</td><td>5 (4.5%)</td><td>0</td><td>8 (6.3%)</td></tr><tr><td>Not specified (NS)</td><td>2 (18.2%)</td><td>28 (25.5%)</td><td>2 (40.0%)</td><td>32 (25.4%)</td></tr><tr><td><strong>When</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>During use (DU)</td><td>7 (63.6%)</td><td>73 (66.4%)</td><td>3 (60.0%)</td><td>83 (65.9%)</td></tr><tr><td>Before use (BU)</td><td>0</td><td>11 (10.0%)</td><td>0</td><td>11 (8.7%)</td></tr><tr><td>In design (ID)</td><td>2 (18.2%)</td><td>4 (3.6%)</td><td>2</td><td>8 (6.3%)</td></tr><tr><td>Not specified (NS)</td><td>2 (18.2%)</td><td>22 (20.0%)</td><td>0</td><td>24 (19.0%)</td></tr><tr><td><strong>How</strong></td><td></td><td></td><td></td><td></td></tr><tr><td><strong>Theoretical model</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>TAM (TAM)</td><td>0</td><td>44 (40.0%)</td><td>2 (40.0%)</td><td>46 (36.5%)</td></tr><tr><td>Adapted TAM (TAM+)</td><td>0</td><td>45 (40.9%)</td><td>1 (20.0%)</td><td>46 (36.5%)</td></tr><tr><td>UTAUT (UTAUT)</td><td>1 (9.1%)</td><td>6 (5.5%)</td><td>1 (20.0%)</td><td>8 (6.3%)</td></tr><tr><td>Another model (AM)</td><td>3 (27.3%)</td><td>6 (5.5%)</td><td>1 (20.0%)</td><td>10 (7.9%)</td></tr><tr><td>More than two models (2M+)</td><td>0</td><td>5 (4.5%)</td><td>0</td><td>5 (4.0%)</td></tr><tr><td>Not specified (NS)</td><td>7 (63.6%)</td><td>4 (3.6%)</td><td>0</td><td>11 (8.7%)</td></tr><tr><td><strong>Méthode de recueil de données</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Questionnaire (Qn)</td><td>10 (90.9%)</td><td>89 (80.9%)</td><td>5 (100.0%)</td><td>104 (82.5%)</td></tr><tr><td>Individual interview (IE)</td><td>3 (27.3%)</td><td>5 (4.5%)</td><td>0</td><td>8 (6.3%)</td></tr><tr><td>Observation (Ob)</td><td>1 (9.1%)</td><td>4 (3.6%)</td><td>0</td><td>5 (4.0%)</td></tr><tr><td>Group interview (GE)</td><td>2 (18.2%)</td><td>1 (0.9%)</td><td>0</td><td>3 (2.4%)</td></tr><tr><td>Physiological measures (PM)</td><td>1 (9.1%)</td><td>0</td><td>0</td><td>1 (0.8%)</td></tr><tr><td>Literature (Lt)</td><td>2 (18.2%)</td><td>22 (20.0%)</td><td>0</td><td>24 (19.0%)</td></tr><tr><td><strong>Technological tool</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>None (Nn)</td><td>9 (81.8%)</td><td>89 (80.9%)</td><td>4 (80.0%)</td><td>102 (81.0%)</td></tr><tr><td>E-mail (Em)</td><td>0</td><td>20 (18.2%)</td><td>0</td><td>20 (15.9%)</td></tr><tr><td>Telephone (Tp)</td><td>1 (9.1%)</td><td>0</td><td>1 (20.0%)</td><td>2 (1.6%)</td></tr><tr><td>Virtual Reality (VR)</td><td>1 (9.1%)</td><td>0</td><td>0</td><td>1 (0.8%)</td></tr><tr><td>Audiovisual (Av)</td><td>0</td><td>1 (0.9%)</td><td>0</td><td>1 (0.8%)</td></tr><tr><td><strong>Method of analysis</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Quantitative (Qt)</td><td>7 (63.6%)</td><td>91 (82.7%)</td><td>5 (100%)</td><td>103 (81.7%)</td></tr><tr><td>Qualitative (Ql)</td><td>3 (27.3%)</td><td>3 (2.7%)</td><td>0</td><td>6 (4.8%)</td></tr><tr><td>Not specified (NS)</td><td>1 (9.1%)</td><td>16 (14.5%)</td><td>0</td><td>17 (13.5%)</td></tr><tr><td></td><td>11</td><td>110</td><td>5</td><td>126</td></tr></tbody></table>