# Perceptions of infection control professionals toward electronic surveillance software supporting inpatient infections: A mixed methods study

Alexandre Baudet, Marie-Jo Brennstuhl, Julie Lizon, Marie Regad, Nathalie Thilly, Béatrice Demoré, Arnaud Florentin

<figure><img src="image_1.png" /></figure>

## To cite this version:

Alexandre Baudet, Marie-Jo Brennstuhl, Julie Lizon, Marie Regad, Nathalie Thilly, et al. Perceptions of infection control professionals toward electronic surveillance software supporting inpatient infections: A mixed methods study. International Journal of Medical Informatics, 2024, 186, pp.105419. 10.1016/j.ijmedinf.2024.105419. hal-04637537

HAL Id: hal-04637537

https://hal.science/hal-04637537v1

Submitted on 6 Jul 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

# Perceptions of infection control professionals toward electronic surveillance software supporting inpatient infections: A mixed methods study

Alexandre Baudet<sup>a,b,*</sup>, Marie-Jo Brennstuhl<sup>a,c</sup>, Julie Lizon<sup>b</sup>, Marie Regad<sup>a,b</sup>, Nathalie Thilly<sup>a,b</sup>,
Béatrice Demoré<sup>a,b</sup>, Arnaud Florentin<sup>a,b</sup>

<figure><img src="image_4.png" /></figure>

Check for
updates

## ARTICLE INFO

**Keywords:**

Infection prevention and control
Electronic surveillance software
Mixed-methods study

Semistructured interview

Usability

## ABSTRACT

**Background:** Electronic surveillance software (ESS) collects multiple patient data from hospital software to assist infection control professionals in the prevention and control of hospital-associated infections. This study aimed to understand the perceptions of end users (i.e., infection control professionals) and the facilitators and barriers related to a commercial ESS named ZINC and to assess its usability.

**Methods:** A mixed-method research approach was adopted among infection control professionals 10 months after the implementation of commercial ESS in the university hospital of Nancy, France. A qualitative analysis based on individual semistructured interviews was conducted to collect professionals' perceptions of ESS and to understand barriers and facilitators. Qualitative data were systematically coded and thematically analyzed. A quantitative analysis was performed using the System Usability Scale (SUS).

**Results:** Thirteen infection control professionals were included. Qualitative analysis revealed technical, organizational and human barriers to the installation and use stages and five significant facilitators: the relevant design of the ESS, the improvement of infection prevention and control practices, the designation of a champion/superuser among professionals, training, and collaboration with the developer team. Quantitative analysis indicated that the evaluated ESS was a "good" system in terms of perceived ease of use, with an overall median SUS score of 85/100.

**Conclusions:** This study shows the value of ESS to support inpatient infections as perceived by infection control professionals. It reveals barriers and facilitators to the implementation and adoption of ESS. These barriers and facilitators should be considered to facilitate the installation of the software in other hospitals.

## 1. Introduction

Surveillance of hospital-associated infections (HAIs) is fundamental for infection prevention and control (IPC). It provides the epidemiological information needed to measure the level of infectious risk in hospitals, to define the prevention policies to be implemented by hospital IPC teams, to assess the effectiveness of these prevention policies, to disseminate surveillance data to health care professionals, and to improve the quality of care and patient safety [1,2].

Traditional manual methods of HAI surveillance are time-consuming and resource intensive for infection control professionals (ICPs) and

generate data of variable quality [3]. In our hospital, these manual methods consisted in e-mails send by microbiology laboratory to ICPs when a bacterium with epidemic potential or possible HAI was detected. Then ICPs studied the electronic patient records, made a survey with paper questionnaire in the ward and used in-house software to identify other patients in contact with patient infected by XDR bacteria [4]. Due to technology improvement, the methods and practices of ICPs for HAI surveillance have evolved. Electronic surveillance software (ESS) systems have been created to supplement electronic patient records using links from various hospital software to collect multiple types of data, such as clinical, microbiological, therapeutic and administrative data.

<footnote><sup>a</sup> Université de Lorraine, Inserm, INSPIIRE, F-54000 Nancy, France</footnote>

<footnote><sup>b</sup> Université de Lorraine, CHRU-Nancy, F-54000 Nancy, France</footnote>

<footnote><sup>c</sup> Université de Lorraine, UFR Sciences Humaines et Sociales, Metz, France</footnote>

<footnote>* Corresponding author at: Faculté d'Odontologie de Lorraine. 7 avenue de la forêt de Haye, 54500 Vandœuvre-lès-Nancy Cedex, France.
E-mail address: alexandre.baudet@univ-lorraine.fr (A. Baudet).</footnote>
ESS processes several algorithms and performs an automated recognition of hospital infections, data collection, analysis and interpretation. ICPs can be assisted by ESS to detect, report and monitor HAIs [3,5,6].

Compared to traditional manual methods, ESS decreases performance variability, improves data accuracy and reliability, increases the efficiency of ICPs and allows ICPs to focus less on infection detection and more on IPC [7]. The utility and performance of ESS have been highlighted in several studies, and it has been suggested that hospitals should develop and implement ESS to support inpatient infections [5]. ICPs also report that improving information technology solutions, such as ESS, is a higher priority [8]. Although ESS is recommended and commercially available on the market, its adoption is slow [9]. Many hospitals do not have ESS to assist their ICPs in HAI surveillance, and several hospitals have developed in-house ESS rather than employing commercial ESS [3,10,11].

Many studies have focused on technical appropriateness and end-users' experience to understand factors related to the adoption of computerized decision support systems for drug management [12]. However, to the best of our knowledge, no study has investigated end-users' perceptions of the usability of an ESS to support inpatient infections. To carry out studies on the user experience of software is crucial to adapt it to the needs, expectations and specificities of end-users. Hebden suggested that the slow implementation of an ESS was due to human factors and indicated the need for qualitative research on ESS adoption and barriers related to its implementation [9].

This study aimed to understand the perceptions of end-users (i.e., infection control professionals) in relation to a commercial ESS to support inpatient infections that was recently implemented in our hospital. The secondary aims were to assess usability and to compare usability scores according to participants' characteristics (age, gender, position...).

## 2. Methods

This study was part of a larger quasiexperimental study that evaluated the impact of an ESS to support inpatient infections and a computerized decision support system to facilitate antibiotic stewardship in the prescription of antibiotics at the university hospital of Nancy, France (ClinicalTrials.gov Identifier: NCT04976829) [4].

### 2.1. Study design

A mixed-method approach was adopted to qualitatively gather the perceptions of end-users and to quantitatively measure the usability of an ESS using individual semistructured interviews and two structured questionnaires. This study followed the COREQ reporting guidelines (Supplementary Table S1) [13].

### 2.2. Setting and participants

The study was conducted among the ICPs of the university hospital of Nancy (France), a 1500-bed hospital. The ICPs started to use the ESS in October 2022, and the qualitative and quantitative investigations were conducted in July and August 2023 after 10 months of use to give the staff time to adapt to the software.

All ESS end-users were invited to participate in the study by The inclusion criteria consisted of ICPs from medical and paramedical staff who used the ESS for at least one month and who signed an informed consent form. The exclusion criteria consist of IPCs who refused to participate in the study and who did not personally use the ESS (e.g., secretaries, the data manager, and students in externship).

### 2.3. ESS implementation

The ESS named ZINC (Lumed®, Sherbrooke, Québec, Canada) was designed in 2016 and was first installed in Canada in 2019 and in our

hospital in France in 2021. Thus, it has been actively developed for French hospitals since September 2021 through near-weekly meetings of ICPs on our hospital medical staff and members of the Lumed® developer team. Since October 2022, the ESS has been implemented for routine clinical use by ICPs in place of the in-house software they previously used [4].

The ESS monitors the microbiological information of inpatients and facilitates the surveillance of their infection and diseases with epidemic potential, especially multidrug-resistant bacteria and other pathogens, such as *Clotridoides difficile*, COVID-19 and measles. All relevant administrative, clinical and microbiological electronic hospital data are automatically sent to the ESS. It allows for real-time identification and follow-up of inpatients suspected of having a contagious illness or HAI. Moreover, it easily identifies a list of contact patients based on a definition of contact time with the infected patient. Alerts are generated by the ESS for the identification of microorganisms requiring additional precautions (Figs. 1 and 2). The ICPs screen the alerts daily (08:30 to 18:30), except on weekends, when the alerts are treated the next Monday. The ICPs contact the ward in charge of positive or contact patients to inform the ward and double-check the implementation of hygiene precautions and screening tests according to the French guidelines.

### 2.4. Qualitative data: collection and analysis

The qualitative part of the study was based on semistructured face-to-face individual interviews with eligible IPC team members. All interviews were conducted by a researcher (AB) who was a PhD student trained for this qualitative research and who had previously worked with the IPC team. The interviews lasted 12 to 30 min (median: 22 min) and were recorded in a closed hospital office. No one was present other than the participant and the researcher. The interviews were conducted using a semistructured interview guide adapted from Jung et al. [14]. The interview guide consisted of open-ended questions to explore the perceptions of ICPs about the use of the implemented ESS and to understand barriers and facilitators related to its use (Supplementary Table S2). During the interviews, main questions could be rephrased when the participants did not understand the initial question and additional follow-up questions were asked to allow participants to explain and detail their statements (e.g., ‘What do you mean...?’, ‘Can you clarify...?’). Data saturation was reached when no new information that could inform the domains of discussion emerged from the last two interviews (data obtained during the last interviews with a medical staff member and a paramedical staff member did not lead to new themes).

All interviews were recorded using an audio recorder, transcribed verbatim and deidentified. NVivo 11 (QSR International, Burlington, MA) was used to analyze the open-ended questions and to guide the thematic analysis. The analysis of transcripts was subject to a grounded and inductive thematic analysis [15,16]. A preliminary coding scheme was subsequently developed using the interview guide questions after data familiarization with the first few transcripts. Then, line-by-line coding was performed by adding emerging codes. Finally, the coding scheme was refined, and a last cycle of coding was performed. The coding scheme was performed by a researcher (AB) and reviewed by another researcher (MJB).

### 2.5. Quantitative data: collection and analysis

Prior to the interviews, the participants completed a short questionnaire including sociodemographic questions and the French version of the System Usability Scale (F-SUS). The F-SUS is a translated and validated version of the System Usability Scale (SUS), which is one of the most widely used scales to measure the perceived ease of use of interactive systems [17]. The SUS is a ten-item scale developed by Brooke to quickly and easily provide a global view of the user's subjective assessments of usability [18]. The items alternate between positive and
<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Screenshot of the ESS end-user interface. 1) Classification of alerts using icons, allowing users to see the notification easily. 2) Type of alert (microorganism) and start date of the follow-up. 3) Patient identification. 4) Status of associated survey.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Screenshot of a patient overview on the ESS. 1) Patient identification. 2) Type of alert and start date of the follow-up. 3) Current infectious status. 4) Last screening tests. 5) Overview of hospital stays. 6) Type of infection (sample results, associated infectious status and investigation).</figcaption></figure>

negative statements. The respondent states his or her position on a 5-point Likert scale ranging from “1 = Strongly Disagree” to “5 = Strongly Agree”. The global SUS score, ranging from 0 (worse usability) to 100 (best usability), is calculated in three steps: 1) for positively worded items 1, 3, 5, 7 and 9, one point is subtracted from the score given by the respondent; 2) for negatively worded items 2, 4, 6, 8 and 10, the calculation is 5 minus the score given by the respondent; and 3) the 10 recalculated scores are summed and multiplied by 2.5 [19]. The global SUS score is interpreted as indicated by Bangor et al. according to the seven-item adjective rating scale [20,21].

The quantitative data were collected with Microsoft® Excel (Microsoft Corporation, Redmond, WA, USA) and analyzed using RStudio® (RStudio Inc., Boston, MA, USA). The data are described as numbers and percentages for categorical variables and as medians with ranges (min-max) for continuous variables. The association between the SUS score and participants' characteristics were tested using Mann-Whitney *U* test or Kruskal-Wallis test for categorical variables and with Spearman's correlation coefficient for continuous variables. The statistical significance was fixed at *p* < 0.05.

## 2.6. Ethical considerations

This study was granted approval by the ethics committee of the university hospital of Nancy in August 2021 (approval No. 322). The participants were informed about the purpose of the study, and all

participants signed an informed consent form prior to the start of each interview. Participation was voluntary and not compensated.

## 3. Results

### 3.1. Participant characteristics

Table 1 presents the demographic and professional characteristics of all interviewed ICPs. Among the 13 participants, seven were medical staff (M1 to M7) and six were paramedical staff (PM1 to PM6). The participants had a median age of 38 years (range: 25–56), and they had worked as ICPs for an average of 2.5 years (range: 2 months-14 years). Among them, 11 used the ESS every day when they worked in the IPC ward (only the department head physician and the health executive midwife used the ESS less often, each week and less than once a week, respectively).

### 3.2. ICPs' perceived barriers

ESS end-users reported technical, organizational and human barriers to the installation and use stages. Most of the illustrative verbatim text related to these three themes of barriers is summarized in Supplementary Tables S3, S4 and S5.
<table><caption>**Table 1**

Characteristics of the participants (N = 13).</caption><thead><tr><th>Characteristics</th><th>n</th><th>%</th></tr></thead><tbody><tr><td><strong>Sex</strong></td><td></td><td></td></tr><tr><td>Female</td><td>10</td><td>77</td></tr><tr><td>Male</td><td>3</td><td>23</td></tr><tr><td><strong>Age (years)</strong></td><td></td><td></td></tr><tr><td>25-34</td><td>5</td><td>38</td></tr><tr><td>35-44</td><td>4</td><td>31</td></tr><tr><td>≥45</td><td>4</td><td>31</td></tr><tr><td><strong>Years of hospital experience (years)</strong></td><td></td><td></td></tr><tr><td>&lt;5</td><td>3</td><td>23</td></tr><tr><td>5-10</td><td>4</td><td>31</td></tr><tr><td>&gt;10</td><td>6</td><td>46</td></tr><tr><td><strong>Years of experience in IPC ward (years)</strong></td><td></td><td></td></tr><tr><td>&lt;1</td><td>4</td><td>31</td></tr><tr><td>1-5</td><td>6</td><td>46</td></tr><tr><td>&gt;5</td><td>3</td><td>23</td></tr><tr><td><strong>Position</strong></td><td></td><td></td></tr><tr><td><strong>Medical staff</strong></td><td></td><td></td></tr><tr><td>Physician</td><td>2</td><td>15</td></tr><tr><td>Pharmacist</td><td>3</td><td>23</td></tr><tr><td>Resident in medicine</td><td>1</td><td>8</td></tr><tr><td>Resident in pharmacy</td><td>1</td><td>8</td></tr><tr><td><strong>Paramedical staff</strong></td><td></td><td></td></tr><tr><td>Nurse</td><td>4</td><td>31</td></tr><tr><td>Midwife</td><td>1</td><td>8</td></tr><tr><td>Pharmacist technician</td><td>1</td><td>8</td></tr><tr><td><strong>Use of the ESS during working days</strong></td><td></td><td></td></tr><tr><td>Every day</td><td>11</td><td>85</td></tr><tr><td>&lt;1/day and ≥1/week</td><td>1</td><td>8</td></tr><tr><td>&lt;1/week</td><td>1</td><td>8</td></tr><tr><td><strong>Use of electronic tools at home</strong></td><td></td><td></td></tr><tr><td>Every day</td><td>11</td><td>85</td></tr><tr><td>&lt;1/day and ≥1/week</td><td>0</td><td>0</td></tr><tr><td>&lt;1/week</td><td>2</td><td>15</td></tr></tbody></table>

### 3.2.1. Technical barriers

At the time of installation, the two medical staff members in charge of ESS implementation noted technical barriers concerning the data flow interfacing and the information technology equipment. The main barrier was the data flow interfacing from electronic hospital data (medical, microbiological and administrative patient records) to the ESS due to internal and external difficulties. On the one hand, the hospital information technology team had a lack of 'understanding of what we're actually capable of extracting from hospital software and what we're not' (M3). On the other hand, the opening up of new data flows depends on the cooperation of the companies that supply the hospital software: 'Information technology people (...) rely a lot on companies to get things done (...)'. We're very dependent on their willingness to respond' (M3).

ICPs reported four significant technical barriers to use: i) the need to consult the electronic patient record in addition to the ESS to see textual data ('I have to go through the [electronic patient record] because that's where I'll see the clinical elements, the patient's follow-up, and find out whether, for example, the patient is symptomatic or not' (M1)); ii) duplicate work for clinical investigation on the ESS ('The doctor will do his pre-investigation, will note, for example (...) the date of the alert (...)'; I'll fill in my part. These items aren't put in automatically, and we're obliged to put them in. So I think it's a waste of time' (PM6)); iii) inadequate information technology equipment among medical staff ('My hospital computer was crap' (M7)) and paramedical staff ('the aim was to carry out audits directly in the wards, but the equipment isn't suitable. In other words, what we need are tablets, not computers' (PM5)); and iv) the absence of graphics in the ESS interface ('to be able to make our own epidemic curves' (M7)).

### 3.2.2. Organizational barriers

At the time of installation, two organizational barriers were largely discussed: the need for dedicated times and the validation of the data flows. The involvement of people was crucial for the implementation and adoption of the ESS: 'The need, when deploying the software, is to realize that it's going to take dedicated time for (...) the teams involved in deploying it' (M3). For example, the lack of dedicated time for microbiologists on the project generated a cascade of barriers, such as a microbiological data flow that was not validated and consolidated by the microbiology laboratory, which was a limitation to confidence in this data flow ('The software's biggest intrinsic problem remains the lack of validation of biological flows (...)'. This is what somewhat limits my confidence in the software' (M3)) and possible difficulties regarding rare infectious diseases ('Take Ebola, for example (...)'. If one day a prescription is made, I'm not sure we'll be able to recognize it correctly' (M3)).

In use, the main organizational barrier was the workflow transition of the IPC team. The ESS modified IPC activities ('In the past, we had much less investigation to do' (M6)) with important modifications of team practices ('It completely changes the way we work' (M7)) and an adjustment period ('It was a bit of a cacophony at the beginning' (M7)). The workflow transition was complexified by the growing number of alerts requiring prioritization of ICP tasks ('The number of alerts has exploded compared to what we had before. We don't necessarily have the human resources to respond to all alerts, so we've implemented a prioritization rule' (M4)) and the absence of management tools included in the ESS ('to supervise what is done via this software' (M3)).

### 3.2.3. Human barriers

At the time of installation, the main human barrier was the low involvement of some people, which could have compromised all the implementation: 'It's [because of] the [lack of] involvement of a microbiologist in the deployment of the software that we missed out on so many things' (M7)). The lack of involvement could be explained by the lack of dedicated time to the ESS installation (organizational barriers) and the lack of explanation of the ESS, which can scare end-users and generate reluctance: 'I was rather reluctant because (...) I was a bit afraid at first that I wouldn't be able to do it with this software because I hadn't taken the time to understand how it worked' (M4). Furthermore, several members of the paramedical staff indicated poor computer skills: 'I think I have trouble using computers' (PM1).

In use, the main human barrier was the difficulty of treating all the alerts generated by the ESS, and the ICPs reported: 'We have a lot more data to process' (M2) and 'In the past, we had much less investigation to do' (M6). A more anecdotal human barrier was the language barrier with the foreign developer team: 'We had a videoconference with the Canadians (...)'. It's really hard with the accent and the differences in language' (PM5).

### 3.3. ICPs' perceived facilitators

Despite barriers, the ICPs were very enthusiastic about the ESS, especially medical staff members: 'It's a really interesting tool' (M2), 'I'm totally in favor of this type of software' (M4), or even 'It's super well thought-out' (M5).

Five themes concerned facilitators of ESS adoption: i) the ESS was well designed, ii) it improved IPC practices, iii) a champion/superuser was designated among ICPs, iv) training sessions were provided, and v) the developer team was helpful. The most illustrative verbatims related to these facilitator themes are summarized in Supplementary Table S6.

#### 3.3.1. A well-designed ESS

According to the ICPs, the relevant design of the ESS contributed strongly to its adoption. Ease of learning, ease of use and navigation in the ESS interface ('to be able to make our own epidemic curves' (M7)).
were important facilitators for end-users. Most ICPs qualified the ESS as 'intuitive' (8/13); this word was used 14 times during the interviews. Many of the ICPs noted the ease of use ('I find this software easy to use' (M1)) and highlighted that it was smooth to navigate ('I find navigation extremely easy, fluid and intuitive' (M6)). The visual appearance of the software contributed to the ease of use and to its adoption: 'The choice of colors, the choice of graphics; in fact, it's pretty intuitive, and it's nice' (M5).

### 3.3.2. The ESS improved IPC practices

ICPs reported: 'It's a software that has enabled us to improve our practices in terms of infectious risk prevention' (M4). One medical staff member stated: 'In our day-to-day operations, it has enabled us to gain a great deal in terms of responsiveness, investigation, contact with wards and alert management' (M3).

The recently installed commercial ESS was a sign of progress compared to the in-house software previously used by ICPs: '[The ESS] is a database that's updated in real time and doesn't crash. Our [in-house] database had its limitations and wasn't easy to use' (PM5), and 'We had an in-house software that already met certain expectations, but I'd say we've taken it to a higher level with ZINC software' (M4).

The main improvements brought by ESS included the best coverage of microorganisms and HAI surveillance ('We've improved the exhaustiveness of our monitoring of the panel of healthcare-associated infections' (M3)) and the centralization of useful data ('It has enabled us to bring together all the data we need in one place' (M4)). These improvements contributed to saving time ('Now, we have everything in one place, which saves time' (M2)) and helped to increase responsiveness and efficiency ('This allows us to be more reactive' (PM3)). ICPs also declared improvements regarding communication with care wards ('I think it improves the relationship with them because they call and they feel that there's someone who's really dealt with the case' (M1)) and the quality of care ('The consequences this can have on the quality of care (...) I think it has had a very positive impact on our team but also on patients' (M6)).

### 3.3.3. A designated champion/superuser among ICPs

One member of the IPC team was identified as champion/superuser to assist the other ICPs with the workflow transition and collaborated weekly with the developer team to make decisions regarding system design and configuration: '[The superuser] meets with the developer team every Friday. So, she sees the problems and discusses them as they arise' (PM1). This medical champion/superuser was particularly appreciated by paramedical team members to help them with the ESS: 'We would go and see [the champion] for 5 min. She would explain, and then you would understand as you went along' (PM5).

### 3.3.4. Training sessions

To learn to use the ESS, training sessions conducted by both the developer team and the champion/superuser were appreciated: 'You need training sessions and, above all, practice' (M4) and '[The champion] gave us a little training again' (PM3).

### 3.3.5. A helpful developer team

The ICPs highlighted the great importance of the developer team in ESS adoption. They noted that the developer team was attentive, reactive and engaged in the development and improvement of the ESS: 'They want to improve (...) the software' (M5) and 'Improvements are made as and when comments are received' (PM5).

## 3.4. Usability of the ESS

Table 2 presents the usability test results. The overall median score was 85/100 (range = 60–95), indicating a 'good' system in terms of perceived ease of use (Fig. 3). Half of the items obtained an optimal median score (items 1, 2, 3, 6 and 8). The median SUS score was higher for medical staff than for paramedical staff (90 vs. 80, p = 0.002), and it decreased with the age of the end-user (r = -0.7, p = 0.006). The median

<table><caption>**Table 2**
Usability test results.</caption><thead><tr><th>Items rated on a 5-point Likert scale</th><th>Median</th><th>Q1-Q3</th><th>Range</th></tr></thead><tbody><tr><td>I would like to continue using this system frequently</td><td>5</td><td>5–5</td><td>4–5</td></tr><tr><td>I found this system unnecessarily complex</td><td>1</td><td>1–1</td><td>1–3</td></tr><tr><td>I thought this system was easy to use</td><td>5</td><td>4–5</td><td>2–5</td></tr><tr><td>I need the support of a technical person to be able to use this system</td><td>2</td><td>1–3</td><td>1–4</td></tr><tr><td>I found the various functions in this system were well integrated</td><td>4</td><td>4–4</td><td>2–5</td></tr><tr><td>I thought there was too much inconsistency in this system</td><td>1</td><td>1–2</td><td>1–4</td></tr><tr><td>I would imagine that most people would learn to use this system very quickly</td><td>4</td><td>4–5</td><td>2–5</td></tr><tr><td>I found the system very cumbersome to use</td><td>1</td><td>1–2</td><td>1–3</td></tr><tr><td>I felt very confident using the system</td><td>4</td><td>4–5</td><td>2–5</td></tr><tr><td>I needed to learn a lot of things before I could get going with this system</td><td>2</td><td>1–3</td><td>1–5</td></tr></tbody></table>

SUS score was not associated with the gender (p = 0.2), the frequencies of use of the ESS during working days (p = 0.3) or of electronic tools at home (p = 0.4). It was not correlated with years of experience in the hospital (p = 0.3) or in the IPC ward (p = 0.1).

## 4. Discussion

This study employed a mixed-method research approach that identified several significant findings on the barriers and facilitators reported by end-users for an ESS that supported inpatient infections. Barriers included technical, organizational and human difficulties to the installation and use stages. Five main facilitators were identified: the relevant design of the ESS, the improvement of IPC practices, the designation of a champion/superuser among ICPs, training, and collaboration with the developer team. The quantitative analysis, including a usability scale, was in line with the qualitative analysis.

### 4.1. Barriers to ESS adoption

A previous qualitative study on the adoption of computerized decision support systems for prescription drug management identified several barriers: alert fatigue, lack of accuracy, poor user interface design, and lack of customizability [14]. Thus, to improve the adoption of computerized decision support systems, it was recommended to focus the design on usability, ergonomics, workflow integration and transparency of the decision-making process [22]. These points were taken into account by the developer team of ZINC in collaboration with our IPC team (especially with the champion/superuser) to produce a 'good' system according to the SUS median score obtained (85/100). However, other barriers were reported by end-users in our qualitative study, including technical, organizational and human barriers. Yusof et al. identified and developed frameworks for technical, organizational and human factors for health information systems through a review of the literature [23,24].

Technical barriers reported at the time of installation generated delays in the implementation of the commercial ESS, more than one year in our hospital after administrative approval of the project. This delay was long but not surprising for the implementation of this type of software. Indeed, the successful implementation of a new system, such as a computerized decision support system or ESS, can be challenging due to the hospital environment, which is characterized by a complex interaction of systems associated with technological, human and organizational factors [24,25]. Wisniewski et al. reported that the development of a clinical data warehouse for a hospital IPC took two years [26], and Apte et al. reported almost two years to develop and integrate an electronic system studying HAIs [27].

In use, a barrier reported in our study and by Hebden was the
<figure><img src="image_7.png" /><figcaption>Fig. 3. System usability scale (SUS) scores in relation to the adjective ratings.</figcaption></figure>

abundance of data available from the system, which may lead to an expansion of surveillance activities [9]. The workflow adaptation of the IPC team is an unavoidable issue when ICPs switch from traditional manual surveillance of HAIs to an ESS. Thus, tasks and their distribution were modified among medical and paramedical staff members in our study, and prioritization of the workflow was implemented. It is crucial to define and standardize the process to ensure that each end user can easily identify which tasks need to be completed to avoid forgetting to take HAIs into account and to avoid duplication of work [9].

Medical staff members are familiar with the use of hospital software while several paramedical staff members indicated poor computer skills in our study. This lower information technology skills of the paramedical staff compared to the medical staff and older age were barriers to ESS adoption because they generated a significant decrease in the perceived ease of use evaluated by SUS. A previous study has already highlighted that the age significantly affects the SUS score with a negative impact on the usability of the interfaces [20]. No institutional barriers were reported in our study because they occurred after the implementation stage. Before the installation of an ESS, institutional approval should be obtained and may involve severe barriers. Institutional concerns include security, confidentiality and medico-legal problems [26]. Economic considerations with return-on-investment expectations are also determinants of the decision to implement, maintain, or discontinue an ESS [9,28].

## 4.2. Facilitators to ESS adoption

In their systematic review, Gagon et al. highlighted factors that influence the adoption of information and communication technologies in a medical context. The main facilitators identified were perceived usefulness, compatibility, ease of use, participation of end-users in the design, presence of champions/superusers, training, the management of implementation and the impact on clinical uncertainty [29]. Most of these facilitators were also reported during our semistructured interviews, which highlighted five significant facilitator themes: the relevant design of the ESS, the improvement of IPC practices (e.g., it gathered useful information, it saved time, and it improved responsiveness, efficiency, communication with care wards and quality of care), the designation of a champion/superuser among ICPs, training, and collaboration with the developer team. The design of ZINC was praised during the semistructured interviews and was confirmed by the SUS score, especially by the good results on its complexity and cumbersomeness of use.

In our study, the ICPs perceived that the ESS enabled them to save time and improve practices compared with traditional manual surveillance. This was objectively demonstrated in other studies: the ESS produced a mean reduction in time spent to surveillance of 74% [3,30] with a 61% reduction in time spent on surveillance for surgical site infection [31]. Moreover, it provided epidemiological and statistical data that

were priority elements for ICPs [8]. ESS adoption also contributed to the quality of care by increasing appropriate isolation precaution rates for patients with antibiotic-resistant bacteria [32,33]. Moreover, ESS adoption in hospitals was significantly and positively associated with the implementation of evidence-based practices for HAIs (including methicillin-resistant *Staphylococcus aureus* and ventilator-associated pneumonia) and for IPCs (including hand hygiene, contact precautions and surgical care improvement) [34]. Improved practices are enabled by the performance of this type of system. The ESS presents good sensitivity (range: 63 to 91 %) and excellent specificity (range: 87 to >99 %) for laboratory-based surveillance of HAIs [35].

The success of the workflow transition of the IPC team was facilitated by the identification of a champion/superuser to assist the other team members with the ESS and to discuss the design and configuration of the ESS with the developer team. Hebden et al. noted the importance of a superuser and of the assessment of the experience and skill set of each ICP before implementing the ESS to facilitate the workflow transition of the IPC team [9]. This champion should be involved in testing the system, taking on the role of experts and superusers when the system is introduced to lead the project and promoting the use of the new software [29,36].

Finally, ICP end-users reported their enthusiasm about the commercial ESS during their interviews and indicated that they would 'strongly agree' (Q1-Q3 score: 5-5) to continue using this system (item 1 of the SUS). In hospitals in the United States, end-users of IPC software also reported greater satisfaction with the IPC abilities of ESS than electronic patient records and a higher likelihood of continuing to use the same ESS than nonusers [6].

## 4.3. Strengths and weaknesses of the study

To our knowledge, this is the first study to analyze barriers and facilitators of ESS both at installation time and in use in association with the usability of the ESS among IPC end-users. As a limitation, this study investigated the adoption of one ESS named ZINC® by only one IPC team in a university hospital. However, the inclusion of all eligible IPC team members permitted us to reach data saturation in qualitative analysis. The inclusion of these 13 end-users was sufficient for quantitative analysis because Tullis and Stetson showed that the SUS increases to approximately 75 % accuracy at a sample size of 8 and may reach an asymptote of 100 % at a sample size of 12 [37]. Therefore, the main limitation of this monocentric study is its low external validity and the possible lack of generalizability of the results to other hospitals. Nevertheless, many of our results are in line with other studies, demonstrating the possible dissemination of several of our results to another context, especially in other hospitals which plan the installation of this ESS in the future.
# 5. Conclusions

The ESS is a valuable system for supporting inpatient infection surveillance and improving the work of ICPs and the quality of care of patients. Nevertheless, the difficulties in implementing and using new software to collect multiple patient data from various hospital software programs should not be underestimated. The barriers and facilitators highlighted in this study should be considered to facilitate the adoption of software in other hospitals. The main conditions for success are the relevant design of the ESS, the improvement of IPC practices enabled by the software, the time slots dedicated to software implementation, the designation of a champion/superuser among professionals, training, collaboration with the developer team and the hospital's information technology, and the adaptation of the workflow.

# 6. Summary table

What was already known on the topic:

* Traditional manual methods of surveillance for hospital-associated infections are time-consuming and resource intensive for infection control professionals, they are progressively replaced by electronic surveillance software.

* Electronic surveillance software allows a significant reduction in time spent to monitor hospital-associated infections and surgical site infections.

* Adoption of new health information systems in hospital depends on technical, organizational and human factors.

What this study added to our knowledge:

* Electronic surveillance software is a valuable system to support inpatient infections surveillance.

* Infection control professionals are very enthusiastic about the electronic surveillance software because it can improve their work and the quality of care of patients.

* Moving from a manual method of healthcare associated infection surveillance to a computer method with electronic surveillance software requires time, training and adaptation of the workflow.

# Funding

BioMérieux, a partner of Lumed, funded the installation and maintenance of the ESS during the evaluation.

# 8. Ethics approval statement

The study was approved by the Ethics Committee of the University Hospital of Nancy.

## CRediT authorship contribution statement

**Alexandre Baudet:** Writing – review & editing, Writing – original draft, Visualization, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.
**Marie-Jo Brennstuhl:** Writing – review & editing, Methodology, Investigation, Formal analysis, Conceptualization.
**Julie Lizon:** Writing – review & editing, Visualization, Software.
**Marie Regad:** Writing – review & editing, Software.
**Nathalie Thilly:** Writing – review & editing, Methodology.
**Béatrice Demoré:** Writing – review & editing, Supervision, Conceptualization.
**Arnaud Florentin:** Writing – review & editing, Supervision, Software, Project administration, Methodology, Conceptualization.

## Declaration of competing interest

The authors declare that they have no known competing financial

interests or personal relationships that could have appeared to influence the work reported in this paper.

# Acknowledgments

The authors thank BioMérieux and Lumed for the installation of the ESS. They thank all of the interviewees who participated in the study. The sponsor was CHRU de Nancy (Direction de la Recherche et de l'Innovation).

# Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.ijmedinf.2024.105419.

# References

[1] E.M. Cooke, R. Coello, J. Sedgwick, V. Ward, J. Wilson, A. Charlett, et al., A national surveillance scheme for hospital associated infections in England, J. Hosp. Infect. 46 (2000) 1–3, https://doi.org/10.1053/jhin.2000.0801.

[2] R.P. Gaynes, R. Platt, Monitoring patient safety in health care: building the case for surrogate measures, Jt. Comm. J. Qual. Patient Saf. 32 (2006) 95–101, https://doi.org/10.1016/s1553-7250(06)32013-2.

[3] P.L. Russo, R.Z. Shaban, D. Macbeth, A. Carter, B.G. Mitchell, Impact of electronic healthcare-associated infection surveillance software on infection prevention resources: a systematic review of the literature, J. Hosp. Infect. 99 (2018) 1–7, https://doi.org/10.1016/j.jhin.2017.09.002.

[4] A. Baudet, N. Agrinier, A. Charmillon, C. Pulcini, A. Lozniewski, N. Aissa, et al., Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study, BMJ Open 12 (2022) e056125.

[5] R. Freeman, L.S.P. Moore, L. García Álvarez, A. Charlett, A. Holmes, Advances in electronic surveillance for healthcare-associated infections in the 21st Century: a systematic review, J. Hosp. Infect. 84 (2013) 106–119, https://doi.org/10.1016/j.jhin.2012.11.031.

[6] M. Masnick, D.J. Morgan, M.-O. Wright, M.Y. Lin, L. Pineles, A.D. Harris, et al., Survey of infection prevention informatics use and practitioner satisfaction in US hospitals, Infect. Control Hosp. Epidemiol. 35 (2014) 891–893, https://doi.org/10.1086/676870.

[7] J.S. de Bruin, W. Seeling, C. Schuh, Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review, J. Am. Med. Inform. Assoc. JAMIA 21 (2014) 942–951, https://doi.org/10.1136/amiajnl-2013-002089.

[8] B.G. Mitchell, L. Hall, D. MacBeth, A. Gardner, K. Halton, Hospital infection control units: staffing, costs, and priorities, Am. J. Infect. Control 43 (2015) 612–616, https://doi.org/10.1016/j.ajic.2015.02.016.

[9] J.N. Hebden, Slow adoption of automated infection prevention surveillance: are human factors contributing? Am. J. Infect. Control 43 (2015) 559–562, https://doi.org/10.1016/j.ajic.2015.02.007.

[10] P.G. Grota, P.W. Stone, S. Jordan, M. Pogorzelska, E. Larson, Electronic surveillance systems in infection prevention: organizational support, program characteristics, and user satisfaction, Am. J. Infect. Control 38 (2010) 509–514, https://doi.org/10.1016/j.ajic.2009.10.007.

[11] P.L. Russo, A.C. Cheng, M. Richards, N. Graves, L. Hall, Variation in health care-associated infection surveillance practices in Australia, Am. J. Infect. Control 43 (2015) 773–775, https://doi.org/10.1016/j.ajic.2015.02.029.

[12] M. Laka, A. Milazzo, T. Merlin, Factors that impact the adoption of clinical decision support systems (CDSS) for antibiotic management, Int. J. Environ. Res. Public Health 18 (2021) 1901, https://doi.org/10.3390/ijerph18041901.

[13] B.C. O'Brien, I.B. Harris, T.J. Beckman, D.A. Reed, D.A. Cook, Standards for reporting qualitative research: a synthesis of recommendations, Acad. Med. 89 (2014) 1245, https://doi.org/10.1097/ACM.0000000000000388.

[14] S.Y. Jung, H. Hwang, K. Lee, H.-Y. Lee, E. Kim, M. Kim, et al., Barriers and facilitators to implementation of medication decision support systems in electronic medical records: mixed methods approach based on structural equation modeling and qualitative analysis, JMIR Med. Inform. 8 (2020) e18758.

[15] J.M. Corbin, A. Strauss, Grounded theory research: Procedures, canons, and evaluative criteria, Qual. Sociol. 13 (1990) 3–21, https://doi.org/10.1007/BF00988593.

[16] C. Pope, S. Ziebland, N. Mays, Qualitative research in health care. Analysing qualitative data, BMJ 320 (2000) 114–116, https://doi.org/10.1136/bmj.320.7227.114.

[17] G. Gronier, A. Baudet, Psychometric evaluation of the F-SUS: creation and validation of the french version of the system usability scale, Int J Human-Computer Interact 37 (2021) 1571–1582, https://doi.org/10.1080/10447318.2021.1898828.

[18] J. Brooke, SUS - A quick and dirty usability scale, CRC Press, Usability Eval. Ind., 1996.

[19] J.S.U.S. Brooke, A retrospective, J. Usab. Stud. 8 (2013) 29–40.
[20] A. Bangor, P.T. Kortum, J.T. Miller, An empirical evaluation of the system usability scale, Int J Hum-Comput Interact 24 (2008) 574-594, https://doi.org/10.1080/10447310802205776.

[21] A. Bangor, P. Kortum, J. Miller, Determining what individual SUS scores mean: adding an adjective rating scale, JUX - J User Exp 4 (2009) 114-123.

[22] G. Catho, N.S. Centemero, H. Catho, A. Ranzani, C. Balmelli, C. Landelle, et al., Factors determining the adherence to antimicrobial guidelines and the adoption of computerised decision support systems by physicians: A qualitative study in three European hospitals, Int. J. Med. Inf. 141 (2020) 104233, https://doi.org/10.1016/j.ijmedinf.2020.104233.

[23] M.M. Yusof, A. Papazafeiropoulou, R.J. Paul, L.K. Stergioulas, Investigating evaluation frameworks for health information systems, Int. J. Med. Inf. 77 (2008) 377-385, https://doi.org/10.1016/j.ijmedinf.2007.08.004.

[24] M.M. Yusof, J. Kuljis, A. Papazafeiropoulou, L.K. Stergioulas, An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit), Int. J. Med. Inf. 77 (2008) 386-398, https://doi.org/10.1016/j.ijmedinf.2007.08.011.

[25] E. Kilsdonk, L.W. Peute, M.W.M. Jaspers, Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis, Int. J. Med. Inf. 98 (2017) 56-64, https://doi.org/10.1016/j.ijmedinf.2016.12.001.

[26] M.F. Wisniewski, P. Kieszkowski, B.M. Zagorski, W.E. Trick, M. Sommers, R. A. Weinstein, Development of a clinical data warehouse for hospital infection control, J Am Med Inform Assoc JAMIA 10 (2003) 454-462, https://doi.org/10.1197/jamia.M1299.

[27] M. Apte, M. Neidell, E.Y. Furuya, D. Caplan, S. Glied, E. Larson, Using electronically available inpatient hospital data for research, Clin. Transl. Sci. 4 (2011) 338-345, https://doi.org/10.1111/j.1752-8062.2011.00353.x.

[28] J.P. Furuno, M.L. Schweizer, J.C. McGregor, E.N. Perencevich, Economics of infection control surveillance technology: cost-effective or just cost? Am. J. Infect. Control 36 (2008) S12-S17, https://doi.org/10.1016/j.ajic.2007.06.002.

[29] M.-P. Gagnon, M. Desmartis, M. Labrecque, J. Car, C. Pagliari, P. Pluye, et al., Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals, J. Med. Syst. 36 (2012) 241-277, https://doi.org/10.1007/s10916-010-9473-4.

[30] S.C. Keller, D.R. Linkin, N.O. Fishman, E. Lautenbach, Variations in identification of healthcare-associated infections, Infect. Control Hosp. Epidemiol. 34 (2013) 678-686, https://doi.org/10.1086/670999.

[31] A. Chalfine, D. Cauet, W.C. Lin, J. Gonot, N. Calvo-Verjat, F.-E. Dazza, et al., Highly sensitive and efficient computer-assisted system for routine surveillance for surgical site infection, Infect. Control Hosp. Epidemiol. 27 (2006) 794-801, https://doi.org/10.1086/506393.

[32] E. Larson, M. Behta, B. Cohen, H. Jia, E.Y. Furuya, B. Ross, et al., Impact of electronic surveillance on isolation practices, Infect. Control Hosp. Epidemiol. 34 (2013) 694-699, https://doi.org/10.1086/671001.

[33] A.N. Kho, P.R. Dexter, J.S. Warvel, A.W. Belsito, M. Commiskey, S.J. Wilson, et al., An effective computerized reminder for contact isolation of patients colonized or infected with resistant organisms, Int. J. Med. Inf. 77 (2008) 194-198, https://doi.org/10.1016/j.ijmedinf.2007.02.005.

[34] H. Halpin, S.M. Shortell, A. Milstein, M. Vanneman, Hospital adoption of automated surveillance technology and the implementation of infection prevention and control programs, Am. J. Infect. Control 39 (2011) 270-276, https://doi.org/10.1016/j.ajic.2010.10.037.

[35] J. Leal, K.B. Laupland, Validity of electronic surveillance systems: a systematic review, J. Hosp. Infect. 69 (2008) 220-229, https://doi.org/10.1016/j.jhin.2008.04.030.

[36] M.A. Martinez, T. Kind, E. Pezo, K.L. Pomerantz, An evaluation of community health center adoption of online health information, Health Promot. Pract. 9 (2008) 59-67, https://doi.org/10.1177/1524839906294804.

[37] T. Tully, J.N. Stetson, A comparison of questionnaires for assessing website usability, 13th Annual UPA Conference, Minneapolis, Minnesota, USA, 2004.