# Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study

Alexandre Baudet, Nelly Agrinier, Alexandre Charmillon, Céline Pulcini, Alain Lozniewski, Nejla Aissa, Julie Lizon, Nathalie Thilly, Béatrice Demoré, Arnaud Florentin

► To cite this version:

Alexandre Baudet, Nelly Agrinier, Alexandre Charmillon, Céline Pulcini, Alain Lozniewski, et al. Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study. BMJ Open, 2022, 12 (4), pp.e056125. 10.1136/bmjopen-2021-056125. hal-03735973

HAL Id: hal-03735973

https://hal.univ-lorraine.fr/hal-03735973

Submitted on 15 Apr 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study

Alexandre Baudet, Nelly Agrinier, Alexandre Charmillon, Céline Pulcini, Alain Lozniewski, Nejla Aissa, Julie Lizon, Nathalie Thilly, Béatrice Demoré, Arnaud Florentin

<figure><img src="image_1.png" /></figure>

To cite this version:

Alexandre Baudet, Nelly Agrinier, Alexandre Charmillon, Céline Pulcini, Alain Lozniewski, et al. Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study. BMJ Open, 2022, 12 (4), pp.e056125. 10.1136/bmjopen-2021-056125. hal-03735973

HAL Id: hal-03735973

https://hal.univ-lorraine.fr/hal-03735973v1

Submitted on 15 Apr 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# BMJ Open Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study

Alexandre Baudet,¹,² Nelly Agrinier,¹,³ Alexandre Charmillon,⁴ Céline Pulcini,¹,⁴
Alain Lozniewski,⁵,⁶ Nejla Aissa,⁵ Julie Lizon,⁷ Nathalie Thilly,¹,⁸ Béatrice Demoré,¹,⁹
Arnaud Florentin ¹,⁷

**To cite:** Baudet A, Agrinier N, Charmillon A, et al. Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study. *BMJ Open* 2022;12:e056125. doi:10.1136/bmjopen-2021-056125

► Prepublication history for this paper is available online. To view these files, please visit the journal online (http://dx.doi.org/10.1136/bmjopen-2021-056125).

Received 04 August 2021
Accepted 01 March 2022

## ABSTRACT

**Introduction** Antibiotic resistance is one of the most pressing health threats that mankind faces now and in the coming decades. Antibiotic resistance leads to longer hospital stays, higher medical costs and increased mortality. In order to tackle antibiotic resistance, we will implement in our tertiary care university hospital a computerised-decision support system (CDSS) facilitating antibiotic stewardship and an electronic surveillance software (ESS) facilitating infection prevention and control activities. We describe the protocol to evaluate the impact of the CDSS/ESS combination in adult inpatients.

**Methods and analysis** We conduct a pragmatic, prospective, single-centre, before-after uncontrolled study with an interrupted time-series analysis 12 months before and 12 months after the introduction of the CDSS for antibiotic stewardship (APSS) and ESS for infection surveillance (ZINC). APSS and ZINC will assist, respectively, the antibiotic stewardship and the infection prevention and control teams of Nancy University Hospital (France). We will evaluate the impact of the CDSS/ESS on the antibiotic use in adult (≥18 years) inpatients (hospitalised ≥48 hours). The primary outcome is the prescription rate by all healthcare professionals from the hospital of all systemic antibiotics expressed in defined daily doses/1000 patients/ month. Concurrently, we will assess the safety of the intervention, its impact on the appropriateness of antibiotic prescriptions and on additional precautions (isolation precautions) as recommended in guidelines, and on bacterial epidemiology (multidrug-resistant bacteria and *Clostridioides difficile* infections) in the hospital. Finally, we will evaluate the users' satisfaction and the cost of this intervention from the hospital perspective.

**Ethics and dissemination** The protocol has been approved by the Ethics Committee of Nancy University Hospital and registered on the ClinicalTrials platform. Results will be disseminated through conferences' presentations and publications in peer-reviewed journals.

**Trial registration number** NCT04976829.

## Strengths and limitations of this study

► A large data set with monthly data over a 12-month before and 12-month after period will provide a precise estimation of the effects of the intervention.

► A multidisciplinary collaboration approach—with infectious disease physicians, microbiologists, pharmacists, infection prevention and control practitioners and computer scientists—will be reinforced throughout the intervention.

► This would be a first of its kind evaluation of an- tibiotic stewardship and healthcare-associated in- fection computerised-decision support system in France, evaluating its effectiveness, safety, accept- ability and costs.

► The study is limited to only one hospital in France and there is no control group in the interrupted time series quasi-experimental analysis.

## INTRODUCTION

### Background and rationale

Antibiotics are drugs that can prevent and treat bacterial infections. Unfortunately, in response to their use—and accelerated by their misuse—bacteria adapt by developing antibiotic resistance. Without urgent action to change the way healthcare professionals prescribe antibiotics, common infections and minor injuries could once again kill. If we do not find proactive solutions now to slow down the rise of antimicrobial resistance, the impact of drug-resistant infections would lead to 10 million people dying every year by 2050 and would cost up to US$100 trillion across the world.¹

Several studies have estimated the morbidity and mortality of infections due to the most common multidrug-resistant bacteria (MDRB) from the European Antimicrobial

<figure><img src="image_2.png" /></figure>

Check for updates

© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

For numbered affiliations see end of article.

**Correspondence to**
Dr Alexandre Baudet;
alexandre.baudet@univ-lorraine.fr
# BMJ Open Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study

Alexandre Baudet,¹,² Nelly Agrinier,¹,³ Alexandre Charmillon,⁴ Céline Pulcini,¹,⁴
Alain Lozniewski,⁵,⁶ Nejla Aissa,⁵ Julie Lizon,⁷ Nathalie Thilly,¹,⁸ Béatrice Demoré,¹,⁹
Arnaud Florentin ¹,⁷

**To cite:** Baudet A, Agrinier N, Charmillon A, et al. Evaluating antibiotic stewardship and healthcare-associated infections surveillance assisted by computer: protocol for an interrupted time series study. *BMJ Open* 2022;12:e056125. doi:10.1136/bmjopen-2021-056125

► Prepublication history for this paper is available online. To view these files, please visit the journal online (http://dx.doi.org/10.1136/bmjopen-2021-056125).

Received 04 August 2021
Accepted 01 March 2022

## ABSTRACT

**Introduction** Antibiotic resistance is one of the most pressing health threats that mankind faces now and in the coming decades. Antibiotic resistance leads to longer hospital stays, higher medical costs and increased mortality. In order to tackle antibiotic resistance, we will implement in our tertiary care university hospital a computerised-decision support system (CDSS) facilitating antibiotic stewardship and an electronic surveillance software (ESS) facilitating infection prevention and control activities. We describe the protocol to evaluate the impact of the CDSS/ESS combination in adult inpatients.

**Methods and analysis** We conduct a pragmatic, prospective, single-centre, before-after uncontrolled study with an interrupted time-series analysis 12 months before and 12 months after the introduction of the CDSS for antibiotic stewardship (APSS) and ESS for infection surveillance (ZINC). APSS and ZINC will assist, respectively, the antibiotic stewardship and the infection prevention and control teams of Nancy University Hospital (France). We will evaluate the impact of the CDSS/ESS on the antibiotic use in adult (≥18 years) inpatients (hospitalised ≥48 hours). The primary outcome is the prescription rate by all healthcare professionals from the hospital of all systemic antibiotics expressed in defined daily doses/1000 patients/ month. Concurrently, we will assess the safety of the intervention, its impact on the appropriateness of antibiotic prescriptions and on additional precautions (isolation precautions) as recommended in guidelines, and on bacterial epidemiology (multidrug-resistant bacteria and *Clostridioides difficile* infections) in the hospital. Finally, we will evaluate the users' satisfaction and the cost of this intervention from the hospital perspective.

**Ethics and dissemination** The protocol has been approved by the Ethics Committee of Nancy University Hospital and registered on the ClinicalTrials platform. Results will be disseminated through conferences' presentations and publications in peer-reviewed journals.

**Trial registration number** NCT04976829.

## Strengths and limitations of this study

► A large data set with monthly data over a 12-month before and 12-month after period will provide a precise estimation of the effects of the intervention.

► A multidisciplinary collaboration approach—with infectious disease physicians, microbiologists, pharmacists, infection prevention and control practitioners and computer scientists—will be reinforced throughout the intervention.

► This would be a first of its kind evaluation of an- tibiotic stewardship and healthcare-associated in- fection computerised-decision support system in France, evaluating its effectiveness, safety, accept- ability and costs.

► The study is limited to only one hospital in France and there is no control group in the interrupted time series quasi-experimental analysis.

## INTRODUCTION

### Background and rationale

Antibiotics are drugs that can prevent and treat bacterial infections. Unfortunately, in response to their use—and accelerated by their misuse—bacteria adapt by developing antibiotic resistance. Without urgent action to change the way healthcare professionals prescribe antibiotics, common infections and minor injuries could once again kill. If we do not find proactive solutions now to slow down the rise of antimicrobial resistance, the impact of drug-resistant infections would lead to 10 million people dying every year by 2050 and would cost up to US$100 trillion across the world.¹

Several studies have estimated the morbidity and mortality of infections due to the most common multidrug-resistant bacteria (MDRB) from the European Antimicrobial

<figure><img src="image_3.png" /></figure>

Check for updates

© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

For numbered affiliations see end of article.

**Correspondence to**
Dr Alexandre Baudet;
alexandre.baudet@univ-lorraine.fr
<figure><img src="image_3.png" /></figure>

Resistance Surveillance Network data. In Europe, burden assessments of infections caused in humans by eight MDRB revealed the increase of antibiotic resistance over the last years. Between 2007 and 2015, the estimated attributable yearly deaths to these infections increased from 25 000 to 33 110.<sup>2,3</sup> For example, percentage of *Escherichia coli* from invasive isolates with resistance to third-generation cephalosporins increased significatively from 14.6% in 2015 to 15.1% in 2018, and 58.3% of *E. coli* isolated in 2018 were resistant to at least one of the anti-biotic groups.<sup>4</sup> In France, the number of infections and deaths due to MDRB was estimated at respectively 124 806 and 5543 in 2015.<sup>3</sup> Antibiotic resistance has also increased for other bacteria than *E. coli*. For example, percentage of *Klebsiella pneumoniae* from invasive isolates with resistance to third-generation cephalosporins increased from 4.1% to 28.8% between 2005 and 2017.<sup>5</sup> The development of antibiotic resistance is correlated with antibiotic use, so it is crucial that antibiotics are only used when indicated and that the most appropriate antibiotic regimen is used.<sup>6</sup> In 2018, France had the fourth-highest human consumption of antibacterials in Europe: 25.3 defined daily doses (DDD) per 1000 inhabitants per day were consumed in the community and hospital sector.<sup>7</sup>

The antimicrobial resistance global action plan of WHO has five strategic objectives including the strengthening of surveillance and research, the reduction of infection incidence through hygiene and infection prevention and control (IPC) measures and the optimisation of the use of antimicrobial medicines.<sup>8</sup> Antimicrobial stewardship (AMS) programmes are defined as ‘a coherent set of actions which promote using antimicrobials responsibly’. They are encouraged to improve and measure the appropriate use of antibiotics by promoting the selection of the optimal drug regimen including dosing, duration of therapy and routes of administration.<sup>9</sup> In the hospital setting, the emergence of electronic medical records and other electronic patient data has facilitated the use of computerised decision support systems (CDSS) for AMS<sup>10</sup> and electronic surveillance software (ESS) for healthcare-associated infections (HCAI) surveillance.<sup>11</sup> CDSS and ESS match electronic patient data—such as clinical, microbiological, pharmaceutical and administrative patient records—with computerised knowledge base (rules based on expert opinion and clinical guidelines) for the purpose of organising and presenting the appropriate and updated information to users.<sup>10 11</sup> Healthcare workers who use CDSS may then make clinical decisions with reduced error and increased accuracy.<sup>10</sup> CDSS targeting prescribers are recommended<sup>12</sup>: they appear beneficial for improving the appropriateness of prescriptions and for reducing the average hospital length of stay and the antimicrobial spending.<sup>10 13</sup> Implementation of IPC measures is also essential, and has been associated with a decrease in MDRB.<sup>14</sup> ESS for HCAI are automated methods which enable continuous and hospital-wide surveillance for the purpose of presenting the appropriate information to IPC team who may then focus less

on infection detection and more on infection prevention.<sup>11 15</sup> ESS maximises the effectiveness of IPC activities<sup>15</sup> and reduces the amount of time that IPC teams spend on surveillance activities.<sup>16</sup> The implementation, in 2022, of a CDSS supporting AMS with an ESS supporting HCAI surveillance (CDSS/ESS) in our hospital is an opportunity to evaluate the CDSS/ESS impact. On the one hand, several studies had evaluated the effectiveness of CDSS in decision-making of physicians on the prescription of antibiotics.<sup>17</sup> On the other hand, the studies evaluating ESS were mostly either retrospective and prospective cohorts.<sup>16</sup> But, to the best of our knowledge, no study evaluated simultaneously the CDSS/ESS effectiveness, safety, acceptability and impact on costs using a quasi-experimental design study. Quasi-experimental studies are recommended to evaluate AMS interventions.<sup>18 19</sup>

## Objectives

The aim of this study is to evaluate the impact of the CDSS/ESS on the overall use of antibiotics (DDDs/1000 patients/month) prescribed to adult inpatients.

The secondary objectives are to assess: (I) the safety of the CDSS/ESS use on (i) all-cause intrahospital mortality, (ii) length of stay and (iii) the incidence of nosocomial epidemics and of the most frequent HCAI (*E. coli*, *Staphylococcus aureus*, *Enterococcus faecalis*, *Pseudomonas aeruginosa* and *K. pneumoniae*); (II) the impact of the CDSS/ESS on (i) the use of specific antibiotics and therapeutic classes with potential for high selective pressure, (ii) the appropriateness of prescriptions of antibiotics and of additional precautions as recommended in guidelines and (iii) the bacterial epidemiology (MDRB and *Clostridioides difficile* infections); (III) the costs associated with the implementation and the use of the CDSS/ESS, as well as cost savings due to potential antibiotic use decrease and length of stay reduction, from the hospital perspective; and (IV) the users' satisfaction and acceptability of the CDSS/ESS.

## METHODS AND ANALYSIS

### Study design

This quasi-experimental study is a pragmatic, single centre, interventional, before-after uncontrolled study using interrupted time series (ITS) analyses. The study will cover a retrospective period of 12 months before and a prospective period of 12 months after the CDSS/ESS implementation.

### Setting and participants

Nancy University Hospital (France) is a 1497-bed tertiary-care hospital including two main geographical sites. In 2019, it counted 9600 employees (including 1514 full-time equivalents (FTE) for healthcare professionals and interns) and 162 624 hospital stays.<sup>20</sup> An AMS team is present—including infectious disease physicians (1.2 FTE), clinical pharmacists (1.2 FTE) and clinical microbiologists (0.1 FTE)—as well as an IPC team including IPC physicians and pharmacists (2.0 FTE), and nurses
<figure><img src="image_4.png" /></figure>

Resistance Surveillance Network data. In Europe, burden assessments of infections caused in humans by eight MDRB revealed the increase of antibiotic resistance over the last years. Between 2007 and 2015, the estimated attributable yearly deaths to these infections increased from 25 000 to 33 110.²,³ For example, percentage of *Escherichia coli* from invasive isolates with resistance to third-generation cephalosporins increased significatively from 14.6% in 2015 to 15.1% in 2018, and 58.3% of *E. coli* isolated in 2018 were resistant to at least one of the anti-biotic groups.⁴ In France, the number of infections and deaths due to MDRB was estimated at respectively 124 806 and 5543 in 2015.³ Antibiotic resistance has also increased for other bacteria than *E. coli*. For example, percentage of *Klebsiella pneumoniae* from invasive isolates with resistance to third-generation cephalosporins increased from 4.1% to 28.8% between 2005 and 2017.⁵ The development of antibiotic resistance is correlated with antibiotic use, so it is crucial that antibiotics are only used when indicated and that the most appropriate antibiotic regimen is used.⁶ In 2018, France had the fourth-highest human consumption of antibacterials in Europe: 25.3 defined daily doses (DDD) per 1000 inhabitants per day were consumed in the community and hospital sector.⁷

The antimicrobial resistance global action plan of WHO has five strategic objectives including the strengthening of surveillance and research, the reduction of infection incidence through hygiene and infection prevention and control (IPC) measures and the optimisation of the use of antimicrobial medicines.⁸ Antimicrobial stewardship (AMS) programmes are defined as ‘a coherent set of actions which promote using antimicrobials responsibly’. They are encouraged to improve and measure the appropriate use of antibiotics by promoting the selection of the optimal drug regimen including dosing, duration of therapy and routes of administration.⁹ In the hospital setting, the emergence of electronic medical records and other electronic patient data has facilitated the use of computerised decision support systems (CDSS) for AMS¹⁰ and electronic surveillance software (ESS) for healthcare-associated infections (HCAI) surveillance.¹¹ CDSS and ESS match electronic patient data—such as clinical, microbiological, pharmaceutical and administrative patient records—with computerised knowledge base (rules based on expert opinion and clinical guidelines) for the purpose of organising and presenting the appropriate and updated information to users.¹⁰ ¹¹ Healthcare workers who use CDSS may then make clinical decisions with reduced error and increased accuracy.¹⁰ CDSS targeting prescribers are recommended¹²: they appear beneficial for improving the appropriateness of prescriptions and for reducing the average hospital length of stay and the antimicrobial spending.¹⁰ ¹³ Implementation of IPC measures is also essential, and has been associated with a decrease in MDRB.¹⁴ ESS for HCAI are automated methods which enable continuous and hospital-wide surveillance for the purpose of presenting the appropriate information to IPC team who may then focus less

on infection detection and more on infection prevention.¹¹ ¹⁵ ESS maximises the effectiveness of IPC activities¹⁵ and reduces the amount of time that IPC teams spend on surveillance activities.¹⁶ The implementation, in 2022, of a CDSS supporting AMS with an ESS supporting HCAI surveillance (CDSS/ESS) in our hospital is an opportunity to evaluate the CDSS/ESS impact. On the one hand, several studies had evaluated the effectiveness of CDSS in decision-making of physicians on the prescription of antibiotics.¹⁷ On the other hand, the studies evaluating ESS were mostly either retrospective and prospective cohorts.¹⁶ But, to the best of our knowledge, no study evaluated simultaneously the CDSS/ESS effectiveness, safety, acceptability and impact on costs using a quasi-experimental design study. Quasi-experimental studies are recommended to evaluate AMS interventions.¹⁸ ¹⁹

## Objectives

The aim of this study is to evaluate the impact of the CDSS/ESS on the overall use of antibiotics (DDDs/1000 patients/month) prescribed to adult inpatients.

The secondary objectives are to assess: (I) the safety of the CDSS/ESS use on (i) all-cause intrahospital mortality, (ii) length of stay and (iii) the incidence of nosocomial epidemics and of the most frequent HCAI (*E. coli*, *Staphylococcus aureus*, *Enterococcus faecalis*, *Pseudomonas aeruginosa* and *K. pneumoniae*); (II) the impact of the CDSS/ESS on (i) the use of specific antibiotics and therapeutic classes with potential for high selective pressure, (ii) the appropriateness of prescriptions of antibiotics and of additional precautions as recommended in guidelines and (iii) the bacterial epidemiology (MDRB and *Clostridioides difficile* infections); (III) the costs associated with the implementation and the use of the CDSS/ESS, as well as cost savings due to potential antibiotic use decrease and length of stay reduction, from the hospital perspective; and (IV) the users' satisfaction and acceptability of the CDSS/ESS.

## METHODS AND ANALYSIS

### Study design

This quasi-experimental study is a pragmatic, single centre, interventional, before-after uncontrolled study using interrupted time series (ITS) analyses. The study will cover a retrospective period of 12 months before and a prospective period of 12 months after the CDSS/ESS implementation.

### Setting and participants

Nancy University Hospital (France) is a 1497-bed tertiary-care hospital including two main geographical sites. In 2019, it counted 9600 employees (including 1514 full-time equivalents (FTE) for healthcare professionals and interns) and 162 624 hospital stays.²⁰ An AMS team is present—including infectious disease physicians (1.2 FTE), clinical pharmacists (1.2 FTE) and clinical microbiologists (0.1 FTE)—as well as an IPC team including IPC physicians and pharmacists (2.0 FTE), and nurses
(4.5 FTE). The University Hospital of Nancy meets all the seven Core Elements of Hospital Antibiotic Stewardship Programs developed by the US Centers for Disease Control and Prevention²¹ and meets the essential healthcare facility core elements for AMS programmes developed by the WHO.²² It owns an advanced IPC level according to the IPC assessment framework²³ which support the WHO guidelines on core components of IPC programmes.²⁴ In 2017, the point prevalence on one single day of patients treated with antibiotics was 24.4% in the University Hospital of Nancy versus 21.5% (18.9%–24.4%) in comparable French University Hospitals. In the same nationwide point-prevalence survey, the prevalence of HCAI was 7.8% in the University Hospital of Nancy versus 7.0% (6.0%–7.9%) in comparable French University Hospitals. In this nationwide point-prevalence survey, 25 University and Regional Hospitals were randomly select in France (including the University Hospital of Nancy) and constituted a representative sample.²⁵ Total antibiotic consumption was 591 DDD/1000 hospital days in 2017 in the French University Hospital of Nancy and the median consumption was 518 DDD/1000 patients in comparable French University Hospitals.²⁶ The five microorganisms responsible for over 55% of HCAI are: *E. coli*, *S. aureus*, *E. faecalis*, *P. aeruginosa* and *K. pneumoniae*.²⁵

The study includes the hospital departments of medicine (intensive care, cardiology, hepato-gastroenterology, haematology, pneumology, infectious diseases, neurology, ...), surgery, long-term care and rehabilitation care. The paediatric and gynaeco-obstetric activities are excluded.

To assess the impact on antibiotic use and intrahospital mortality, length of stay, incidence of HCAI, all inpatients ≥18 years and hospitalised ≥48 hours in Nancy University Hospital will be considered in the study. To assess the impact of the CDSS/ESS on the appropriateness of antibiotic prescriptions and of additional IPC precautions, inpatients will be randomly selected from the study adult population receiving respectively antibiotic and additional precautions prescriptions before and after the CDSS/ESS implementation (due to the wide number of patients receiving antibiotic and additional precautions prescriptions, a random sample will be used to allow the analysis of the patient files by AMS and IPC teams). To assess the impact of the CDSS/ESS on bacterial epidemiology, patients will be randomly selected from identification of bacterial isolates in the study population. To assess the hospital costs due to antibiotic use, all the study adult population treated by antibiotics will be considered. Finally, to assess the users' satisfaction and acceptability, AMS and IPC teams will be interviewed.

## Study organisation and patient involvement

This study is promoted and coordinated by the University Hospital of Nancy. A scientific committee is in charge of supervising all scientific aspects and organisational issues occurring during the study period. This committee is multidisciplinary, including one pharmacist (BD), two infectious diseases specialists (AC and CP), two clinical

microbiologists (AL and NAi), two IPC practitioners (one MD (AF) and one PharmD (JL)) and two epidemiologists and public health specialists (NAg and NT). This evaluation is coordinated by both a PhD student (AB) and a project manager. The committee will meet regularly: at least one meeting before the study starts to define the protocol, at least two meetings during the study period to discuss potential organisational issues and at least one meeting after the end of the study period to present and discuss the results.

Patients were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

## Intervention

The intervention consists of the implementation of a CDSS supporting AMS and an ESS supporting HCAI surveillance, in 2022, to help respectively AMS and IPC teams. These teams will continue their usual practices but with the assistance of the CDSS/ESS. All inpatients will potentially benefit from the intervention, there is no random allocation nor control group.

## AMS team activities and implementation of the CDSS supporting AMS

The AMS team aims at reducing inappropriate and unnecessary antibiotic prescribing. The AMS programme in the University Hospital of Nancy was described in 2011 by Bevilacqua et al²⁷ and evaluated in two studies.²⁸ ²⁹ Currently, the AMS team reviews antibiotic prescriptions without a specific software: hospital pharmacists analyse all electronic antibiotic prescriptions one by one using the electronic patient's medical record. Given the number of patients hospitalised each day, this prescription analysis is time consuming for pharmacists, and only a limited number of antibiotic prescriptions are analysed at D0, D3 and D7 of therapy with an infectious disease physician (mainly last-line and broad-spectrum antibiotics)³⁰. Since 2017, two infectious disease physicians of the AMS team conduct regularly (at least weekly) ward rounds in several targeted units in the hospital; they also call physicians managing the patients to discuss difficult cases with them and advise on a therapeutic strategy and regular follow-up. The microbiologists send in real-time emails to the infectious disease physicians of the AMS team to inform them of positive bacterial blood cultures. Infectious disease physicians also meet daily with microbiologists in the microbiological department—except during the weekends—to discuss bacterial identification and antibiotic susceptibility tests of positive blood cultures and other relevant samples.

The implementation of the CDSS supporting AMS—named APSS (Lumed, Sherbrooke, Québec, Canada)—will help the AMS team in its daily activities. APSS is an antimicrobial stewardship system which monitors the relevant clinical information of inpatients to facilitate the evaluation of the appropriateness of antibiotic prescriptions by AMS team members (age, gender, respiratory rate, temperature, body mass index, systolic and diastolic
(4.5 FTE). The University Hospital of Nancy meets all the seven Core Elements of Hospital Antibiotic Stewardship Programs developed by the US Centers for Disease Control and Prevention²¹ and meets the essential healthcare facility core elements for AMS programmes developed by the WHO.²² It owns an advanced IPC level according to the IPC assessment framework²³ which support the WHO guidelines on core components of IPC programmes.²⁴ In 2017, the point prevalence on one single day of patients treated with antibiotics was 24.4% in the University Hospital of Nancy versus 21.5% (18.9–24.4%) in comparable French University Hospitals. In the same nationwide point-prevalence survey, the prevalence of HCAI was 7.8% in the University Hospital of Nancy versus 7.0% (6.0%–7.9%) in comparable French University Hospitals. In this nationwide point-prevalence survey, 25 University and Regional Hospitals were randomly select in France (including the University Hospital of Nancy) and constituted a representative sample.²⁵ Total antibiotic consumption was 591 DDD/1000 hospital days in 2017 in the French University Hospital of Nancy and the median consumption was 518 DDD/1000 patients in comparable French University Hospitals.²⁶ The five microorganisms responsible for over 55% of HCAI are: *E. coli*, *S. aureus*, *E. faecalis*, *P. aeruginosa* and *K. pneumoniae*.²⁵

The study includes the hospital departments of medicine (intensive care, cardiology, hepato-gastroenterology, haematology, pneumology, infectious diseases, neurology, ...), surgery, long-term care and rehabilitation care. The paediatric and gynaeco-obstetric activities are excluded.

To assess the impact on antibiotic use and intrahospital mortality, length of stay, incidence of HCAI, all inpatients ≥18 years and hospitalised ≥48 hours in Nancy University Hospital will be considered in the study. To assess the impact of the CDSS/ESS on the appropriateness of antibiotic prescriptions and of additional IPC precautions, inpatients will be randomly selected from the study adult population receiving respectively antibiotic and additional precautions prescriptions before and after the CDSS/ESS implementation (due to the wide number of patients receiving antibiotic and additional precautions prescriptions, a random sample will be used to allow the analysis of the patient files by AMS and IPC teams). To assess the impact of the CDSS/ESS on bacterial epidemiology, patients will be randomly selected from identification of bacterial isolates in the study population. To assess the hospital costs due to antibiotic use, all the study adult population treated by antibiotics will be considered. Finally, to assess the users' satisfaction and acceptability, AMS and IPC teams will be interviewed.

## Study organisation and patient involvement

This study is promoted and coordinated by the University Hospital of Nancy. A scientific committee is in charge of supervising all scientific aspects and organisational issues occurring during the study period. This committee is multidisciplinary, including one pharmacist (BD), two infectious diseases specialists (AC and CP), two clinical

microbiologists (AL and NAi), two IPC practitioners (one MD (AF) and one PharmD (JL)) and two epidemiologists and public health specialists (NAg and NT). This evaluation is coordinated by both a PhD student (AB) and a project manager. The committee will meet regularly: at least one meeting before the study starts to define the protocol, at least two meetings during the study period to discuss potential organisational issues and at least one meeting after the end of the study period to present and discuss the results.

Patients were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

## Intervention

The intervention consists of the implementation of a CDSS supporting AMS and an ESS supporting HCAI surveillance, in 2022, to help respectively AMS and IPC teams. These teams will continue their usual practices but with the assistance of the CDSS/ESS. All inpatients will potentially benefit from the intervention, there is no random allocation nor control group.

## AMS team activities and implementation of the CDSS supporting AMS

The AMS team aims at reducing inappropriate and unnecessary antibiotic prescribing. The AMS programme in the University Hospital of Nancy was described in 2011 by Bevilacqua et al²⁷ and evaluated in two studies.²⁸ ²⁹ Currently, the AMS team reviews antibiotic prescriptions without a specific software: hospital pharmacists analyse all electronic antibiotic prescriptions one by one using the electronic patient's medical record. Given the number of patients hospitalised each day, this prescription analysis is time consuming for pharmacists, and only a limited number of antibiotic prescriptions are analysed at D0, D3 and D7 of therapy with an infectious disease physician (mainly last-line and broad-spectrum antibiotics)³⁰. Since 2017, two infectious disease physicians of the AMS team conduct regularly (at least weekly) ward rounds in several targeted units in the hospital; they also call physicians managing the patients to discuss difficult cases with them and advise on a therapeutic strategy and regular follow-up. The microbiologists send in real-time emails to the infectious disease physicians of the AMS team to inform them of positive bacterial blood cultures. Infectious disease physicians also meet daily with microbiologists in the microbiological department—except during the weekends—to discuss bacterial identification and antibiotic susceptibility tests of positive blood cultures and other relevant samples.

The implementation of the CDSS supporting AMS—named APSS (Lumed, Sherbrooke, Québec, Canada)—will help the AMS team in its daily activities. APSS is an antimicrobial stewardship system which monitors the relevant clinical information of inpatients to facilitate the evaluation of the appropriateness of antibiotic prescriptions by AMS team members (age, gender, respiratory rate, temperature, body mass index, systolic and diastolic
<figure><img src="image_4.png" /></figure>

blood pressures, cardiac frequency, absolute neutro-
phil count and white blood cell count as indicators of
immune system status, creatinine clearance as an indi-
cator of renal function, ...) and to assist them in the post
prescription review process. As new information becomes
available, it automatically checks in real time if the
ongoing treatment seems appropriate or not according
to predefined rules in its knowledge base based on the
local ‘Antibioguide’ guidelines.³¹ The knowledge base
contains rules of contraindications related to drug–drug
interactions, drug bug or drug-laboratory mismatches,
cheaper alternatives, maximum daily dose, maximum
and minimum dose and frequency, maximum duration
and route of administration. APSS generate automatic
alerts sent to the AMS team when a prescription seems
inappropriate. These alerts are prioritised by APSS using
rules which generate a score to target patients who are
most in need of post prescription review. The approach
used by APSS to review antimicrobial prescriptions was
previously described by Beaudoin *et al*.³² The AMS team
will screen these alerts daily (08:30 to 18:30)—except
during the weekends, where the alerts will be treated the
next Monday—according to their prioritised levels. If the
number of alerts is too high, the lowest levels of alert
will not be reviewed. The AMS team will accept or refuse
the alerts based on clinical relevance, after a review of
the electronic medical record if needed. For each alert,
the AMS team will indicate in APSS whether the alert is
over-ridden or considered clinically relevant or irrele-
vant. If the alert is found to be relevant, the AMS team
will contact prescribers to discuss their prescriptions and
advise them on potential improvements. Prescribers will
remain free to change their prescriptions or not.

IPC team activities and implementation of the ESS supporting HCAI

The IPC team is a team including mainly hygiene-
specialised physicians, pharmacists and nurses which
aim at improving the standard and additional IPC
precautions, and to reduce the HCAI rate, including
MDRB infections. Currently, the IPC team achieves its
mission without a specific software by receiving email
alerts concerning MDRB and epidemic micro-organisms
identified by the microbiological department. Then,
the IPC team checks the electronic medical record of
these patients and looks for contact patients (ie, patients
managed in the same hospital ward and during the same
period of time as the infected patient) using a software
developed by the IPC team. The IPC team searches for
the location of each contact patient—one-by-one—by
reviewing their electronic medical records. Finally, the
IPC team informs healthcare professionals managing the
patients to discuss the implementation of hygiene precau-
tions and screening tests.

The ESS supporting HCAI surveillance—named ZINC (Lumed, Sherbrooke, Québec, Canada)—will be used by the IPC team. ZINC is an ESS which monitors the microbiological information of inpatients and facilitates the surveillance of HCAI and diseases with epidemic

potential (all MDRB including extensively drug-resistant
and pandrug-resistant bacteria, and other bacteria and
viruses with epidemic potential like *C. difficile* and measles
for example). All relevant clinical and microbiological
electronic hospital data are automatically sent to ZINC.
ZINC allows for real-time identification and follow-up
of patients suspect of having a HCAI. Moreover, it easily
identifies the list of contact patients based on a definition
of contact time with the infected patient. Finally, it recog-
nises all MDRB from the results of the antibiotic suscepti-
bility testing results.³³ Alerts will be generated by ZINC for
all identification of micro-organisms requiring additional
precautions. The IPC team will screen the alerts daily
(08:30 to 18:30)—except during the weekends, where
the alerts will be treated the next Monday. The IPC team
will contact the physician in charge of positive or contact
patients to inform him/her and double-check the imple-
mentation of hygiene precautions and/or the screening
tests according to French guidelines.³⁴⁻³⁶

Outcomes measures, data sources

The primary outcome is the prescription rate by all healthcare professionals from the Nancy University Hospital of all systemic antibiotics (J01 code according to the Anatomical Therapeutic Chemical—2017 classification) expressed in DDDs/1000 patients/month, calculated monthly during the 12-month before and 12-month after period. These data are routinely collected through the hospital pharmacy database.

The secondary outcomes—presented in table 1—will
be collected monthly during the whole 24-month study
period, except the proportion of prescriptions compliant
with guidelines (evaluated twice during the study period),
the users' satisfaction and the costs of the implementa-
tion of the CDSS/ESS which will be evaluated respectively
after 6 and 12 months of CDSS/ESS implementation.
Concerning the assessment of the users' satisfaction, a
qualitative study based on individual and semi-directive
interviews will be performed by an investigator (AB) with
all the members of AMS and IPC teams who will use the
software.

The intervention is planned for 2022. Data for the
before period will be collected retrospectively, while data
for the after period may be collected prospectively.

Confounding factors, such as other antibiotic steward-
ship or IPC interventions during the study period will be
collected and taken into account in the statistical anal-
ysis. Changes in the configuration of the CDSS/ESS, their
reasons and dates of occurrence will also be collected.

The number of hospitalisation episodes and of inpatients receiving antibiotics will be collected—with inpatients' gender and age—to describe the studied population. To describe the AMS team activity, the number of reviewed antibiotic prescriptions, notably concerning last-line and broad-spectrum antibiotics will be collected. The number of recommendations made by AMS and IPC teams following an APSS or ZINC alert will be reported, as well as the type of AMS's intervention (dosing adjustment,
blood pressures, cardiac frequency, absolute neutro-
phil count and white blood cell count as indicators of
immune system status, creatinine clearance as an indi-
cator of renal function, ...) and to assist them in the post
prescription review process. As new information becomes
available, it automatically checks in real time if the
ongoing treatment seems appropriate or not according
to predefined rules in its knowledge base based on the
local ‘Antibioguide’ guidelines.³¹ The knowledge base
contains rules of contraindications related to drug–drug
interactions, drug bug or drug-laboratory mismatches,
cheaper alternatives, maximum daily dose, maximum
and minimum dose and frequency, maximum duration
and route of administration. APSS generate automatic
alerts sent to the AMS team when a prescription seems
inappropriate. These alerts are prioritised by APSS using
rules which generate a score to target patients who are
most in need of post prescription review. The approach
used by APSS to review antimicrobial prescriptions was
previously described by Beaudoin *et al*.³² The AMS team
will screen these alerts daily (08:30 to 18:30)—except
during the weekends, where the alerts will be treated the
next Monday—according to their prioritised levels. If the
number of alerts is too high, the lowest levels of alert
will not be reviewed. The AMS team will accept or refuse
the alerts based on clinical relevance, after a review of
the electronic medical record if needed. For each alert,
the AMS team will indicate in APSS whether the alert is
over-ridden or considered clinically relevant or irrele-
vant. If the alert is found to be relevant, the AMS team
will contact prescribers to discuss their prescriptions and
advise them on potential improvements. Prescribers will
remain free to change their prescriptions or not.

IPC team activities and implementation of the ESS supporting HCAI

The IPC team is a team including mainly hygiene-
specialised physicians, pharmacists and nurses which
aim at improving the standard and additional IPC
precautions, and to reduce the HCAI rate, including
MDRB infections. Currently, the IPC team achieves its
mission without a specific software by receiving email
alerts concerning MDRB and epidemic micro-organisms
identified by the microbiological department. Then,
the IPC team checks the electronic medical record of
these patients and looks for contact patients (ie, patients
managed in the same hospital ward and during the same
period of time as the infected patient) using a software
developed by the IPC team. The IPC team searches for
the location of each contact patient—one-by-one—by
reviewing their electronic medical records. Finally, the
IPC team informs healthcare professionals managing the
patients to discuss the implementation of hygiene precau-
tions and screening tests.

The ESS supporting HCAI surveillance—named ZINC
(Lumed, Sherbrooke, Québec, Canada)—will be used
by the IPC team. ZINC is an ESS which monitors the
microbiological information of inpatients and facilitates
the surveillance of HCAI and diseases with epidemic

potential (all MDRB including extensively drug-resistant
and pandrug-resistant bacteria, and other bacteria and
viruses with epidemic potential like *C. difficile* and measles
for example). All relevant clinical and microbiological
electronic hospital data are automatically sent to ZINC.
ZINC allows for real-time identification and follow-up
of patients suspect of having a HCAI. Moreover, it easily
identifies the list of contact patients based on a definition
of contact time with the infected patient. Finally, it recog-
nises all MDRB from the results of the antibiotic suscepti-
bility testing results.³³ Alerts will be generated by ZINC for
all identification of micro-organisms requiring additional
precautions. The IPC team will screen the alerts daily
(08:30 to 18:30)—except during the weekends, where
the alerts will be treated the next Monday. The IPC team
will contact the physician in charge of positive or contact
patients to inform him/her and double-check the imple-
mentation of hygiene precautions and/or the screening
tests according to French guidelines.³⁴⁻³⁶

Outcomes measures, data sources

The primary outcome is the prescription rate by all
healthcare professionals from the Nancy University
Hospital of all systemic antibiotics (J01 code according
to the Anatomical Therapeutic Chemical—2017 classifi-
cation) expressed in DDDs/1000 patients/month, calcu-
lated monthly during the 12-month before and 12-month
after period. These data are routinely collected through
the hospital pharmacy database.

The secondary outcomes—presented in table 1—will
be collected monthly during the whole 24-month study
period, except the proportion of prescriptions compliant
with guidelines (evaluated twice during the study period),
the users' satisfaction and the costs of the implementa-
tion of the CDSS/ESS which will be evaluated respectively
after 6 and 12 months of CDSS/ESS implementation.
Concerning the assessment of the users' satisfaction, a
qualitative study based on individual and semi-directive
interviews will be performed by an investigator (AB) with
all the members of AMS and IPC teams who will use the
software.

The intervention is planned for 2022. Data for the
before period will be collected retrospectively, while data
for the after period may be collected prospectively.

Confounding factors, such as other antibiotic steward-
ship or IPC interventions during the study period will be
collected and taken into account in the statistical anal-
ysis. Changes in the configuration of the CDSS/ESS, their
reasons and dates of occurrence will also be collected.

The number of hospitalisation episodes and of inpatients receiving antibiotics will be collected—with inpatients' gender and age—to describe the studied population. To describe the AMS team activity, the number of reviewed antibiotic prescriptions, notably concerning last-line and broad-spectrum antibiotics will be collected. The number of recommendations made by AMS and IPC teams following an APSS or ZINC alert will be reported, as well as the type of AMS's intervention (dosing adjustment,
<table><caption>**Table 1** Description of secondary outcomes</caption><thead><tr><th>Outcome</th><th>Outcome measures</th><th>Measured by</th><th>Population</th><th>Data source</th></tr></thead><tbody><tr><td rowspan="3">Safety</td><td>All-cause intrahospital mortality rate.</td><td>All causes of death/1000 inpatients/month.</td><td>All adult inpatients.</td><td>Medical records department.</td></tr><tr><td>Average length of stay.</td><td>Total length of stay for each inpatient/number of stays/month.</td><td>All adult inpatients.</td><td>Medical records department.</td></tr><tr><td>Incidence of nosocomial epidemics and of the most frequent HCAI.*</td><td>New nosocomial epidemics and new cases of HCAI/ number of inpatients/month.</td><td>All adult inpatients.</td><td>IPC team.</td></tr><tr><td rowspan="5">Impact</td><td>Use of overall antibiotics and by therapeutic classes.</td><td>DDDs/1000 patients/month.</td><td>All adult inpatients.</td><td>Hospital pharmacy.</td></tr><tr><td>Proportion of the antibiotic prescriptions compliant with guidelines.</td><td>Number of antibiotic prescriptions compliant with guidelines/number of antibiotic prescriptions evaluated by AMS team.</td><td>Sample of adult inpatients receiving antibiotics.</td><td>Medical files sampled by the medical records department. Medical files reviewed by the members of the AMS team.</td></tr><tr><td>Proportion of the additional precaution prescriptions compliant with hospital recommendations.</td><td>Number of additional precaution prescriptions compliant with hospital recommendations/number of additional precaution prescriptions evaluated by IPC team.</td><td>Sample of adult inpatients required additional precautions.</td><td>Medical files sampled by the medical records department. Medical files examined by the members of the IPC team.</td></tr><tr><td>Proportion of Clostridioides difficile.</td><td>Number of stays with C. difficile infections/number of stays/month.</td><td>All adult inpatients.</td><td>ZINC (microbiological data).</td></tr><tr><td>Proportion of MDRB.</td><td>Number of stays with MDRB identification by bacterial sample done after 48 hours of hospitalisation/number of stays with identification of bacterial isolates/month.</td><td>All adult inpatients with identification of bacterial isolates.</td><td>ZINC (microbiological data).</td></tr><tr><td rowspan="3">Hospital costs</td><td>Costs of antibiotics.</td><td>Costs (in €) of all oral and intravenous antibiotics delivered by hospital pharmacy/month.</td><td>All adult inpatients treated by antibiotics.</td><td>Hospital pharmacy.</td></tr><tr><td>Average cost of hospital stays.</td><td>Costs (in €) of hospital stays for each inpatient/number of stays/month.</td><td>All adult inpatients treated by antibiotics.</td><td>Financial department.</td></tr><tr><td>Costs of the implementation of the CDSS/ESS and of the purchased equipment.</td><td>Costs (in €) of installation and maintenance of the CDSS/ESS after 12 months of implementation.</td><td>/</td><td>Information technology service.</td></tr><tr><td rowspan="3">Users' satisfaction and acceptability</td><td>Satisfaction.</td><td>Qualitative study after 6 months of use.</td><td>All members of AMS and IPC teams.</td><td>Individual semi-structured interviews.</td></tr><tr><td>Proportion of APSS' alerts accepted by the AMS team.</td><td>Number of APSS' alerts accepted by the AMS team/ number of alerts generated by APSS/month.</td><td>All alerts generated by APSS.</td><td>APSS.</td></tr><tr><td>Proportion of AMS team's recommendations accepted by prescribers.</td><td>Number of AMS team's recommendations accepted by prescribers/number of recommendations given by the AMS team to prescribers/month.</td><td>All recommendations made by the AMS team.</td><td>APSS.</td></tr></tbody></table>

switch from intravenous to oral therapy, immediate discontinuation of treatment, planned end of treatment, adjustment based on antimicrobial serum levels or adjustment according to microbiology results for example).

## Statistical analysis

### Power and sample size calculations

Based on the study conducted by Nault *et al*,¹³ we estimate that the effect size (expected intervention effect over its SE) of the CDSS supporting AMS intervention on our primary outcome will be ≥2 for both level and trend changes. According to the simulation-based power calculation for designing ITS of Zhang *et al*⁸⁷—with both level and trend changes assuming effect size ≥2, equal pre-intervention and post-intervention time periods and statistical significance level=0.05—a 24-time periods study is sufficient to obtain a power >0.9. The time period of 1 month is currently used in ITS.³⁸ A 12-month before and 12-month after intervention period allows the adequate evaluation of seasonal variations.³⁹

Due to potential COVID-19 impact on several outcomes (it had notably affected the use of some antibiotics in several French hospitals⁴⁰), a preliminary study regarding COVID-19 impact on the antibiotic use will be performed in our hospital to take into account the possible variation during the different waves of COVID-19. The results of this preliminary study will be used to improve the statistical analysis and to discuss the local results.

To increase the power study and to assess the long-term sustainability, an extended study which will cover a 24-month before and 24-month after period will be performed at a later stage using the same outcomes.

## Descriptive analysis

Inpatient characteristics, prescriptions and other descriptive data will be described as numbers, percentages and CIs for categorical variables and as means and SD or as medians and IQRs—depending on the distribution—for continuous variables.

<footnote>*The most frequent healthcare associated infections (HCAI) are infections acquired by patients more than 48 hours after their admission to the hospital (and not present or incubated at admission) and caused by *Escherichia coli*, *Staphylococcus aureus*, *Enterococcus faecalis*, *Pseudomonas aeruginosa* and *Klebsiella pneumoniae*.</footnote>

<footnote>AMS, antimicrobial stewardship; CDSS, computerised decision support systems; DDD, defined daily dose; ESS, electronic surveillance software; IPC, infection prevention and control; MDRB, multidrug-resistant bacteria.</footnote>
<table><caption>Table 1 Description of secondary outcomes</caption><thead><tr><th>Outcome</th><th>Outcome measures</th><th>Measured by</th><th>Population</th><th>Data source</th></tr></thead><tbody><tr><td rowspan="3">Safety</td><td>All-cause intrahospital mortality rate.</td><td>All causes of death/1000 inpatients/month.</td><td>All adult inpatients.</td><td>Medical records department.</td></tr><tr><td>Average length of stay.</td><td>Total length of stay for each inpatient/number of stays/month.</td><td>All adult inpatients.</td><td>Medical records department.</td></tr><tr><td>Incidence of nosocomial epidemics and of the most frequent HCAI.*</td><td>New nosocomial epidemics and new cases of HCAI/ number of inpatients/month.</td><td>All adult inpatients.</td><td>IPC team.</td></tr><tr><td rowspan="5">Impact</td><td>Use of overall antibiotics and by therapeutic classes.</td><td>DDDs/1000 patients/month.</td><td>All adult inpatients.</td><td>Hospital pharmacy.</td></tr><tr><td>Proportion of the antibiotic prescriptions compliant with guidelines.</td><td>Number of antibiotic prescriptions compliant with guidelines/number of antibiotic prescriptions evaluated by AMS team.</td><td>Sample of adult inpatients receiving antibiotics.</td><td>Medical files sampled by the medical records department. Medical files reviewed by the members of the AMS team.</td></tr><tr><td>Proportion of the additional precaution prescriptions compliant with hospital recommendations.</td><td>Number of additional precaution prescriptions compliant with hospital recommendations/number of additional precaution prescriptions evaluated by IPC team.</td><td>Sample of adult inpatients required additional precautions.</td><td>Medical files sampled by the medical records department. Medical files examined by the members of the IPC team.</td></tr><tr><td>Proportion of Clostridioides difficile.</td><td>Number of stays with C. difficile infections/number of stays/month.</td><td>All adult inpatients.</td><td>ZINC (microbiological data).</td></tr><tr><td>Proportion of MDRB.</td><td>Number of stays with MDRB identification by bacterial sample done after 48 hours of hospitalisation/number of stays with identification of bacterial isolates/month.</td><td>All adult inpatients with identification of bacterial isolates.</td><td>ZINC (microbiological data).</td></tr><tr><td rowspan="3">Hospital costs</td><td>Costs of antibiotics.</td><td>Costs (in €) of all oral and intravenous antibiotics delivered by hospital pharmacy/month.</td><td>All adult inpatients treated by antibiotics.</td><td>Hospital pharmacy.</td></tr><tr><td>Average cost of hospital stays.</td><td>Costs (in €) of hospital stays for each inpatient/number of stays/month.</td><td>All adult inpatients treated by antibiotics.</td><td>Financial department.</td></tr><tr><td>Costs of the implementation of the CDSS/ESS and of the purchased equipment.</td><td>Costs (in €) of installation and maintenance of the CDSS/ESS after 12 months of implementation.</td><td>/</td><td>Information technology service.</td></tr><tr><td rowspan="3">Users' satisfaction and acceptability</td><td>Satisfaction.</td><td>Qualitative study after 6 months of use.</td><td>All members of AMS and IPC teams.</td><td>Individual semi-structured interviews.</td></tr><tr><td>Proportion of APSS' alerts accepted by the AMS team.</td><td>Number of APSS' alerts accepted by the AMS team/ number of alerts generated by APSS/month.</td><td>All alerts generated by APSS.</td><td>APSS.</td></tr><tr><td>Proportion of AMS team's recommendations accepted by prescribers.</td><td>Number of AMS team's recommendations accepted by prescribers/number of recommendations given by the AMS team to prescribers/month.</td><td>All recommendations made by the AMS team.</td><td>APSS.</td></tr></tbody></table>

switch from intravenous to oral therapy, immediate discontinuation of treatment, planned end of treatment, adjustment based on antimicrobial serum levels or adjustment according to microbiology results for example).

## Statistical analysis

### Power and sample size calculations

Based on the study conducted by Nault *et al*,¹³ we estimate that the effect size (expected intervention effect over its SE) of the CDSS supporting AMS intervention on our primary outcome will be ≥2 for both level and trend changes. According to the simulation-based power calculation for designing ITS of Zhang *et al*⁸⁷—with both level and trend changes assuming effect size ≥2, equal pre-intervention and post-intervention time periods and statistical significance level=0.05—a 24-time periods study is sufficient to obtain a power >0.9. The time period of 1 month is currently used in ITS.³⁸ A 12-month before and 12-month after intervention period allows the adequate evaluation of seasonal variations.³⁹

Due to potential COVID-19 impact on several outcomes (it had notably affected the use of some antibiotics in several French hospitals⁴⁰), a preliminary study regarding COVID-19 impact on the antibiotic use will be performed in our hospital to take into account the possible variation during the different waves of COVID-19. The results of this preliminary study will be used to improve the statistical analysis and to discuss the local results.

To increase the power study and to assess the long-term sustainability, an extended study which will cover a 24-month before and 24-month after period will be performed at a later stage using the same outcomes.

## Descriptive analysis

Inpatient characteristics, prescriptions and other descriptive data will be described as numbers, percentages and CIs for categorical variables and as means and SD or as medians and IQRs—depending on the distribution—for continuous variables.

<footnote>*The most frequent healthcare associated infections (HCAI) are infections acquired by patients more than 48 hours after their admission to the hospital (and not present or incubated at admission) and caused by *Escherichia coli*, *Staphylococcus aureus*, *Enterococcus faecalis*, *Pseudomonas aeruginosa* and *Klebsiella pneumoniae*.</footnote>

<footnote>AMS, antimicrobial stewardship; CDSS, computerised decision support systems; DDD, defined daily dose; ESS, electronic surveillance software; IPC, infection prevention and control; MDRB, multidrug-resistant bacteria.</footnote>
Qualitative analysis

The interviews will be fully transcribed. Then a duplicate
analysis of the data will be performed by two investigators
using an inductive method derived from the grounded
theory (a qualitative research method focused on the
identification of concepts that emerge from study inter-
views or observation<sup>41</sup>). NVivo software (QSR Interna-
tional, Melbourne, Australia) will be used to facilitate the
coding process of the transcripts.

Interrupted time series analysis

Assessment of the CDSS/ESS impact will be realised with
ITS analysis. ITS analysis is the best approach for quasi-
experimental studies assessing longitudinal effects of
interventions introduced at a specific time point where a
randomised trial is infeasible or unethical.<sup>39 42 43</sup> Before—
after quasi-experimental studies using ITS analysis is the
most common study design used for evaluation of anti-
biotic stewardship interventions for hospital inpatients.<sup>44</sup>
ITS analysis is also recommended to evaluate the effects
of IPC interventions according to Effective Practice and
Organisation of Care and ORION guidelines (guide-
lines for transparent reporting of Outbreak Reports and
Intervention studies Of Nosocomial infection).<sup>45 46</sup> In this
study, we comply with ITS guidance<sup>38 47</sup> and recommenda-
tions to evaluate the impact of AMS interventions with two
limits: the study is limited to a single centre and there is
no control group.<sup>18</sup> This study will not include an internal
control group for practical and ethical reasons. Indeed,
the implementation of the CDSS will need a change in
practice to support routine work of the AMS team who
will not continue its current way of operation in a control
group for efficiency reasons. Moreover, the implementa-
tion of the ESS will include all hospital wards for ethical
reasons because it should allow a better prevention and
management of the HCAI which can result in serious
illness, prolong hospital stay, drain healthcare resources
and may result in loss of life. Moreover, the use of an
external control group (like another hospital) seems not
appropriated because, in ITS, the control group should
be exposed to any such co-interventions or events that
might also affect the intervention group.<sup>48</sup> This is not
possible with another hospital as control.

The segmented regression analysis will be used to eval-
uate the longitudinal impact of the CDSS/ESS imple-
mentation on the outcomes of interest. Segmented
regression is used to measure statistically the changes in
level (immediate change in the rate of the outcome) and
slope (changes in the trend) during the before period
as compared with the after period.<sup>38 39 42</sup> Segmented
regression analysis is a powerful statistical method for
estimating intervention effects in ITS studies.<sup>39 49</sup> Before
this analysis, we will graph the mean/percentage and
SD/CI scores for all outcomes presented as monthly data
to determine the structure and trend of the data. We
will test for the presence of the autocorrelation (extent
to which the data are dependent on each other) using
the Durbin-Watson statistic<sup>50 51</sup> and, if necessary, we will

adjust for autocorrelation and seasonality. The transi-
tion period—during which the CDSS/ESS will be imple-
mented, and the healthcare users will be trained—will
not be included in the analysis. Such a lag period is
currently applied in ITS study.⁵² Statistical analysis will be
conducted using RStudio (RStudio, Boston, Massachu-
setts, USA)—an integrated development environment
developed for R.

Author affiliations

**Acknowledgements** The authors are grateful to Stéphanie Bonnay who is the project manager and coordinate this study. The sponsor is CHRU de Nancy (Direction de la Recherche et de l'Innovation).

**Contributors** AB, AF, NAg and NT significantly contributed to designing the study. AB, CP and NT drafted the manuscript. AF and JL worked on the implementation of the ESS supporting HCAI. AC, AL, NAi and BD worked on the implementation of the CDSS supporting AMS. AC, AF, AL, BD, CP, JL, NAg, NAi and NT are members of the scientific committee in charge of the study supervision. AB is responsible for data set preparation and analysis. All authors have read and approved the final manuscript.

**Funding** BioMérieux—partner of Lumed—funds the installation and the maintenance of APSS and ZINC software during the evaluation time (grant number is not applicable).

**Competing interests** None declared.

**Patient and public involvement** Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

**Patient consent for publication** Not applicable.

**Provenance and peer review** Not commissioned; externally peer reviewed.

**Open access** This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

ORCID iDs

Alexandre Baudet http://orcid.org/0000-0003-2905-0530

Arnaud Florentin http://orcid.org/0000-0003-2011-4016

REFERENCES

1 O'Neill J. Tackling drug-resistant infections globally: final report and recommendations, 2016. Available: https://amr-review.org/sites/default/files/160518_Final%20paper_with%20cover.pdf [Accessed April 2021].

2 European Centre for Disease Prevention and Control (ECDC). European Medicines agency (EMA). the bacterial challenge: time to react, 2009. Available: https://www.ecdc.europa.eu/sites/default/files/media/en/publications/Publications/0909_TERM_The_Bacterial_Challenge_Time_to_R react.pdf [Accessed April 2021].

3 Cassini A, Högberg LD, Plachouras D, et al. Attributable deaths and disability-adjusted life-years caused by infections with antibiotic-resistant bacteria in the EU and the European economic area in

<footnote n="1">¹APEMAC, Université de Lorraine, Nancy, France</footnote>

<footnote n="2">²Service d'odontologie, CHRU de Nancy, Nancy, France</footnote>

<footnote n="3">³CIC, Epidémiologie Clinique, CHRU de Nancy, Nancy, France</footnote>

<footnote n="4">⁴Service des maladies infectieuses, CHRU de Nancy, Nancy, France</footnote>

<footnote n="5">⁵Service de microbiologie, CHRU de Nancy, Nancy, France</footnote>

<footnote n="6">⁶Stress Immunity Pathogens unit (SIMPA) EA 7300, Université de Lorraine, Nancy, France</footnote>

<footnote n="7">⁷Département territorial d'hygiène et prévention du risque infectieux, CHRU de Nancy, Nancy, France</footnote>

<footnote n="8">⁸Département Méthodologie, Promotion, Investigation, CHRU de Nancy, Nancy, France</footnote>

<footnote n="9">⁹Pharmacie, CHRU de Nancy, Nancy, France</footnote>
Qualitative analysis

The interviews will be fully transcribed. Then a duplicate
analysis of the data will be performed by two investigators
using an inductive method derived from the grounded
theory (a qualitative research method focused on the
identification of concepts that emerge from study inter-
views or observation<sup>41</sup>). NVivo software (QSR Interna-
tional, Melbourne, Australia) will be used to facilitate the
coding process of the transcripts.

Interrupted time series analysis

Assessment of the CDSS/ESS impact will be realised with
ITS analysis. ITS analysis is the best approach for quasi-
experimental studies assessing longitudinal effects of
interventions introduced at a specific time point where a
randomised trial is infeasible or unethical.<sup>39 42 43</sup> Before—
after quasi-experimental studies using ITS analysis is the
most common study design used for evaluation of anti-
biotic stewardship interventions for hospital inpatients.<sup>44</sup>
ITS analysis is also recommended to evaluate the effects
of IPC interventions according to Effective Practice and
Organisation of Care and ORION guidelines (guide-
lines for transparent reporting of Outbreak Reports and
Intervention studies Of Nosocomial infection).<sup>45 46</sup> In this
study, we comply with ITS guidance<sup>38 47</sup> and recommenda-
tions to evaluate the impact of AMS interventions with two
limits: the study is limited to a single centre and there is
no control group.<sup>18</sup> This study will not include an internal
control group for practical and ethical reasons. Indeed,
the implementation of the CDSS will need a change in
practice to support routine work of the AMS team who
will not continue its current way of operation in a control
group for efficiency reasons. Moreover, the implementa-
tion of the ESS will include all hospital wards for ethical
reasons because it should allow a better prevention and
management of the HCAI which can result in serious
illness, prolong hospital stay, drain healthcare resources
and may result in loss of life. Moreover, the use of an
external control group (like another hospital) seems not
appropriated because, in ITS, the control group should
be exposed to any such co-interventions or events that
might also affect the intervention group.<sup>48</sup> This is not
possible with another hospital as control.

The segmented regression analysis will be used to evaluate the longitudinal impact of the CDSS/ESS implementation on the outcomes of interest. Segmented regression is used to measure statistically the changes in level (immediate change in the rate of the outcome) and slope (changes in the trend) during the before period as compared with the after period.<sup>38 39 42</sup> Segmented regression analysis is a powerful statistical method for estimating intervention effects in ITS studies.<sup>39 49</sup> Before this analysis, we will graph the mean/percentage and SD/CI scores for all outcomes presented as monthly data to determine the structure and trend of the data. We will test for the presence of the autocorrelation (extent to which the data are dependent on each other) using the Durbin-Watson statistic<sup>50 51</sup> and, if necessary, we will

adjust for autocorrelation and seasonality. The transition period—during which the CDSS/ESS will be implemented, and the healthcare users will be trained—will not be included in the analysis. Such a lag period is currently applied in ITS study.⁵² Statistical analysis will be conducted using RStudio (RStudio, Boston, Massachusetts, USA)—an integrated development environment developed for R.

Author affiliations

**Acknowledgements** The authors are grateful to Stéphanie Bonnay who is the project manager and coordinate this study. The sponsor is CHRU de Nancy (Direction de la Recherche et de l'Innovation).

**Contributors** AB, AF, NAg and NT significantly contributed to designing the study. AB, CP and NT drafted the manuscript. AF and JL worked on the implementation of the ESS supporting HCAI. AC, AL, NAi and BD worked on the implementation of the CDSS supporting AMS. AC, AF, AL, BD, CP, JL, NAg, NAi and NT are members of the scientific committee in charge of the study supervision. AB is responsible for data set preparation and analysis. All authors have read and approved the final manuscript.

**Funding** BioMérieux—partner of Lumed—funds the installation and the maintenance of APSS and ZINC software during the evaluation time (grant number is not applicable).

**Competing interests** None declared.

**Patient and public involvement** Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

**Patient consent for publication** Not applicable.

**Provenance and peer review** Not commissioned; externally peer reviewed.

**Open access** This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

ORCID iDs

Alexandre Baudet http://orcid.org/0000-0003-2905-0530

Arnaud Florentin http://orcid.org/0000-0003-2011-4016

REFERENCES

1 O'Neill J. Tackling drug-resistant infections globally: final report and recommendations, 2016. Available: https://amr-review.org/sites/default/files/160518_Final%20paper_with%20cover.pdf [Accessed April 2021].

2 European Centre for Disease Prevention and Control (ECDC). European Medicines agency (EMA). the bacterial challenge: time to react, 2009. Available: https://www.ecdc.europa.eu/sites/default/files/media/en/publications/Publications/0909_TERM_The_Bacterial_Challenge_Time_to_R react.pdf [Accessed April 2021].

3 Cassini A, Högberg LD, Plachouras D, et al. Attributable deaths and disability-adjusted life-years caused by infections with antibiotic-resistant bacteria in the EU and the European economic area in

<footnote n="1">¹APEMAC, Université de Lorraine, Nancy, France</footnote>

<footnote n="2">²Service d'odontologie, CHRU de Nancy, Nancy, France</footnote>

<footnote n="3">³CIC, Epidémiologie Clinique, CHRU de Nancy, Nancy, France</footnote>

<footnote n="4">⁴Service des maladies infectieuses, CHRU de Nancy, Nancy, France</footnote>

<footnote n="5">⁵Service de microbiologie, CHRU de Nancy, Nancy, France</footnote>

<footnote n="6">⁶Stress Immunity Pathogens unit (SIMPA) EA 7300, Université de Lorraine, Nancy, France</footnote>

<footnote n="7">⁷Département territorial d'hygiène et prévention du risque infectieux, CHRU de Nancy, Nancy, France</footnote>

<footnote n="8">⁸Département Méthodologie, Promotion, Investigation, CHRU de Nancy, Nancy, France</footnote>

<footnote n="9">⁹Pharmacie, CHRU de Nancy, Nancy, France</footnote>
2015: a population-level modelling analysis. *Lancet Infect Dis* 2019;19:56–66.

4 European center for disease prevention and control (ECDC). surveillance of antimicrobial resistance in Europe, annual report 2018. *Stockholm: ECDC* 2019 https://www.ecdc.europa.eu/sites/ default/files/documents/surveillance-antimicrobial-resistance- Europe-2018.pdf

5 European Center for Disease Prevention and Control (ECDC), Santé Publique France. EARS-Net France. Contribution de la France Au réseau Européen de surveillance de la résistance bactérienne aux antibiotiques, 2002-2017. Available: https://www. santepubliquefrance.fr/media/files/01-maladies-et-traumatismes/ infections-associees-aux-soins-et-resistance-aux-antibiotiques/ resistance-aux-antibiotiques/synthese_ears_net_france_2002-2017 [Accessed April 2021].

6 Bell BG, Schellevis F, Stobberingh E, et al. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. *BMC Infect Dis* 2014;14:13.

7 European Center for Disease Prevention and Control (ECDC). Antimicrobial consumption in EU/EEA, annual epidemiological report for 2018. Stockholm: ECDC, 2019. https://www.ecdc.europa.eu/ sites/default/files/documents/Antimicrobial-consumption-EU-EEA. pdf

8 World Health Organization (WHO). Global action plan on antimicrobial resistance [Internet]. World Health Organization, 2015. http://www. who.int/iris/bitstream/10665/193736/1/9789241509763_eng.pdf? ua=1

9 Dyar OJ, Huttner B, Schouten J. Escmid Study Group for antimicrobial stewardship). What is antimicrobial stewardship? *Clin Microbiol Infect* 2017;23:793–8.

10 Thursky K. Use of computerized decision support systems to improve antibiotic prescribing. *Expert Rev Anti Infect Ther* 2006;4:491–507.

11 de Bruin JS, Seeling W, Schuh C. Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review. *J Am Med Inform Assoc* 2014;21:942–51.

12 Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an antibiotic stewardship program: guidelines by the infectious diseases Society of America and the Society for healthcare epidemiology of America. *Clin Infect Dis* 2016;62:e51–77.

13 Nault V, Pepin J, Beaudoin M, et al. Sustained impact of a computer-assisted antimicrobial stewardship intervention on antimicrobial use and length of stay. *J Antimicrob Chemother* 2017;72:933–40.

14 Mahamat A, MacKenzie FM, Brooker K, et al. Impact of infection control interventions and antibiotic use on hospital MRSA: a multivariate interrupted time-series analysis. *Int J Antimicrob Agents* 2007;30:169–76.

15 Freeman R, Moore LSP, García Álvarez L, et al. Advances in electronic surveillance for healthcare-associated infections in the 21st century: a systematic review. *J Hosp Infect* 2013;84:106–19.

16 Russo PL, Shaban RZ, Macbeth D, et al. Impact of electronic healthcare-associated infection surveillance software on infection prevention resources: a systematic review of the literature. *J Hosp Infect* 2018;99:1–7.

17 Carracedo-Martinez E, Gonzalez-Gonzalez C, Teixeira-Rodrigues A, et al. Computerized clinical decision support systems and antibiotic prescribing: a systematic review and meta-analysis. *Clin Ther* 2019;41:552–81.

18 Schweitzer VA, van Werkhoven CH, Rodríguez Baño J, et al. Optimizing design of research to evaluate antibiotic stewardship interventions: consensus recommendations of a multinational Working group. *Clin Microbiol Infect* 2020;26:41–50.

19 de Kraker MEA, Abbas M, Huttner B, et al. Good epidemiological practice: a narrative review of appropriate scientific methods to evaluate the impact of antimicrobial stewardship interventions. *Clin Microbiol Infect* 2017;23:819–25.

20 Centre Hospitalier Régional Universitaire de Nancy. Chiffres clés 2019, 2020. Available: http://www.chu-nancy.fr/images/CHRU/ Chiffres-cles-2019-web.pdf [Accessed April 2021].

21 Centers for Disease Control and Prevention. The core elements of hospital antibiotic stewardship programs, 2019, 2019. Available: https://www.cdc.gov/antibiotic-use/healthcare/pdfs/hospital-core- elements-H.pdf [Accessed April 2021].

22 World Health Organization (WHO). Antimicrobial stewardship programmes in health-care facilities in low- and middle-income countries: a who practical toolkit. Geneva: who, 2019. Available: https://apps.who.int/iris/handle/10665/329404 [Accessed April 2021].

23 World Health Organization (WHO). Infection prevention and control assessment framework at the facility level. Geneva: WHO, 2018.

Available: https://apps.who.int/iris/handle/10665/330072 [Accessed April 2021].

24 World Health Organization (WHO). Guidelines on core components of infection prevention and control programmes at the National and acute health care facility level. Geneva: who, 2016. Available: https:// www.who.int/infection-prevention/publications/ipc-components- guidelines/en/ [Accessed April 2021].

25 France SP. Enquête nationale de prévalence des infections nosocomiales et des traitements anti-infectieux en établissements de santé. *mai-juin* 2019;2017 https://www.santepubliquefrance.fr/ content/download/189000/2328245

26 France SP. Surveillance de la consommation des antibiotiques – Réseau ATB-Raisin, France. Résultats 2017. 2019. Available: https:// www.santepubliquefrance.fr/content/download/209355/2415496 [Accessed April 2021].

27 Bevilacqua S, Demoré B, Boschetti E, et al. 15 years of antibiotic stewardship policy in the Nancy teaching hospital. *Med Mal Infect* 2011;41:532–9.

28 Demoré B, Humbert P, Boschetti E, et al. Evaluation of effects of an operational multidisciplinary team on antibiotic use in the medium to long term at a French university hospital. *Int J Clin Pharm* 2017;39:1061–9.

29 Bevilacqua S, Demoré B, Erpelding M-L, et al. Effects of an operational multidisciplinary team on hospital antibiotic use and cost in France: a cluster controlled trial. *Int J Clin Pharm* 2011;33:521–8.

30 Agence nationale de sécurité Du médicament et des produits de santé. Liste des antibiotiques critiques. *Actualisation* 2016;2015 https://www.omedit-grand-est.ars.sante.fr/system/files/2017-08/ ATBC-antibiotiques-critiques-actualisation2015.pdf

31 AntibioEst. AntibioEst - Conseil, information, bon usage des antibiotiques. Available: https://www.antibioest.org/ [Accessed April 2021].

32 Beaudoin M, Kabanza F, Nault V, et al. An antimicrobial prescription surveillance system that learns from experience. *AI Mag* 2014;35:15.

33 LUMED. ZINC Monitoring System: Prevent & Control Healthcare Related Infections. Available: https://lumed.ca/en/zinc/ [Accessed April 2021].

34 Haut Conseil de Santé Publique (HCSP). Actualisation des recommandations relatives La maîtrise de la diffusion des bactéries hautement résistantes aux antibiotiques émergentes (BHRe), 2019. Available: https://www.hcsp.fr/Explore.cgi/Telecharger?NomFichier= hcsp20191211gardesrecommarrelatиваuxbhe.pdf [Accessed April 2021].

35 Société Français d'Hygiène Hospitalière (SF2H). Recommandations nationales - Prévention de la transmission croisée : précautions complémentaires contact. Consensus formalisé d'experts. HygièneS, 2009. Available: https://www.sf2h.net/wp-content/uploads/2009/01/ SF2H_prevention-transmission-croisee-2009.pdf [Accessed April 2021].

36 Société Français d'Hygiène Hospitalière (SF2H). Recommandations nationales - Prévention de la transmission croisée par voie respiratoire : Air ou Gouttelettes. Recommandations pour la pratique clinique (RPC). HygièneS, 2013. Available: https://www.sf2h.net/ wp-content/uploads/2013/03/SF2H_recommandations_Air-ou- gouttelettes_2013.pdf [Accessed April 2021].

37 Zhang F, Wagner AK, Ross-Degnan D. Simulation-Based power calculation for designing interrupted time series analyses of health policy interventions. *J Clin Epidemiol* 2011;64:1252–61.

38 Ramsay CR, Matowe L, Grilli R, et al. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. *Int J Technol Assess Health Care* 2003;19:613–23.

39 Wagner AK, Soumerai SB, Zhang F, et al. Segmented regression analysis of interrupted time series studies in medication use research. *J Clin Pharm Ther* 2002;27:299–309.

40 Chabaud A, Jouzeau A, Dugravot L. Antibiotic use and antibiotic resistance in French healthcare facilities in 2020: data from the National spares network. *Bull Epidémiol Hebd* 2021;18:342–50.

41 Corbin JM, Strauss A. Grounded theory research: procedures, canons, and evaluative criteria. *Qual Sociol* 1990;13:3–21.

42 Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. *Acad Pediatr* 2013;13:S38–44.

43 Kontopantelis E, Doran T, Springate DA, et al. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. *BMJ* 2015;350:h2750.

44 Davey P, Marwick CA, Scott CL, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. *Cochrane Database Syst Rev* 2017;2:CD003543.

45 Cochrane Effective Practice and Organisation of Care (EPOC). What study designs can be considered for inclusion in an EPOC
2015: a population-level modelling analysis. *Lancet Infect Dis* 2019;19:56–66.

4 European center for disease prevention and control (ECDC). surveillance of antimicrobial resistance in Europe, annual report 2018. *Stockholm: ECDC* 2019 https://www.ecdc.europa.eu/sites/ default/files/documents/surveillance-antimicrobial-resistance- Europe-2018.pdf

5 European Center for Disease Prevention and Control (ECDC), Santé Publique France. EARS-Net France. Contribution de la France Au réseau Européen de surveillance de la résistance bactérienne aux antibiotiques, 2002-2017. Available: https://www. santepubliquefrance.fr/media/files/01-maladies-et-traumatismes/ infections-associees-aux-soins-et-resistance-aux-antibiotiques/ resistance-aux-antibiotiques/synthese_ears_net_france_2002-2017 [Accessed April 2021].

6 Bell BG, Schellevis F, Stobberingh E, et al. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. *BMC Infect Dis* 2014;14:13.

7 European Center for Disease Prevention and Control (ECDC). Antimicrobial consumption in EU/EEA, annual epidemiological report for 2018. Stockholm: ECDC, 2019. https://www.ecdc.europa.eu/ sites/default/files/documents/Antimicrobial-consumption-EU-EEA. pdf

8 World Health Organization (WHO). Global action plan on antimicrobial resistance [Internet]. World Health Organization, 2015. http://www. who.int/iris/bitstream/10665/193736/1/9789241509763_eng.pdf? ua=1

9 Dyar OJ, Huttner B, Schouten J. Escmid Study Group for antimicrobial stewardship). What is antimicrobial stewardship? *Clin Microbiol Infect* 2017;23:793–8.

10 Thursky K. Use of computerized decision support systems to improve antibiotic prescribing. *Expert Rev Anti Infect Ther* 2006;4:491–507.

11 de Bruin JS, Seeling W, Schuh C. Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review. *J Am Med Inform Assoc* 2014;21:942–51.

12 Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an antibiotic stewardship program: guidelines by the infectious diseases Society of America and the Society for healthcare epidemiology of America. *Clin Infect Dis* 2016;62:e51–77.

13 Nault V, Pepin J, Beaudoin M, et al. Sustained impact of a computer-assisted antimicrobial stewardship intervention on antimicrobial use and length of stay. *J Antimicrob Chemother* 2017;72:933–40.

14 Mahamat A, MacKenzie FM, Brooker K, et al. Impact of infection control interventions and antibiotic use on hospital MRSA: a multivariate interrupted time-series analysis. *Int J Antimicrob Agents* 2007;30:169–76.

15 Freeman R, Moore LSP, García Álvarez L, et al. Advances in electronic surveillance for healthcare-associated infections in the 21st century: a systematic review. *J Hosp Infect* 2013;84:106–19.

16 Russo PL, Shaban RZ, Macbeth D, et al. Impact of electronic healthcare-associated infection surveillance software on infection prevention resources: a systematic review of the literature. *J Hosp Infect* 2018;99:1–7.

17 Carracedo-Martinez E, Gonzalez-Gonzalez C, Teixeira-Rodrigues A, et al. Computerized clinical decision support systems and antibiotic prescribing: a systematic review and meta-analysis. *Clin Ther* 2019;41:552–81.

18 Schweitzer VA, van Werkhoven CH, Rodríguez Baño J, et al. Optimizing design of research to evaluate antibiotic stewardship interventions: consensus recommendations of a multinational Working group. *Clin Microbiol Infect* 2020;26:41–50.

19 de Kraker MEA, Abbas M, Huttner B, et al. Good epidemiological practice: a narrative review of appropriate scientific methods to evaluate the impact of antimicrobial stewardship interventions. *Clin Microbiol Infect* 2017;23:819–25.

20 Centre Hospitalier Régional Universitaire de Nancy. Chiffres clés 2019, 2020. Available: http://www.chu-nancy.fr/images/CHRU/ Chiffres-cles-2019-web.pdf [Accessed April 2021].

21 Centers for Disease Control and Prevention. The core elements of hospital antibiotic stewardship programs, 2019, 2019. Available: https://www.cdc.gov/antibiotic-use/healthcare/pdfs/hospital-core- elements-H.pdf [Accessed April 2021].

22 World Health Organization (WHO). Antimicrobial stewardship programmes in health-care facilities in low- and middle-income countries: a who practical toolkit. Geneva: who, 2019. Available: https://apps.who.int/iris/handle/10665/329404 [Accessed April 2021].

23 World Health Organization (WHO). Infection prevention and control assessment framework at the facility level. Geneva: WHO, 2018.

Available: https://apps.who.int/iris/handle/10665/330072 [Accessed April 2021].

24 World Health Organization (WHO). Guidelines on core components of infection prevention and control programmes at the National and acute health care facility level. Geneva: who, 2016. Available: https:// www.who.int/infection-prevention/publications/ipc-components- guidelines/en/ [Accessed April 2021].

25 France SP. Enquête nationale de prévalence des infections nosocomiales et des traitements anti-infectieux en établissements de santé. *mai-juin* 2019;2017 https://www.santepubliquefrance.fr/ content/download/189000/2328245

26 France SP. Surveillance de la consommation des antibiotiques – Réseau ATB-Raisin, France. Résultats 2017. 2019. Available: https:// www.santepubliquefrance.fr/content/download/209355/2415496 [Accessed April 2021].

27 Bevilacqua S, Demoré B, Boschetti E, et al. 15 years of antibiotic stewardship policy in the Nancy teaching hospital. *Med Mal Infect* 2011;41:532–9.

28 Demoré B, Humbert P, Boschetti E, et al. Evaluation of effects of an operational multidisciplinary team on antibiotic use in the medium to long term at a French university hospital. *Int J Clin Pharm* 2017;39:1061–9.

29 Bevilacqua S, Demoré B, Erpelding M-L, et al. Effects of an operational multidisciplinary team on hospital antibiotic use and cost in France: a cluster controlled trial. *Int J Clin Pharm* 2011;33:521–8.

30 Agence nationale de sécurité Du médicament et des produits de santé. Liste des antibiotiques critiques. *Actualisation* 2016;2015 https://www.omedit-grand-est.ars.sante.fr/system/files/2017-08/ ATBC-antibiotiques-critiques-actualisation2015.pdf

31 AntibioEst. AntibioEst - Conseil, information, bon usage des antibiotiques. Available: https://www.antibioest.org/ [Accessed April 2021].

32 Beaudoin M, Kabanza F, Nault V, et al. An antimicrobial prescription surveillance system that learns from experience. *AI Mag* 2014;35:15.

33 LUMED. ZINC Monitoring System: Prevent & Control Healthcare Related Infections. Available: https://lumed.ca/en/zinc/ [Accessed April 2021].

34 Haut Conseil de Santé Publique (HCSP). Actualisation des recommandations relatives La maîtrise de la diffusion des bactéries hautement résistantes aux antibiotiques émergentes (BHRe), 2019. Available: https://www.hcsp.fr/Explore.cgi/Telecharger?NomFichier= hcsp20191211gardesrecommarrelatиваuxbhe.pdf [Accessed April 2021].

35 Société Français d'Hygiène Hospitalière (SF2H). Recommandations nationales - Prévention de la transmission croisée : précautions complémentaires contact. Consensus formalisé d'experts. HygièneS, 2009. Available: https://www.sf2h.net/wp-content/uploads/2009/01/ SF2H_prevention-transmission-croisee-2009.pdf [Accessed April 2021].

36 Société Français d'Hygiène Hospitalière (SF2H). Recommandations nationales - Prévention de la transmission croisée par voie respiratoire : Air ou Gouttelettes. Recommandations pour la pratique clinique (RPC). HygièneS, 2013. Available: https://www.sf2h.net/ wp-content/uploads/2013/03/SF2H_recommandations_Air-ou- gouttelettes_2013.pdf [Accessed April 2021].

37 Zhang F, Wagner AK, Ross-Degnan D. Simulation-Based power calculation for designing interrupted time series analyses of health policy interventions. *J Clin Epidemiol* 2011;64:1252–61.

38 Ramsay CR, Matowe L, Grilli R, et al. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. *Int J Technol Assess Health Care* 2003;19:613–23.

39 Wagner AK, Soumerai SB, Zhang F, et al. Segmented regression analysis of interrupted time series studies in medication use research. *J Clin Pharm Ther* 2002;27:299–309.

40 Chabaud A, Jouzeau A, Dugravot L. Antibiotic use and antibiotic resistance in French healthcare facilities in 2020: data from the National spares network. *Bull Epidémiol Hebd* 2021;18:342–50.

41 Corbin JM, Strauss A. Grounded theory research: procedures, canons, and evaluative criteria. *Qual Sociol* 1990;13:3–21.

42 Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. *Acad Pediatr* 2013;13:S38–44.

43 Kontopantelis E, Doran T, Springate DA, et al. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. *BMJ* 2015;350:h2750.

44 Davey P, Marwick CA, Scott CL, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. *Cochrane Database Syst Rev* 2017;2:CD003543.

45 Cochrane Effective Practice and Organisation of Care (EPOC). What study designs can be considered for inclusion in an EPOC
review and what should they be called? Cochrane effective practice
and organisation of care, 2017. Available: https://epoc.cochrane.
org/sites/epoc.cochrane.org/files/public/uploads/Resources-for-
authors2017/what_study designs should be included in an_epoc_
review.pdf [Accessed April 2021].

46 Stone SP, Cooper BS, Kibbler CC, et al. The ORION statement: guidelines for transparent reporting of outbreak reports and intervention studies of nosocomial infection. *Lancet Infect Dis* 2007;7:282–8.

47 Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. *Int J Epidemiol* 2017;46:348–55.

48 Lopez Bernal J, Cummins S, Gasparrini A. The use of controls in interrupted time series studies of public health interventions. *Int J Epidemiol* 2018;47:2082–93.

49 Abeysinghe T, Balasooriya U, Tsui A. Small-Sample forecasting regression or Arima models? *J Quant Econ* 2003;1:103–13.

50 Durbin J, Watson GS. Testing for serial correlation in least squares regression. I. *Biometrika* 1950;37:409-28.

51 Durbin J, Watson GS. Testing for serial correlation in least squares regression. II. *Biometrika* 1951;38:159-78.

52 Lopez Bernal J, Soumerai S, Gasparrini A. A methodological framework for model selection in interrupted time series studies. *J Clin Epidemiol* 2018;103:82-91.
review and what should they be called? Cochrane effective practice
and organisation of care, 2017. Available: https://epoc.cochrane.
org/sites/epoc.cochrane.org/files/public/uploads/Resources-for-
authors2017/what_study designs should be included in an_epoc_
review.pdf [Accessed April 2021].

46 Stone SP, Cooper BS, Kibbler CC, et al. The ORION statement: guidelines for transparent reporting of outbreak reports and intervention studies of nosocomial infection. *Lancet Infect Dis* 2007;7:282–8.

47 Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. *Int J Epidemiol* 2017;46:348–55.

48 Lopez Bernal J, Cummins S, Gasparrini A. The use of controls in interrupted time series studies of public health interventions. *Int J Epidemiol* 2018;47:2082–93.

49 Abeysinghe T, Balasooriya U, Tsui A. Small-Sample forecasting regression or Arima models? *J Quant Econ* 2003;1:103–13.

50 Durbin J, Watson GS. Testing for serial correlation in least squares regression. I. *Biometrika* 1950;37:409-28.

51 Durbin J, Watson GS. Testing for serial correlation in least squares regression. II. *Biometrika* 1951;38:159-78.

52 Lopez Bernal J, Soumerai S, Gasparrini A. A methodological framework for model selection in interrupted time series studies. *J Clin Epidemiol* 2018;103:82-91.