SepsiTest assay for rapidly identifying bloodstream bacteria and fungi

Evidence-based recommendations on the SepsiTest assay for rapidly identifying bloodstream bacteria and fungi.

# Recommendation
There is currently insufficient evidence to recommend the routine adoption in the NHS of the SepsiTest assay for rapidly identifying bloodstream bacteria and fungi. The test shows promise and further research to provide robust evidence is encouraged, particularly to demonstrate the value of using the test results in clinical decisionmaking (see sections 5.18 to 5.22). # Clinical need and practice

# The problem addressed
In current practice, people who are clinically unwell and who have a suspected bloodstream infection have empirically prescribed broadspectrum antibiotics, that is, antibiotics that are prescribed based on clinical presentation, until the identity of the pathogen causing the infection is known. Broadspectrum antibiotics and, if appropriate, antifungals, are used because they are effective against a wide range of bacterial and fungal pathogens and are likely to achieve a therapeutic response. But, although clinically effective, broadspectrum antibiotic use is associated with people developing superinfection and with antimicrobial resistance. Rapidly identifying the bacterial and fungal pathogen may allow earlier targeted treatment and shorten the length of use of broadspectrum antibiotics and antifungals, which may help antimicrobial stewardship by conserving the effectiveness of existing antimicrobials.
Three molecular tests, the LightCycler SeptiFast Test MGRADE, SepsiTest and IRIDICA BAC BSI assay, were identified during scoping as relevant to the assessment (see section 3 for additional details). These tests are designed to rapidly detect and identify bacterial and fungal DNA that may be in the bloodstream in people who are suspected of having sepsis. These tests are intended to be used with clinical assessment and established microbiology techniques that provide information on which antimicrobials are likely to be effective against the identified pathogen. The tests are designed to be run on whole blood samples and without the prior incubation or the preculture steps that are needed for tests used in current standard practice. The absence of these steps means that pathogens may be identified earlier. It is possible that blood culture would still be needed to give definitive antimicrobialsusceptibility data, if this is not provided by the rapid diagnostic test. The rapid detection and identification of bacterial and fungal DNA may be particularly beneficial in people who are suspected of having a severe infection and who need quick medical intervention.
The purpose of this assessment is to evaluate the clinical and cost effectiveness of using the LightCycler SeptiFast Test MGRADE, SepsiTest and IRIDICA BAC BSI assay for rapidly identifying bloodstream bacteria and fungi in the NHS.

# The condition

## Sepsis and bloodstream infection
Sepsis is a lifethreatening condition characterised by the body's inflammatory response to an infection. According to the Surviving Sepsis Campaign's International guidelines for the management of severe sepsis and septic shock, sepsis is diagnosed if there is evidence of systemic inflammation, in addition to a documented or presumed infection in the body. Systemic illness often happens if bacteria invade normally sterile parts of the body. One example of this is when bacteria or fungi invade the bloodstream (bloodstream infection); a process that often causes an inflammatory immune response.
Bacterial infections are the most common cause of sepsis and bloodstream infection, but they can also be caused by fungal infections, and less commonly by viral infections. The most common sites of infection associated with sepsis are the lungs, urinary tract, abdomen and pelvis. Other sources of infection leading to sepsis include skin infections (such as cellulitis), postsurgical infections and infections of the nervous system (such as meningitis or encephalitis).
People who have recently been admitted to hospital are at risk of getting hospitalacquired infections that can lead to sepsis and bloodstream infection. The increased use of invasive procedures, such as catheterisation and life support measures, as well as immunosuppressive therapy and antibiotic therapy may have resulted in more healthcareassociated bloodstream infections. Communityacquired bloodstream infections may also occur in people who have not had recent contact with healthcare services. The pathogens infecting these people may differ from those associated with hospitalacquired bloodstream infection.
The bacteria most commonly associated with bloodstream infection in adults include gramnegative species such as Escherichia coli, Klebsiella and Pseudomonas, and grampositive species such as Staphylococcus aureus, non-pyogenic streptococci, Enterococcus and Streptococcus pneumoniae. The types of pathogens causing bloodstream infection can differ in children compared with those causing infection in adults, and can include Neisseria meningitidis. Polymicrobial infection and anaerobic bacteraemia are also thought to occur less often in children.

# The diagnostic and care pathways

## Diagnosing sepsis and bloodstream infection
Diagnostic criteria for sepsis are listed in the Surviving Sepsis Campaign's International guidelines for the management of severe sepsis and septic shock. In summary, regular observations of all vital signs should be taken and recorded, kidney and liver function tests should be done, and inflammatory biomarkers and serum lactate should be measured. These guidelines state that a diagnosis of sepsis should be based on infection, documented or suspected, with hyperthermia or hypothermia, tachycardia and at least 1 indication of altered organ function.
The guidelines also make the following specific recommendations relating to detecting localised and bloodstream infection:
At least 2 samples for blood culture should be collected (aerobic and anaerobic) before antimicrobial therapy is started if such cultures do not cause significant delay (greater than 45 minutes) in the start of antimicrobial administration. At least 1 sample should be drawn percutaneously and 1 drawn through each vascular access device, unless the device was recently (less than 48 hours) inserted. The blood cultures can be drawn at the same time if they are taken from different sites. Cultures from other sites that may be the source of infection, such as urine, cerebrospinal fluid, wounds, respiratory secretions or other bodily fluids, should be collected before starting antimicrobial therapy, if doing so does not cause significant delay in the start of antimicrobial treatment.
Imaging studies such as CT or Xray should be done to confirm a potential source of infection.
Assays to diagnose systemic fungal infection should be used if available and invasive candidiasis is suspected.

## Blood cultures
Public Health England's standards for the investigation of blood cultures are available. A blood culture set for diagnosing bloodstream infection is defined as 1 aerobic and 1 anaerobic bottle. For adults it is recommended that 2030 ml of blood is cultured per set, and that 2 consecutive blood culture sets from 2 separate venepuncture sites should be collected during any 24hour period for each septic episode. The first set should be taken before starting antimicrobial treatment because the presence of antibiotics or antifungals may inhibit the growth of pathogens in blood culture. Blood culture sample collection differs for infants and neonates, for whom a single aerobic bottle or lowvolume blood culture bottle may be requested. The criterion for calculating total bloodculture volume in neonates and children is based on weight rather than age and relates to total patient blood volume. It has been suggested that the volume of blood drawn should be no more than 1% of the patient's total blood volume. In infants and children, the level of bacteraemia is usually higher than in adults and so the sensitivity of detection is not thought to be substantially reduced by a lower bloodtomedium ratio.
Blood culture bottles should be incubated within 4 hours of the blood sample being taken. Many laboratories now use automated culture systems that alert laboratory staff once growth has been detected.
When a blood culture has been detected as positive it is recommended that:
Gram staining and rapid antigen testing should be done within 2 hours.
Direct or automated isolate identification should be done within 24 hours (extending to 48 hours if traditional microbiology techniques such as morphological identification are used). Rapid species identification may be done after blood culture using techniques such as MALDITOF mass spectrometry.
Identification should be followed by sensitivity testing to determine the antimicrobials that the identified pathogen is susceptible to. If direct or automated sensitivity testing is used, a report should be made within 24 hours, extended to 48 hours if traditional techniques, such as the disc diffusion method, are used.
A preliminary positive report is made within 2 hours of identification and sensitivity testing, and a final positive report should be made within 5 days of the sample arriving in the laboratory.
If a blood culture is negative, it is recommended that a preliminary negative report is provided within 48 hours of the sample arriving in the laboratory and a final negative report should be issued within 5 days unless extended culture is being done, such as if fungi or unusual, fastidious or slow growing organisms are suspected.

## Treating sepsis and bloodstream infection
Sepsis treatment varies based on the initial infection, the organs affected and the extent of tissue damage. The management of severe sepsis and septic shock is described in the Surviving Sepsis Campaign's International guidelines for the management of severe sepsis and septic shock.
The guidelines recommend that effective intravenous antimicrobials should be given within the first hour of recognising severe sepsis and septic shock. Initial empirical antimicrobial therapy should include 1 or more drugs that have activity against all likely pathogens (bacterial, fungal or viral) and that penetrate in adequate concentrations into the tissues thought to be the source of sepsis. Frequently used broadspectrum antibiotics for more serious infections include cephalosporins and aminoglycosides.
The guidelines recommend that the choice of empirical antimicrobial therapy be based on:
the patient's history, including drug intolerances
recent antibiotic treatments (previous 3 months)
underlying disease
the clinical syndrome
susceptibility patterns of pathogens in the community and hospital
previous microbiology reports identifying pathogens that have previously colonised or infected the patient.
Clinicians prescribing antimicrobial therapy should take into account the Department of Health's guidance on antimicrobial stewardship, which is based on the 'start smart then focus' strategy. The guidance recommends that when empirical antimicrobials are prescribed, the clinical diagnosis should be reviewed after 48 to 72 hours to allow an antimicrobial prescribing decision to be made. This decision should take into account available microbiology results to determine if therapy can be stopped or changed; that is, the deescalation, substitution or addition of antimicrobial agents to the treatment plan.
Narrowing the spectrum of antimicrobial coverage and shortening the duration of therapy may reduce the risk of a person developing a superinfection, and reduce treatmentrelated adverse events. Adverse events associated with using broadspectrum antimicrobials may include diarrhoea, nausea, vomiting, hearing loss, damage to the kidneys and an increased risk of superinfection with Clostridium difficile. Narrowing the spectrum of antimicrobial coverage may also be associated with an increase in treatment efficacy in some scenarios.
Reducing the spectrum of antimicrobial coverage and duration of antibiotic therapy may also contribute to antimicrobial stewardship and protect the effectiveness of existing antibiotics. Surveillance data for England for the period 2010 to 2013 suggest that rates of methicillinresistant Staphylococcus aureus (MRSA) have fallen while the incidence of bloodstream infections caused by resistant gram-negative Enterobacteriaceae bacteria, such as Klebsiella and Escherichia coli, has increased (English surveillance programme for antimicrobial utilisation and resistance, 2014). Of particular concern in some regions of England is the increasing resistance to carbapenem antibiotics, which are often used as a last resort for treating severe infections when other antibiotics have not brought the infection under control.# The diagnostic tests

# The interventions

## The LightCycler SeptiFast Test MGRADE
The LightCycler SeptiFast Test MGRADE (Roche Diagnostics) is a CEmarked, in vitro, diagnostic, realtime polymerase chain reaction (PCR) test that simultaneously detects and identifies DNA from 25 bacterial and fungal pathogens. The test needs 1.5 ml of ethylenediaminetetraacetic acid (EDTA)treated whole blood.
The LightCycler SeptiFast Test MGRADE involves 3 distinct processes: specimen preparation by mechanical lysis and purification of DNA, realtime PCR amplification of target DNA in 3 parallel reactions (grampositive bacteria, gramnegative bacteria and fungi), and detection using fluorescencelabelled probes specific to the target DNA. The test takes a minimum of 6 hours, depending on laboratory workflow.
The SeptiFast Identification Software set v2.0 analyses the samples and generates a report, which contains all the relevant laboratory data and details of the identified species. The software also includes a crossing point cutoff rule, which is intended to reduce the positive rate for coagulasenegative Staphylococci and Streptococcus species based on the assumption that they are contaminants and not causal agents when the crossing point value is less than 20.
If Staphylococcus aureus is identified in a sample, an aliquot of the SeptiFast Test MGRADE eluate can be further tested for the MecA gene using the LightCycler SeptiFast MecA Test MGRADE. The test can determine the likely methicillin resistance of Staphylococcus aureus through PCR, using the LightCycler 2.0 instrument.
The test has an analytical sensitivity of 100 colonyforming units/ml for coagulasenegative Staphylococci, Streptococcus agalactiae, Streptococcus pyogenes, Streptococcus pneumonia and Streptococcus mitis. The minimum analytical sensitivity for all other pathogens detected by the LightCycler SeptiFast test MGRADE is 30 colonyforming units/ml.

## SepsiTest
SepsiTest (Molzym Molecular Diagnostics) is a CEmarked PCR in vitro test for detecting bacterial and fungal DNA in 1 ml of kEDTA- or citratetreated whole blood. The test is able to identify species from more than 200 genera of bacteria and 65 genera of fungi.
The SepsiTest involves 3 distinct processes: extracting and purifying microbial DNA using centrifugation, universal PCR and sequencing. The PCR result, which is available after 4 hours, indicates whether bacteria or fungi are present in the sample. Amplicons from positive samples are then sequenced to confirm the PCR result and to determine which bacteria or fungi species are present. If readable sequences are available from sequence analysis, bacteria and fungi can be identified using the SepsiTestBLAST online tool. Sequencing results may be available in 3 to 4 hours depending on the analyser used.
The analytical sensitivity of SepsiTest ranges from 10 to 80 colonyforming units/ml, depending on the target species.

## IRIDICA BAC BSI assay
The IRIDICA BAC BSI assay (Abbott Laboratories) is a CEmarked, in vitro, diagnostic test for detecting and identifying DNA from bacteria and candida in 5 ml of whole blood treated with EDTA. The test can also detect the MecA (Staphylococcusspecific methicillin resistance), vanA and vanB (Enterococcusspecific vancomycin resistance), and KPC (gramnegative associated carbapenem resistance) genes, which are associated with antibiotic resistance.
The test is designed for use with the IRIDICA system, which combines broadrange PCR with electrospray ionisation timeofflight mass spectrometry to amplify and detect pathogens. The estimated time to result is at least 5 hours and 55 minutes.
The IRIDICA analysis computer consists of a proprietary database and software, which identifies the organism present in the sample by comparing the sequence of the sample with a library of known sequences.
The BAC BSI assay is able to identify more than 780 bacteria and candida. The mean limit of detection for the assay is 39 colonyforming units/ml, with a range of 0.25 to 128 colonyforming units/ml depending on the target species.

# The comparators
Two comparators are included, blood culture alone and blood culture with MALDITOF mass spectrometry:
Blood culture alone refers to the incubation of whole blood followed by the identification of pathogens by traditional microbiology techniques.
Blood culture with MALDITOF mass spectrometry refers to the incubation of whole blood followed by the identification of pathogens using MALDITOF mass spectrometry.# Outcomes
The Diagnostics Advisory Committee (section 6) considered evidence from a number of sources (section 7). Full details of all the evidence are in the committee papers.

# How outcomes were assessed
The assessment consisted of a systematic review of the evidence on test performance and clinicaleffectiveness data for the LightCycler SeptiFast Test MGRADE, SepsiTest and IRIDICA BAC BSI and comparator tests.
Studies were included if they evaluated 1 of the interventions, compared with either blood culture or blood culture with MALDITOF mass spectrometry (MS), to analyse whole blood samples collected from people being treated for suspected sepsis. Studies that compared 1 of the interventions with another intervention were also included. In total, 66 studies met the inclusion criteria. Diagnosticaccuracy data were reported in 62 of the 66 studies and were included in metaanalyses, which were based on a bivariate normal model with Markov Chain Monte Carlo simulation. Interstudy heterogeneity was explored using metaregression. Intermediate or clinical outcome measures were reported in 41 of the 66 studies and were included in a narrative analysis.
Sixty four of the 66 studies were singleindex test, singlegate studies, that is, studies in which only patients with the target condition (suspected sepsis) were recruited. Three of these 64 studies were randomised controlled trials (RCTs). The remaining 2 studies were singlegate studies that reported results for both the LightCycler SeptiFast Test MGRADE and SepsiTest.
Only 3 of the 66 studies included patients from the UK. Most of the studies were done in other European countries. Of the studies that included patients from the UK, 1 study (Dark et al. 2009) used the SeptiFast assay to test 50 patients and 1 other study (Vincent et al. 2015) used the IRIDICA assay to test 529 patients from 6 European countries. The third UK study (Warhurst et al. 2015) reported the use of SeptiFast in 795 patients with sepsis and was judged to be the highest quality and most applicable included study.
All studies were assessed using the QUADAS2 tool. The results of 65 of the 66 studies were considered to be at risk of bias and may not be applicable to the decision problem. The issues of greatest uncertainty included patient selection and blinding to the index test or reference standard. The External Assessment Group also reported concerns about 21 of the 66 studies, which did not report whether the blood samples for the index test and reference standard were drawn at the same time, and 6 of the 66 studies, which used a mixture of reference standards. In addition, only 28 of the 66 studies reported using blood sampling and test methods that were in accordance with the company's instructions for use. Studies also reported different units of analysis for diagnosticaccuracy data, such as per patient, per sample, per episode of sepsis, and species or pathogen level.

# Diagnostic accuracy
Of the 62 studies that reported diagnosticaccuracy data, 55 reported data for the LightCycler SeptiFast Test MGRADE; 5 reported data for SepsiTest; and 4 reported data for the IRIDICA BAC BSI assay. Two of the 62 studies that reported data for both the Light Cycler SeptiFast Test MGRADE and SepsiTest were counted as individual studies for each test.

## LightCycler SeptiFast Test MGRADE
There were 54 studies that compared the LightCycler SeptiFast Test MGRADE with blood culture and were combined in a metaanalysis. The pooled estimate for sensitivity was 0.65 (95% credible interval  0.60 to 0.71; 95% prediction interval 0.29 to 0.90) and for specificity was 0.86 (95% CrI 0.84 to 0.89; 95% prediction interval 0.62 to 0.96). The proportion of discordant results varied across studies from 6% to 46% (median 17%).
One study (Tafelski et al. 2015) compared the LightCycler SeptiFast Test MGRADE with blood culture plus MALDITOF MS. It reported a sensitivity of 0.58 (95% confidence interval  0.30 to 0.86) and a specificity of 0.74 (95% CI 0.64 to 0.85).
Reasons for heterogeneity in sensitivity and specificity estimates between studies were explored using metaregression for clinically relevant variables. The following variables were explored:
age (neonates and children)
exposure to antibiotics before blood sample collection
suspected community- or healthcareacquired infection
febrile neutropenia
studies with inclusion or exclusion of contaminants.There was no evidence that sensitivity and specificity estimates were affected by these variables.

## SepsiTest
Four studies compared SepsiTest with blood culture and were combined in a metaanalysis. The pooled estimate for sensitivity was 0.48 (95% CrI 0.21 to 0.74; 95% prediction interval 0.07 to 0.90) and for specificity was 0.86 (95% CrI 0.78 to 0.92; 95% prediction interval 0.66 to 0.95). The proportion of discordant results varied between studies and ranged from 14% to 26% (median 22%).
One study (Loonen et al. 2014) compared SepsiTest with blood culture plus MALDITOF MS. The study reported a sensitivity of 0.11 (95% CI 0.00 to 0.23) and specificity of 0.96 (95% CI 0.92 to 1.00). No subgroup analyses were possible for the SepsiTest.

## IRIDICA BAC BSI
Four studies compared the IRIDICA BAC BSI assay with blood culture and were combined in a metaanalysis. Two of these studies reported data using an earlier version of the IRIDICA PCR/ESIMS analyser known as the PLEXID system, which has different desalter and mass spectrometry modules. The pooled estimate for sensitivity was 0.81 (95% CrI 0.69 to 0.90; 95% prediction interval 0.55 to 0.94) and for specificity was 0.84 (95% CrI 0.71 to 0.92; 95% prediction interval 0.50 to 0.96). The proportion of discordant results varied between studies and ranged from 7% to 30% (median 18%).
No studies compared the IRIDICA BAC BSI assay with blood culture plus MALDITOF MS and no subgroup analyses were possible for this intervention.

# Intermediate and clinical outcomes
There were 41 studies included that reported data relating to the time to pathogen identification for the index test, time to treatment, testfailure rate, mortality, duration of intensive care unit or hospital stay, duration of antibiotic therapy or reported changes in antimicrobial treatment plan. None of the included studies reported data on readmission rates, adverse events associated with broadspectrum antimicrobial use, morbidity, changes in disease severity over time, rates of superinfection, rates of resistant infection, or healthrelated quality of life.

## Light Cycler SeptiFast Test MGRADE
There were 37 studies that reported data on intermediate and clinical outcomes for the LightCycler SeptiFast Test MGRADE. In addition, 1 study (Schreiber et al. 2013) reported data for both the LightCycler SeptiFast Test MGRADE and SepsiTest. No studies compared the LightCycler SeptiFast Test MGRADE with the IRIDICA BAC BSI assay.
There were 21 studies using the LightCycler SeptiFast Test MGRADE that reported turnaround times of a minimum of 4 hours to a median of 26.25 hours for pathogen identification. Some of these studies also reported the time for pathogen identification using blood cultures, which ranged from a turnaround time of a minimum of 24 hours to a median of 80 hours.
Time-to-treatment change for the LightCycler SeptiFast Test MGRADE was reported in 3 RCTs:
Tafelski et al. (2015) reported a mean time of 18.8 hours (standard deviation  5.6) from taking the blood sample to changing treatment using the LightCycler SeptiFast Test MGRADE and a mean time of 38.3 hours (SD 14.5) using blood culture and MALDITOF MS.
Rodrigues et al. (2013) reported a mean time of 9.7 hours from taking the blood sample to a change in treatment using the LightCycler SeptiFast Test MGRADE compared with a mean time of 50.1 hours using blood culture (p=0.004).
Idelevich et al. (2015) reported a mean time to changing treatment of 21.4 hours (range 16.2 to 46.3 hours) in the LightCycler SeptiFast Test MGRADE group compared with 47.5 hours (range 7.3 to 59.2 hours) in the blood culture group (p=0.018).
There were 7 studies that reported testfailure rates for the LightCycler SeptiFast Test MGRADE, which ranged from 1.5% to 24.2%. It is not clear why there is a large variation in failure rates between studies.
Duration of stay in an intensive care unit, or hospital, or both were reported in 13 studies that compared the LightCycler SeptiFast Test MGRADE with blood culture. In most of these studies, it was unclear if the duration of stay was recorded from before, during or after blood sampling. Also, most of the studies did not present comparative data. Of the 4 studies that did report between group differences, 1 study (Alvarez et al. 2012) reported a statistically significant difference (p<0.05) in intensive care unit and hospital duration of stay in favour of the LightCycler SeptiFast Test MGRADE. Three other studies (Idelevich et al. 2014; Mancini et al. 2014; Rodrigues et al. 2013) reported no significant difference in duration of stay.
One RCT (Tafelski et al. 2015) reported a duration of empirical antimicrobial therapy (antibiotics which are prescribed based on clinical presentation) of 18.8 hours (SD 5.6) for patients in the LightCycler SeptiFast Test MGRADE group compared with 38.3 hours (SD 14.5) for patients in the blood culture with MALDITOF MS group.
There were 14 studies that reported details of change in antimicrobial treatment, 10 of which did not report comparative data. Three studies compared the LightCycler SeptiFast Test MGRADE with blood culture. One RCT (Rodrigues et al. 2013) reported that therapy was adjusted for 35% of patients in the LightCycler SeptiFast Test MGRADE group compared with 24% of patients in the blood culture group. In contrast, a further RCT (Idelevich et al. 2015) reported that 9.5% of patients in the LightCycler SeptiFast Test MGRADE had an adjustment to therapy compared with 10.5% in the blood culture group. One study based on propensity score matching (Mancini et al. 2014) reported no differences in management.
One RCT (Tafelski et al. 2015) compared the LightCycler SeptiFast Test MGRADE with blood culture plus MALDITOF MS. Testing with the LightCycler SeptiFast Test MGRADE resulted in a change of treatment for 9.8% of patients compared with 13.5% of patients in the blood culture plus MALDITOF MS group.
Mortality data were reported in 17 studies, 12 of which reported data on a cohort level only. The mortality rates reported ranged from 4% to 61%; but the length of followup was highly variable across the studies. One study (Alvarez et al. 2012) reported no statistically significant differences between the LightCycler SeptiFast Test MGRADE and blood culture for both 28day and 6month mortality. One other study (Rodrigues et al. 2013) also reported no statistically significant difference in 28day mortality.
One propensity score matching study (Mancini et al. 2014) reported no statistically significant difference in mortality (p=0.39) between a prospective cohort (LightCycler SeptiFast Test MGRADE) and retrospective cohort (blood culture). Although, when more strict matching criteria were applied, the LightCycler SeptiFast Test MGRADE was associated with a statistically significant reduction in mortality (3.13% compared with 14.71%; p=0.04). A reduction in mortality associated with using the LightCycler SeptiFast Test MGRADE was reported in 2 further studies (Idelevich et al. 2015; Tafelski et al. 2015), but the reductions were not statistically significant.

## SepsiTest
One study (Loonen et al. 2014) reported a mortality rate of 3.2% for the study cohort but the duration of followup was not reported. In addition, Schreiber et al. (2013) reported an intensive care unit mortality rate of 16% and a 28day mortality rate of 24% for the study cohort.
No other intermediate or clinicaloutcome data were reported for the SepsiTest.

## IRIDICA BAC BSI
One study, which was unpublished at the time of guidance development, reported data relevant to testfailure rates for the IRIDICA BAC BSI assay. These data are considered to be academic in confidence and cannot be reported at this time.
One study (Vincent et al. 2015) reported that an adjudication panel of 3 clinical experts retrospectively recommended a change in management based on the IRIDICA BAC BSI assay for 41% of all patients. This increased to 57% of patients when the IRIDICA BAC BSI assay was positive and blood culture was negative.
One study (Vincent et al. 2015) reported a mortality rate of 29% for the study cohort, but did not report the duration of followup.

# Costs and cost effectiveness
The External Assessment Group conducted a search to identify studies investigating the cost effectiveness of the LightCycler SeptiFast Test MGRADE, SepsiTest or the IRIDICA BAC BSI assay. The External Assessment Group also constructed a conceptual economic model to determine the cost effectiveness of the technologies.

## Systematic review of cost-effectiveness evidence
Four studies were included and were assessed according to their relevance to the decision problem: 3 studies included the LightCycler SeptiFast Test MGRADE, 2 of which were withinstudy costminimisation analyses (that is, a costminimisation analysis conducted within a clinical study), and 1 was a costeffectiveness analysis. The remaining study included a costminimisation analysis of the IRIDICA PLEXID hybrid assay. The target population, condition and setting varied across the 4 studies.
The 2 studies that were withinstudy costminimisation analyses of using the LightCycler SeptiFast Test MGRADE when compared with blood culture reported cost savings of 178.75 per sample (Mancini et al. 2014) and 183.00 per patient (Alvarez et al. 2012). The third study, Lehmann et al. (2010), reported incremental costeffectiveness ratios (ICERs) of 11,477 per incremental survivor and 3,107 per qualityadjusted life year (QALY) gained when using the LightCycler SeptiFast Test MGRADE compared with blood culture. When the use of an IRIDICAPLEXID hybrid system was compared with blood culture, Bilkovski et al. (2014) reported cost savings of $1,123,372 per 422 tests. None of the studies considered the effect of a potential reduction in antibiotic resistance. The External Assessment Group concluded that the existing economic evaluations had limited relevance to either the UK or the decision problem because of differences in patient populations, costs of the interventions and standard care. In particular, Mancini et al. (2014) included haematology patients for whom relatively expensive empirical antifungals were prescribed that are unlikely to be representative of the UK treatment pathway.

## Economic analysis
The External Assessment Group developed a conceptual economic model designed to explore the cost effectiveness of the LightCycler SeptiFast Test MGRADE, SepsiTest and the IRIDICA BAC BSI assay. The population included in the model was hospitalised patients with suspected bloodstream infection.
The model comprised a decision tree with a lifetime time horizon and took the perspective of the NHS and personal social services. The key clinical outcomes included in the model were 30day mortality, duration of stay in intensive care unit, duration of hospital stay and antimicrobial treatment.
Data on the diagnostic accuracy of the interventions, intermediate outcomes and clinical outcomes were taken from the clinicaleffectiveness systematic review when possible. Expert opinion was also sought to populate key clinical outcomes and supplement the data available from the systematic review. Routine sources of costs and prevalence data were also used when appropriate. A discount rate of 3.5% per annum was applied to both costs and effects. The potential effect of the tests on antimicrobial stewardship was not included in the model, because there was insufficient evidence to show how the tests would affect antimicrobial use and the subsequent development of resistant organisms.
The incremental cost per test was calculated using the cost of the test, the net effect on duration of intensive care unit and hospital stay, and changes in the costs of antimicrobial treatment. The estimated cost per day for an intensive care unit bed was 1057 and for a general ward bed was 275. A course of empirical antimicrobial treatment was estimated to cost 350.
It was assumed that the cost per test was dependent upon both test throughput and whether laboratory equipment needed to be bought to use the tests. The range of technology costs included in the model were as follows:
LightCycler SeptiFast Test MGRADE 153.67 to 205.54
SepsiTest 108.30 to 149.53
IRIDICA BAC BSI 197.35 to 314.61
MALDITOF MS 6.94 to 232.39.
Incremental QALYs were calculated by assuming 11.32 discounted QALYs per 30day mortality avoided, based on the estimated number of discounted life years for an adult patient with sepsis and the estimated quality of life after an episode of sepsis. The model assumed a mean age of 58 years and that 60% of the cohort were male. Patients were assumed to have a utility value of 0.68 at 5 years after an episode of severe sepsis (Cuthbertson et al. 2013) unless the utility value predicted for the age and sex profile of a patient in the general population was lower. In these instances, the lower utility value was applied.

## Economic-analysis results
Five deterministic analyses were done:
Base case 1: interventions compared with blood culture, with clinicaloutcome data taken from the systematic review.
Base case 2: interventions compared with blood culture, with clinicaloutcome estimates taken from expert opinion.
Threshold analyses.
Interventions compared with MALDITOF MS.
Data taken from studies comparing more than 1 intervention.
The following assumptions were common to all analyses:
The only parameter to affect QALY gain or loss was 30day mortality rate.
Negative rapid tests did not affect any of the 4 key outcomes.
Failed rapid tests did not affect any of the 4 key outcomes.
If 2.4 tests per day were run, laboratories ran tests Monday to Friday only, with 3 times the number of tests run on Monday to account for samples building up over a weekend.
If 17 or 68 tests per day are run, laboratories did 3 runs per day and worked 24 hours a day, 7 days a week.
The purchase cost of machines needed for the interventions and comparators was equally divided over 7 years of use.
It was assumed that no additional staff costs or laboratory estate costs were incurred when using the interventions.
The time scale of testing was 1 year although discounted QALYs accrued in subsequent years were included.
Incremental QALYs were accrued through the number of avoided 30day mortalities.
If accuracy data from Warhurst et al. (2015) were used, the LightCycler SeptiFast Test MGRADE had a failure rate of 6.9%. A failure rate of 1.4% was assumed when pooled accuracy data was used.
IRIDICA BAC BSI had a failure rate of 1.9%.
SepsiTest had a failure rate of 0%.
Patients were treated with either 18 g per day of piperacillin/tazobactam or 3 g per day of meropenem for 7 days.
day mortality rates were assumed to be either 13% or 29%.
MALDITOF MS was only used on positive samples (8.7% of all blood cultures).
MALDITOF MS had a sensitivity of 79.8% at species level compared with blood culture.
LightCycler SeptiFast test MGRADE diagnostic-accuracy data were derived from Warhurst et al. (2015) unless otherwise specified.
SepsiTest and IRIDICA BAC BSI diagnostic-accuracy data were derived from the External Assessment Group's metaanalyses unless otherwise specified.
In this analysis, clinicaloutcome data from the clinicaleffectiveness review were included. This resulted in the assumption that there were no clinical benefits associated with the interventions for 30day mortality, duration of stay in the intensive care unit or duration of stay in hospital. The costs of antimicrobials were also unchanged in this analysis. All interventions were compared with blood culture only.
The results of the analysis showed that all the interventions were dominated by blood culture (that is, blood culture was less expensive and more effective than all of the interventions). Regardless of the test throughput assumed in different scenarios, the interventions remained dominated (more expensive with no additional clinical benefit) because of the lack of QALYs gained.
In addition, a threshold analysis was done for base case 1 to assess the reduction in antimicrobial costs that would be needed for each intervention to be cost neutral. The results suggested that the reductions needed would be 44% to 59% for the LightCycler SeptiFast Test MGRADE, 31% to 43% for the SepsiTest and 56% to 90% for the IRIDICA BAC BSI, although the rate of positive tests associated with each intervention suggested that their costs could not be offset solely by a reduction in antimicrobial therapy use.
In this analysis, the key clinicaloutcome parameters were populated using an average of estimated values provided by clinical experts. The External Assessment Group used these values in a range of scenarios that assumed a 30day mortality rate of either 13% or 29%, a throughput of 2.4, 17 or 68 tests per day and a maximum acceptable ICER of 20,000 or 30,000 per QALY gained. The comparator used in this analysis was blood culture.
For each scenario, the net monetary benefit of each intervention was estimated. A positive net monetary benefit suggests that the benefits associated with the intervention outweigh the costs, and the intervention with the largest net monetary benefit is estimated to be the most cost effective. MALDITOF MS was also included in the analysis to estimate the relative cost effectiveness between the 2 comparators included in the assessment.
In all scenarios modelled, MALDITOF MS produced a positive net benefit compared with blood culture. In 1 scenario (30day mortality rate 13%, 2.4 tests per day, maximum acceptable ICER of 20,000 per QALY gained), SepsiTest had the highest net monetary benefit when it was assumed that equipment to run the test had to be bought. In the same scenario, the IRIDICA BAC BSI assay had the highest net monetary benefit when only the test reagents and consumables were purchased. In all other modelled scenarios, the IRIDICA BAC BSI assay had the highest net monetary benefit.
ICERs were also calculated using the data from expert opinion. When it was assumed that no additional equipment had to be bought or the 30day mortality rate was 29%, the ICERs became more favourable because of either a decrease in incremental costs or an increase in incremental QALY gain.
The External Assessment Group also explored the effect of applying the pooled estimates of sensitivity and specificity from the metaanalyses to the LightCycler SeptiFast Test MGRADE. This assumption produced more favourable ICERs for the LightCycler SeptiFast Test MGRADE through increasing the estimated sensitivity of the test (65% pooled estimate compared with 51% from Warhurst et al. 2015), while maintaining specificity at 86%.
The External Assessment Group used a range of threshold analyses to explore the effect of key clinical outcomes. In all analyses, it was assumed that the comparator equipment had already been bought but that the equipment for the interventions needed to be bought. The threshold levels resulting from the analyses, which assumed 2.4 tests run per day and a maximum acceptable ICER of 20,000 per QALY gained, suggested reductions in 30day mortalities ranging from 0.09 to 0.14 per 100 tests would be needed for the interventions to be considered cost effective compared with blood culture. Antimicrobial costs would need to reduce by 149.53 to 314.61 per 100 tests. The results were similar when the interventions were compared with MALDITOF MS. The threshold analyses that assumed either 17 or 68 tests run per day produced lower threshold values. The values of the reductions needed were also lower when a maximum acceptable ICER of 30,000 per QALY gained was assumed.
The External Assessment Group also explored the cost effectiveness of both the LightCycler SeptiFast Test MGRADE and SepsiTest compared with MALDITOF MS, based on data from 2 studies (Tafelski et al. 2015; Loonen et al. 2014) that used MALDITOF MS in addition to blood culture. The effect estimates based on expert opinion were also included in the analysis. It was assumed that both interventions had a failure rate of 0% and that equipment to run the tests needed to be bought. The results of these analyses suggested that when compared with MALDITOF MS (and blood culture), the LightCycler SeptiFast Test MGRADE dominated (less costly and more effective) MALDITOF MS (and blood culture), and SepsiTest had ICERs ranging from 23,375 to 34,848 per QALY gained with a 30day mortality rate of 13% and from 10,479 to 15,621 per QALY gained with a 30day mortality rate of 29%.
An analysis was run using data from 2 studies (Schreiber et al. 2013; Leitner et al. 2013), which evaluated both the LightCycler SeptiFast Test MGRADE and SepsiTest with blood culture. The analysis was done to compare the relative costeffectiveness estimates with those derived in base case 2 that were based on indirect comparisons of the relative effectiveness of the interventions from expert opinion. The analysis assumed a 0% testfailure rate for both interventions and that equipment to run the tests needed to be bought. A range of scenarios were presented with 30day mortality rates of 13% or 29% and a throughput of 2.4, 17 or 68 tests per day. In all scenarios, the ICER for the LightCycler SeptiFast Test MGRADE was greater than 30,000 per QALY gained when compared with SepsiTest.# Considerations
The Diagnostics Advisory Committee reviewed the evidence available on the clinical and cost effectiveness of using the LightCycler SeptiFast Test MGRADE, SepsiTest and IRIDICA BAC BSI assay to rapidly identify bloodstream bacteria and fungi in people with a suspected bloodstream infection.

# Clinical effectiveness
The Committee considered the evidence for the diagnostic accuracy of each of the rapid molecular tests compared with blood culture. It noted that 54 studies reported data for the LightCycler SeptiFast Test MGRADE, 6 of which included children or neonates, 4 reported data for SepsiTest and 4 reported data for the IRIDICA BAC BSI assay. The Committee noted that most of the included studies were considered to have unclear risks of bias, particularly about details of the reference standard and the populations included in the studies. The Committee considered that the unclear risk of bias was attributable to poor reporting in the studies, and concluded that it was not possible to adequately assess the quality of the studies included in the diagnostic accuracy metaanalyses.
The Committee noted that 2 studies compared either the LightCycler SeptiFast Test MGRADE or SepsiTest with MALDITOF mass spectrometry (MS). It considered that both of these studies had relatively small sample sizes and it is likely that the results are not applicable to the UK because of differences in clinical practice in Europe, where the studies were done. The Committee concluded that there was insufficient evidence to establish either the diagnostic accuracy or the clinical utility of the rapid molecular tests against this comparator. Also, because of insufficient clinical data, the Committee concluded that there was too much uncertainty in the analyses for it to be confident that the rapid molecular tests would be cost effective compared with MALDITOF MS.
The Committee questioned the assumption in the diagnostic accuracy metaanalyses that blood culture is 100% accurate and noted that clinical specialists consider it to be an imperfect reference standard. It heard from the External Assessment Group that on this basis it was possible that the pooled estimates of sensitivity and specificity had been underestimated in the analysis, and so the rate of false positive and false negative results may also have been overestimated. The Committee discussed the reasons for false positive results with the rapid molecular tests. It noted that false positives may be real false positives in situations in which the rapid molecular test detects DNA from contaminant organisms in the blood sample which may result from the testing process. But it also heard from clinical specialists that it is possible that the rapid molecular tests may give more accurate results in some scenarios, such as the detection of fastidious organisms that may not grow in culture. The Committee also heard from clinical specialists that the rapid molecular tests may detect transient bacteraemia in some people, but that the clinical implications of this are not fully understood. It is possible that people may have extended courses of antibiotics and stay in hospital for longer if transient bacteraemia is detected. The Committee concluded that although the sensitivity and specificity may have been underestimated in the metaanalyses, the absence of data on the clinical significance of discordant results means that the size of any underestimation cannot be determined.
The Committee discussed the number of positive blood cultures in the included diagnostic accuracy studies and their prevalence in clinical practice. It heard from clinical specialists that blood culture is often negative in practice, with only around 10% of blood cultures being positive. The Committee considered that the low prevalence of positive blood cultures was likely to mean that there would be a relatively low number of false negative rapid molecular test results in routine practice. Also, the absolute rate of falsepositive rapid molecular test results is likely to be high because of the greater prevalence of negative blood cultures. The Committee concluded that although the absolute number of falsenegative rapid molecular test results was likely to be low in practice, the consequences of changing antimicrobial treatment in this group could be severe.
The Committee discussed the studies included in the clinicaloutcomes systematic review. It noted that fewer studies reported clinicaloutcome data compared with diagnosticaccuracy data, and that studies typically reported data for the LightCycler SeptiFast Test MGRADE only. The Committee noted that most of the studies were done in Europe or the USA and questioned the applicability of the clinicaloutcome studies to the UK. It heard from clinical specialists that although the treatment of sepsis is based on international guidelines, clinical outcomes such as duration of intensive care unit stay and duration of antimicrobial therapy cannot usually be applied to the UK from international studies because of differences in antibiotic prescribing practices. The Committee concluded that although the included studies provide some indication of the likely effect of the rapid molecular tests on clinical outcomes, additional UKbased studies are needed to show the clinical utility of the tests in practice.
The Committee considered the test turnaround times reported in the studies and heard from clinical specialists that the shorter times seen in research studies are unlikely to be seen in routine clinical practice, unless a molecular service is available 24 hours a day. It noted that 24hour services may become available if microbiology laboratories are joined into networks or centralised, but that this was unlikely to happen in the very near future. The Committee also noted that in some studies, the reported testfailure rates for the LightCycler SeptiFast Test MGRADE were high (up to 24.2%) and considered that this could further affect its potential to rapidly deliver information for clinical decisionmaking. The Committee questioned why the reported testfailure rates were high in some studies but heard from the External Assessment Group that the reasons for the failed tests were not reported. The Committee concluded that faster reporting of results is highly dependent on laboratory infrastructure and that the turnaround times needed to gain benefits from the rapid molecular tests are unlikely to be achieved in routine practice.
The Committee considered the data for mortality and duration of intensive care unit or hospital stay and noted that the studies were unlikely to have had sufficient power to detect statistically significant differences for these clinical endpoints. The Committee also noted that most of the studies did not report statistically significant differences between the rapid molecular tests and standard practice. Also, it heard from clinical specialists that both mortality and duration of stay among people with suspected bloodstream infection are likely to be influenced by multiple factors, and that any differences are unlikely to be solely because of the use of a rapid molecular test. The Committee concluded that mortality and duration of stay may not be appropriate primary clinical outcomes for studies, and suggested that future studies should consider using change in antimicrobial prescribing as a surrogate clinical outcome.
The Committee discussed the plausibility of the rapid molecular tests having an effect on antimicrobial prescribing. It noted that the results of the clinicaleffectiveness analysis suggested that only small numbers of people, if any, would have changes made to their antimicrobial treatment plan. The Committee heard from clinical specialists that there may be some situations in which the rapid molecular tests could affect patient management. Also, it heard that these situations would be restricted to instances in which the rapid test was positive, because the current accuracy of the tests was not sufficient to convince clinicians to withdraw antibiotic therapy on the basis of a negative test result. The Committee concluded that although the rapid molecular tests might give results more quickly, it was unlikely that the information they give would have an effect on patients' treatment plans and antimicrobial prescribing at present.

# Cost effectiveness
The Committee discussed the results of the economic analyses and questioned whether the use of an imperfect reference standard to calculate the estimates of diagnostic accuracy for the rapid molecular tests could have introduced bias. The Committee heard from the External Assessment Group that negative results were assumed not to have an effect on outcomes and that falsepositive results were associated with benefits in the model. The Committee concluded that any underestimate of pooled diagnostic accuracy in the clinicaleffectiveness analysis is unlikely to have a substantial effect on the results of the economic model.
The Committee questioned the assumptions made about the number of tests processed per day. It heard from clinical experts that the estimates based on 68 tests per day were unrealistic and that a large service laboratory would be unlikely to get more than 40 blood cultures per day. The Committee noted that the External Assessment Group had also produced estimates based on 2.4 and 17 tests per day. The Committee heard from the External Assessment Group that an assumption of 68 tests per day was included as an extreme scenario to show the effect on the results of the economic analyses. The Committee concluded that the most representative scenarios in the economic analyses were those that assumed either 2.4 or 17 tests per day.
The Committee discussed the differences in the results produced in the 2 different base cases of the economic analyses. It noted that the main difference between the 2 base cases came from the difference in data source for clinical outcomes: base case 1 used data taken from the systematic review, and base case 2 used data based on expert opinion. The Committee noted that the systematic review suggested that the rapid molecular tests had no effect on clinical outcomes, but some of the clinical experts thought that the tests may be beneficial, although their estimates of the size of the benefit varied widely. The Committee concluded that the tests may offer clinical benefit, but there is too much uncertainty in the size of the benefit to determine the effect of introducing the tests into clinical practice. The Committee also noted that the incremental costeffectiveness ratios (ICERs) in base case 2 ranged from the rapid molecular tests being more costly and equally effective (dominated) than blood culture, to being less costly and more effective (dominant) than blood culture alone, when using estimates from individual clinicians. The Committee considered that the wide range of ICERs resulted from the high level of variation between the clinicians' estimates. The Committee concluded that the effect of introducing the rapid molecular tests on NHS resources was highly uncertain and that the results of the economic analyses were subject to substantial uncertainty.
The Committee considered the likely effect of the costs and outcomes that had been excluded from the economic analyses and noted that these included laboratory overhead and additional staff costs, and clinical benefits that may be accrued through improved antimicrobial stewardship. The Committee noted that because the results of the clinicaleffectiveness analysis suggested that the effect of the rapid molecular tests on antimicrobial prescribing was highly uncertain, it would have been inappropriate to extrapolate the clinical outcomes to estimate an effect on antimicrobial stewardship. It also noted that the rapid molecular tests would most likely increase laboratory overhead costs, and possibly staff costs, and concluded that because of the clinical uncertainties their absence from the economic analyses was unlikely to have a substantial effect.
The Committee considered the results of the threshold analyses and noted the reductions in antimicrobial costs that would be needed for the tests to be considered cost effective. The Committee noted that this ranged from 823.34 to 1482.28 per 100 positive tests, depending on whether the rapid molecular tests were compared with blood culture or blood culture plus MALDITOF MS. The Committee concluded that because of the prevalence of positive tests in clinical practice, the costs of the rapid molecular tests were unlikely to be offset by reduced antimicrobial costs alone.
The Committee noted that the economic analyses did not include neonates and children, and that the model was based on an adult population with a mean age of 58 years. The Committee considered that the estimated qualityadjusted life year (QALY) gain by avoided 30day mortalities would be greater for children and neonates because of their greater number of life years remaining, but accepted that there were insufficient clinicalutility data for this population for an economic analysis.

# Additional considerations
The Committee considered the potential benefits of the interventions in practice. It heard from clinical experts that because the tests can be used directly on whole blood samples, they may be able to give information on a pathogen's identity earlier in the care pathway than tests that need incubated blood samples or samples from culture plates, which could be beneficial for antimicrobial stewardship. Also, it heard that the information from the rapid molecular tests may be used to modify a person's antimicrobial therapy, particularly when empirical antimicrobial therapy (antibiotics which are prescribed based on clinical presentation) has been prescribed. The Committee concluded that one of the key claimed benefits of the rapid molecular tests is their potential to contribute towards antimicrobial stewardship.
The Committee considered that because the rapid molecular tests need to be used in addition to blood culture for antimicrobial susceptibility testing, they may be less suitable for use in neonates and children. The Committee heard from clinical experts that this is a particular issue for tests that need a large volume of whole blood. The Committee also heard from clinical specialists that using a lower volume of blood from these patients for the molecular tests may have an adverse effect on the test's sensitivity and concluded that further exploration of these analytical issues should be encouraged.

# Research considerations
The Committee discussed the value of developing research recommendations for the rapid molecular tests. The Committee considered that for the tests to have clinical utility in both research settings and routine practice, clinicians would need to be certain that the tests are sufficiently accurate, and be confident that basing antimicrobial prescribing decisions on the results of the tests would not lead to adverse outcomes for people. The Committee noted that the reported accuracy data from the systematic review were unlikely to be sufficient for clinical decisionmaking at present. The Committee concluded that further research in the UK is needed to determine the clinical scenarios in which the tests may offer most benefit in clinical decisionmaking and to quantify their clinical utility. The Committee also considered that future studies should investigate using the rapid molecular tests in conjunction with other biomarkers, such as procalcitonin, and diagnostic tests that may be used to assess people with suspected sepsis.
The Committee considered that, conceptually, the molecular tests show promise for the early identification of fungal pathogens in people who are thought to be at increased risk of developing invasive fungal infections. The Committee concluded that if the accuracy of the tests was sufficient to guide clinical decisionmaking in this population, they could offer substantial value and address a clinically unmet need. The Committee encouraged future studies in this population and highlighted that the studies should aim to quantify the clinical utility of the rapid molecular tests, including their effect on antifungal prescribing. The Committee noted that studies planned by the National Institute for Health Research Health Technology Assessment Programme may investigate the use of rapid tests for identifying fungal pathogens.
The Committee considered the utility of further research to quantify the levels of certainty about the results of rapid molecular tests, which clinicians need to have before they decide to change treatment and level of care for patients. The Committee noted that the results of an elicitation exercise could be used to guide the development of future diagnostic tests that are designed to be used to change treatment plans for patients who are acutely unwell, and wished to encourage this research.
The Committee considered that because an increasing number of microbiology laboratories are adopting MALDITOF MS for rapidly identifying bloodstream bacteria and fungi, future studies aiming to establish the clinical utility of rapid molecular tests should include this technology as a comparator when possible.
The Committee noted that there was insufficient evidence to determine whether the tests were clinically effective in children and neonates. It wished to encourage the inclusion of these populations in future research studies, and noted that particular consideration should be given to establishing whether the blood volumes needed for the tests in this assessment are suitable for these populations.