Evaluation of commercial molecular screening platforms for the detection of food-borne 

bacterial pathogens by FSIS Field Service Laboratories 

Abstract 

Four commercially available molecular screening platforms were evaluated against the 

FSIS/OPHS Microbiology Laboratory Guidebook (MLG) reference cultural methods for 

Salmonella sp., Listeria monocytogenes, Campylobacter sp., and Shiga toxin-producing 

Escherichia coli (STEC) composed of E. coli O157 and non-O157 STECs. A variety of 

representative food product samples were inoculated with target microorganisms at a 

fractional recovery range of 20-80%. Each screening platform was used by FSIS/OPHS Field 

Service Laboratory (FSL) personnel to analyze a total of 440 Salmonella inoculated samples, 120 

L. monocytogenes inoculated samples, 120 Campylobacter inoculated samples, and 180 Shiga 

toxin-producing E. coli (STEC) inoculated samples. Sixty of the STEC-specific samples were 

inoculated with E. coli O157. Limits of detection, inclusivity, and exclusivity panels were 

examined along with an internal evaluation of the strengths and weaknesses of each 

technology during use in the FSIS FSL high-throughput microbiology screening environment. All 

platforms evaluated had a high degree of concordance when compared to the FSIS MLG 

reference cultural methods, and met the performance expectations for predictive value, 

sensitivity, and selectivity. 

Introduction 

USDA FSIS assessed several high-throughput, multi-analyte molecular screening platforms for 

the detection of bacterial pathogens in food products and food producing environments 

regulated under the Poultry Inspection Act (PIA), Egg Products Inspection Act (EPIA), and the 

Federal Meat Inspection Act (FMIA), including Siluriformes fish, as added under the 2008 and 

2014 Farm Bills. The targeted microorganisms included: Salmonella, Shiga toxin-producing 

Escherichia coli serotype O157:H7 and non-O157 (STEC), Listeria monocytogenes, and 

Campylobacter spp. 

Based upon the criteria devised by FSIS for the selection of the screening technology, for each 

targeted microorganism, USDA/FSIS tested a minimum of three (3) food product matrices with 

sixty (60) samples per method-. Samples were artificially inoculated with the target 

microorganism at fractional recovery levels (20-80% recovery). A paired-study design was used 

to compare the MLG reference method to the alternative molecular screening method 

associated with the tested commercial instruments. Laboratory evaluations included a review 

of limit of detection, exclusivity and inclusivity panels, and a review of each technology for 

fitness-for-use as defined by the bench analysts at the three FSIS Field Services Laboratories 

The USDA FSIS Field Service Laboratories (FSLs) laboratories evaluated the performance of each 

platform compared to the cultural methods described in the FSIS MLG (reference method[s]). 

For each microorganism, USDA FSIS tested a minimum of three food product types with sixty 

(60) samples per method. Food product samples were inoculated with the target 
 

 

 

 

 

 

 

 

 

 

 

microorganisms at fractional recovery levels. A paired-study design was used to compare the 

performance of each platform (predictive value, sensitivity, selectivity) to the MLG reference 

cultural methods. 

Materials and Methods 

Microbiology Laboratory Guidebook (MLG): 

https://www.fsis.usda.gov/wps/portal/fsis/topics/science/laboratories-andprocedures/guidebooks-and-methods/microbiology-laboratory-guidebook/microbiologylaboratory-guidebook 

Each instrument was assigned a letter designation prior to the start of the evaluation to prevent 

any secondary bias when assessing the results. The instruments were evaluated with food 

product matrices inoculated with target microorganisms at fractional recovery levels or the 

reference cultural methods (FSIS MLG Chapters 4, 5, 5B, 8 and 41). Fractional recovery is 

defined as 20-80% positive cultural reference MLG method results from the inoculated samples 

If a set of inoculated samples did not meet an acceptable fractional recovery range of 20-80% 

based on the MLG cultural reference method, a new set of samples were inoculated. 

Evaluated instruments were identified using a lettering scheme consistent in each FSIS 

laboratory. Each laboratory was assigned a matrix to evaluate along with the appropriate 

number of spiked sample (see Table 1 for a full list of food product and other included 

matrices). In addition, at least twenty of each sample type from the field were analyzed by the 

laboratories for the analyses appropriate to the food product matrix: ready-to-eat (RTE), 

ground beef, chicken rinses, and swabs. Data was not be used from any spiked set of samples 

that did not meet the appropriate fractional recovery range for the MLG cultural reference 

methods. 

Each set of 10 samples included an un-inoculated food product matrix control, a 

positive control, a media sterility control, and in some cases a negative control as 

outlined in the MLG chapter for each microorganism: Salmonella enterica Typhimurium 

(Microbiologics Sal54 01223 UV-V FDA Sal 5694); Escherichia coli O157:H7 

(Microbiologics EC43 01227 UV-V FDA ESC 1177); E. coli O26:H2 (USDA, ARS #TB285); 

E. coli O45:H2 (USDA, ARS #96-3285); Listeria monocytogenes (Microbiologics 

01248UV-V); L. innocua (ATCC 33090, negative control); and Campylobacter jejuni 

(ATCC 33291). 

Sample Preparation. 

Food product sample sizes and enrichment volumes were consistent with MLG protocols. For 

MLG 8 (L. monocytogenes), samples were enriched with UVM, and carried forward to MOPSBLEB. All MOPS-BLEB tubes are plated onto MOX and carried forward to HL; no direct plating to 

MOX was performed. 
 

 

 

 

 

 

 

 

 

 

 

 

 

Inoculum preparation 

Samples were spiked by culturing each organism 18-24h on TSA/SB. See MLG chapters 4, 5, 8, 

and 41 for control culture preparation. Suspensions of the target microorganisms were 

prepared in 0.85% saline of 0.5 McFarland (approximately 108-109 cells per mL). Serial dilutions 

were prepared to attain stock suspensions used to inoculate the food product matrix at the 

appropriate concentration to achieve fractional recovery. For Salmonella, Listeria, and STECs, 

the stock inoculum was plated on PetrifilmTM APC in duplicate and incubated at 35 ± 1°C for 24 ± 

2 hours to determine actual levels. For Campylobacter, 0.1 ml of serial dilutions was plated 

from the stock solution in duplicate on TSA/SB plates and incubated for 24 ± 2 hours at 42 ± 

1.0oC under microaerobic conditions to determine the actual level. 

MLG cultural method endpoint 

The MLG cultural reference comparison terminated for each sample set at the following points 

in each method; MLG Chapter 4: DMLIA and BGS plating; MLG Chapter 5 and 5B: latex 

agglutination; MLG Chapter 8: HL plating; MLG Chapter 41: microscopic examination and latex 

agglutination. These points of termination were chosen as the best points to assure recovery of 

the microorganisms to determine the false negative and false positive rates of each technology 

compared to the reference methods. 

Inclusivity/Exclusivity determination 

To evaluate inclusivity of each instrument and screening method, at least 30 strains of the 

target microorganism for each method were chosen and tested using pure culture. To evaluate 

exclusivity of each technology, at least 20 strains of the target analyte for each method were 

chosen and tested using pure culture. When available, strains were chosen that have nontypical characteristics (for example, stx2e, stx2g, Salmonella enterica subspecies III, non-β 

hemolytic Listeria monocytogenes strains, stx-negative E. coli O157:H7). The pure culture work 

was performed directly from colonies on the plating medium as well as from DNA templates 

prepared from pure culture. 

Limits of Detection 

The limit of detection (LOD) was evaluated for each assay by spiking incubated, 

screen-negative enrichment media with estimated target levels prepared from serially diluted 

inoculum. The enrichment medium was inoculated with levels ranging from 102-107 cells/ml 

and tested using the assay kit appropriate for each target. The test at each inoculum level was 

performed in triplicate. The LOD was the inoculum dilution at which 100% detection was no 

longer achieved. The lowest pure culture inoculum was plated in triplicate on plating medium 

or Petrifilm to determine the 95% Confidence Interval (CI) for LOD. 
 

 

 

 

 

 

 

 

 

Results 

Instrument/molecular screen comparison. 

Four molecular screening instruments were evaluated against the applicable MLG reference 

cultural methods. Technology A was evaluated for Salmonella, L. monocytogenes, E. coli O157, 

and Campylobacter. Technology B was evaluated for Salmonella, STEC, and L. monocytogenes. 

Technologies C and D were evaluated for Salmonella, STEC, L. monocytogenes, and 

Campylobacter. Samples were prepared from a variety of representative food products 

(“matrices”) (Tables 1 through 5) and inoculum levels (CFU) for each matrix were at fractional 

recovery levels for each method. Each result was compared against the MLG reference cultural 

method to evaluate false positive and false negative rates at fractional recovery levels. Any 

inoculated samples that did not meet fractional recovery of 20-80% as verified by the MLG 

reference cultural method were discarded and a new set of samples were prepared. If a 

technology provided a result described as indeterminate, inhibited, or error it was treated as a 

positive for the purposes of this evaluation. 

Positive results for each technology are listed in Tables 1-5 by microorganism and matrix along 

with the final results of the MLG reference cultural method. Statistical analysis was performed 

on the data using Kappa Coefficient, a statistical test of agreement, to evaluate recovery for 

each technology against the existing MLG reference cultural methods for similarity. Kappa 

values of 0.80 – < 1.00 were interpreted as very good agreement. The Kappa Coefficient values 

of each method are listed below for each instrument by microorganism (Table 6). Statistical 

analysis was also performed on the data using McNemar test for Dependent/Paired Samples, 

comparing each technology with the culture reference method (Table 6). A p-value of < 0.05 

indicates results from the technology are statistically different than the MLG reference culture 

method. A p- value of ≥ 0.05 indicates results are statistically similar to the MLG reference 

culture method. 

Inclusivity/exclusivity evaluation 

Inclusivity and exclusivity panels were gathered from pure culture DNA from bacterial strains 

collected from USDA FSIS products. Each technology was evaluated using the strains to assess 

the ability of the technology to distinguish between a known positive (True Positive) and a 

known negative (True Negative). Overall, the technologies identified True Positives equally well 

across the organisms tested. Any of the False Positives (i.e. screen positive result for a known 

negative) had late cycle threshold (Ct) values (>35 cycles) and were likely artifacts or potential 

carry over (Table 7). As the isolates were being tested from purified DNA, it is expected that 

the isolate should be positive well before 35 cycles for a real-time PCR assay. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Limits of detection evaluation 

The limits of detection for the microorganisms of interest were examined for each screening 

instrument. Matrices were enriched and incubated according to the appropriate MLG method. 

A 0.5 McFarland saline solution (~108 cfu/mL) was made for each organism. Each of these 

solutions was used to prepare a 1:10 dilution series in saline buffer from 107 to 102 

concentrations. The solutions were plated in triplicate for the 104 to 102 dilution tubes. The 107 

tube for each organism was then used to make a 106-101 dilution series in triplicate using the 

enriched matrices. The dilution series were then analyzed on each screening instrument along 

with the appropriate controls to provide the LOD data (Table 8). 

During the evaluation the technologies provided results described as inhibited, indeterminate 

or unidentifiable. Following the current business rules of the USDA FSIS laboratories, these 

inhibited, indeterminate or unidentifiable results cannot be used to rule out the presence of the 

target analyte in the sample. Samples with these results would be carried forward like a 

positive result, using the cultural method for confirmation. Inhibited, indeterminate or 

unidentifiable results could indicate matrix interference, questionable test kit performance, or 

that random errors should be expected using a technology. 

Discussion 

The USDA FSIS laboratories are high throughput laboratories requiring technologies that are 

reliable and high-throughput, in order to protect the safety of the consumer while minimizing 

the commercial impact for the producers. The USDA FSIS laboratories seek to utilize 

technologies that show a high degree of accuracy in a qualitative screen coupled with ease of 

use and robustness in a high-throughput setting. According to the data collected during 

extensive comparisons by the FSIS Field Service Laboratories, the performance of each 

molecular screening instrument was not statistically significantly different from the MLG 

reference cultural methods. The Kappa values indicated a good to very good agreement of all 

instruments to each MLG reference cultural method. Accuracy calculations, which took into 

account the false positives and false negatives were greater than 90% for a majority of the 

instruments. Overall, the molecular screening technologies and associated screening methods 

correctly classified the organisms tested in the exclusivity and inclusivity panels. 
 

 

 

 

 

 

 

 

 

Table 1: Comparison of Positive Salmonella Results for Each Technology Compared Against 

the MLG Reference Culture Method Using Samples Inoculated for Fractional Recovery 

Matrix 

Sample 

Number 

Inoculum 

Level 

(CFU) 

A B C D 

Cultural 

Method 

MLG 4 

Raw Beef Products* 60 0.40-0.80 27 25 29 27 31 

Raw Poultry Products* 60 0.69-0.74 27 29 28 27 30 

Ready to Eat (RTE) 

Products* 

60 0.52-0.59 23 23 23 24 23 

Turkey Sponges 20 0.63-1.1 8 7 9 7 7 

Ground Chicken 20 0.50-1.3 7 5 5 5 9 

Smoked Catfish 20 0.50 7 7 6 8 8 

Environmental Sponges 20 0.69 7 7 7 7 7 

Meat Carcass Sponges 20 0.69-0.71 6 5 5 5 5 

Whole Eggs 20 0.60-0.73 7 7 14 5 7 

Raw Catfish 20 0.43-0.52 9 9 12 10 10 

Raw Pork 20 1.8 16 15 15 11 15 

Cecal Beef 30 0.40-1.0 14 16 16 17 16 

Cecal Pork 30 0.40-0.63 16 15 15 11 15 

Cecal Turkey 20 0.40-1.0 5 9 8 7 9 

Cecal Chicken 20 0.60-0.90 14 6 15 15 15 

*Raw Beef Products were composed of raw ground beef and raw beef trim 

Raw Poultry Products were composed of ground turkey, chicken rinses, and chicken part rinse 

RTE Products were composed of hot dogs and chicken nuggets 

Table 2: Comparison of Positive L. monocytogenes Results for Each Technology Compared Against the 

MLG Reference Culture Method Using Samples Inoculated for Fractional Recovery 

Matrix Sample 

Number 

Inoculum 

Level 

(CFU) 

A B C D 

Cultural 

Method 

MLG 8 

RTE Products* 60 1.2-1.5 36 36 36 38 36 

Environmental Sponges 20 0.63-0.72 4 4 4 4 4 

Smoked Catfish 20 1.3 10 10 10 14 15 

Whole Eggs 20 4.7-5.6 10 10 9 11 10 

*RTE Products were composed of hot dogs and chicken nuggets 

Table 3: Comparison of Positive Campylobacter Results for Each Technology Compared Against the 

MLG Reference Culture Method Using Samples Inoculated for Fractional Recovery 

Matrix Sample 

Number 

Inoculum 

Level (CFU) C D 

Cultural 

Method 

MLG 41 

Raw Poultry Products* 60 1.5-3.5 29 25 27 

Turkey Sponges 20 

0.34-3.0 

10 9 8 

Ground Chicken 20 6 6 6 

Environmental Sponges 20 7 7 7 

*Raw Poultry Products were composed of ground turkey, chicken rinses, and chicken part rinse 
 

 

 

 

 

Table 4: Comparison of Positive STEC Results for Each Technology Compared Against the MLG 

Reference Culture Method Using Samples Inoculated for Fractional Recovery 

Matrix Sample 

Number 

Inoculum Level 

(CFU) B C D 

Cultural 

Method 

MLG 5 & 5B 

Raw Beef Products* 60 0.70-1.4 22 27 23 28 

Raw Pork 60 0.97-1.0 34 35 31 37 

Environmental Sponges 60 0.69-0.71 34 30 34 34 

*Raw Beef Products were composed of raw ground beef and raw beef trim 

Table 5: Comparison of Positive E. coli O157 Results for Each Technology Compared Against the MLG 

Reference Culture Method Using Samples Inoculated for Fractional Recovery 

Matrix Sample 

Number 

Inoculum Level 

(CFU) A B C D 

Cultural 

Method 

MLG 5 

Raw Beef Products* 60 0.70-1.4 8 10 10 5 10 

Raw Pork 60 0.97-1.0 14 10 12 8 13 

Environmental Sponges 60 0.63-0.71 11 11 11 11 11 

*Raw Beef Products were composed of raw ground beef and raw beef trim 
 

 

 

 

 

 

Table 6: Statistical Comparison of Microorganism Recovery from Each Screening Technology (A-D) 

against the MLG Reference Cultural Methods 

McNemar Test Positive Test Negative Kappa p-value 

Salmonella A Positive 189 (42.95%) 5 (1.14%) .89 0.0043 (MLG 4) 

Negative 19 (4.32%) 227 (51.59%) 

B Positive 181 (41.14%) 5 (1.14%) .85 0.00010 

Negative 27 (6.14%) 227 (51.59%) 

C Positive 195 (44.32%) 15 (3.41%) .87 0.71 

Negative 13 (2.95%) 217 (49.32%) 

D Positive 185 (42.05%) 7 (1.59%) .86 0.0035 

Negative 23 (5.23%) 225 (5.23%) 

McNemar Test Positive Test Negative Kappa p-value 

A Positive 60 (50.00%) 0 (0.00%) .92 0.025 

Listeria Negative 5 (4.17%) 55 (45.83%) monocytogenes 

(MLG 8) B Positive 60 (50.00%) 0 (0.00%) .92 0.025 

Negative 5 (4.17%) 55 (45.83%) 

C Positive 59 (49.17%) 2 (1.67%) .87 0.16 

Negative 6 (5.00%) 53 (44.17%) 

D Positive 64 (53.33%) 1 (0.83%) .97 1 

Negative 1 (0.83%) 54 (45.00%) 

McNemar Test Positive Test Negative Kappa p-value 

Campylobacter C Positive 46 (38.33%) 6 (5.00%) .86 0.16 (MLG 41) 

Negative 2 (1.67%) 66 (55.00%) 

D Positive 43 (35.83%) 4 (3.33%) .84 0.74 

Negative 5 (4.17%) 68 (56.67%) 

McNemar Test Positive Test Negative Kappa p-value 

STEC B Positive 89 (49.44%) 1 (0.56%) .88 0.0067 (MLG 5 and 5B) 

Negative 10 (5.56%) 80 (44.44%) 
 

 

 

C Positive 

Negative 

D Positive 

Negative 

E. coli O157:H7 

(MLG 5) A Positive 

Negative 

B Positive 

Negative 

C Positive 

Negative 

D Positive 

Negative 

91 (50.56%) 

8 (4.44%) 

88 (48.89%) 

11 (6.11%) 

Test Positive 

32 (53.33%) 

2 (3.33%) 

29 (48.33%) 

5 (8.33%) 

32 (53.33%) 

2 (3.33%) 

24 (40.00%) 

10 (16.67%) 

1 (0.56%) .90 0.020 

80 (44.44%) 

0 (0.00%) .88 0.00090 

81 (45.00%) 

McNemar Test Negative Kappa p-value 

1 (1.67%) . 90 0.56 

25 (41.67%) 

1 (1.67%) .80 0.10 

25 (41.67%) 

1 (1.67%) .90 0.56 

25 (41.67%) 

0 (0.00%) .68 0.0016 

26 (43.33%) 

Table 7: Exclusivity and Inclusivity Results 
Listeria 

monocytogenes 

Test Test 

Positive Negative 

Technology 

A Inclusivity (Positive) 30 0 

Exclusivity (Negative) 0 20 

Technology 

B Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 19 

Technology 

C Inclusivity (Positive) 30 0 

Exclusivity (Negative) 0 20 

Technology 

D Inclusivity (Positive) 30 0 

Exclusivity (Negative) 6* 14 

Notes: * Late Ct value false positives (>35 cycles) 

Test Test 

Positive Negative 

Technology 

A Inclusivity (Positive) 52 0 

Exclusivity (Negative) 0 20 

Technology 

B Inclusivity (Positive) 52 0 

Exclusivity (Negative) 5* 15 

Technology 

C Inclusivity (Positive) 52 0 

Exclusivity (Negative) 7* 13 

Technology 

D Inclusivity (Positive) 52 0 

Exclusivity (Negative) 2* 18 

Notes: * Late Ct value false positives (>35 cycles) 

Salmonella 

Test Test 

Positive Negative 

Technology 

Campylobacter C Inclusivity (Positive) 30 0 

Exclusivity (Negative) 0 20 

Technology 

D Inclusivity (Positive) 30 0 

Exclusivity (Negative) 0 20 

Test Test 

Positive Negative 

Technology 

E. coli O157:H7 A Inclusivity (Positive) 28 1 

(O-group) Exclusivity (Negative) 0 20 
 

Technology 

B Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 20 

Technology 

C Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 20 

Technology 

D Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 20 

Technology 

E. coli O157:H7 B Inclusivity (Positive) 29 0 

(stx) Exclusivity (Negative) 0 20 

Technology 

C Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 20 

Technology 

D Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 20 

Technology 

E. coli O157:H7 B Inclusivity (Positive) 29 0 

(eae) Exclusivity (Negative) 0 20 

Technology 

C Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 20 

Technology 

D Inclusivity (Positive) 29 0 

Exclusivity (Negative) 0 20 

E. coli non-O157 

(O-group) 

Technology 

B 

Technology 

C 

Technology 

D 

E. coli non- O157 

(stx) 

Technology 

B 

Technology 

C 

Technology 

D 

Inclusivity (Positive) 

Exclusivity (Negative) 

Inclusivity (Positive) 

Exclusivity (Negative) 

Inclusivity (Positive) 

Exclusivity (Negative) 

Inclusivity (Positive) 

Exclusivity (Negative) 

Inclusivity (Positive) 

Exclusivity (Negative) 

Inclusivity (Positive) 

Exclusivity (Negative) 

Inclusivity (Positive) 

Test 

Positive 

24** 

N/A** 

Test 

Negative 

0 

N/A** 

64 

0 

0 

20 

64 0 

0 20 

64 

2* 

0 

18 

64 

3* 

0 

17 

64 

0 

64 

0 

20 

0 
 

 

E. coli non-O157 Technology 

(eae) B Exclusivity (Negative) 0 20 

Technology Inclusivity (Positive) 64 0 

C Exclusivity (Negative) 2 18 

Technology Inclusivity (Positive) 64 0 

D Exclusivity (Negative) 1 19 

Notes: * Late Ct value false positives (>35 cycles) 

** Technology Failure, no remaining kits 

Table 8: Limit of Detection Evaluation of Evaluated Screening Technologies using MLG methods 

Organism Matrix Technology A Technology B Technology C Technology D 

Raw Beef 104 104 104 104 

Salmonella RTE Catfish 

Cecal 

105 

104 

105 

103 

104 

103 

104 

104 

Turkey Sponges 104 

104 

104 

NA* 

NA* 

NA* 

103 

105 

104 

105 

NA* 

NA* 

102 

104 

102 

105 

103 

102 

104 

103 

104 

104 

104 

104 

L. monocytogenes RTE Catfish 

E. coli O157:H7 Raw Beef 

STEC Raw Beef 

Campylobacter Ground Chicken 

Turkey Sponges 

*NA=organism is not applicable for technology