General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 1  

2013 Johne’s Disease Fecal Proficiency Panel

General Summary

October 3, 2013

Overview

A total of 63 laboratories participated in the 2013 Johne’s Disease Fecal Proficiency Panel (7

Canadian, 3 European Union, 1 New Zealand, 1 Australian and 51 USA laboratories). Overall, the number

of laboratories that requested individual proficiency panels for direct PCR and liquid culture methods

increased from 2012 and decreased for solid culture methods. Requests for pooled proficiency panels

increased for direct PCR and solid culture methods, and decreased for liquid culture methods. Table 1

details the number of individual and pooled panels shipped and the overall pass/fail status for each

method. Laboratories could order multiple panels for each method and were notified of their

preliminary pass/fail status upon submission of their results. A total of 176 panels were requested;

results were not returned for 3 of them.  No panels were reported to be faulty this year. If preliminary

results indicated that the laboratory had failed, they were given the opportunity to retake the

proficiency panel provided the results be completed by September 30th, 2013. The results provided in

Table 1 include these retests. Laboratories that only used reagents from a single manufacturer, either

Tetracore or Applied Biosystems, are listed separately. Laboratories that use either in‐house reagents,

other commercial kits not marketed in the US, or mix commercial reagents are listed under the “In

House” category.

Table 1. Summary results of the 2013 Johne’s Disease Fecal Proficiency Panel. In order to pass results must meet the

criteria listed in the 2010 Uniform Program Standards for the Voluntary Bovine Johne's Disease Control Program.

# passed 

1st attempt 

(%)

# failed 

1st attempt 

(%)

# passed 

2nd attempt 

(%)

# failed 

2nd attempt 

(%)

# Kits 

not 

retested

Total 

Shipped

Total shipped in 

2012 (%change)

Individual Panel

Direct PCR (all) 48 (84%) 9 (16%) 2 (67%) 1 (33%) 4 61 62 (‐2%)

Tetracore 22 (92%) 2 (8%) 1 (100%) 25 23 (9%)

Li fe Technologies 15 (94%) 1 (6%) 16 25 (‐36%)

In‐House 11 (65%) 6 (35%) 1 (50%) 1 (50%) 19 13 (46%)

Liquid Systems (all) 23 (82%) 5 (18%) 2 (100%) 3 30 31 (‐3%)

MGIT 960 5 (56%) 4 (44%) 2 (100%) 11 8 (38%)

TREK 18 (95%) 1 (5%) 19 22 (‐14%)

HEY Solid Media (all) 18 (95%) 1 (5%) 1 19 21 (‐10%)

Individual Panel Total 89 (86%) 15 (14%) 4 (80%) 1 (20%) 8 110 114 (‐4%)

Pooling Panel

Direct PCR (all) 36 (97%) 1 (3%) 1 (100%) 39 36 (8%)

Liquid 19 (95%) 1 (5%) 1 21 23 (‐9%)

HEY 6 (100%) 6 6 (0%)

Pooled Panel Total 61 (97%) 2 (3%) 1 (100%) 1 66 65 (2%)
   

General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 2  

Individual Panel Description  

Each individual panel consisted of 25 unknown samples and one positive control. Positive

samples were collected from naturally infected cows, and negative samples were from individual

animals residing in non‐infected herds. Approximately 4 liters of fecal material were collected rectally

per animal, shipped to NVSL, aliquoted as soon as possible in individual vials, and stored at ‐70°C until

kits were distributed. Panels were assembled in groups, each with a different key (See Table 9 at the end

of this report for the key). Table 2 shows the categorical (positive/negative) performance for each

identification method by animal ID. Samples from cow 10‐08285 were very challenging (red numbers)

with only 59% of the samples classified correctly. Because 10‐08285 samples failed to meet the required

70% pass rate, they were excluded from the official panel grading, but are still included in this analysis.

Table 2. Composition of the 2012 Johne’s Disease Fecal Proficiency Panel, and the overall categorical summary

results per cow for each method performed by laboratories.  

All Kits HEY TREK MGIT Life Tech Tetracore In-House

Cow ID 1092 19 19 11 16 25 19

09-02559 (ND) 1 Critical- Neg 99% 100% 95% 100% 100% 100% 95%

10-05134 (OH) 2 Critical- Neg 99% 100% 100% 95% 100% 98% 97%

10-06315 (OH) 2 Critical- Neg 98% 100% 100% 100% 97% 100% 92%

10-08285 (OH) 2 Low 59%3 78% 87% 55% 38% 50% 45%

11-06357 (IA) 1 Low 83% 94% 100% 64% 81% 76% 74%

11-09383 (MT) 2 Low 84% 92% 95% 68% 84% 80% 82%

12-03915 (ND) 2 Low 96% 100% 95% 82% 100% 100% 92%

12-03914 (ND) 2 Low 96% 97% 97% 86% 100% 100% 89%

12-00957 (KS) 2 Moderate 99% 100% 100% 91% 100% 100% 97%

12-02530 (MT) 2 Moderate 96% 100% 100% 86% 100% 98% 89%

12-00953 (KS)4 2 Moderate 99% 100% 100% 95% 100% 98% 100%

12-03430 (ND) 2 Critical- High 99% 100% 100% 91% 100% 100% 97%

12-03427 (ND) 2 Critical- High 99% 97% 100% 91% 100% 100% 100%

12-03428 (ND) 2 Critical- High 99% 100% 100% 95% 100% 100% 97%

2

Number of proficiency panels submitted per method.

3

In order to be considered valid, more than 70% of the samples from an animal must be correctly classified.

4

The positive sample was one of the two from this animal.

Liquid Media Direct PCR

Percent of Samples Correctly Classified

1

In order to pass, laboratories must correctly classify critical samples. A critical sample is any negative sample or a 

sample that is identified as a heavy shedder by more than 50% of the laboratories using solid media. 

# Vials 

/Panel

Shedding 

Status1
   

General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 3  

Samples from 4 animals were used in both the 2012 and 2013 panels and their performance

compared. Table 3 shows the 2012 and 2013 categorical (positive/negative) performance for each

identification method by animal ID. There is a reduced percent of samples correctly classified this year

compared to 2012 using direct PCR and MGIT methods in the samples from the low shedding animals,

while solid culture improved and TREK held steady. The constant or slight increase seen with the HEY

and TREK methods for the samples from low shedding animals suggests there was no degradation in the

quality of the samples over the year.

Table 3. Comparison between four animals used in both the 2012 and 2013 Johne’s Disease Fecal Proficiency

Panels with the overall categorical summary results per cow for each method performed by laboratories.  

All Kits HEY TREK MGIT Life Tech Tetracore In-House

2012 1121 21 22 8 23 25 13

Cow ID 2013 109 19 19 11 16 25 19

11-06357 (IA) 3 Low 88% 92% 100% 79% 87% 79% 90%

11-06357 (IA) 1 Low 83% 94% 100% 64% 81% 76% 74%

11-09383 (MT) 2 Low 91% 88% 95% 81% 91% 90% 96%

11-09383 (MT) 2 Low 84% 92% 95% 68% 84% 80% 82%

12-00957 (KS) 2 Moderate 100% 100% 100% 94% 100% 100% 100%

12-00957 (KS) 2 Moderate 99% 100% 100% 91% 100% 100% 97%

12-02530 (MT) 3 Moderate 99% 100% 100% 100% 97% 97% 100%

12-02530 (MT) 2 Moderate 96% 100% 100% 86% 100% 98% 89% 1

Number of proficiency panels submitted per method.

# Vials 

/Panel

Shedding 

Status

Percent of Samples Correctly Classified

Liquid Media Direct PCR

The performance of each method was further evaluated by determining the number of samples

that were misclassified (Figure 1). Please note that this analysis included all 25 samples, including the

two samples from cow 10‐08285 that were not included in the official grading. In this analysis TREK

outperformed all other methods with 63% of the kits correctly classifying all 25 samples. Thirty‐six

percent of laboratories using Life Technologies direct PCR correctly classified all samples, and 47% of the

laboratories using solid media correctly classified all samples.
   

General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 4  

Figure1. Percentage of 2013 Johne’s disease fecal proficiency panels by number of samples misclassified for the

three culture (TREK liquid media, solid media and MGIT 960 liquid media) and three direct PCR (Tetracore, Life

Technologies, and In‐House) methods. A panel consisted of 25 fecal samples.

According to the 2010 Johne’s Disease Uniform Methods and Rules, laboratories must correctly

classify all critical high shedding samples as positive, all negative samples as negative and misidentify

less than 30% of the remaining, valid, non‐critical samples. Table 4 lists the specific reasons laboratories

failed to pass the proficiency panel for each method.  

Table 4. Reasons laboratories failed the 2013 Johne’s Disease Fecal Proficiency Panel. Laboratories were required

to correctly identify all the negative samples as negative and all the critical high shedding samples as positive

(critical samples). They also were required to correctly classify at least 70% of the remaining samples.  

Direct PCR 

(Tetracore)

Direct PCR 

(Life Tech)

Direct PCR 

(In-House)

TREK 

liquid 

media

MGIT 

liquid 

media

HEY 

solid 

media

Misclassified a negative sample as 

positive

1 1 3 1 10

Missed 3 or more low / moderate 

shedders (lack of sensitivity) 1 0 2 0 00

Misclassified a high shedding sample 

as negative 0 0 2 0 21

Multiple reasons cited above 0 0 0 0 1 0

Total failed kits 2 (8%) 1 (6%) 7 (37%) 1 (5%) 4 (36%) 1 (5%)

Total kitstested 25 16 19 19 11 19
   

General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 5  

As more and more laboratories use direct PCR as their primary organism detection assay, the

performance of that assay across laboratories becomes more important. Variation in reported cycle

times (Ct) of the direct PCR methods was investigated in Figure 2 by comparing the average reported Ct

for the positive samples. Only valid Ct values from each panel were used in this comparison and include

samples categorized as negative but that had valid Ct scores reported (e.g. negative but a Ct of 39.9).

However, any Ct reported as a range (e.g. >38) or as a text entry (e.g. undetermined) were excluded

from this analysis. The overall means of all three groups were remarkably similar with the average Ct

score between the methods for each animal differing by less than 1.75. These differences were not

statistically significant. Not only were averages similar but standard deviations were also similar among

the methods for each animal. Of note is that the standard deviations for the In‐House method group

ranged from 7‐13 in 2012 but ranged from 2‐3 this year. This was the biggest improvement among the

methods from last year and makes all three direct PCR methods very similar.  As stated above, samples

from four animals were used in both the 2012 and 2013 panels. Comparing the average Ct from year to

year for each of the four animals and method showed that each differed by less than 1 Ct, and most

were less than 0.5, which is remarkably consistent.  This consistency also suggests the quality of these

samples is very consistent from last year to this year.

Figure 2. Average reported Ct of 2012 Johne’s disease fecal proficiency panel animals for the three direct PCR

methods (ABI, Tetracore, and In House). Shedding status is listed below the animal ID.

False positive results with either direct fecal PCR or confirmatory culture PCR continue to be the

most common cause of failure. While none of the non‐infected cows have been used in the check test

previously, fecal material from animals in these herds has been used as negative samples in the

proficiency panel in previous years. Although we did not include any samples from animals that were

shedding over 7,000 CFU per tube this year, we did have one that contained 3,350 CFU per tube (2

samples) and another that  contained 5700 CFU per tube (2 samples). Table 5 examines the number of

negative samples reported with Ct values by PCR method; this includes laboratories that had Ct values

but correctly reported them as negative. Errors were evenly distributed amongst the samples. There

were a total of 5 laboratories that reported Ct values for negative samples; of those two reported more

than one negative sample with Ct values. Two kits were tested using SYBR Green technology, which

resulted in Ct values due to non‐specific amplification in most of the samples. The false‐positive results
   

General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 6  

for these two kits were not included in Table 5. There were a total of 8 laboratories that failed the PT

(see Table 4) by calling negative samples positive, the same as last year. Three of those laboratories

chose to retake the PT and 2 successfully passed the panel on the retest. The only laboratory that failed

the retest identified different animals as negative.

Table 5. The number of samples from non‐infected cows

reported with Ct values (regardless of their categorical

positive/negative results) by direct PCR method.  

Tetracore 

(60 tested)

ABI 

(120 tested)

In-House 

(120 tested)

09-02559 (ND) 0 0 2

10-05134 (OH) 101

10-06315 (OH) 013

Pooling Panel Description  

Twenty five individual samples were provided with instructions regarding which 5 samples to

pool together, for a total of 5 pooled samples. Table 6 lists the contents of each pool, and Table 8 lists

the vial numbers associated with each pool. Laboratories were required to correctly classify the negative

pool and the two pools that contained a high‐shedding animal (10‐08425 & 12‐03427) in order to pass.

Laboratories were allowed to misclassify one of the other two pools (moderate or mod‐high) and still

pass the panel.

Table 6. Composition of the 2013 Johne’s Disease Fecal Pooling

Proficiency Panel.

Cow ID

Avg. 

CFU/ tube*

1 Very High, 4 Negative samples 10-08425 >10000

1 High, 4 Negative samples 12-03427 3350

1 Mod-High, 4 Negative samples 11-06361 86

1 Moderate, 4 Negative samples 12-02530 20

5 Negative samples

Positive sample(s) 

description

*Refers to the positive samples, not the pooled sample

Table 7 further describes the performance of each method used in the pooled proficiency test.

No laboratory failed because they misclassified negative pools as positive or classified pools with a high

shedding animal as negative. It is commendable that no laboratory misclassified the negative pool and

that only two laboratories failed out of 64 kits. All laboratories that submitted results using solid culture

methods passed, with all but one of those correctly classifying all the pools. Direct PCR and Liquid

culture methods had one laboratory each fail due to misclassifying both non‐critical pools.  
   

General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 7  

Table 7. Performance of each method used in the Johne’s Disease 2013 Fecal Pooling

Proficiency Panel. A total of 5 pooled samples were in each panel.  

Direct

PCR

Liquid

media

Solid

media

Identi fied the negative pool as positive 0 0 0

Identi fied a high ‐shedding pool as negative 0 0 0

Two non‐critical pools were identi fied as negative 1 1 0

Failed due to multiple criteria 0 0 0

One non‐critical pool was misidenti fied as negative 10 3 1

All 5 pools were identi fied correctly 27 16 5

Total Failed Pooled Kits 1 (3%) 1 (5%) 0 (0%)

Total 38 20 6

Panels

that

passed

Panels

that

failed

A current listing of all the approved laboratories is available in the NVLS web site:

http://www.aphis.usda.gov/animal_health/lab_info_services/approved_labs.shtml .  

Remaining sample vials from the 2013 Proficiency Panel are available to laboratories for

validation or research purposes. Available samples can be viewed in the reagents catalog under Johne’s

positive/negative fecal samples on the NVSL web site:

http://www.aphis.usda.gov/animal_health/lab_info_services/reagents.shtml.  

Table 8. 2013 Johne’s Disease Pooled Fecal Proficiency Panel key by kit number

Pool Description

Kit#     

1‐25

Kit#     

25‐50

Kit#     

51‐75

5 Negative samples 2 5 3

1 moderate (12-02530), 4 Negative samples 4 1 5

1 mod-high (11-06361), 4 Negative samples 3 4 1

1 high (12-03427), 4 Negative samples 1 2 4

1 very high (10-08425), 4 Negative samples 5 3 2

Pool Sample Number
   

General Summary of the 2013 Johne’s Disease Fecal Proficiency Panel Page | 8  

Table 9. 2013 Johne’s Disease Individual Fecal Proficiency Panel key by kit number. Samples are coded by

color according to shedding status as follows: Negative, Noncritical positive samples, Critical – high shedding

samples. Sample 26 was the positive control.

Vial # 1-25 26-50 51-75 76-100 101-125

1 12‐03430 (ND) 10‐08285 (OH) 10‐05134 (OH) 12‐00953 (KS) 12‐03915 (ND)

2 11‐09383 (MT) 12‐03430 (ND) 12‐03428 (ND) 12‐00957 (KS) 10‐08285 (OH)

3 10‐05134 (OH) 12‐03427 (ND) 11‐09383 (MT) 09‐02559 (ND) 10‐06315 (OH)

4 12‐03428 (ND) 10‐05134 (OH) 12‐02530 (MT) 12‐03428 (ND) 12‐03427 (ND)

5 12‐03914 (ND) 12‐03915 (ND) 12‐03430 (ND) 12‐03427 (ND) 12‐02530 (MT)

6 12‐02530 (MT) 12‐02530 (MT) 10‐06315 (OH) 10‐05134 (OH) 12-00953 (KS)

7 10‐08285 (OH) 12‐03914 (ND) 12‐03914 (ND) 10‐08285 (OH) 12‐03914 (ND)

8 12‐03915 (ND) 12‐03428 (ND) 12‐00953 (KS) 12‐03430 (ND) 12‐00957 (KS)

9 10‐05134 (OH) 09‐02559 (ND) 12‐00957 (KS) 12‐03915 (ND) 10‐05134 (OH)

10 12‐03428 (ND) 12-03914 (ND) 12‐03430 (ND) 11‐06357 (IA) 12‐03428 (ND)

11 10‐06315 (OH) 12‐00957 (KS) 10‐08285 (OH) 12‐03427 (ND) 10‐06315 (OH)

12 12-03430 (ND) 12‐03915 (ND) 11‐09383 (MT) 10‐06315 (OH) 11‐06357 (IA)

13 12‐03915 (ND) 12‐00957 (KS) 12‐00957 (KS) 12‐03914 (ND) 12‐00957 (KS)

14 12‐00957 (KS) 12‐03430 (ND) 09‐02559 (ND) 11-09383 (MT) 11‐09383 (MT)

15 12‐00957 (KS) 10‐05134 (OH) 12‐03427 (ND) 12‐02530 (MT) 10‐05134 (OH)

16 12‐03427 (ND) 11‐09383 (MT) 12‐03915 (ND) 12‐03430 (ND) 12‐03427 (ND)

17 10‐06315 (OH) 12‐02530 (MT) 11-06357 (IA) 10‐05134 (OH) 12‐02530 (MT)

18 11‐06357 (IA) 11‐09383 (MT) 12‐03428 (ND) 12‐02530 (MT) 12‐03430 (ND)

19 12‐03914 (ND) 12‐03428 (ND) 10‐05134 (OH) 10‐08285 (OH) 10‐08285 (OH)

20 12‐00953 (KS) 10‐06315 (OH) 12‐03914 (ND) 12‐03915 (ND) 12‐03428 (ND)

21 12‐02530 (MT) 10‐08285 (OH) 10‐08285 (OH) 12‐03914 (ND) 09‐02559 (ND)

22 12‐03427 (ND) 12‐00953 (KS) 12‐03915 (ND) 10‐06315 (OH) 12‐03430 (ND)

23 09‐02559 (ND) 11‐06357 (IA) 12‐02530 (MT) 12‐03428 (ND) 12‐03914 (ND)

24 10‐08285 (OH) 12‐03427 (ND) 10‐06315 (OH) 12‐00957 (KS) 11‐09383 (MT)

25 11‐09383 (MT) 10‐06315 (OH) 12‐03427 (ND) 11‐09383 (MT) 12‐03915 (ND)

26 12‐00953 (KS) 12‐00953 (KS) 12‐00953 (KS) 12‐00953 (KS) 12‐00953 (KS)

Any questions or comments can be directed to the Diagnostic Bacteriology Laboratory at 515.337.7388.  

Report was prepared by:

Kevin D. Stokes, PhD

USDA/APHIS/NVSL

Mycobacteria /Brucella Section

Kevin.D.Stokes@USDA.APHIS.GOV