General Summary of the 2012 Johne’s Disease Fecal Proficiency Panel Page | 1  

2012 Johne’s Disease Fecal Proficiency Panel

General Summary

October 4, 2012

Overview

A total of 59 laboratories participated in the 2012 Johne’s Disease Fecal Proficiency Panel (7

Canadian, 3 European Union, 1 New Zealand and 48 USA laboratories). Overall, the number of

laboratories that requested individual proficiency panels for direct PCR, liquid culture, and solid culture

testing slightly decreased from 2011. Requests for pooled proficiency panels decreased slightly for direct

PCR, held steady for liquid culture, and slightly increased for solid culture. Table 1 details the number of

individual and pooled panels shipped and the overall pass/fail status for each method. Laboratories

could order multiple panels for each method and were notified of their preliminary pass/fail status upon

submission of their results. A total of 179 panels were requested; results were not returned for 3 of

them, and one was faulty and replaced. If preliminary results indicated that the laboratory had failed,

they were given the opportunity to retake the proficiency panel provided the results could be

completed by September 28th, 2012. The results provided in Table 1 include these retests. Laboratories

that only used reagents from a single manufacturer, either Tetracore or Applied Biosystems, are listed

separately. Laboratories that use either in‐house reagents, other commercial kits not marketed in the

US, or mix commercial reagents are listed under the “In House” category.

Table 1. Summary results of the 2012 Johne’s Disease Fecal Proficiency Panel. In order to pass results must meet the

criteria listed in the 2010 Uniform Program Standards for the Voluntary Bovine Johne's Disease Control Program.

# passed 

1st attempt 

(%)

# failed 

1st attempt 

(%)

# passed 

2nd attempt 

(%)

# failed 

2nd attempt 

(%)

# labs 

not 

retesting

Total 

Shipped

Total shipped in 

2011 (%change)

Individual Panel

Direct PCR (all) 44 (83%) 9 (17%) 7 (88%) 1 (13%) 1 62 63 (‐2%)

Tetracore 15 (79%) 4 (21%) 4 (100%) 23 31 (‐26%)

Applied Biosystems 18 (86%) 3 (14%) 3 (75%) 1 (25%) 25 19 (32%)

In‐House 11 (85%) 2 (15%) 13 13 (0%)

Liquid Systems (all) 28 (97%) 1 (3%) 1 (100%) 31 32 (‐3%)

MGIT 960 8 (100%) 8 9 (‐11%)

TREK 20 (95%) 1 (5%) 1 (100%) 22 23 (‐4%)

HEY Solid Media (all) 18 (86%) 3 (14%) 3 21 25 (‐16%)

Individual Panel Total 91 (84%) 13 (13%) 8 (89%) 1 (11%) 4 114 120 (‐5%)

Pooling Panel

Direct PCR (all) 28 (88%) 4 (13%) 2 (50%) 2 (50%) 36 38 (‐5%)

Liquid 21 (95%) 1 (5%) 1 23 23 (0%)

HEY 5 (100%) 6 4 (50%)

Pooled Panel Total 54 (92%) 5 (8%) 2 (50%) 2 (50%) 1 65 65 (0%)
   

General Summary of the 2012 Johne’s Disease Fecal Proficiency Panel Page | 2  

Individual Panel Description  

Each individual panel consisted of 25 unknown samples and one positive control. Positive

samples were collected from naturally infected cows, and negative samples were from individual

animals residing in non‐infected herds. Approximately 4 liters of fecal material were collected rectally

per animal, shipped to NVSL, aliquoted as soon as possible in individual vials, and stored at ‐70°C until

kits were distributed. Panels were assembled in groups, each with a different key (See Table 8 at the end

of this report for the key). Table 2 shows the categorical (positive/negative) performance for each

identification method by animal ID. Samples from cow 11‐09382 were very challenging (red numbers)

with only 43% of the samples classified correctly. Because 11‐09382 samples failed to meet the required

70% pass rate, they were excluded from the official panel grading, but are still included in this analysis.

All laboratories performing MGIT culture passed, the only method to do so. However, this method was

the least sensitive for low shedding samples.

Table 2. Composition of the 2012 Johne’s Disease Fecal Proficiency Panel, and the overall categorical summary

results per cow for each method performed by laboratories.  

All Kits HEY TREK MGIT 

Applied 

Biosystems Tetracore In-House

Cow ID 1122 21 22 8 23 25 13

10-04922 (OH) 1 Critical- Neg 98% 100% 100% 100% 100% 96% 92%

10-04923 (OH) 3 Critical- Neg 99% 100% 98% 100% 97% 99% 100%

10-04999 (OH) 2 Critical- Neg 99% 100% 100% 100% 100% 98% 96%

11-09382 (MT) 2 Low 43%3 36% 59% 13% 26% 62% 42%

11-06357 (IA) 3 Low 88% 92% 100% 79% 87% 79% 90%

11-09383 (MT) 2 Low 91% 88% 95% 81% 91% 90% 96%

11-06361 (IA) 2 Low 100% 100% 100% 100% 100% 100% 100%

12-00957 (KS) 2 Moderate 100% 100% 100% 94% 100% 100% 100%

12-02530 (MT) 3 Moderate 99% 100% 100% 100% 97% 97% 100%

11-09381 (MT) 2 Critical- High 99% 95% 100% 100% 100% 100% 100%

11-06359 (IA) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

12-00954 (KS) 2 Critical- High 100% 100% 100% 100% 98% 100% 100%

2

Number of proficiency panels submitted per method.

3

In order to be considered valid, more than 70% of the samples from an animal must be correctly classified.

Shedding 

Status1

Liquid Media Direct PCR

Percent of Samples Correctly Classified

1

In order to pass, laboratories must correctly classify critical samples. A critical sample is any negative sample or a sample 

that is identified as a heavy shedder by more than 50% of the laboratories using solid media. 

# 

Vials/

Panel

The performance of each method was further evaluated by determining the number of samples

that were misclassified (Figure 1). Please note that this analysis included all 25 samples, including the

two samples from cow 11‐09382 that were not included in the official grading. In this analysis Tetracore

outperformed all other methods with 44% of the kits correctly classifying all 25 samples. This is the first

year where a direct PCR method out performed liquid culture. Thirty‐two percent of laboratories using

TREK liquid culture correctly classified all samples, and 19% of the laboratories using solid media

correctly classified all samples.
   

General Summary of the 2012 Johne’s Disease Fecal Proficiency Panel Page | 3  

Figure1. Percentage of 2012 Johne’s disease fecal proficiency panels by number of samples misclassified for the

three culture (TREK liquid media, solid media and MGIT 960 liquid media) and three direct PCR (Tetracore, ABI, and

In‐House) methods. A panel consisted of 25 fecal samples.

According to the 2010 Johne’s Disease Uniform Methods and Rules, laboratories must correctly

classify all critical high shedding samples as positive, all negative samples as negative and misidentify

less than 30% of the remaining, valid, non‐critical samples. Table 3 lists the specific reasons laboratories

failed to pass the proficiency panel for each method.  

Table 3. Reasons laboratories failed the 2012 Johne’s Disease Fecal Proficiency Panel. Laboratories were required

to correctly identify all the negative samples as negative and all the critical high shedding samples as positive

(critical samples). They also were required to correctly classify at least 70% of the remaining samples.  

Direct PCR 

(Tetracore)

Direct PCR 

(ABI)

Direct PCR 

(In-House)

TREK 

liquid 

media

MGIT 

liquid 

media

HEY solid 

media

Misclassified a negative sample as 

positive

3 2 2 1 00

Missed 3 or more low / moderate 

shedders (lack of sensitivity) 1 1 0 0 01

Misclassified a high shedding sample 

as negative 0 1 0 0 02

Multiple reasons cited above 0 0 0 0 0 0

Total failed kits 4 (16%) 4 (17%) 2 (15%) 1 (5%) 0 (0%) 3 (14%)

Total kitstested 25 23 13 22 8 21

As more and more laboratories use direct PCR as their primary organism detection assay, the

performance of that assay across laboratories becomes more important. Variation in reported cycle

times (Ct) of the direct PCR methods was investigated in Figure 2 by comparing the average reported Ct

for the positive samples. Only valid Ct values from each panel were used in this comparison and includes

samples categorized as negative but had valid Ct scores reported (e.g. negative but a Ct of 39.9).

However, any Ct reported as a range (e.g. >38) or as a text entry (e.g. undetermined) were excluded

from this analysis. The overall means of all three groups were remarkably similar. The averages differed
   

General Summary of the 2012 Johne’s Disease Fecal Proficiency Panel Page | 4  

by less than 2 Ct for samples from each animal except 11‐02530 (2.1 Ct) and 11‐06359 (2.9 Ct) but the

differences were not statistically significant. Not only were averages similar but standard deviations

were also similar for the ABI and Tetracore methods, which indicates these two methods performed

remarkably consistently between laboratories and technicians. The standard deviations of the In‐House

methods were much larger and somewhat concerning even though there are a wide range of methods

categorized within this group. Laboratories using these methods may want to look at ways to reduce

variation if it occurred, which can be found in their individual laboratory reports released separately.  

Figure2. Average reported Ct of 2012 Johne’s disease fecal proficiency panel animals for the three direct PCR

methods (ABI, Tetracore, and In House). Shedding status is listed below the animal ID.

False positive results with either direct fecal PCR or confirmatory culture PCR continue to be the

most common cause of failure. While the non‐infected 10‐04922 and 10‐04923 cows had not been used

in the check test previously, 10‐04999 has been. Also, fecal material from animals in these herds has

been used as negative samples in the proficiency panel in previous years. Although we did not include

any samples from animals that were shedding over 7,000 CFU per tube this year, we did have one that

contained 2,600 CFU per tube (2 samples) and another that  contained 4000 CFU per tube (2 samples).

Table 4 examines the number of negative samples reported with Ct values by PCR method; this includes

laboratories that had Ct values but correctly reported them as negative. Errors were evenly distributed

amongst the samples.  There were a total of 8 laboratories (7 with direct PCR and one with the

confirmatory PCR) that failed the initial PT (see table 3) by calling negative samples positive as compared

to 9 last year. Seven of those laboratories chose to retake the PT and 5 successfully passed the panel on

the retest. The only laboratory that failed the retest identified different animals as negative.
   

General Summary of the 2012 Johne’s Disease Fecal Proficiency Panel Page | 5  

Table 4. The number of samples from non‐infected cows

reported with Ct values (regardless of their categorical

positive/negative results) by direct PCR method.  

Tetracore 

(138 tested)

ABI 

(150 tested)

In-House 

(78 tested)

10-04922 (OH) 1 0 2

10-04923 (OH) 1 2 0

10-04999 (OH) 1 0 2

Pooling Panel Description  

Twenty five individual samples were provided with instructions regarding which 5 samples to

pool together, for a total of 5 pooled samples. Table 5 lists the contents of each pool and Table 7 lists

the vial numbers associated with each pool. Laboratories were required to correctly classify the two

negative pools and the one pool that contained the high‐shedding animal (10‐08425). Laboratories were

allowed to misclassify one of the other two pooled samples and still pass the panel.  

Table 5. Composition of the 2012 Johne’s Disease Fecal Pooling

Proficiency Panel.

Cow ID

Avg. 

CFU/ 

tube*

1 high, 4 Negative samples 10-08282 2,275

1 moderate, 4 Negative samples 12-02530 20

2 low, 3 Negative samples 10-08285 1

5 Negative samples

5 Negative samples

Positive sample(s) 

description

*Refers to the positive samples, not the pooled sample

Table 6 further describes the performance of each method used in the pooled proficiency test.

For direct PCR, 4 laboratories failed because they misclassified negative pools as positive, one failed

because they misclassified both non‐critical pools, and one misclassified both negative samples and the

two non‐critical pools. Only 1 laboratory failed using a liquid culture method and misclassified a negative

pool. None of the laboratories using solid media failed the proficiency panel.  
   

General Summary of the 2012 Johne’s Disease Fecal Proficiency Panel Page | 6  

Table 6. Performance of each method used in the Johne’s Disease 2012 Fecal Pooling

Proficiency Panel. A total of 5 pooled samples were in each panel.  

Direct

PCR

Liquid

media

Solid

media

Identi fied a negative pool as positive 4 1 0

Identi fied the high ‐shedding pool as negative 0 0 0

Two non‐critical pools were identi fied as negative 1 0 0

Failed due to multiple criteria 1 0 0

One non‐critical pool was misidenti fied as negative 11 3 3

All 5 pools were identi fied correctly 19 18 2

Total Failed Pooled Kits 6 (17%) 1 (5%) 0 (0%)

Total 36 22 5

No. panels

Panels

that

failed

Panels

that

passed

A current listing of all the approved laboratories is available in the NVLS web site:

http://www.aphis.usda.gov/animal_health/lab_info_services/approved_labs.shtml .  

Remaining sample vials from the2012 Proficiency Panel are available to laboratories for

validation or research purposes. Available samples can be viewed in the reagents catalog under Johne’s

positive/negative fecal samples on the NVSL web site:

http://www.aphis.usda.gov/animal_health/lab_info_services/reagents.shtml.  

Table 7. 2012 Johne’s Disease Pooled Fecal Proficiency Panel key by kit number

 Pool Sample Number 

Pool Description

Kit#     

1‐25

Kit#    

25‐50

Kit#    

51‐75

5 Negative samples 1 1 1 

5 Negative samples 4 4 4 

2 low (10-08285), 3 Negative samples 5 5 5 

1 moderate (12-02530), 4 Negative samples 2 2 2 

1 high (10-08282), 4 Negative samples 3 3 3 
   

General Summary of the 2012 Johne’s Disease Fecal Proficiency Panel Page | 7  

Table 8. 2012 Johne’s Disease Individual Fecal Proficiency Panel key by kit number. Samples are coded by

color according to shedding status as follows: Negative, Noncritical positive samples, Critical – high shedding

samples. Sample 26 was the positive control.

Vial # 1-25 26-50 51-75 76-100 101-125

1 11‐06361 (IA) 11‐06357 (IA) 11‐09383 (MT) 11‐09382 (MT) 12‐02530 (MT)

2 10‐04923 (OH) 11‐09383 (MT) 10‐04923 (OH) 10‐04999 (OH) 10‐04923 (OH)

3 11‐06359 (IA) 12‐00954 (KS) 11‐06359 (IA) 12‐00954 (KS) 11‐09381 (MT)

4 12‐00957 (KS) 10‐04999 (OH) 11‐09381 (MT) 11‐06357 (IA) 11‐06361 (IA)

5 11‐06357 (IA) 11‐09383 (MT) 10‐04923 (OH) 11‐06359 (IA) 11‐09382 (MT)

6 12‐02530 (MT) 12‐02530 (MT) 11‐06361 (IA) 10‐04999 (OH) 10-04922 (OH)

7 11‐09383 (MT) 10‐04923 (OH) 12‐00957 (KS) 12‐02530 (MT) 12‐00954 (KS)

8 10‐04923 (OH) 11‐09381 (MT) 11‐06357 (IA) 11‐06361 (IA) 11‐06359 (IA)

9 11‐06359 (IA) 11‐06359 (IA) 12‐00954 (KS) 11‐09383 (MT) 10‐04923 (OH)

10 11‐06357 (IA) 10-04922 (OH) 10‐04999 (OH) 12‐00957 (KS) 11‐09382 (MT)

11 11‐09381 (MT) 11‐09382 (MT) 12‐02530 (MT) 10‐04923 (OH) 11‐09381 (MT)

12 10-04922 (OH) 12‐00957 (KS) 12‐00957 (KS) 11‐09381 (MT) 10‐04999 (OH)

13 11‐09381 (MT) 10‐04999 (OH) 11‐06357 (IA) 11‐06359 (IA) 12‐00957 (KS)

14 12‐02530 (MT) 11‐09381 (MT) 11‐09382 (MT) 10-04922 (OH) 11‐06357 (IA)

15 10‐04999 (OH) 11‐09382 (MT) 12‐02530 (MT) 11‐06361 (IA) 11‐09383 (MT)

16 11‐06357 (IA) 11‐06361 (IA) 11‐06359 (IA) 11‐09382 (MT) 11‐06357 (IA)

17 11‐09382 (MT) 12‐00957 (KS) 10-04922 (OH) 12‐02530 (MT) 12‐00957 (KS)

18 12‐00954 (KS) 11‐06357 (IA) 11‐09381 (MT) 11‐06357 (IA) 12‐02530 (MT)

19 10‐04999 (OH) 11‐06361 (IA) 10‐04999 (OH) 12‐00957 (KS) 11‐06361 (IA)

20 12‐00957 (KS) 12‐02530 (MT) 11‐09383 (MT) 12‐02530 (MT) 12‐02530 (MT)

21 12‐02530 (MT) 11‐06359 (IA) 11‐06357 (IA) 11‐06357 (IA) 11‐06359 (IA)

22 11‐06361 (IA) 10‐04923 (OH) 12‐02530 (MT) 11‐09383 (MT) 10‐04999 (OH)

23 11‐09382 (MT) 12‐02530 (MT) 11‐06361 (IA) 10‐04923 (OH) 11‐09383 (MT)

24 11‐09383 (MT) 11‐06357 (IA) 11‐09382 (MT) 11‐09381 (MT) 11‐06357 (IA)

25 10‐04923 (OH) 10‐04923 (OH) 10‐04923 (OH) 10‐04923 (OH) 10‐04923 (OH)

26 12‐00954 (KS) 12‐00954 (KS) 12‐00954 (KS) 12‐00954 (KS) 12‐00954 (KS)

Any questions or comments can be directed to the Diagnostic Bacteriology Laboratory at 515.337.7388.  

Report was prepared by:

Kevin D. Stokes, PhD

USDA/APHIS/NVSL

Mycobacteria /Brucella Section

Kevin.D.Stokes@USDA.APHIS.GOV