2015 Johne’s Disease Fecal Proficiency Panel

General Summary

October 8, 2015

Overview

A total of 61 laboratories participated in the 2015 Johne’s Disease Fecal Proficiency Panel (7

Canadian, 4 European Union, 1 New Zealand, 1 Australian and 48 USA laboratories). Compared to 2014, 

the number of individual proficiency panel requesting laboratories increased for direct PCR, decreased 

for liquid, and remained the same for solid culture methods. Requests for pooled proficiency panels 

increased for direct PCR, and remained constant for liquid and solid culture methods. Table 1 details the 

number of individual and pooled panels shipped and the overall pass/fail status for each method. 

Laboratories could order multiple panels for each method and were notified of their preliminary 

pass/fail status upon submission of their results. A total of 160 panels were requested; results were not 

returned for 5. None of the kits were reported to be faulty this year. If preliminary results indicated that 

the laboratory had failed, it was given the opportunity to retake the proficiency panel provided the 

results were completed by September 30th, 2015. The results provided in Table 1 include these retests.

Laboratories that only used reagents from a single manufacturer, either Tetracore or Life Technologies, 

are listed separately. Laboratories that use either in-house reagents, other commercial kits not 

marketed in the US, or mix commercial reagents are listed under the “In House” category. One 

laboratory used in-house liquid culture reagent, which is grouped with the laboratories using the MGIT 

system. All laboratories using solid media were grouped together even though two laboratories used inhouse solid media.

Table 1. Summary results of the 2015 Johne’s Disease Fecal Proficiency Panel. In order to pass results must meet the 

criteria listed in the 2010 Uniform Program Standards for the Voluntary Bovine Johne's Disease Control Program.

# passed 

1st attempt 

(%)

# failed 

1st attempt 

(%)

# passed 

2nd attempt 

(%)

# failed 

2nd attempt 

(%)

# Kits 

not 

retested

Total 

Shipped

Total shipped in 

2014 (%change)

Individual tanel

Direct tCR (all) 52 (91%) 5 (9%) 2 (100%) 3 60 63 (-5%

Tetracore 21 (91%) 2 (9%) 2 (100%) 25 24 (+4%)

Life Technologies 20 (95%) 1 (5%) 21 19 (+11%)

In-House 11 (85%) 2 (15%) 13 17 (-24%)

Liquid Systems (all) 21 (95%) 1 (5%) 1 24 29 (-17%)

aGIT 960 4 (80%) 1 (20%) 5 7 (-29%

TREK 17 (100%) 17 22 (-23%)

HEY Solid aedia (all) 11 (92%) 1 (8%) 1 12 13 (-8%

Individual tanel Total 84 (92%) 7 (8%) 2 (100%) 5 96 105 (-9%)

tooling tanel

Direct tCR (all) 40 (95%) 2 (5%) 1 (100%) 1 44 41 (+7%

Liquid 16 (100%) 17 18 (-6%)

HEY 3 (100%) 3 4 (-25%)

tooled tanel Total 59 (97%) 2 (3%) 1 (100%) 1 64 63 (+2%)

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 1
Individual Panel Description 

Each individual panel consisted of 25 unknown samples and one positive control. Positive 

samples were collected from naturally infected cows, and negative samples were from individual 

animals residing in non-infected herds. Approximately 4 liters of fecal material were collected rectally 

per animal, shipped to NVSL, aliquoted as soon as possible in individual vials, and stored at -70°C until 

kits were distributed. Panels were assembled in groups, each with a different key (See Table 9 at the end 

of this report for the key). Table 2 shows the categorical (positive/negative) performance for each 

identification method by animal ID. This year all animals met the required 70% pass rate to be 

considered valid. Numbers in red indicate percentages that were less than the required 70%.

Table 2. Composition of the 2015 Johne’s Disease Fecal Proficiency Panel, and the overall categorical summary 

results per cow for each method performed by laboratories. 

All Kits HEY TREK MGIT Life Tech Tetracore In-House

Cow ID 932 12 17 5 21 25 13

14-02868 (IA) 2 Critical- Neg 100% 100% 100% 100% 100% 100% 100%

13-01420 (IA) 3 Critical- Neg 99% 100% 100% 100% 98% 100% 97%

13-00354 (IA) 1 Critical- Neg 99% 92% 100% 75% 100% 100% 100%

13-00349 (IA) 1 Critical- Neg 98% 100% 100% 100% 100% 92% 100%

14-02866 (IA) 1 Critical- Neg 99% 100% 100% 100% 98% 98% 100%

12-03917 (ND) 2 Low 98% 100% 97% 63% 98% 98% 100%

12-03913 (ND) 2 Low 96% 83% 91% 75% 98% 100% 88%

12-03914 (ND) 2 Moderate 99% 100% 100% 100% 100% 100% 92%

13-06801 (IA) 2 Moderate 99% 100% 100% 100% 100% 100% 100%

14-03358 (IA) 2 Critical- High 99% 100% 100% 88% 98% 98% 100%

13-08115 (ID)3 2 Critical- High 99% 100% 100% 100% 100% 98% 100%

11-09754 (MT) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

12-03428 (ND) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

12-03916 (ND) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

2

Number of proficiency panels submitted per method.

3

The positive control was one of the two from this animal.

Percent of Samples Correctly Classified

# Vials 

/Panel

Shedding 

Status1

Liquid Media Direct PCR

1

In order to pass, laboratories must correctly classify critical samples. A critical sample is any negative sample or a 

sample that is identified as a heavy shedder by more than 50% of the laboratories using solid media. 

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 2
Samples from 6 animals had been used in prior years, 2 in 2013 and 4 in 2014, and their 

performance compared. Table 3 shows the respective year panels’ categorical (positive/negative) 

performance for each identification method by animal ID. 

Table 3. Comparison between four animals used in both the 2012 and 2013 Johne’s Disease Fecal Proficiency 

Panels with the overall categorical summary results per cow for each method performed by laboratories. 

All Kits HEY TREK MGIT Life Tech Tetracore In-House

2013 1091 19 19 11 16 25 19

2014 105 13 22 7 19 24 17

Cow ID 2015 93 12 17 5 21 25 13

13-00349 (IA) 2014 2 Critical-Neg 99% 100% 100% 100% 97% 100% 97%

13-00349 (IA) 2015 1 Critical-Neg 98% 100% 100% 100% 100% 92% 100%

12-03917 (ND) 2014 2 Low 96% 92% 100% 75% 97% 98% 97%

12-03917 (ND) 2015 2 Low 98% 100% 97% 63% 98% 98% 100%

12-03913 (ND) 2014 2 Low 89% 85% 100% 63% 92% 90% 88%

12-03913 (ND) 2015 2 Low 96% 83% 91% 75% 98% 100% 88%

12-03914 (ND) 2013 2 Low 96% 97% 97% 86% 100% 100% 89%

12-03914 (ND) 2015 2 Moderate 99% 100% 100% 100% 100% 100% 92%

11-09754 (MT) 2014 2 Critical-High 99% 100% 100% 100% 100% 98% 100%

11-09754 (MT) 2015 2 Critical-High 100% 100% 100% 100% 100% 100% 100%

12-03428 (ND) 2013 2 Critical-High 99% 100% 100% 95% 100% 100% 97%

12-03428 (ND) 2015 2 Critical-High 100% 100% 100% 100% 100% 100% 100%

1

Number of proficiency panels submitted per method.

Panel 

Year

# Vials 

/Panel

Shedding 

Status

Percent of Samples Correctly Classified

Liquid Media Direct PCR

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 3
The performance of each method was further evaluated by determining the number of samples 

that were misclassified (Figure 1). In this analysis all three direct PCR methods and TREK performed 

comparably and very well. Eighty-six percent of laboratories using Life Technologies direct PCR method 

correctly classified all the samples. Eighty-two percent of laboratories using the TREK system correctly 

classified all samples, and 58% of the laboratories using solid media correctly classified all samples.

Figure1. Percentage of 2015 Johne’s disease fecal proficiency panels by number of samples misclassified for the

three culture (TREK liquid media, solid media and MGIT 960 liquid media) and three direct PCR (Tetracore, Life 

Technologies, and In-House) methods. A panel consisted of 25 fecal samples.

According to the 2010 Johne’s Disease Uniform Methods and Rules, laboratories must correctly 

classify all critical-high shedding samples as positive, all negative samples as negative and misidentify 

less than 30% of the remaining, valid, non-critical samples. Table 4 lists the specific reasons laboratories 

failed to pass the proficiency panel for each method. 

Table 4. Reasons laboratories failed the 2015 Johne’s Disease Fecal Proficiency Panel. 

Laboratories were required to correctly identify all the negative samples as negative and 

all the critical high shedding samples as positive (critical samples). They also were required 

to correctly classify at least 70% of the remaining samples. 

Direct PCR 

(Tetracore)

Direct PCR 

(Life Tech)

Direct PCR 

(In-House)

TREK 

liquid 

media

MGIT 

liquid 

media

HEY 

solid 

media

Misclassified a negative sample as 

positive

0 2 1 0 1 1

Missed 4 or more low / moderate 

shedders (lack of sensitivity) 1 0 0 0 0 0

Misclassified a high shedding sample 

as negative 0 0 0 0 0 0

Multiple reasons cited above 1 0 0 0 0 0

Total failed kits 2 (8%) 2 (10%) 2 (8%) 0 (0%) 1 (20%) 1 (8%)

Total kits tested 25 21 13 17 5 12

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 4
As more laboratories use direct PCR as their primary organism detection assay, the performance 

of that assay across laboratories becomes more important. Variation in reported cycle threshold (Ct) of 

the direct PCR methods was investigated in Figure 2 by comparing the average reported Ct for the 

positive samples. Only valid Ct values from each panel were used in this comparison and include

samples categorized as negative but that had valid Ct scores reported (e.g. negative but a Ct of 39.9).

The overall means of all three groups were statistically similar with the average Ct score between the 

methods for each animal differing by less than 3. Despite life technologies having the most laboratories 

correctly classify all of the samples, this method resulted in higher mean Ct values. 

Figure 2. Average reported Ct of 2015 Johne’s disease fecal proficiency panel animals for the three direct PCR

methods (ABI, Tetracore, and In House). Shedding status is listed below the animal ID.

False positive results with either direct fecal PCR or confirmatory culture PCR continue to be the 

most common cause of failure. While animal 13-00349 (IA) is the only non-infected cow used in previous 

check tests, fecal material from animals in this herd has been used in previous years’ proficiency panels.

This year the highest shedding animals we included were one animal that contained 8,500 CFU per tube 

(2 samples) and another that contained 5,700 CFU per tube (2 samples). Table 5 examines the number 

of negative samples reported with Ct values by PCR method; this includes laboratories that had Ct values 

but correctly reported them as negative. Errors were relatively evenly distributed amongst three of the 

negative animals that were used in this year’s panel. There were a total of 5 laboratories that reported 

Ct values one negative sample each. Of those 5 laboratories, 4 failed the PT (see Table 4) by calling the 

negative sample positive, which is half the number of failing laboratories as the last three years.

Table 5. The number of samples from non-infected cows 

reported with Ct values (regardless of their categorical 

positive/negative results) by direct PCR method. 

Tetracore ABI In-House

13-01420 (IA) 1 0 1

13-00354 (IA) 0 0 1

13-00349 (IA) 0 2 0

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 5
Pooling Panel Description 

Twenty five individual samples were provided with instructions regarding which 5 samples to 

pool together, for a total of 5 pooled samples. Table 6 lists the contents of each pool, and Table 8 lists 

the pool numbers associated with each lot of kits. Laboratories were required to correctly classify the 

negative pool and the two pools that contained a high-shedding animal (12-03427 & 13-08115) in order 

to pass. Laboratories were allowed to misclassify one of the other two pools (12-00953 & 14-03358) and 

still pass the panel.

Table 6. Composition of the 2015 Johne’s Disease Fecal Pooling 

Proficiency Panel.

Cow ID

Avg. 

CFU/ tube*

1 High, 4 Negative samples 12-03427 3,350

1 High, 4 Negative samples 13-08115 500

1 Moderate, 4 Negative samples 12-00953 44

1 Moderate, 4 Negative samples 14-03358 38

5 Negative samples

Positive sample(s) 

description

*Refers to the positive samples, not the pooled sample

Table 7 further describes the performance of each method used in the pooled proficiency test. It 

is commendable that all but two laboratories passed the pooled panel. The two laboratories that failed 

were using a direct PCR method; one laboratory misclassified the negative pool and other misclassified a 

pool with a high shedding animal. 

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 6
Table 7. Performance of each method used in the Johne’s Disease 2015 Fecal Pooling 

Proficiency Panel. A total of 5 pooled samples were in each panel.

Direct 

PC

Liquid 

media

 Solid 

media

Identified the negative pool as positive 1 0 0

Identified a high -shedding pool as negative 1 0 0

Two non-critical pools were identified as negative 0 0 0

Failed due to multiple criteria 0 0 0

One non-critical pool was misidentified as negative 2 0 0

All 5 pools were identified correctly 39 16 3

Total Failed Pooled Kits 2 (5%) 0 (0%) 0 (0%

Total 43 16 3

No. panels

Panels

that 

failed

Panels

that 

passed

A current listing of all the approved laboratories is available in the 

NVLS web site: Approved laboratories.

Remaining sample vials from the 2015 Proficiency Panel are

available to laboratories for validation or research purposes.

Available samples can be viewed in the reagents catalog under 

Johne’s positive/negative fecal samples on the NVSL web site:

Reagent Catalog. 

Table 8. 2015 Johne’s Disease Pooled Fecal Proficiency Panel key by kit number

Pool Descriptio

Kit# 

1-20

Kit# 

21-40

Kit# 

41-60

Kit# 

61-80

D NegaPive samples 4 1 

1 moderaPe (14-033D8), 4 NegaPive sampls D 4

1 moderaPe (12-009D3), 4 NegaPive samples2 D

1 high (13-0811D), 4 NegaPive samples 3 

1 high (12-03427), 4 NegaPive samples 1 3 2 

Pool Sample Number

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 7
Table 9. 2015 Johne’s Disease Individual Fecal Proficiency Panel key by kit number. Samples are coded by 

color according to shedding status as follows: Negative, Noncritical positive samples, Critical – high shedding 

samples. Sample 26 was the positive control.

Vial # 1-25 26-50 51-75 76-100

1 12-03428 (N5) 14-03358 (IA) 13-01420 (IA) 12-03916 (N

2 13-00349 (IA) 12-03428 (N5) 12-03913 (N5) 12-03914 (

3 13-01420 (IA) 14-02868 (IA) 12-03917 (N5) 13-00354 (IA

4 12-03917 (N5) 12-03916 (N5) 12-03916 (N5) 13-06801 (

5 13-08115 (I5) 13-01420 (IA) 12-03914 (N5) 12-03917 (

6 13-06801 (IA) 12-03913 (N5) 14-03358 (IA) 14-02868 (IA

7 12-03917 (N5) 12-03914 (N5) 13-08115 (I5) 13-06801 (

8 14-02868 (IA) 12-03917 (N5) 11-09754 (aT) 13-01420 (I

9 12-03913 (N5) 13-00349 (IA) 13-00354 (IA) 12-03913 (N

10 11-09754 (aT) 11-09754 (MT) 12-03913 (N5) 12-03428 (

11 13-06801 (IA) 13-08115 (I5) 12-03917 (N5) 11-09754 (

12 12-03914 (ND) 14-03358 (IA) 13-06801 (IA) 12-03913 (N5

13 14-02868 (IA) 13-06801 (IA) 13-00349 (IA) 13-01420 (IA)

14 14-02866 (IA) 13-01420 (IA) 12-03914 (N5) 11-09754 (MT

15 14-03358 (IA) 12-03913 (N5) 14-02868 (IA) 12-03917 (N

16 13-01420 (IA) 13-00354 (IA) 12-03428 (N5) 14-02868 (IA

17 11-09754 (aT) 12-03917 (N5) 11-09754 (MT) 14-02866 (I

18 12-03916 (N5) 14-02868 (IA) 14-03358 (IA) 14-03358 (IA

19 12-03913 (N5) 14-02866 (IA) 14-02868 (IA) 13-08115 (I

20 14-03358 (IA) 12-03914 (N5) 14-02866 (IA) 12-03916 (N

21 13-00354 (IA) 11-09754 (aT) 13-06801 (IA) 13-01420 (IA

22 12-03916 (N5) 13-01420 (IA) 13-01420 (IA) 12-03428 (N

23 13-01420 (IA) 12-03428 (N5) 12-03916 (N5) 12-03914 (

24 12-03428 (N5) 13-06801 (IA) 13-01420 (IA) 13-00349 (IA

25 12-03914 (N5) 12-03916 (N5) 12-03428 (N5) 14-03358 (

26 13-08115 (I5) 13-08115 (I5) 13-08115 (I5) 13-08115 

Any questions or comments can be directed to the Diagnostic Bacteriology Laboratory at 515.337.7388. 

Report was prepared by:

Kevin D. Stokes, PhD

USDA/APHIS/STAS/NVSL

Mycobacteria /Brucella Section

Kevin.D.Stokes@USDA.APHIS.GOV

General Summary of the 2015 Johne’s Disease Fecal Proficiency Panel Page | 8