General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 1

2014 Johne’s Disease Fecal Proficiency Panel

General Summary

October 8, 2014

Overview

A total of 59 laboratories participated in the 2014 Johne’s Disease Fecal Proficiency Panel (7

Canadian, 4 European Union, 1 New Zealand, 1 Australian and 46 USA laboratories). Compared to 2013, 

the number of requesting laboratories increased for individual proficiency panels for direct PCR and 

decreased for liquid and solid culture methods. Requests for pooled proficiency panels increased for 

direct PCR, and decreased for liquid and solid culture methods. Table 1 details the number of individual 

and pooled panels shipped and the overall pass/fail status for each method. Laboratories could order 

multiple panels for each method and were notified of their preliminary pass/fail status upon submission

of their results. A total of 168 panels were requested; results were not returned for 6 of them. There 

was 1 kit reported to be faulty this year. If preliminary results indicated that the laboratory had failed, 

they were given the opportunity to retake the proficiency panel provided the results were completed by 

September 30th, 2014. The results provided in Table 1 include these retests. Laboratories that only used 

reagents from a single manufacturer, either Tetracore or Life Technologies, are listed separately.

Laboratories that use either in-house reagents, other commercial kits not marketed in the US, or mix 

commercial reagents are listed under the “In House” category.

Table 1. Summary results of the 2014 Johne’s Disease Fecal Proficiency Panel. In order to pass results must meet the 

criteria listed in the 2010 Uniform Program Standards for the Voluntary Bovine Johne's Disease Control Program.

# passed 

1st attempt 

(%)

# failed 

1st attempt 

(%)

# passed 

2nd attempt 

(%)

# failed 

2nd attempt 

(%)

# Kits 

not 

retested

Total 

Shipped

Total shipped in 

2013 (%change)

Individual Panel

Direct PCR (all) 53 (91%) 5 (9%) 2 (100%) 3 63 61 (3%)

Tetracore 23 (96%) 1 (4%) 1 2 4 25 (-4%)

Life Technologies 16 (89%) 2 (11%) 1 (100%) 1 1 9 16 (19%)

In-House 14 (88%) 2 (13%) 1 (100%) 1 1 7 19 (-11%)

Liquid Systems (all) 24 (92%) 2 (8%) 1 (100%) 1 29 30 (-3%)

MGIT 960 7 (100%) 7 11 (-36%)

TREK 17 (89%) 2 (11%) 1 (100%) 1 2 2 19 (16%)

HEY Solid Media (all) 12 (92%) 1 (8%) 1 13 19 (-32%)

Individual Panel Total 89 (92%) 8 (8%) 3 (100%) 5 105 110 (-5%)

Pooling Panel

Direct PCR (all) 37 (93%) 3 (8%) 1 (100%) 2 41 39 (5%)

Liquid 17 (100%) 18 21 (-14%)

HEY 4 (100%) 4 6 (-33%)

Pooled Panel Total 58 (95%) 3 (5%) 1 (100%) 2 63 65 (-3%)
General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 2

Individual Panel Description 

Each individual panel consisted of 25 unknown samples and one positive control. Positive 

samples were collected from naturally infected cows, and negative samples were from individual 

animals residing in non-infected herds. Approximately 4 liters of fecal material were collected rectally 

per animal, shipped to NVSL, aliquoted as soon as possible in individual vials, and stored at -70°C until 

kits were distributed. Panels were assembled in groups, each with a different key (See Table 9 at the end 

of this report for the key). Table 2 shows the categorical (positive/negative) performance for each 

identification method by animal ID. Samples from cow 11-09382 were very challenging (red numbers) 

with only 46% of the samples classified correctly. Because 11-09382 samples failed to meet the required 

70% pass rate, they were excluded from the official panel grading, but are still included in this analysis.

Table 2. Composition of the 2014 Johne’s Disease Fecal Proficiency Panel, and the overall categorical summary 

results per cow for each method performed by laboratories.

All Kits HEY TREK MGIT Life Tech Tetracore In-House

Cow ID 105213 22 7 19 24 17

10-05134 (OH) 2 Critical- Neg 99% 100% 100% 100% 100% 98% 100%

10-06315 (OH) 2 Critical- Neg 98% 100% 92% 100% 100% 100% 97%

13-01419 (IA) 2 Critical- Neg 99% 96% 100% 100% 97% 100% 100%

13-00349 (IA) 2 Critical- Neg 99% 100% 100% 100% 97% 100% 97%

11-09382 (MT) 2 Low 46%3

42% 58% 13% 42% 44% 56%

12-03913 (ND) 2 Low 89% 85% 100% 63% 92% 90% 88%

12-03917 (ND) 2 Low 96% 92% 100% 75% 97% 98% 97%

12-00956 (KS) 2 Moderate 99% 100% 100% 100% 100% 98% 97%

12-00953 (KS) 2 Moderate 97% 100% 100% 100% 100% 94% 94%

11-07393 (IA) 2 High 98% 92% 100% 88% 100% 100% 100%

12-03430 (ND)4 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

11-09754 (MT) 2 Critical- High 99% 100% 100% 100% 100% 98% 100%

12-03427 (ND) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

2Number of proficiency panels submitted per method.

3

In order to be considered valid, more than 70% of the samples from an animal must be correctly classified.

4The positive control was one of the two from this animal.

Liquid Media Direct PCR

Percent of Samples Correctly Classified

1

In order to pass, laboratories must correctly classify critical samples. A critical sample is any negative sample or a 

sample that is identified as a heavy shedder by more than 50% of the laboratories using solid media. 

# Vials 

/Panel

Shedding 

Status1
General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 3

Reported values for animal 12-00956 (KS) differ between methods and should be noted. Table 3

shows the averaged value reported for each of the methods and shedding animals. The averaged values 

of animal 12-00956 (KS) are most similar to the high-shedding animals for liquid culture and direct PCR. 

However, it is more similar to a low- or moderate-shedding animal using solid culture. In our laboratory 

we isolated both cattle and ‘bison’ strains of Mycobacterium avium subsp. Paratuberculosis (MAP) from 

12-00956 (KS), which is consistent with the results shown in Table 3 since ‘bison’ MAP strains do not 

grow well, if at all, on HEY media. For those laboratories conducting strain differentiation assays, the 

‘bison’ MAP strain should be the dominant strain recovered from the liquid culture systems and the 

cattle strain from the HEY media.

Table 3. A comparison of the averaged result values among the three methods for shedding animals.

HEY1TREK MGIT Life Tech Tetracore In-House

Colonies 

per Tube

Days to 

Positive

Days to 

Positive Ct Ct Ct

Cow ID 13 22 7 19 24 17

11-09382 (MT) Low 0.3 41 41 37.6 37.6 36.6

12-03913 (ND) Low 2.3 36 37 34.8 34.7 33.6

12-03917 (ND) Low 6.0 34 32 33.9 33.6 32.5

12-00956 (KS) Moderate 10.5 20 18 27.1 27.2 27.5

12-00953 (KS) Moderate 21.3 26 24 31.5 30.5 30.3

11-07393 (IA) High 10.6 18 14 24.6 27.6 25.0

12-03430 (ND) Critical- High 8.6 22 15 28.0 27.3 27.4

11-09754 (MT) Critical- High 10.7 19 16 26.4 25.0 24.6

12-03427 (ND) Critical- High 4.9 18 13 25.4 25.0 24.5

1Results shown include reported values only. Reports that do not include Ct values for direct PCR, daysto-positive for Liquid culture, colonies per tube or list Too-Numerous-To-Count (TNTC) for solid culure 

are not included; this especially skews the values down for the solid culture of high-shedding animals.

Shedding 

Status

Average Result Values for Shedding Animals

Liquid Media Direct PCR

When evaluating laboratory performance for each animal there was an unusual level of 

variability noticed among duplicate samples from animal 11-07393 (IA). While this animal was classified 

as a critical-high shedder due to NVSL’s CFU counts, several laboratories reported one sample as high 

shedding and the other as low to moderate. This finding was independent of method and appeared to 

be inherent in the sample. Because of this unusual level of variability reported, we reclassified these 

samples as non-critical high. This reclassification allowed 2 laboratories who would have failed the 

proficiency test to pass. Unfortunately this variability was not detected until roughly 50% of the 

laboratories reported their results.
General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 4

The performance of each method was further evaluated by determining the number of samples 

that were misclassified (Figure 1). Please note that this analysis included all 25 samples, including the 

two samples from cow 11-09382 (MT) that were not included in the official grading. In this analysis the 

In House Direct PCR methods outperformed all other methods with 41% of the kits correctly classifying 

all 25 samples. Thirty-two percent of laboratories using the TREK system correctly classified all samples,

and 15% of the laboratories using solid media correctly classified all samples.

Figure1. Percentage of 2014 Johne’s disease fecal proficiency panels by number of samples misclassified for the

three culture (TREK liquid media, solid media and MGIT 960 liquid media) and three direct PCR (Tetracore, Life 

Technologies, and In-House) methods. A panel consisted of 25 fecal samples.

According to the 2010 Johne’s Disease Uniform Methods and Rules, laboratories must correctly 

classify all critical-high shedding samples as positive, all negative samples as negative and misidentify 

less than 30% of the remaining, valid, non-critical samples. Table 4 lists the specific reasons laboratories 

failed to pass the proficiency panel for each method. 

Table 4. Reasons laboratories failed the 2014 Johne’s Disease Fecal Proficiency Panel. 

Laboratories were required to correctly identify all the negative samples as negative and 

all the critical high shedding samples as positive (critical samples). They also were required 

to correctly classify at least 70% of the remaining samples. 

Direct PCR 

(Tetracore)

Direct PCR 

(Life Tech)

Direct PCR 

(In-House)

TREK 

liquid 

media

MGIT 

liquid 

media

HEY 

solid 

media

Misclassified a negative sample as 

positive

1 2 2 2 0 1

Missed 3 or more low / moderate 

shedders (lack of sensitivity) 0 0 0 0 0 0

Misclassified a high shedding sample 

as negative 1 0 0 0 0 0

Multiple reasons cited above 1 0 0 0 0 0

Total failed kits 1 (4%) 2 (11%) 2 (12%) 2 (11%) 0 (0%) 1 (8%)

Total kits tested 24 19 17 20 6 13
General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 5

As more laboratories use direct PCR as their primary organism detection assay, the performance 

of that assay across laboratories becomes more important. Variation in reported cycle threshold (Ct) of 

the direct PCR methods was investigated in Figure 2 by comparing the average reported Ct for the 

positive samples. Only valid Ct values from each panel were used in this comparison and include

samples categorized as negative but that had valid Ct scores reported (e.g. negative but a Ct of 39.9).

However, any Ct reported as a range (e.g. >38) or as a text entry (e.g. undetermined) were excluded

from this analysis. Except for animal 11-07393 (IA), the animal where inconsistencies were noted among 

the samples, the overall means of all three groups were remarkably similar with the average Ct score

between the methods for each animal differing by less than 1.8. These differences were not statistically 

significant, even for 11-07393 (IA) with a difference of 2.97.

Figure 2. Average reported Ct of 2014 Johne’s disease fecal proficiency panel animals for the three direct PCR

methods (ABI, Tetracore, and In House). Shedding status is listed below the animal ID.

False positive results with either direct fecal PCR or confirmatory culture PCR continue to be the 

most common cause of failure. While none of the non-infected cows have been used in the check test 

previously, fecal material from animals these herds has been used as negative samples in the proficiency 

panel in previous years. Although we did not include any samples from animals that were shedding over 

7,000 CFU per tube this year, we did have one that contained 5,700 CFU per tube (2 samples) and 

another that contained 3,350 CFU per tube (2 samples). Table 5 examines the number of negative 

samples reported with Ct values by PCR method; this includes laboratories that had Ct values but 

correctly reported them as negative. Errors were relatively evenly distributed amongst the samples. 

There were a total of 8 laboratories that reported Ct values for negative samples; of those two reported 

more than one negative sample with Ct values. There were a total of 8 laboratories that failed the PT 

(see Table 4) by calling negative samples positive, the same as the last two years. Although a larger 

percentage of laboratories correctly called all the samples using In House methods, the laboratories 

using the same In House methods also reported more Ct values for the negative samples.
General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 6

Table 5. The number of samples from non-infected cows 

reported with Ct values (regardless of their categorical 

positive/negative results) by direct PCR method. 

Tetracore ABI In-House

10-05134 (OH) 2 0 0

10-06315 (OH) 0 0 2

13-01419 (IA) 0 1 1

13-00349 (IA) 1 1 3

Pooling Panel Description 

Twenty five individual samples were provided with instructions regarding which 5 samples to 

pool together, for a total of 5 pooled samples. Table 6 lists the contents of each pool, and Table 8 lists 

the vial numbers associated with each pool. Laboratories were required to correctly classify the 

negative pool and the two pools that contained a high-shedding animal (11-06361 & 12-00953) in order 

to pass. Laboratories were allowed to misclassify one of the other two pools (moderate or mod-high)

and still pass the panel.

Table 6. Composition of the 2014 Johne’s Disease Fecal Pooling 

Proficiency Panel.

Cow ID

Avg. 

CFU/ tube*

1 High, 4 Negative samples 12-03427 3,350

1 High, 4 Negative samples 11-09754 1,950

1 Mod-High, 4 Negative samples 11-06361 86

1 Moderate, 4 Negative samples 12-00953 43

5 Negative samples

Positive sample(s) 

description

*Refers to the positive samples, not the pooled sample

Table 7 further describes the performance of each method used in the pooled proficiency test.

Interestingly, the only laboratories that failed misclassified a pool containing a high shedding animal and 

were using a direct PCR method. It is commendable that no laboratory misclassified the negative pool 

and that only three laboratories failed out of 63 kits. All laboratories that submitted results using liquid 

and solid culture methods passed and also correctly classified all the pools.
General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 7

Table 7. Performance of each method used in the Johne’s Disease 2014 Fecal Pooling 

Proficiency Panel. A total of 5 pooled samples were in each panel.

Direct 

PCR

Liquid 

media

 Solid 

media

Identified the negative pool as positive 0 0 0

Identified a high -shedding pool as negative 3 0 0

Two non-critical pools were identified as negative 0 0 0

Failed due to multiple criteria 0 0 0

One non-critical pool was misidentified as negative 0 0 0

All 5 pools were identified correctly 3 8 1 6 4

Total Failed Pooled Kits 3 (7%) 0 (0%) 0 (0%)

Total 4 1 1 6 4

Panels 

that 

passed

No. panels

Panels 

that 

failed

A current listing of all the approved laboratories is available in the 

NVLS web site: Approved laboratories.

Remaining sample vials from the 2014 Proficiency Panel are

available to laboratories for validation or research purposes.

Available samples can be viewed in the reagents catalog under 

Johne’s positive/negative fecal samples on the NVSL web site:

Reagent Catalog. 

Table 8. 2014 Johne’s Disease Pooled Fecal Proficiency Panel key by kit number

Pool Description

Kit# 

1-25

Kit# 

25-50

Kit# 

51-75

5 Negative samples 5 1 3

1 moderate (12-00953), 4 Negative samples 2 4 1

1 mod-high (11-06361), 4 Negative samples 1 2 4

1 high (11-09754), 4 Negative samples 3 5 2

1 very high (12-03427), 4 Negative samples 4 3 5

Pool Sample Number
General Summary of the 2014 Johne’s Disease Fecal Proficiency Panel Page | 8

Table 9. 2014 Johne’s Disease Individual Fecal Proficiency Panel key by kit number. Samples are coded by 

color according to shedding status as follows: Negative, Noncritical positive samples, Critical – high shedding 

samples. Sample 26 was the positive control.

Vial # 1-25 26-50 51-75 76-100 101-125

1 13-00349 (IA) 12-03917 (ND) 12-03427 (MT) 10-06315 (OH) 11-09754 (ND)

2 11-09754 (ND) 11-07393 (ND) 12-03913 (ND) 10-05134 (KS) 13-01419 (IA)

3 10-05134 (KS) 13-01419 (IA) 11-09382 (MT) 11-09754 (ND) 11-09382 (MT)

4 11-07393 (ND) 12-03430 (IA) 10-05134 (KS) 13-00349 (IA) 12-00956 (KS)

5 10-06315 (OH) 10-06315 (OH) 12-00956 (KS) 11-07393 (ND) 12-03917 (ND)

6 11-09382 (MT) 12-00953 (OH) 11-09754 (ND) 12-03913 (ND) 12-03427 (MT)

7 12-03427 (MT) 12-00956 (KS) 10-05134 (KS) 12-00953 (OH) 13-01419 (IA)

8 13-01419 (IA) 12-03913 (ND) 12-00953 (OH) 12-03917 (ND) 11-07393 (ND)

9 12-03917 (ND) 11-09382 (MT) 12-03430 (IA) 11-09754 (ND) 10-06315 (OH)

10 12-00953 (OH) 13-00349 (IA) 13-01419 (IA) 13-00349 (IA) 12-03427 (MT)

11 12-00956 (KS) 11-09754 (ND) 12-03427 (MT) 12-00956 (KS) 13-00349 (IA)

12 12-03913 (ND) 10-05134 (KS) 13-00349 (IA) 11-07393 (ND) 12-03913 (ND)

13 12-00953 (OH) 11-09754 (ND) 11-07393 (ND) 10-05134 (KS) 12-03917 (ND)

14 12-03917 (ND) 13-00349 (IA) 10-06315 (OH) 11-09382 (MT) 12-03430 (IA)

15 11-07393 (ND) 12-00956 (KS) 13-01419 (IA) 12-03913 (ND) 10-06315 (OH)

16 13-01419 (IA) 12-03913 (ND) 12-03917 (ND) 12-03427 (MT) 12-00953 (OH)

17 12-00956 (KS) 12-03917 (ND) 11-09754 (ND) 13-01419 (IA) 11-09754 (ND)

18 12-03427 (MT) 12-00953 (OH) 10-06315 (OH) 12-00953 (OH) 10-05134 (KS)

19 10-06315 (OH) 11-07393 (ND) 11-09382 (MT) 12-03917 (ND) 11-07393 (ND)

20 11-09382 (MT) 13-01419 (IA) 12-00956 (KS) 12-03430 (IA) 10-05134 (KS)

21 12-03913 (ND) 12-03427 (MT) 13-00349 (IA) 13-01419 (IA) 11-09382 (MT)

22 13-00349 (IA) 10-05134 (KS) 11-07393 (ND) 11-09382 (MT) 12-03913 (ND)

23 12-03430 (IA) 12-03427 (MT) 12-00953 (OH) 12-00956 (KS) 13-00349 (IA)

24 10-05134 (KS) 10-06315 (OH) 12-03913 (ND) 12-03427 (MT) 12-00953 (OH)

25 11-09754 (ND) 11-09382 (MT) 12-03917 (ND) 10-06315 (OH) 12-00956 (KS)

26 12-03430 (IA) 12-03430 (IA) 12-03430 (IA) 12-03430 (IA) 12-03430 (IA)

Any questions or comments can be directed to the Diagnostic Bacteriology Laboratory at 515.337.7388. 

Report was prepared by:

Kevin D. Stokes, PhD

USDA/APHIS/STAS/NVSL

Mycobacteria /Brucella Section

Kevin.D.Stokes@USDA.APHIS.GOV