General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 1  

2016 Johne’s Disease Fecal Proficiency Panel

General Summary

October 3, 2016

Overview

A total of 62 laboratories participated in the 2016 Johne’s Disease Fecal Proficiency Panel (7

Canadian, 4 European Union, 1 New Zealand, 1 Australian and 49 USA laboratories). Compared to 2015,

the number of individual proficiency panel requesting laboratories increased for direct PCR, decreased

for liquid, and remained the same for solid culture methods. Requests for pooled proficiency panels

increased for direct PCR, decreased for liquid, and remained constant solid culture methods.  Table 1

details the number of individual and pooled panels shipped and the overall pass/fail status for each

method. Laboratories could order multiple panels for each method and were notified of their

preliminary pass/fail status upon submission of their results. A total of 172 panels were requested, with

one reported to be incomplete and needing replaced.  Results were not returned for 3 panels. If

preliminary results indicated the laboratory failed, it was given the opportunity to retake the proficiency

panel provided the results were completed by September 30th, 2016. The results provided in Table 1

include these retests. Laboratories that only used reagents from a single manufacturer, either Tetracore

or Life Technologies, are listed separately. Laboratories that use either in‐house reagents, other

commercial kits not marketed in the US, or mix commercial reagents are listed under the “In House”

category.  One laboratory used in‐house liquid culture reagents and is grouped with the laboratories

using the MGIT system.  All laboratories using solid media were grouped together, including one

laboratory that used in‐house solid media.

Table 1. Summary results of the 2016 Johne’s Disease Fecal Proficiency Panel. In order to pass results must meet the

criteria listed in the 2010 Uniform Program Standards for the Voluntary Bovine Johne's Disease Control Program.

# passed 

1st attempt 

(%)

# failed 

1st attempt 

(%)

# passed 

2nd attempt 

(%)

# failed 

2nd attempt 

(%)

# Kits 

not 

retested

Total 

Shipped

Total shipped in 

2015 (%change)

Individual Panel

Direct PCR (all) 52 (80%) 13 (20%) 5 (83%) 1 (17%) 7 71 60 (+18%)

Tetracore 21 (84%) 4 (16%) 2 (100%) 2 27 25 (+8%)

Li fe Technologies 21 (88%) 3 (13%) 1 (50%) 1 (50%) 1 26 21 (+24%)

In‐House 10 (63%) 6 (38%) 2 (100%) 4 18 13 (+38%)

Liquid Systems (all) 16 (76%) 5 (24%) 5 22 24 (‐8%)

MGIT 960 3 (75%) 1 (25%) 1 4 5 (‐20%)

TREK 13 (76%) 4 (24%) 4 17 17 (+0%)

HEY Solid Media (all) 10 (100%) 11 12 (‐8%)

Individual Panel Total 78 (81%) 18 (19%) 5 (83%) 1 (17%) 12 104 96 (+8%)

Pooling Panel

Direct PCR (all) 46 (98%) 1 (2%) 1 (100%) 48 44 (+9%)

Liquid 14 (88%) 2 (13%) 2 16 17 (‐6%)

HEY 2 (100%) 3 4 (‐25%)

Pooled Panel Total 62 (95%) 3 (5%) 1 (100%) 2 67 63 (+6%)
   

General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 2  

Individual Panel Description  

Each individual panel consisted of 25 unknown samples and one positive control. Positive

samples were collected from naturally infected cows, and negative samples were from individual

animals residing in non‐infected herds. Approximately 4 liters of fecal material were collected rectally

per animal, shipped to NVSL, aliquoted as soon as possible in individual vials, and stored at ‐70°C until

kits were distributed. This year, material from two animals was mixed with material from a different

shedding animal before being aliquoted. These two animals were 15‐00427 (ID) that was mixed with

material from 15‐00426 (ID) and 15‐00628 (IA) that was mixed with material from 15‐00471 (FL).

Although animal 15‐00628 (IA) is characterized as a high shedder, it was included in the non‐critical

shedding animal category since a full evaluation of the mixture was not completed before assembly of

the panels. Panels were assembled in groups, each with a different key (See Table 9 at the end of this

report for the key). Table 2 shows the categorical (positive/negative) performance for each identification

method by animal ID. This year all animals met the required 70% pass rate to be considered valid.

Table 2. Composition of the 2016 Johne’s Disease Fecal Proficiency Panel, and the overall categorical summary

results per cow for each method performed by laboratories.  

All Kits HEY TREK MGIT Life Tech Tetracore In-House

Cow ID 1022 10 13 3 26 27 18

13-00352 (IA) 1 Critical- Neg 98% 100% 100% 100% 100% 96% 94%

13-00353 (IA) 1 Critical- Neg 98% 100% 100% 100% 96% 100% 94%

14-02869 (IA) 4 Critical- Neg 97% 100% 94% 100% 96% 99% 97%

14-02870 (IA) 2 Critical- Neg 99% 100% 100% 100% 100% 100% 100%

12-00956 (KS) 2 Moderate 100% 100% 100% 100% 100% 100% 100%

15-00427 (ID) 2 Moderate 97% 100% 97% 100% 96% 96% 97%

16-01645 (IA)3 2 Mod-High 99% 100% 97% 100% 98% 100% 97%

15-00628 (IA) 2 High 100% 100% 97% 100% 100% 100% 100%

15-00426 (ID) 2 Critical- High 97% 100% 97% 100% 98% 100% 89%

13-08115 (ID) 2 Critical- High 100% 100% 100% 100% 100% 98% 100%

16-01648 (IA) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

15-00471 (FL) 2 Critical- High 99% 100% 100% 88% 100% 98% 100%

16-01646 (IA) 2 Critical- High 100% 100% 100% 100% 100% 100% 97%

2

Number of proficiency panels submitted per method.

3

The positive control was one of the two from this animal.

Percent of Samples Correctly Classified

# Vials 

/Panel

Shedding 

Status1

Liquid Media Direct PCR

1

In order to pass, laboratories must correctly classify critical samples. A critical sample is any negative sample or a 

sample that is identified as a heavy shedder by more than 50% of the laboratories using solid media. 
   

General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 3  

Samples from 2 animals have been used in prior years, 1 in 2014 and 1 in 2015, and their

performance compared. Table 3 shows the respective year panels’ categorical (positive/negative)

performance for each identification method by animal ID.  

Table 3. Comparison between two animals used in the 2014 and 2015 Johne’s Disease Fecal Proficiency Panels with

the overall categorical summary results per cow for each method performed by laboratories.  

All Kits HEY TREK MGIT Life Tech Tetracore In-House

2014 1051 13 22 7 19 24 17

2015 93 12 17 5 21 25 13

Cow ID 2016 102 10 13 3 26 27 18

12-00956 (KS) 2014 2 Moderate 99% 100% 100% 100% 100% 98% 97%

12-00956 (KS) 2016 2 Moderate 100% 100% 100% 100% 100% 100% 100%

13-08115 (ID) 2015 2 Low 99% 100% 100% 100% 100% 98% 100%

13-08115 (ID) 2016 2 Low 100% 100% 100% 100% 100% 98% 100%

1

Number of proficiency panels submitted per method.

Percent of Samples Correctly Classified

Liquid Media Direct PCR

Panel 

Year

# Vials 

/Panel

Shedding 

Status

Table 4 shows the averaged value reported for each of the methods and shedding animals.  

Interestingly, the reported values for animal 12‐00956 (KS) differ between methods. The averaged

values of animal 12‐00956 (KS) are most similar to the high‐shedding animals for liquid culture and

direct PCR.  However, it is more similar to a low‐ or moderate‐shedding animal using solid culture.  

These results are similar to those found in 2014 when this animal was also used in the panels and is

included at the bottom of Table 4.  In our laboratory we isolated both cattle and ‘bison’ strains of

Mycobacterium avium subsp. paratuberculosis (MAP) from 12‐00956 (KS), which is consistent with the

results shown in Table 4 since ‘bison’ MAP strains do not grow well, if at all, on HEY media. For those

laboratories conducting strain differentiation assays, the ‘bison’ MAP strain should be the dominant

strain recovered from the liquid culture systems and the cattle strain from the HEY media.

Table 4.  A comparison of the averaged result values among the three methods for shedding animals.  

HEY1 TREK MGIT Life Tech Tetracore In-House

Colonies 

per Tube

Days to 

Positive

Days to 

Positive Ct Ct Ct

Cow ID 10 13 3 26 27 18

12-00956 (KS) Moderate 11.2 22 12 27.2 25.7 26.3

15-00427 (ID) Moderate 11.0 32 28 32.8 30.1 31.2

16-01645 (IA) Mod-High 24.6 25 26 32.0 29.5 29.7

15-00628 (IA) High 43.8 21 19 27.4 25.3 25.6

15-00426 (ID) Critical- High 22.7 23 17 29.7 29.0 26.9

13-08115 (ID) Critical- High TNTC 20 16 28.2 26.7 26.5

16-01648 (IA) Critical- High 30.0 18 15 26.3 23.1 23.3

15-00471 (FL) Critical- High TNTC 15 13 23.6 22.0 21.6

16-01646 (IA) Critical- High TNTC 14 10 22.3 26.8 20.4

2014

12-00956 (KS) Moderate 10.5 20 18 27.1 27.2 27.5

Average Result Values for Shedding Animals

Liquid Media Direct PCR

Shedding 

Status

1

Results shown include reported values only. Reports that do not include Ct values for direct PCR, daysto-positive for Liquid culture, colonies per tube or list Too-Numerous-To-Count (TNTC) for solid culure 

are not included; this especially skews the values down for the solid culture of high-shedding animals.
   

General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 4  

The performance of each method was further evaluated by determining the number of samples

that were misclassified (Figure 1). In this analysis 100% of the laboratories using solid media correctly

classified all samples. All three direct PCR methods and TREK performed comparably. Eighty‐five percent

of laboratories using Life Technologies direct PCR method correctly classified all the samples and 76% of

laboratories using the TREK system correctly classified all samples.

Figure1. Percentage of 2016 Johne’s disease fecal proficiency panels by number of samples misclassified for the

three culture (TREK liquid media, solid media and MGIT 960 liquid media) and three direct PCR (Tetracore, Life

Technologies, and In‐House) methods. A panel consisted of 25 fecal samples.

According to the 2010 Johne’s Disease Uniform Methods and Rules, laboratories must correctly

classify all critical‐high shedding samples as positive, all negative samples as negative and misidentify

less than 30% of the remaining, valid, non‐critical samples. Table 5 lists the specific reasons laboratories

failed to pass the proficiency panel for each method.  

Table 5. Reasons laboratories failed the 2016 Johne’s Disease Fecal Proficiency Panel.

Laboratories were required to correctly identify all the negative samples as negative and

all the critical high shedding samples as positive (critical samples). They also were required

to correctly classify at least 70% of the remaining samples.  

Direct PCR 

(Tetracore)

Direct PCR 

(Life Tech)

Direct PCR 

(In-House)

TREK 

liquid 

media

MGIT 

liquid 

media

HEY 

solid 

media

Misclassified a negative sample as 

positive

2 3 4 3 00

Missed 4 or more low / moderate 

shedders (lack of sensitivity) 0 0 0 1 00

Misclassified a high shedding sample 

as negative 1 0 2 0 10

Multiple reasons cited above 1 1 0 0 0 0

Total failed kits 1 (4%) 4 (15%) 6 (33%) 4 (24%) 1 (25%) 0 (0%)

Total kitstested 27 26 18 17 4 10
   

General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 5  

As more laboratories use direct PCR as their primary organism detection assay, the performance

of that assay across laboratories becomes more important. Variation in reported cycle threshold (Ct) of

the direct PCR methods was investigated in Figure 2 by comparing the average reported Ct for the

positive samples. Only valid Ct values from each panel were used in this comparison and include

samples categorized as negative but that had valid Ct scores reported (e.g. negative but a Ct of 39.9).

The overall means of all three groups were statistically similar with the average Ct score between the

methods for each animal differing by less than 3, except 16‐01646 that varies by >6. Despite life

technologies having the most laboratories correctly classify all of the samples, this method resulted in

generally higher mean Ct values.  

Figure 2. Average reported Ct of 2016 Johne’s disease fecal proficiency panel animals for the three direct PCR

methods (ABI, Tetracore, and In House). Shedding status is listed below the animal ID.

False positive results with either direct fecal PCR or confirmatory culture PCR continue to be the

most common cause of failure. Table 6 examines the number of negative samples reported with Ct

values by PCR method; this includes laboratories that had Ct values but correctly reported them as

negative. Also shown are the number of panels where at least one Ct is reported. Errors were relatively

evenly distributed amongst the four negative animals that were used in this year’s panel when

considering the number of vials included. There were a total of 12 laboratories that reported Ct values

on at least one negative sample. Of those 12 laboratories, 11 failed the PT (see Table 5) by calling a

negative sample positive and this is a significant increase from year’s past.

Table 6. The number of samples from non‐infected cows

reported with Ct values (regardless of their categorical

positive/negative results) by direct PCR method.  

Tetracore ABI In-House

13-00352 (IA) 1 2

13-00353 (IA) 1 1

14-02869 (IA) 1 4 4

14-02870 (IA) 1 1 3

Num. panels 

reporting Ct 354
   

General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 6  

Pooling Panel Description  

Twenty five individual samples were provided with instructions regarding which 5 samples to

pool together, for a total of 5 pooled samples. Table 7 lists the contents of each pool, and Table 9 lists

the pool numbers associated with each lot of panels.  Laboratories were required to correctly classify

the two negative pools and the two pools that contained a high‐shedding animal (13‐08115 & 16‐01648)

in order to pass. Laboratories were allowed to misclassify the other pool (16‐01645) and still pass the

panel.

Table 7. Composition of the 2016 Johne’s Disease Fecal Pooling

Proficiency Panel.

Cow ID

Avg. 

CFU/ tube*

1 High, 4 Negative samples 16-01648 ~4000

1 High, 4 Negative samples 13-08115 ~500

1 Moderate, 4 Negative samples 16-01645 29

5 Negative samples

5 Negative samples

Positive sample(s) 

description

*Refers to the positive samples, not the pooled sample.

Table 8 further describes the performance of each method used in the pooled proficiency test. It

is commendable that all but three laboratories passed the pooled panel.  The laboratory that failed using

a direct PCR method misclassified a negative pool. Of the laboratories using liquid culture one

misclassified the negative pool and another misclassified a pool with a high shedding animal.  

Table 8. Performance of each method used in the Johne’s Disease 2016 Fecal Pooling

Proficiency Panel. A total of 5 pooled samples were in each panel.  

Direct

PCR

Liquid

media

Solid

media

Identi fied the negative pool as positive 1 1 0

Identi fied a high ‐shedding pool as negative 0 1 0

Two non‐critical pools were identi fied as negative 0 0 0

Failed due to multiple criteria 0 0 0

One non‐critical pool was misidenti fied as negative 1 1 0

All 5 pools were identi fied correctly 46 13 2

Total Failed Pooled Kits 1 (2%) 2 (13%) 0 (0%)

Total 48 16 2

No. panels

Panels

that

failed

Panels

that

passed
   

General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 7  

A current listing of all the approved laboratories is available in the

NVLS web site:  Approved laboratories.  

Remaining sample vials from the 2016 Proficiency Panel are

available to laboratories for validation or research purposes.

Available samples can be viewed in the reagents catalog under

Johne’s positive/negative fecal samples on the NVSL web site:

Reagent Catalog.  

Table 9. 2016 Johne’s Disease Pooled Fecal Proficiency Panel key by kit number.

Pool Description

Kit#     

1‐20

Kit#     

21‐40

Kit#     

41‐60

Kit#     

61‐65

5 Negative samples 2 1 2 3

5 Negative samples 3 2 5 2

1 mod-high (16-01645), 4 Negative samples 4 5 4 1

1 high (13-08115), 4 Negative samples 5 3 1 4

1 high (16-01648), 4 Negative samples 1 4 3 5

Pool Sample Number
   

General Summary of the 2016 Johne’s Disease Fecal Proficiency Panel Page | 8  

Table 10. 2016 Johne’s Disease Individual Fecal Proficiency Panel key by kit number. Samples are coded by

color according to shedding status as follows: Negative, Noncritical positive samples, Critical – high shedding

samples. Sample 26 was the positive control.

Vial # 1-25 26-50 51-75 76-100 100-105

1 13‐08115 (ID) 14‐02870 (IA) 15‐00628 (IA) 15‐00426 (ID) 14‐02870 (IA)

2 13‐00352 (IA) 16‐01645 (IA) 15‐00426 (ID) 15‐00471 (FL) 16‐01646 (IA)

3 14‐02870 (IA) 15‐00427 (ID) 16‐01646 (IA) 14‐02869 (IA) 14‐02869 (IA)

4 15‐00471 (FL) 13‐08115 (ID) 14‐02869 (IA) 12‐00956 (KS) 16‐01646 (IA)

5 14‐02869 (IA) 13‐00353 (IA) 15‐00427 (ID) 15‐00628 (IA) 15‐00426 (ID)

6 15‐00426 (ID) 15‐00471 (FL) 15‐00471 (FL) 16‐01646 (IA) 15-00628 (IA)

7 15‐00471 (FL) 14‐02869 (IA) 14‐02870 (IA) 14‐02869 (IA) 15‐00427 (ID)

8 14‐02869 (IA) 16‐01648 (IA) 16‐01645 (IA) 16‐01648 (IA) 14‐02870 (IA)

9 16‐01645 (IA) 15‐00628 (IA) 15‐00471 (FL) 14‐02870 (IA) 12‐00956 (KS)

10 15‐00427 (ID) 12-00956 (KS) 14‐02869 (IA) 15‐00426 (ID) 13‐08115 (ID)

11 15‐00628 (IA) 15‐00426 (ID) 15‐00628 (IA) 12‐00956 (KS) 13‐00353 (IA)

12 16-01648 (IA) 16‐01646 (IA) 12‐00956 (KS) 13‐08115 (ID) 16‐01648 (IA)

13 14‐02869 (IA) 14‐02869 (IA) 13‐08115 (ID) 13‐00353 (IA) 14‐02869 (IA)

14 16‐01648 (IA) 15‐00426 (ID) 13‐00352 (IA) 16-01646 (IA) 16‐01648 (IA)

15 13‐00353 (IA) 13‐08115 (ID) 16‐01646 (IA) 14‐02870 (IA) 15‐00628 (IA)

16 13‐08115 (ID) 13‐00352 (IA) 14‐02869 (IA) 15‐00628 (IA) 15‐00427 (ID)

17 12‐00956 (KS) 15‐00471 (FL) 13-08115 (ID) 16‐01645 (IA) 16‐01645 (IA)

18 14‐02870 (IA) 14‐02869 (IA) 13‐00353 (IA) 15‐00427 (ID) 14‐02869 (IA)

19 15‐00427 (ID) 15‐00628 (IA) 16‐01648 (IA) 15‐00471 (FL) 15‐00471 (FL)

20 15‐00628 (IA) 16‐01646 (IA) 14‐02869 (IA) 14‐02869 (IA) 15‐00426 (ID)

21 15‐00426 (ID) 14‐02869 (IA) 16‐01648 (IA) 13‐08115 (ID) 13‐00353 (IA)

22 16‐01646 (IA) 12‐00956 (KS) 14‐02870 (IA) 13‐00352 (IA) 15‐00471 (FL)

23 14‐02869 (IA) 15‐00427 (ID) 15‐00426 (ID) 16‐01648 (IA) 12‐00956 (KS)

24 16‐01646 (IA) 14‐02870 (IA) 15‐00427 (ID) 14‐02869 (IA) 13‐00353 (IA)

25 12‐00956 (KS) 16‐01648 (IA) 12‐00956 (KS) 15‐00427 (ID) 13‐08115 (ID)

26 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA)

Any questions or comments can be directed to the Diagnostic Bacteriology Laboratory at 515.337.7388.  

Report was prepared by:

Kevin D. Stokes, PhD

USDA/APHIS/STAS/NVSL

Mycobacteria /Brucella Section

Kevin.D.Stokes@USDA.APHIS.GOV