General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 1  

2017 Johne’s Disease Fecal Proficiency Panel 

General Summary 

October 2, 2017 

Overview 

A total of 62 laboratories participated in the 2017 Johne’s Disease Fecal Proficiency Panel (7 

Canadian, 4 European Union, 1 New Zealand, 1 Australian and 49 USA laboratories). Table 1 details the 

number of individual and pooled panels shipped and the overall pass/fail status for each method. 

Laboratories could order multiple panels for each method and were notified of their preliminary 

pass/fail status upon submission of their results. A total of 170 panels were requested, with none 

needing to be replaced. Results were not returned for 6 Individual and 4 Pooled panels. If preliminary 

results indicated the laboratory failed, it was given the opportunity to retake the proficiency panel 

provided the results were completed by September 30th, 2017. The results provided in Table 1 include 

these retests. Laboratories that only used reagents from a single manufacturer, either Tetracore or Life 

Technologies, are listed separately. Laboratories that use either in‐house reagents, other commercial 

kits not marketed in the US, or mix commercial reagents are listed under the “In House” category. One 

laboratory used in‐house liquid culture reagents and is grouped with the laboratories using the MGIT 

system. All laboratories using solid media were grouped together, regardless if they purchased media or 

used in‐house media.  

Table 1. Summary results of the 2017 Johne’s Disease Fecal Proficiency Panel. In order to pass results must meet the 

criteria listed in the 2010 Uniform Program Standards for the Voluntary Bovine Johne's Disease Control Program. 

   

# passed 

1st attempt 

(%)

# failed 

1st attempt 

(%)

# passed 

2nd attempt 

(%)

# failed 

2nd attempt 

(%)

# Kits 

not 

retested

Total 

Shipped

Total shipped in 

2016 (%change)

Individual Panel

Direct PCR (all) 43 (73%) 16 (27%) 9 (90%) 1 (10%) 6 71 71 (+0%)

Tetracore 14 (74%) 5 (26%) 3 (75%) 1 (25%) 1 23 27 (‐15%)

Li fe Technologies 19 (79%) 5 (21%) 4 (100%) 1 28 26 (+8%)

In‐House 10 (63%) 6 (38%) 2 (100%) 4 18 18 (+0%)

Liquid Systems (all) 15 (88%) 2 (12%) 2 20 22 (‐9%)

MGIT 960 5 (100%) 5 4 (+25%)

TREK 10 (83%) 2 (17%) 2 12 17 (‐29%)

HEY Solid Media (all) 8 (89%) 1 (11%) 1 10 11 (‐9%)

Individual Panel Total 66 (78%) 19 (22%) 9 (90%) 1 (10%) 9 101 104 (‐3%)

Pooling Panel

Direct PCR (all) 45 (98%) 1 (2%) 1 48 48 (+0%)

Liquid 11 (73%) 4 (27%) 4 16 16 (+0%)

HEY 4 (100%) 5 3 (+67%)

Pooled Panel Total 60 (92%) 5 (8%) 5 69 67 (+3%)
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 2  

Individual Panel Description  

Each individual panel consisted of 25 blinded samples and one positive control. Positive samples were 

collected from naturally infected cows, and negative samples were from individual animals residing in 

non‐infected herds. When possible, approximately 4 liters of fecal material were collected rectally per 

animal, shipped to NVSL, aliquoted as soon as possible in individual vials, and stored at ‐70°C until kits 

were distributed. Because we lacked moderate shedding samples, two samples were produced where 

material from a high shedding cow was diluted with material from a culture negative cow and mixed 

thoroughly. These samples are labeled 17‐02487D (FL) and 15‐00471D (FL), reflecting the animal that 

the positive material originated. Panels were assembled in groups, each with a different key (See Table 9 

at the end of this report for the key). Vial #18 in the second lot of panels (#26 thru #50) did not meet our 

quality criteria and therefore was omitted from the analysis. Table 2 shows the categorical 

(positive/negative) performance for each identification method by animal ID. According to the 2010 

Uniform Program Standards a sample is considered valid “by a consensus of at least 70 percent of the 

laboratories participating in the fecal culture check testing process.” Animal 15‐00627 (IA) did not meet 

this criteria and was not included in the official grading. Removal of samples from this animal left 5 

samples from valid low and moderate shedding animals. Instead of requiring laboratories to correctly 

classify 70% of these non‐critical valid samples (miss up to 1 sample to pass) we allowed up to 3 

misclassifications to pass. 

Table 2. Composition of the 2017 Johne’s Disease Fecal Proficiency Panel, and the overall categorical summary 

results per cow for each method performed by laboratories.  

All Kits HEY TREK MGIT Life Tech Tetracore In-House

Cow ID 952 9 12 5 28 23 18

17-03780 (IA) 1 Critical- Neg 98% 100% 100% 100% 96% 100% 94%

17-03785 (IA) 1 Critical- Neg 97% 100% 100% 100% 100% 91% 94%

17-03784 (IA) 2 Critical- Neg 96% 100% 96% 100% 98% 93% 94%

17-03787 (IA) 2 Critical- Neg 99% 100% 100% 100% 100% 100% 94%

17-03786 (IA) 2 Critical- Neg 95% 94% 100% 100% 95% 93% 92%

15-00627 (IA) 2 Low 52% 39% 50% 20% 38% 70% 69%

17-03492 (WI) 2 Low-Mod 44% 72% 92% 30% 25% 39% 39%

17-02487D (FL)3 2 Mod 98% 100% 92% 90% 100% 100% 97%

16-01645 (IA)4 2 Mod-High 99% 100% 92% 100% 100% 100% 100%

15-00471D (FL)3 2 Critical- High 99% 100% 100% 100% 100% 98% 100%

16-01648 (IA) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

16-01646 (IA) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

17-02487 (FL) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

15-00471 (FL) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

2

Number of proficiency panels submitted per method.

3

Positive sample diluted with negative material.

4

The positive control was one of the two from this animal.

Percent of Samples Correctly Classified

# Vials 

/Panel

Shedding 

Status1

Liquid Media Direct PCR

1

In order to pass, laboratories must correctly classify critical samples. A critical sample is any negative sample or a 

sample that is identified as a heavy shedder by more than 50% of the laboratories using solid media. 
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 3  

Samples from 5 animals were also used in 2016 and their performance compared. Table 3 shows 

the respective year panels’ categorical (positive/negative) performance for each identification method 

by animal ID.  

Table 3. Comparison between five animals used in the 2017 and 2016 Johne’s Disease Fecal Proficiency Panels with 

the overall categorical summary results per cow for each method performed by laboratories.  

Table 4 shows the averaged value reported for each of the methods and shedding animals.   

Table 4.  A comparison of the averaged result values among the three methods for shedding animals.  

All Kits HEY TREK MGIT Life Tech Tetracore In-House

2016 1021 10 13 3 26 27 18

Cow ID 2017 95 9 12 5 28 23 18

16-01645 (IA) 2016 2 Mod-High 99% 100% 97% 100% 98% 100% 97%

16-01645 (IA) 2017 2 Mod-High 99% 100% 92% 100% 100% 100% 100%

15-00471D (FL) 2016 2 High 100% 100% 97% 100% 100% 100% 100%

15-00471D (FL) 2017 2 Critical- High 99% 100% 100% 100% 100% 98% 100%

16-01648 (IA) 2016 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

16-01648 (IA) 2017 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

16-01646 (IA) 2016 2 Critical- High 100% 100% 100% 100% 100% 100% 97%

16-01646 (IA) 2017 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

15-00471 (FL) 2016 2 Critical- High 99% 100% 100% 88% 100% 98% 100%

15-00471 (FL) 2017 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

1

Number of proficiency panels submitted per method.

Percent of Samples Correctly Classified

Liquid Media Direct PCR

Panel 

Year

# Vials 

/Panel

Shedding 

Status

HEY1 TREK MGIT Life Tech Tetracore In-House

Colonies 

per Tube

Days to 

Positive

Days to 

Positive Ct Ct Ct

Cow ID 9 12 5 28 23 18

15-00627 (IA) Low 0.3 35 40 36.7 35.8 35.3

17-03492 (WI) Low-Mod 0.8 34 40 37.7 36.3 36.4

17-02487D (FL) Mod 2.3 31 35 32.4 30.9 31.3

16-01645 (IA) Mod-High 24.3 26 30 32.1 29.7 30.5

15-00471D (FL) Critical- High 36.0 22 22 26.8 25.7 26.0

16-01648 (IA) Critical- High 51.4 18 17 26.1 23.9 24.1

16-01646 (IA) Critical- High TNTC 13 13 21.9 27.1 22.1

17-02487 (FL) Critical- High 15.5 16 17 23.9 22.9 21.9

15-00471 (FL) Critical- High 38.0 16 16 23.2 22.9 22.1

Average Result Values for Shedding Animals

Liquid Media Direct PCR

Shedding 

Status

1

Results shown include reported values only. Reports that do not include Ct values for direct PCR, daysto-positive for Liquid culture, colonies per tube or list Too-Numerous-To-Count (TNTC) for solid culure 

are not included; this especially skews the values down for the solid culture of high-shedding animals.
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 4  

The performance of each method was further evaluated by determining the number of samples 

that were misclassified (Figure 1). Please note that this analysis included all 25 samples, including the 

two samples from cow 15‐00627 (IA) that were not included in the official grading. In this analysis 33% 

of laboratories using the TREK system correctly classified all the samples. No laboratory using solid 

media correctly classified all the samples but most misclassified only 1 sample. The performance of the 

three direct PCR methods were more variable than expected when compared to the previous two years. 

Twenty‐two percent of laboratories using In‐house direct PCR methods correctly classified all the 

samples.  

Figure1. Percentage of 2017 Johne’s disease fecal proficiency panels by number of samples misclassified for the 

three culture (TREK liquid media, solid media and MGIT 960 liquid media) and three direct PCR (Tetracore, Life 

Technologies, and In‐House) methods. A panel consisted of 25 fecal samples. 
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 5  

According to the 2010 Johne’s Disease Uniform Program Standards, laboratories must correctly 

classify all critical‐high shedding samples as positive, all negative samples as negative and misidentify 3 

or fewer of the remaining, valid, non‐critical samples (modified for this year, see discussion above). 

Table 5 lists the specific reasons laboratories failed to pass the proficiency panel for each method. As in 

previous years the most common reason for failure is misclassifying a negative sample as positive. 

Interestingly, it seemed to be a bigger problem for laboratories using direct PCR methods than liquid or 

solid culture. 

Table 5. Reasons laboratories failed the 2017 Johne’s Disease Fecal Proficiency Panel.  

Because direct PCR is now the most common organism detection assay offered, the 

performance of that assay across laboratories becomes more important. Variation in reported cycle 

threshold (Ct) of the direct PCR methods was investigated in Figure 2 by comparing the average 

reported Ct for the positive samples. Only valid Ct values from each panel were used in this comparison 

and include samples categorized as negative but that had valid Ct scores reported (e.g. negative but a Ct 

of 39.9). The overall means of all three groups were statistically similar, except 16‐01646 where a 

statistically significant difference is seen between laboratories using Life Technologies and Tetracore 

reagents.  

Figure 2. Average reported Ct of 2017 Johne’s disease fecal proficiency panel animals for the three direct PCR 

methods (ABI, Tetracore, and In House). Shedding status is listed below the animal ID. 

Direct PCR 

(Tetracore)

Direct PCR 

(Life Tech)

Direct PCR 

(In-House)

TREK 

liquid 

media

MGIT 

liquid 

media

HEY 

solid 

media

Misclassified a negative sample as 

positive

5 5 61 1

Missed 4 or more low / moderate 

shedders (lack of sensitivity) 1

Misclassified a high shedding sample 

as negative

Multiple reasons cited above 1

Total failed kits 6 (26%) 5 (18%) 6 (33%) 2 (17%) 0 1 (11%)

Total kits tested 23 28 18 12 5 9
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 6  

False positive results with either direct fecal PCR or confirmatory culture PCR continue to be the most 

common cause of failure. Table 6 examines the number of negative samples reported with Ct values by 

PCR method; this includes laboratories that had Ct values but correctly reported them as negative. Also 

shown are the number of panels where at least one Ct is reported. Errors were relatively evenly 

distributed amongst the five negative animals that were used in this year’s panel when considering the 

number of vials included. There were a total of 19 laboratories that reported Ct values on at least one 

negative sample. Of those 19 laboratories, 17 failed the PT (see Table 5) by calling a negative sample 

positive and this is a very significant increase from year’s past. 

Table 6. The number of samples from non‐infected cows 

reported with Ct values (regardless of their categorical 

positive/negative results) by direct PCR method.  

Pooling Panel Description  

  Twenty five individual samples were provided with instructions regarding which 5 samples to 

pool together, for a total of 5 pooled samples. Table 7 lists the contents of each pool, and Table 9 lists 

the pool numbers associated with each lot of panels. Laboratories were required to correctly classify the 

negative pool and the two pools that contained a high‐shedding animal (15-00471& 13-08115) in order 

to pass. Laboratories were allowed to misclassify one of the other pools (16‐01645 & 15‐00427) and still 

pass the panel.

Table 7. Composition of the 2017 Johne’s Disease Fecal Pooling 

Proficiency Panel. 

Tetracore ABI In-House

17-03780 (IA) 2 1

17-03785 (IA) 2 1 2

17-03784 (IA) 4 3 4

17-03787 (IA) 1 2 3

17-03786 (IA) 4 4 4

Num. panels 

reporting Ct 757

Cow ID

Avg. 

CFU/ tube*

1 High, 4 Negative samples 15-00471 ~1250

1 High, 4 Negative samples 13-08115 ~500

1 Moderate, 4 Negative samples 16-01645 25

1 Moderate, 4 Negative samples 15-00427 11

5 Negative samples

Positive sample(s) 

description

*Refers to the positive samples, not the pooled sample.
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 7  

Table 8 further describes the performance of each method used in the pooled proficiency test. It 

is commendable that all laboratories using solid culture passed.  All but one laboratory passed the 

pooled panel using direct PCR methods.  Three laboratories using liquid culture misclassified the 

negative pool and another misclassified multiple pools.  

Table 8. Performance of each method used in the Johne’s Disease 2017 Fecal Pooling 

Proficiency Panel. A total of 5 pooled samples were in each panel.  

A current listing of all the approved laboratories is available in 

the NVLS web site: 

https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/lab‐

info‐services/sa_approved_labs/ct_approved_labs.  

Remaining sample vials from the 2017 Proficiency Panel are 

available to laboratories for validation or research purposes. 

Available samples can be viewed in the reagents catalog under 

Johne’s positive/negative fecal samples on the NVSL web site 

Reagent Catalog at 

https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/lab‐

info‐services/sa_reagents/ct_reagents  

   

Direct 

PCR

Liquid 

media

 Solid 

media

Identi fied the negative pool as positive 3

Identi fied a high ‐shedding pool as negative 1

Two non‐critical pools were identi fied as negative

Failed due to multiple criteria 1

One non‐critical pool was misidenti fied as negative

All 5 pools were identi fied correctly 41 10 4

Total Failed Pooled Kits 1 (2%) 4 (27%)

Total 45 15 4

No. panels

Panels 

that 

failed

Panels 

that 

passed
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 8  

Table 9. 2017 Johne’s Disease Pooled Fecal Proficiency Panel key by kit number. 

Table 10. 2017 Johne’s Disease Individual Fecal Proficiency Panel key by kit number. Samples are coded by 

color according to shedding status as follows: Negative, Noncritical positive samples, Critical – high shedding 

samples. Vial 18 in panels 26‐50 was omitted from grading, see discussion above for details.  Sample 26 was 

the positive control. 

Pool Description

Kit#     

1‐20

Kit#     

21‐40

Kit#     

41‐60

Kit#     

61‐65

5 Negative samples 4 2 1 5

1 low-mod (15-00427), 4 Negative samples 1 5 3 2

1 mod-high (16-01645), 4 Negative samples 5 3 4 1

1 high (13-08115), 4 Negative samples 2 4 5 3

1 high (15-00471), 4 Negative samples 3 1 2 4

Pool Sample Number

Vial # 1-25 26-50 51-75 76-100 100-105

1 15‐00627 (IA) 16‐01648 (IA) 15‐00627 (IA) 17‐03784 (IA) 17‐03492 (WI)

2 15‐00471D (FL) 15‐00627 (IA) 17‐03785 (IA) 15‐00471 (FL) 17‐02487D (FL)

3 16‐01645 (IA) 17‐03784 (IA) 17‐02487D (FL) 17‐03786 (IA) 17‐03780 (IA)

4 17‐03492 (WI) 15‐00471 (FL) 17‐03786 (IA) 17‐03787 (IA) 15‐00471D (FL)

5 16‐01646 (IA) 17‐03786 (IA) 17‐02487 (FL) 17‐03492 (WI) 17‐02487 (FL)

6 17‐02487 (FL) 17‐03780 (IA) 17‐03784 (IA) 17‐02487D (FL) 17-03786 (IA)

7 17‐03786 (IA) 16‐01648 (IA) 15‐00471 (FL) 16‐01645 (IA) 17‐03785 (IA)

8 17‐03784 (IA) 17‐03787 (IA) 17‐03787 (IA) 16‐01646 (IA) 15‐00627 (IA)

9 15‐00471 (FL) 17‐02487D (FL) 15‐00471D (FL) 17‐02487 (FL) 15‐00471D (FL)

10 16‐01648 (IA) 15-00627 (IA) 17‐03492 (WI) 16‐01648 (IA) 17‐02487 (FL)

11 17‐03787 (IA) 17‐03785 (IA) 16‐01645 (IA) 15‐00627 (IA) 16‐01648 (IA)

12 16-01648 (IA) 17‐03492 (WI) 15‐00627 (IA) 17‐03784 (IA) 17‐03784 (IA)

13 17‐02487 (FL) 17‐03784 (IA) 17‐03492 (WI) 17‐03780 (IA) 15‐00471 (FL)

14 17‐03492 (WI) 17‐02487D (FL) 17‐02487 (FL) 15-00471D (FL) 17‐03492 (WI)

15 17‐03780 (IA) 15‐00471 (FL) 16‐01648 (IA) 17‐03786 (IA) 17‐03787 (IA)

16 17‐02487D (FL) 15‐00471D (FL) 17‐03784 (IA) 17‐02487 (FL) 16‐01646 (IA)

17 17‐03784 (IA) 17‐03786 (IA) 16-01646 (IA) 16‐01648 (IA) 16‐01645 (IA)

18 15‐00627 (IA) Omitted 15‐00471 (FL) 17‐03492 (WI) 17‐02487D (FL)

19 15‐00471D (FL) 17‐03787 (IA) 17‐03786 (IA) 15‐00471D (FL) 17‐03784 (IA)

20 17‐03785 (IA) 16‐01646 (IA) 17‐03787 (IA) 15‐00471 (FL) 16‐01648 (IA)

21 16‐01646 (IA) 16‐01645 (IA) 15‐00471D (FL) 17‐03787 (IA) 17‐03786 (IA)

22 17‐03786 (IA) 16‐01646 (IA) 17‐03780 (IA) 17‐02487D (FL) 15‐00471 (FL)

23 15‐00471 (FL) 17‐03492 (WI) 17‐02487D (FL) 15‐00627 (IA) 17‐03787 (IA)

24 17‐03787 (IA) 17‐02487 (FL) 16‐01646 (IA) 17‐03785 (IA) 16‐01646 (IA)

25 17‐02487D (FL) 15‐00471D (FL) 16‐01648 (IA) 16‐01646 (IA) 15‐00627 (IA)

26 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA)
   

General Summary of the 2017 Johne’s Disease Fecal Proficiency Panel Page | 9  

Any questions or comments can be directed to the Diagnostic Bacteriology Laboratory at 515.337.7388.  

Report was prepared by: 

Kevin D. Stokes, PhD 

USDA/APHIS/STAS/NVSL 

Mycobacteria /Brucella Section 

Kevin.D.Stokes@USDA.APHIS.GOV