General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 1  

2018 Johne’s Disease Fecal Proficiency Panel 

General Summary 

October 5, 2018 

Overview 

A total of 60 laboratories participated in the 2018 Johne’s Disease Fecal Proficiency Panel (7 

Canadian, 4 European Union, 1 New Zealand, 2 Australian and 46 USA laboratories). Table 1 details the 

number of individual and pooled panels shipped and the overall pass/fail status for each method. 

Laboratories could order multiple panels for each method and were notified of their preliminary 

pass/fail status upon submission of their results. A total of 164 panels were requested, with 4 replaced 

due to import/export issues. Results were not returned for 3 Individual and 3 Pooled panels. If 

preliminary results indicated the laboratory failed, it was given the opportunity to retake the proficiency 

panel provided the results were completed by September 30th, 2018. The results provided in Table 1 

includes these retests. Laboratories that only used reagents from a single manufacturer, either 

Tetracore or Life Technologies, are listed separately. Laboratories that use either in‐house reagents, 

other commercial kits not marketed in the US, or mix commercial reagents are listed under the “In‐

House” category. One laboratory used in‐house liquid culture reagents and is grouped with the 

laboratories using the MGIT system. All laboratories using solid media were grouped together, 

regardless if they purchased media or used in‐house media (1 laboratory).  

Table 1. Summary results of the 2018 Johne’s Disease Fecal Proficiency Panel. In order to pass results must meet the 

criteria listed in the 2010 Uniform Program Standards for the Voluntary Bovine Johne's Disease Control Program. 

   

2018 # passed 

1st attempt 

(%)

# failed 

1st attempt 

(%)

# passed 

2nd attempt 

(%)

# failed 

2nd attempt 

(%)

# Kits 

not 

retested

Total 

Shipped

Total shipped in 

2017 (%change)

Individual Panel

Direct PCR (all) 49 (80%) 12 (20%) 4 (67%) 2 (33%) 6 69 71 (‐3%)

Tetracore 14 (74%) 5 (26%) 1 (50%) 1 (50%) 3 21 23 (‐9%)

Li fe Technologies 20 (83%) 4 (17%) 2 (100%) 2 26 28 (‐7%)

In‐House 15 (83%) 3 (17%) 1 (50%) 1 (50%) 1 20 18(11%)

Liquid Systems (all) 14 (93%) 1 (7%) 1 16 20 (‐20%)

MGIT 960 4 (100%) 4 5 (‐20%)

TREK 10 (91%) 1 (9%) 1 11 12 (‐8%)

HEY Solid Media (all) 7 (100%) 7 10 (‐30%)

Individual Panel Total 70 (84%) 13 (16%) 4 (67%) 2 (33%) 7 92 101 (‐9%)

Pooling Panel

Direct PCR (all) 45 (92%) 4 (8%) 4 51 48 (+6%)

Liquid 11 (92%) 1 (8%) 1 13 16 (‐19%)

HEY 4 (100%) 4 5 (‐20%)

Pooled Panel Total 60 (92%) 5 (8%) 5 68 69 (‐1%)
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 2  

Individual Panel Description  

Each individual panel consisted of 25 blinded samples and one positive control. Positive samples were 

collected from naturally infected cows, and negative samples were from individual animals residing in 

non‐infected herds. When possible, approximately 4 liters of fecal material were collected rectally per 

animal, shipped to NVSL, aliquoted in to individual vials, and stored at ‐70°C until kits were distributed. 

Because we lacked moderate shedding samples, three samples were produced where material from a 

high shedding cow was diluted with material from a culture negative cow and mixed thoroughly. These 

samples are 15‐00426D (ID), 15‐00471D (FL), and 17‐02487D (FL), reflecting the animal that the positive 

material originated. Panels were assembled in groups, each with a different key (See Table 10 at the end 

of this report for the key). Table 2 shows the categorical (positive/negative) performance for each 

identification method by animal ID. According to the 2010 Uniform Program Standards, a laboratory 

receives a passing score when: all samples from non‐shedding and high shedding animals are correctly 

classified; and they correctly classify 70% of the remaining samples (low and moderate shedding 

animals). This year laboratories were allowed up to 3 misclassifications (67% correctly classified) to pass. 

All samples performed as expected except 17‐03492. The 2010 Uniform Program Standards states that 

an animal is considered valid “by a consensus of at least 70 percent of the laboratories participating in 

the fecal culture check testing process.”  Animal 17‐03492 (WI) had 33% of all vials correctly classified 

(direct PCR and culture) but had 73% correctly classified by laboratories using culture methods, making 

it a valid animal. 

Table 2. Composition of the 2018 Johne’s Disease Fecal Proficiency Panel, and the overall categorical summary 

results per cow for each method performed by laboratories.  

All Kits HEY TREK MGIT Life Tech Tetracore In-House

Cow ID 892 7 11 4 26 21 20

17-03787 (IA) 1 Critical- Neg 94% 100% 100% 100% 100% 81% 95%

17-03786 (IA) 1 Critical- Neg 94% 100% 100% 100% 96% 86% 95%

17-04259 (IA) 3 Critical- Neg 97% 100% 100% 100% 98% 93% 100%

18-01896 (IA) 1 Critical- Neg 98% 100% 100% 100% 94% 98% 95%

17-03492 (WI) 2 Low 33% 71% 73% 75% 21% 17% 20%

18-00942 (ID) 2 Low 98% 95% 97% 100% 96% 98% 100%

17-02487D (FL)3 3 Low 97% 100% 91% 100% 100% 100% 95%

15-00426D (ID)3 1 Moderate 99% 100% 100% 100% 98% 100% 98%

16-01645 (IA)4 2 Mod-High 99% 100% 100% 100% 96% 100% 100%

15-00471D (FL)3 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

17-03210 (FL) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

17-02487 (FL) 3 Critical- High 99% 100% 91% 100% 100% 100% 100%

15-00471 (FL) 1 Critical- High 100% 100% 100% 100% 100% 100% 100%

17-03143 (FL) 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

2

Number of proficiency panels submitted per method.

3

Positive sample diluted with negative material.

4

The positive control was one of the two from this animal.

Percent of Samples Correctly Classified

# Vials 

/Panel

Shedding 

Status1

Liquid Media Direct PCR

1

In order to pass, laboratories must correctly classify critical samples. A critical sample is any negative sample or a 

sample that is identified as a heavy shedder by more than 50% of the laboratories using solid media. 
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 3  

Samples from 9 animals were used in either 2016, 2017, or both and their performance was 

compared. Table 3 shows the respective year panels’ categorical (positive/negative) performance for 

each identification method by animal ID.  

Table 3. Comparison between nine animals used in the 2016, 2017, and 2018 Johne’s Disease Fecal Proficiency 

Panels with the overall categorical summary results per cow for each method performed by laboratories.  

   

All Kits HEY TREK MGIT Life Tech Tetracore In-House

2016 1021 10 13 3 26 27 18

2017 95 9 12 5 28 23 18

Cow ID 2018 87 7 11 4 26 21 18

17-03787 (IA) 2017 2 Critical- Neg 99% 100% 100% 100% 100% 100% 94%

17-03787 (IA) 2018 1 Critical- Neg 94% 100% 100% 100% 100% 81% 95%

17-03786 (IA) 2017 2 Critical- Neg 95% 94% 100% 100% 95% 93% 92%

17-03786 (IA) 2018 1 Critical- Neg 94% 100% 100% 100% 96% 86% 95%

17-03492 (WI) 2017 2 Low-Mod 44% 72% 92% 30% 25% 39% 39%

17-03492 (WI) 2018 2 Low 33% 71% 73% 75% 21% 17% 20%

17-02487D (FL) 2017 2 Moderate 98% 100% 92% 90% 100% 100% 97%

17-02487D (FL) 2018 3 Low 97% 100% 91% 100% 100% 100% 95%

15-00426D (ID) 2016 2 Moderate 97% 100% 97% 100% 96% 96% 97%

15-00426D (ID) 2018 1 Moderate 99% 100% 100% 100% 98% 100% 98%

16-01645 (IA) 2016 2 Mod-High 99% 100% 97% 100% 98% 100% 97%

16-01645 (IA) 2017 2 Mod-High 99% 100% 92% 100% 100% 100% 100%

16-01645 (IA) 2018 2 Mod-High 99% 100% 100% 100% 96% 100% 100%

15-00471D (FL) 2016 2 High 100% 100% 97% 100% 100% 100% 100%

15-00471D (FL) 2017 2 Critical- High 99% 100% 100% 100% 100% 98% 100%

15-00471D (FL) 2018 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

17-02487 (FL) 2017 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

17-02487 (FL) 2018 3 Critical- High 99% 100% 91% 100% 100% 100% 100%

15-00471 (FL) 2016 2 Critical- High 99% 100% 100% 88% 100% 98% 100%

15-00471 (FL) 2017 2 Critical- High 100% 100% 100% 100% 100% 100% 100%

15-00471 (FL) 2018 1 Critical- High 100% 100% 100% 100% 100% 100% 100%

1

Number of proficiency panels submitted per method.

Percent of Samples Correctly Classified

Liquid Media Direct PCR

Panel 

Year

# Vials 

/Panel

Shedding 

Status
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 4  

Table 4 shows the averaged value reported for each of the methods summarized by animal. 

Interestingly, the reported values for animal 17‐03143 (FL) differ between methods with the averaged 

values most similar to the high‐shedding animals for liquid culture and direct PCR.  However, it is more 

similar to a low‐shedding animal using solid culture.  These results are similar to those found in 2014 and 

2016 with animal 12‐00956 (KS), which had a mixed infection of both cattle and ‘bison’ strains of 

Mycobacterium avium subsp. paratuberculosis (MAP).  Reviewing whole genome sequence data from 

our laboratory indicates animal 17‐03143 (FL) likely also has a mixed infection with both cattle and 

‘bison’ strains of MAP, which is consistent with the results shown in Table 4 since ‘bison’ MAP strains do 

not grow well, if at all, on HEY media. For those laboratories conducting strain differentiation assays, the 

‘bison’ MAP strain should be the dominant strain recovered from the liquid culture systems and the 

cattle strain from the HEY media.  

Table 4.  A comparison of the averaged result values among the three methods for shedding animals.  

HEY1 TREK MGIT Life Tech Tetracore In-House

Colonies 

per Tube

Days to 

Positive

Days to 

Positive Ct Ct Ct

Cow ID 9 12 5 28 23 18

17-03492 (WI) Low 0.5 33 34 37.4 37.7 35.9

18-00942 (ID) Low 2.4 32 30 32.1 30.9 31.4

17-02487D (FL) Low 8.5 30 27 32.9 30.9 31.8

15-00426D (ID) Moderate 5.1 31 33 33.1 30.5 31.0

16-01645 (IA) Mod-High 23.8 25 27 31.7 29.8 30.0

15-00471D (FL) Critical- High 39.5 21 16 26.6 25.6 25.4

17-03210 (FL) Critical- High TNTC 18 15 23.6 22.5 22.2

17-02487 (FL) Critical- High 44.0 16 14 23.1 22.2 22.1

15-00471 (FL) Critical- High TNTC 16 13 23.0 22.1 21.7

17-03143 (FL) Critical- High 2.6 16 12 21.9 19.8 20.4

Average Result Values for Shedding Animals

Liquid Media Direct PCR

Shedding 

Status

1

Results shown include reported values only. Reports that do not include Ct values for direct PCR, daysto-positive for Liquid culture, colonies per tube or list Too-Numerous-To-Count (TNTC) for solid culure 

are not included; this especially skews the values down for the solid culture of high-shedding animals.
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 5  

The performance of each method was further evaluated by determining the number of samples 

that were misclassified (Figure 1) using all 25 samples. In this analysis 55% of laboratories using the TREK 

system correctly classified all the samples. For the laboratories using solid media 43% correctly classified 

all the samples. The performance of the three direct PCR methods was similar to last year with even 

fewer laboratories correctly calling all the samples, mainly due to the difficulty of samples from animal 

17‐03492 (WI). Ten percent of laboratories using In‐house direct PCR methods correctly classified all the 

samples.  

Figure1. Percentage of 2018 Johne’s disease fecal proficiency panels by number of samples misclassified for the 

three culture (TREK liquid media, solid media and MGIT 960 liquid media) and three direct PCR (Tetracore, Life 

Technologies, and In‐House) methods. A panel consisted of 25 fecal samples. 
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 6  

According to the 2010 Johne’s Disease Uniform Program Standards, laboratories must correctly 

classify all critical‐high shedding samples as positive, all negative samples as negative and misidentify 3 

or fewer of the remaining, valid, non‐critical samples. Table 5 lists the specific reasons laboratories failed 

to pass the proficiency panel for each method. As in previous years the most common reason for failure 

is misclassifying a negative sample as positive. Interestingly, only laboratories using direct PCR methods 

misclassified negative samples as positive. 

Table 5. Reasons laboratories failed the 2018 Johne’s Disease Fecal Proficiency Panel.  

Because direct PCR is now the most common organism detection assay offered, the 

performance of that assay across laboratories becomes more important. Variation in reported cycle 

threshold (Ct) of the direct PCR methods was investigated in Figure 2 by comparing the average 

reported Ct for the positive samples. Only valid Ct values from each panel were used in this comparison 

and include samples categorized as negative but that had valid Ct scores reported (e.g. negative but a Ct 

of 39.9). The overall means of all three methods for each animal were statistically similar. The diluted 

samples, 15‐00471D (FL), 17‐00426D (FL), and 17‐02487D (FL), all performed similar to natural shedding 

animals. 

Figure 2. Average, and 1 standard deviation, reported Ct of 2018 Johne’s disease fecal proficiency panel animals for 

the three direct PCR methods (ABI, Tetracore, and In House). Shedding status is listed below the animal ID. Animal 

numbers ending in “D” are diluted samples. 

2018 Direct PCR 

(Tetracore)

Direct PCR 

(Life Tech)

Direct PCR 

(In-House)

TREK 

liquid 

media

MGIT 

liquid 

media

HEY 

solid 

media

Misclassified a negative sample as 

positive

6 34

Missed 4 or more low / moderate 

shedders (lack of sensitivity) 1

Misclassified a high shedding sample 

as negative 1

Multiple reasons cited above

Total failed kits 6 (29%) 4 (15%) 4 (20%) 1 (9%) 0 (0%) 0 (0%)

Total kits tested 21 26 20 11 4 7
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 7  

False positive results with either direct fecal PCR or confirmatory culture PCR continue to be the 

most common cause of failure. Table 6 examines the number of negative samples reported with Ct 

values by PCR method; this includes laboratories that had Ct values and correctly reported them as 

negative. Also shown are the number of panels where at least one Ct is reported. Errors were relatively 

evenly distributed amongst the four negative animals that were used in this year’s panel when 

considering the number of vials included. There were a total of 15 laboratories that reported Ct values 

on at least one negative sample. Of those 15 laboratories, 12 failed the PT (see Table 5) by calling a 

negative sample positive and is a slight reduction from last year’s panel (17 of 19 failed).  This continues 

to be a significant issue. 

Table 6. The number of samples from non‐infected cows reported with 

Ct values (regardless of their categorical positive/negative results) by 

direct PCR method.  

Pooling Panel Description  

  Twenty five individual samples were provided with instructions regarding which 5 samples to 

pool together, for a total of 5 pooled samples. Table 7 lists the contents of each pool, and Table 9 lists 

the pool numbers associated with each lot of panels. To pass, laboratories were required to correctly 

classify the negative pool and the two pools that contained a high‐shedding animal (15-00471& 17-

02487). Laboratories were allowed to misclassify one of the other pools (16‐01645) and still pass the 

panel.

Table 7. Composition of the 2018 Johne’s Disease Fecal Pooling 

Proficiency Panel.* 

Tetracore ABI In-House Total

17-03787 (IA) 4 1 5

17-03786 (IA) 4 1 2 7

17-04259 (IA) 5 5 2 12

18-01896 (IA) 1 1 1 3

Num. panels 

reporting Ct 6 4 5 15

Cow ID

Avg. 

CFU/ tube*

1 High, 4 Negative samples 15-00471 ~1250

1 High, 4 Negative samples 17-02487 ~1000

1 Moderate, 4 Negative samples 16-01645 25

1 Moderate, 4 Negative samples 16-01645 25

5 Negative samples

Positive sample(s) 

description

*Refers to the positive samples, not the pooled sample.
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 8  

Table 8 further describes the performance of each method used in the pooled proficiency test. It 

is commendable that all laboratories using solid culture passed.  All but one laboratory passed the 

pooled panel using direct PCR methods.  Three laboratories using liquid culture misclassified the 

negative pool and another misclassified multiple pools.  

Table 8. Performance of each method used in the Johne’s Disease 2018 Fecal Pooling 

Proficiency Panel. A total of 5 pooled samples were in each panel.  

A current listing of all the approved laboratories is available in 

the NVLS web site: 

https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/lab‐

info‐services/sa_approved_labs/ct_approved_labs.  

Remaining sample vials from the 2018 Proficiency Panel are 

available to laboratories for validation or research purposes. 

Available samples can be viewed in the reagents catalog under 

Johne’s positive/negative fecal samples on the NVSL web site 

Reagent Catalog at 

https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/lab‐

info‐services/sa_reagents/ct_reagents  

   

Direct 

PCR

Liquid 

media

 Solid 

media

Identi fied the negative pool as positive 1

Identi fied a high ‐shedding pool as negative 1 1

Two non‐critical pools were identi fied as negative

Failed due to multiple criteria 2

One non‐critical pool was misidenti fied as negative 2

All 5 pools were identi fied correctly 43 11 4

Total Failed Pooled Kits 4 (8%) 1 (8%) 0 (0%)

Total 49 12 4

No. panels

Panels 

that 

failed

Panels 

that 

passed

2018
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 9  

Table 9. 2018 Johne’s Disease Pooled Fecal Proficiency Panel key by kit number. 

Table 10. 2018 Johne’s Disease Individual Fecal Proficiency Panel key by kit number. Samples are coded by 

color according to shedding status as follows: Negative, Non‐critical positive samples, Critical – high 

shedding samples. Sample 26 was the positive control. 

Pool Description

Kit#     

1‐20

Kit#     

21‐40

Kit#     

41‐60

Kit#     

61‐70

5 Negative samples 3 1 4 5

1 mod-high (16-01645), 4 Negative samples 1 3 2 2

1 mod-high (16-01645), 4 Negative samples 4 5 3 4

1 high (17-02487), 4 Negative samples 5 2 1 3

1 high (15-00471), 4 Negative samples 2 4 5 1

Pool Sample Number

Vial # 1-25 26-50 51-75 76-100

1 17‐02487D (FL) 16‐01645 (IA) 17‐02487 (FL) 15‐00426D (ID)

2 17‐03143 (FL) 17‐02487D (FL) 17‐04259 (IA) 18‐00942 (ID)

3 17‐03786 (IA) 17‐03492 (WI) 15‐00471D (FL) 17‐02487 (FL)

4 17‐03210 (FL) 17‐02487 (FL) 18‐00942 (ID) 17‐04259 (IA)

5 17‐02487 (FL) 17‐04259 (IA) 17‐03143 (FL) 17‐03210 (FL)

6 17‐04259 (IA) 15‐00471D (FL) 17‐03786 (IA) 17‐03492 (WI)

7 15‐00471D (FL) 17‐02487D (FL) 16‐01645 (IA) 17‐02487D (FL)

8 17‐03492 (WI) 15‐00471 (FL) 17‐03492 (WI) 17‐03143 (FL)

9 17‐02487 (FL) 17‐03787 (IA) 17‐02487D (FL) 18‐01896 (IA)

10 17‐04259 (IA) 17-03210 (FL) 17‐02487 (FL) 15‐00471D (FL)

11 16‐01645 (IA) 18‐00942 (ID) 17‐04259 (IA) 17‐02487D (FL)

12 17-03143 (FL) 17‐03143 (FL) 17‐03210 (FL) 16‐01645 (IA)

13 18‐01896 (IA) 17‐03786 (IA) 17‐02487D (FL) 17‐02487D (FL)

14 17‐02487D (FL) 17‐02487 (FL) 15‐00426D (ID) 15-00471 (FL)

15 15‐00426D (ID) 17‐04259 (IA) 17‐03492 (WI) 17‐03787 (IA)

16 17‐03492 (WI) 15‐00471D (FL) 17‐03143 (FL) 17‐02487 (FL)

17 18‐00942 (ID) 17‐02487D (FL) 18-01896 (IA) 17‐04259 (IA)

18 15‐00471 (FL) 15‐00426D (ID) 17‐02487 (FL) 15‐00471D (FL)

19 17‐03787 (IA) 18‐00942 (ID) 17‐04259 (IA) 17‐03492 (WI)

20 15‐00471D (FL) 17‐02487 (FL) 17‐03210 (FL) 17‐03143 (FL)

21 17‐02487D (FL) 17‐04259 (IA) 17‐02487D (FL) 17‐03786 (IA)

22 17‐02487 (FL) 17‐03210 (FL) 15‐00471 (FL) 17‐03210 (FL)

23 17‐04259 (IA) 17‐03143 (FL) 17‐03787 (IA) 18‐00942 (ID)

24 17‐03210 (FL) 18‐01896 (IA) 15‐00471D (FL) 17‐02487 (FL)

25 18‐00942 (ID) 17‐03492 (WI) 18‐00942 (ID) 17‐04259 (IA)

26 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA) 16‐01645 (IA)
   

General Summary of the 2018 Johne’s Disease Fecal Proficiency Panel Page | 10  

Any questions or comments can be directed to the Diagnostic Bacteriology and Pathology Laboratory at 

515.337.7388.  

Report was prepared by: 

Kevin D. Stokes, PhD 

USDA/APHIS/DB/NVSL 

Mycobacteria & Brucella Section 

Kevin.D.Stokes@USDA.GOV