# Presentation: 302235

## Enhancing the Credibility of Taxonomic Data: The National Wadeable Streams Assessment

**James B. Stribling**
**Kristen L. Pavlik**
- Tetra Tech, Inc.
- 400 Red Brook Blvd., Suite 200
- Owings Mills, MD 21117-5159

**Susan M. Holdsworth**
- U. S. EPA/Office of Wetlands,         Oceans, and Watersheds
- 1200 Pennsylvania Ave., NW
- Washington, DC 20460

## Purpose of presentation

- Scale and scope of the WSA
- Importance of taxonomic data quality in biological assessments
- Procedure for quantifying consistency, repeatability (i.e., precision) and completeness
- Results
- Comparisons
- Effectiveness of corrective actions
- Lessons learned

## National Wadeable Streams Assessment (WSA)

- Western Dataset
- 750 probability sites
- 50 repeat visits
- 50 reference sites

- Eastern Dataset
- 550 probability sites
- 56 repeat visits
- 142 reference sites

## Taxonomic data

- The identities of organisms contained within a sample
- Foundation of biological assessments
- Multimetric indexes
- O/E models
- For multimetric indexes, we are interested in 2 questions
- What are their names? ***AND***
- How many are there?

## Two types of quality control

- Taxon-based
- How well identification matches with “truth”
- Accuracy
- Traditional approach
- Sample-based
- How well sample characterization matches between two taxonomists
- Precision (consistency)
- Because sample is focus of assessment

## Three performance measures

- % taxonomic disagreement (precision)
- Proportion of specimens in sample non-matching
- Low values, greater precision
- MQO=15%
- % difference in enumeration (precision)
- Proportional difference in sample counts
- Low values, greater precision
- MQO=5%
- % taxonomic completeness (completeness)
- Proportion of specimens in sample id’d to target level
- High values, greater completeness
- No specified MQO, but large majority of samples >90%

## Process

**T1 identifies all samples**

**Intermediary (EPA) randomly selects 10% of all samples, per lab**

**T2 *****re-identifies***** selected whole samples**

**Results directly compared**

**Taxonomic performance characteristics calculated**

**T1 sends selected samples to T2**

## Purpose of process

- Determine inconsistencies
- Where are they?
- What are the causes?
- Develop steps to minimize inconsistencies
- Purpose is _***NOT***_  to say one of the taxonomists is right and one is wrong

## Most common causes of differences

- Damaged specimens
- Early instar/juvenile specimens
- Poor mounts (midges & worms)
-  “Morphotyping” midges and worms
- Incomplete samples
- Incorrect data entry
- Less experience with certain taxa

## Example corrective actions

- Mount and re-identify all midges and worms (Chironomidae and Oligochaeta)
- Reexamine Baetidae and Acari
- Reexamine mollusk shells - ID only when “someone’s home”
- Ensure transferred samples are complete
- Carefully proofread all data entry
- Then, go to Round 2

## Sample distribution

| Laboratory | Number of Samples |  |
| --- | --- | --- |
|  | Round 1 | Round 2 |
| A | 11 | 14 |
| B | 20 | 26 |
| C | 3 | * |
| D | 12 | 12 |
| E | 3 | 3 |
| F | 3 | 3 |
| G | 3 | * |
| H | 10 | 13 |
| I | 3 | 2 |
| J | 4 | na |

- * Samples absorbed by other labs

## PTD (Round 1 vs. 2)

**MQO=15%**

## PTD – partitioned, by lab 

- PTD – partitioned, by lab

*****

*****

*****samples absorbed by other labs for R2

**MQO=15%**

## Effectiveness of corrective actions

| Lab | Number of Samples |  | PTD change |
| --- | --- | --- | --- |
|  | Round 1 | Round 2 |  |
| A | 11 | 14 | -9.4 |
| B | 20 | 25 | -3.3 |
| C | 3 | * | na |
| D | 12 | 12 | +2.4 |
| E | 3 | 3 | -2.2 |
| F | 3 | 3 | -5.5 |
| G | 3 | * | na |
| H | 10 | 13 | INC |
| I | 3 | 2 | -8.2 |
| J | 4 | 4** | 0.0 |

- *samples absorbed by other labs; **results carried over from Round 1

## Other issues/lessons learned

- Problematic taxa may require collapsing to higher level
- Morphotyping of midges and worms
- About 10% improvement in consistency WITH slide-mounting
- Wide variability among taxonomists in morphotyping results
- Excessive damage to specimens
- May need to modify sample/specimen handling procedures to minimize (both field and lab)
- Straight disagreements and hierarchical differences _roughly_ equivalent

## Other issues/lessons learned

- Must have _outside_ laboratory or independent taxonomist for whole sample re-identifications
- Taxonomists (both T1 and T2) need to be well-trained AND experienced
- Confirm through NABS certification (in part)
- Majority of labs/taxonomists viewed process as constructive for strengthening programs
- Can and should be applied to smaller programs