# Presentation: 551510

## National Advisory Committee on Meat and Poultry InspectionAugust 8-9, 2007

**Issue Paper****Data Collection and Analysis at FSIS:****Standard Operating Procedures**

## Overview of presentation

**Purpose of draft document**
**Roles and responsibilities of data collection and analysis teams**
**Process for data collection and analysis at FSIS**
**Stakeholder input**
**Independent peer review**
**Use of data in decision-making**
**Program evaluation**

## Purpose of Draft Document

**To describe the standard operating procedure for data collection and analysis at FSIS**
**Developed in response to stakeholders’ comments **
**Seeking the Committee’s comments:**
**Suggestions for improving data collection and analysis strategy?**
**Suggestions for additional stakeholder input in this process?**
**Suggestions for conducting external peer review?**
**Should NACMPI form an on-going Sub-Committee to assist FSIS in evaluating data issues?  **

## Roles and Responsibilities of the Data Collection and Analysis Teams

**Data Analysis and Integration Group (DAIG)**
**Characterize, coordinate, analyze and integrate data within and across different program areas**
**Data Coordinating Committee (DCC)**
**Senior Agency representatives, serves as a liaison between the various FSIS Program Offices and the DAIG **

## Data Analysis and Integration Group 

**Ensure data analyses are relevant to Program Office business processes and the Agency’s mission**
**Ensure data analyses are consistently of high quality **
**Conduct analyses to inform Agency decisions **
**Provide automated tools to facilitate data analysis and display**

## Data Analysis and Integration Group 

**Conduct analyses to identify data gaps/needs within and across Program Offices**
**Develop sophisticated analytical models to integrate data streams and rapidly identify events, trends and anomalies**
**Ensure data analyses are consistent with FSIS policies and OMB guidelines**

## Process for Data Collection and Analysis

**In response to stakeholder comments, FSIS is formalizing an overall process for data analysis **
**Involves the development of a technical plan, and subsequent technical paper that describes:**
**the problem to be addressed**
**the data collection and analysis strategy**
**the results and interpretation of data analysis **
**Leverage updated IT systems (e.g., PHS)**
**Stakeholder input and peer review incorporated into the process**

## Overview of The Process

**Overview of The Process**

## Problem Definition

**Define issues in terms of questions to be answered **
**Inform decision-making to improve public health**
**Identified by policy managers, data analysts and other Agency officials **
**Purpose and justification for each type of analysis should be stated **
**Include impacts on Agency resources. **
**Stakeholder input**

## Develop Technical Plan

**Summary of issues/questions to be addressed **
**Identify, collect and review existing data **
**Describe data collection strategy**
**Standard/ validated methodology**
**Statistically valid sampling plan**
**Expert elicitation**
**Describe data analysis methods **
**Descriptive statistics **
**Probabilistic methods**
**Program evaluation**
**Stakeholder input and independent peer review**

## Collect and analyze data

**Follow methods described in the technical plan**
**Consider sources of uncertainty and variability in data**
**Discuss validity of assumptions made**
**Determine whether additional data or other analytical techniques are needed**

## Technical Report

**Policy issues driving the analysis**
**The sources and quality of the data **
**Methodology used**
**Results**
**Sources of uncertainty and variability**
**Data gaps and assumptions **
**Internal review **
**External peer review**

## Stakeholder input 

**Integral part of data collection and analysis**
**Problem definition**
**Input into framing and context of issue**
**Review questions to be addressed**
**Purpose and justification for analysis**
**Technical plan**
**Additional sources of existing data**
**Review proposed methods of data collection and analysis**
**Technical report**
**Results of data analysis**
** **

## External Peer Review 

**Ensure scientifically sound data collection/ analysis**
**Peer Reviewers could include:**
**NACMPI**
**NACMCF**
**National Academy of Sciences**
**Subject matter experts **
**Technical plan**
**Data quality issues, e.g., avoid bias in data sets**
**Appropriate methods of analysis used**
**Validity of assumptions**
**Results of data analysis**

## Use data in decision-making

**Based on the results of the analyses, make recommendations how to best address the policy issues/questions identified during the problem definition phase**

## Program evaluation

**Evaluate outcome of program compared with control**
**Consider type of data needed for comparison**
**How well is program achieving its objectives? **
**Is there a need to improve the program?**
**Ideally, measure improvements in public health outcomes**
**E.g., reduction in illness attributable to specific foods**
**Surrogate: **
**Reduction in pathogen prevalence/ levels **
**Reduction in product recalls**
**Use to refining program planning, development, and accountability**

## Questions

**Do you have any suggestions for improving our strategy for data collection and analysis?**
**Do you have other suggestions for stakeholder input in this process?**
**Do you have any other suggestions for conducting peer review?**
**Do you believe it would be worthwhile to form an on-going Sub-Committee to assist FSIS in evaluating various data issues?  **
**If so, please provide a rationale as to why it would be useful and recommendations on how it would be structured and should operate.**

## Contact Information:

**Isabel Walls, Ph.D.**
**Office of Food Defense & Emergency Response**
**Food Safety Inspection Service**
**(202) 690-0834**
**Isabel.walls@fsis.usda.gov**