# Presentation: 551489

## Statistical Analysis Plans and Analysis Databases: Regulatory Perspectives

*Steve Wilson, CAPT, USPHS*
- Deputy Director
- Division of Biometrics II/CDER/FDA
- SESSION 420 ST - STATISTICS
*Developing Best Practices and Regulatory Guidance for Statistical Analysis Plans and Databases*
**40th Annual DIA Meeting**
**Washington, DC, USA**
- Thursday, June 17, 2004
- 8: 30 AM - 10: 00 AM Room 140B

## Developing Guidance for Statistical Analysis Plans and Analysis Dataset Submissions: Regulatory Perspectives

*Steve Wilson, CAPT, USPHS*
- Deputy Director
- Division of Biometrics II/CDER/FDA
- SESSION 420 ST - STATISTICS
*Developing Best Practices and Regulatory Guidance for Statistical Analysis Plans and Databases*
**40th Annual DIA Meeting**
**Washington, DC, USA**
- Thursday, June 17, 2004
- 8: 30 AM - 10: 00 AM Room 140B

## Disclaimer

- Views expressed in this presentation are those of the speaker and not, necessarily, of the Food and Drug Administration

## Acknowledgements

- Jane Boone
- Dave Christiansen
- Ron Helms
- Russ Helms
- Kathy Hollinger
- Becky Kush
- Randy Levin
- Ed Nevius
- Norm Stockbridge
- FDA eStat Team
- FDASA
- CDISC SDS, ADaM and Protocol Teams
- Sponsors

## Outline

- Science, Confirmatory Trials, Review & Guidance
- Statistical Analysis Plans and Review
- Current situation
- Guidance: Progress Report
- Analysis Database and Review
- Current situation
- Guidance: Progress Report
- Questions & Advice

## Science & Confirmatory Trials, 

- Science – use statistics to make regulatory decisions concerning the approvability of drug products/claims
- Confirmatory Trials –
- Expectation #1: analysis pre-specified in a  Statistical Analysis Plan.
- Expectation #2: quality assured, well documented analysis datasets will be submitted for review

## Today’s Clinical Trial

- Site/Trial Preparation
- Sites;Trial Sponsor; IRB

- Protocol Development
- PI or Trial Sponsor

- Research Hypothesis
- PI or Trial Sponsor

-  Subject Recruitment
- Sites & CRO

- Trial Management
- &Data Management
- Trial Sponsor; Sites

- Data Analysis
- Reporting of Results
- Trial Sponsor

- Data Collection
- Sites

- Patient Education
- Sites

- Regulatory Submission
- Trial Sponsor to Agency

*Adapted from*

## Statistical Review

- Protocol/SAP
- Clear specification
- Appropriate methodologies
- Sufficient detail
- NDA – Analysis Database
- Assess quality / completeness of data
- Assess compliance with protocol / blinded analysis plans -- conduct of the study
- Check appropriateness of statistical models and conclusions
- Verify results reported in the NDA
- Answer new review-related questions
- Examine individual patient experience during clinical trial

## Analysis Database:Quality Assurance

- Drill down to data item
- Clear description of what was done to each data element: edits, imputations, partial missing...
- Summary of changes
- Trace the creation of an analysis dataset
**UNAMBIGUOUS documentation**

## Issues

- Statistical Analysis Plans  (planning & evaluation)
- Variable format and content
- Review process and expectations not well defined
- JANUS – proposal to include analysis plans in “warehouse”
- Analysis Databases
- eCTD provides “home”
- Sponsors use variable format and content in submissions
- Review process and expectations not well defined
- Evolving CDISC/SDS/ADaM data standards

## Regulatory Response?

- Ignore and muddle through
-  Regulation
-  Guidance

## Two Guidances

## A Reminder: Guidance

- Represents the Agency’s current thinking
- Not binding on FDA or the public
- An alternative approach may be used if such approach satisfies the requirements of the applicable statutes, regulations or both.
- Guidance does not limit the authority of a Center and should not supplant discussions between Centers and sponsors.

**S. Woollen**

**Notes:**
It is important to make the distinction between  the DRAFT GUIDANCE document which we are introducing here today and the Electronic Record; Electronic Signature Rule which I discussed earlier in the way of background.  The Rule,  21 CFR Part 11, represents Agency regulation which applies when the choice is made to use electronic records or electronic signatures to meet an Agency record or signature requirement.

On the other hand, our document on Computerized Systems Used in Clinical Trials is being developed as guidance --- and what guidance means is indicated here:

Guidance represents the Agency’s current thinking.
It is not binding on FDA or the public.
An alternative approach may be used if such approach satisfies the requirements of the applicable statutes, regulations, or both.
And Guidance does not limit the authority of a Center and should not supplant discussions between Centers and sponsors. 

NOTE:  FR v.62 (2/27/97):  Because a guidance document represents the agency’s current thinking on the subject addressed in the document, FDA’s decision makers will take steps to ensure that their staff do not deviate from the guidance document without appropriate justification and appropriate supervisory concurrent.

## Guidance Process

- Developed (internally) by FDA
- Published initially in the Federal Register as a “Draft”
- Comment period (Docket)
- Final guidance issued in Federal Register  (may modify based on comments)

## Developing a Statistical Analysis Plan Guidance

## CDISC/HL7 Standard Protocol Group

- Includes FDA, NIH, sponsors, vendors,
- Started by Kathy Hollinger (FDA) and Becky Kush (CDISC)
- Structured document, with major headings modeled on ICH E6 – Good Clinical Practice Guidance
- Formed Statistical Analysis Plan (SAP) Workgroup

## Statistical Analysis Plan (SAP) Workgroup

- Steve Wilson (FDA), Jane Boone (CDISC), sponsors, vendors
- Includes representatives from CDISC/ADaM Team
- Structured document with headings and content modeled on synthesis of company documents and data elements (working with CDISC/SDS Team

## CDISC SAP Workgroup: Original Purpose

- Develop a spreadsheet for the Statistical Analysis Plan (SAP) analogous to the protocol spreadsheet to represent the content of a SAP.
- Scoping statement: The SAP contains:
- the details of the planned statistical analyses associated with a clinical study so that the analyses are planned with the desired work product(s) in mind and can be conducted in a consistent, repeatable manner.
- the detailed requirements and parameters for the reporting database, statistical programs/output reports, and any tests of the robustness and sensitivity of the analysis.
- The SAP may include example tables, figures and listings.

## Headings:Structured Document Template

- 1	Introduction
- 2	Study design and Objectives
- 3	General Analysis Definitions
- 4	Demographics and Baseline Characteristics
- 5	Patient disposition
- 6	Study medication and concomitant therapy
- 7	Efficacy analyses
- 8	Safety analyses
- 9	Quality of life
- 10	Pharmacoeconomics
- 11	Pharmacokinetics and pharmacodynamics
- 12	Interim analyses and safety monitoring analyses
- 13	Reference
- 14	Protocol violations

## Sample:Structured Document Template

- 1	Introduction
- 2	Study design and Objectives
- 2.1	 Study objectives
- 2.1.1	Primary objectives
- 2.1.2	Secondary objectives
- 2.2	 Trial Design
- 2.3	 Sample Size Justification

## Sample (con.):Structured Document Template

- 3	General Analysis Definitions
- 3.1	Study period and visit window definitions
- 3.1.1	Study periods
- 3.2	Visit windows
- 3.3	Study populations
- 3.3.1	Intent-to-treat population
- 3.3.2	Per-protocol population
- 3.3.3	Safety population
- 3.3.4	Other populations
- 3.4	Subgroup definitions
- 3.5	Treatment assignment and treatment groups
- 3.6	Center pooling method

## FDA Guidance Development Team

- Includes members from five FDA Centers (CBER, CDER, CDRH, CFSAN & CVM)
- Project of the FDA Statistical Association (FDASA)
- Developing guidance describing:
- Purpose of SAP (regulatory “use case”)
- Format and content of SAP
- Review process
- “Informed” by work of CDISC SAP Workgroup

## Developing a Guidance for the Submission of Analysis Datasets

## Analysis Datasets in the 1999 NDA Submission Guidance

- “You should provide a single SAS transport file for each dataset.
- Confer with the appropriate reviewers on the datasets to be provided and the data elements to be included in the datasets.
- Place all dataset files into folders named for the study report or type of data/ analysis.
- Put these folders in the datasets subfolder of the crt folder.”

## Proposed File Structure:eCTD Module 5 – A “Home” for Analysis Datasets

## eCTD Draft Guidance: Individual Subject Data Listings 

- Data tabulations
- Data tabulations datasets
- Data definitions
- Data listing
- Data listing datasets
- Data definitions
- Analysis datasets
- Analysis datasets
- Analysis programs
- Data definitions

## Analysis Datasets (ADs)

- Datasets submitted in addition to the SDTDs to document sponsor’s analysis or to facilitate review
- Also referred to as analysis files, analysis datasets
- FDA will issue separate guidances for SDTs (through CDISC/HL7) and SADs
- Data listings and patient profiles will be generated by FDA from the SDT datasets

## FDA Guidance Development Team

- eStat Workgroup: presently includes representatives from CDER and CBER
- Dave Christiansen: Special Government Employee
- Draft prepared for review by Centers prior to publication in the Federal Register for public comment

## DRAFT GUIDANCE

**Guidance for Industry**
**Providing Regulatory Submissions in Electronic Format — Statistical Analysis Datasets and Documentation**
***DRAFT GUIDANCE***
***Version 0.7 April 14, 2004***
- 	** **Revision History
**DateVersionSummary of ChangesDistribution**11/13/030.1Original Partial DraftSW11/21/030.2Draft of Complete OutlineSW12/28/030.3Reorganize Sections ala SWSW1/5/040.4Edit and add examplesSW1/9/040.5Edit and add draft FDA Uses (IV.B)SW

## Draft Guidance: Standard Metadata/Documentation

- Analysis
- Analysis Datasets
- Analysis Variables

## Guidance Principles

- Clear communication of content, source and quality
- Utilize standard models, variables and codes where they exist and are applicable
- Develop standard metadata model and documentation language
- If there are competing alternatives, select the one that clearly communicates to the reviewer
- All clinical trials are unique, recognize when no standard is available – do not force fit.

## Designed for Review

- Enable reviewers to understand, replicate, explore, confirm, reuse, etc.
- Clear, _unambiguous_ communication of decisions, analysis and results
- Underlying principles:
- Can a reviewing statistician understand?
- Can a reviewing statistician	efficiently:
- Quality Assure?
- Validate?
- Analyze?

## Analysis Metadata

- Analysis name: A unique name. For tables, figures and listings, the name can be the item name.
- Description: Text describing endpoint(s), methodology used, assumptions made, population, etc.
- Authority: If planned and described in the SAP – should be linked to the SAP. If planned, but modified, the value would be “modified” and linked to description in statistical appendix. “Exploratory” analyses would also be identified and linked.

## Analysis Metadata

- Dataset name(s): Name(s) of the dataset(s) used in the analysis. The ADaM analysis datasets provide standard structures for specific methods.
- Program(s): Analysis programs associated with the analysis should be described here.
- Decision support: Description of link between analysis and reported results.

## déjà vu all over again?Guideline: Format and Content, 1988

*déjà vu all over again?**Guideline: Format and Content, *1988

## Questions & Advice

- Other guidances?
- E3 Study Report Statistical sections (Appendix?)
- Programs
- FDA Town Meeting?
- CDISC/HL7/JANUS
- Implications?
- Transition?
- Where do we get the time?

## THANK YOU

- _[wilsons@cder.fda.gov](mailto:wilsons@cder.fda.gov)_
- 301-827-5583