# Presentation: 066987

## Submission of Microarray Data:Dealing Effectively with Data Quality Issues and Information Content Necessary to Develop an Expression Database  

- Advisory Committee for Pharmaceutical Science
- June 10, 2003

## Challenge: Over 50% of drug failures due to efficacy/tox. 
Problem: 20 million US patients exposed to drugs withdrawn between Sept. ’97 and Sept. ’98
Problem: Only 1 in 10 IND’s become approved drugs  -- Our current methods of candidate characterization are only 10% accurate
Solution: Bridge genomics and chemistry to broadly understand a compound’s effects in genomic terms

**Problem**: 20 million US patients exposed to drugs withdrawn between Sept. ’97 and Sept. ’98
**Problem:** Only 1 in 10 IND’s become approved drugs  -- Our current methods of candidate characterization are only 10% accurate
**Solution:** Bridge genomics and chemistry to broadly understand a compound’s effects in genomic terms

**Vision and Challenge**

## Assumptions  -- Sponsor side

- Sponsor providing data to support IND or NDA
- Data is part of a larger package and not the sole and only evidence provided
- Sponsor has on going array efforts with trained staff **or** will contract with CRO that meets these requirements

## Assumptions – Agency side

- Agency is willing to develop and train staff so that the data is meaningfully interpreted and a balanced view is taken
- Sponsor is concerned about overly reactive view
- Sponsor is concerned about the future liability from public disclosure
- Agency is able to accept data in a community defined standard format and has capacity to assess quality
- Agency is prepared for all current technologies
- Agency will keep up with future technology improvements
- Agency desires to deposit submitted data into an internal database for use by staff during future evaluations
- Agency understands that context is essential for a balanced view

## Array measurements similarities to traditional measurements, but... 

- ALT elevation is associated with liver damage
- Treated group value is 55 ± 7 (SE) [range: 30-75], n=5
- Control group value is 50 ± 4 (SE) [range: 15-110] n=15
**No treated animals outside range of controls**
**Conclusion:** ALT not significantly changed by treatment, consistent with good liver safety.
- Gene expression data on 5 RNAs associated with liver damage
**No treated animal outside range of controls**
**Conclusion:** 5 RNAs associated with liver toxicity not significantly changed, consistent with good liver safety.

|  | RNA A | RNA B | RNA C | RNA D | RNA E |
| --- | --- | --- | --- | --- | --- |
| Ratio to control  | 0.9 | 1.0 | 1.2 | 0.8 | 1.4 |
| SE
N (Treated, Untreated) | 0.1
(5,15) | 0.2
(5,15) | 0.1
(5,15) | 0.4
(5,15) | 0.5
(5,15) |
| Range
(significant at p<0.05) | 1.1-0.8
n.s. | 1.2-0.7
n.s. | 1.4-0.9
n.s. | 0.6-1.1
n.s. | 1.7-0.9
n.s. |

## ...  but, microarray data has differences from conventional measurements

**Both agency and community has lower familiarity with technology**
- Will improve with experience
**Concern that the survey might uncover “confounding” factors**
- Sponsor concerned about a “overly reactive view”
- FDA concerned about sponsor missing important findings
**Less “scientific agreement” about how to interpret and meaning**
- Pattern matching is less familiar to biological community
- Requires a different mindset than single gene focus of the past
**Perception that microarray data is lower quality and noisier**
- Technology has improved
- Carefully conducted experiments are accurate and predictive

## Summary: What should be provided by sponsor to FDA

**Summary: What should be provided by sponsor to FDA**

**MIAME/MAGE-ML compliant descriptions of experiment(s) and electronic submission of all data**
**Minimum experimental design metrics similar to that required for any other biological experiment**
**“****Youth” of technology requires that additional quality data be submitted to demonstrate competency of experimenters**
**Sponsors interpretation of data in “scientific paper style” format**
**Comparison to community accepted RNA biomarkers and compared to Bench Mark Drugs and Toxicants **
- Interpretation vs. context of current drugs failed drugs and toxicants

## Sponsor provided data to assure QUALITY

**Themes**
- Measurement vs. lab historical values
- Measurement vs. external standards
- RNA external standard
- Measurement vs. internal standards
- A few spike-in standards

- Different than other agency submissions due to “youth” of technology
- Need to assure competency of experimenter
- Need to assure internal/external consistency
- Need to assure consistency with historical values

## Experiment is complex but has several points where critical evaluations needed

**286 steps needed to prepare microarray**
**Several quality control check points**
**Independent of platform **

- 

- 

- 

## Experimental design minimums

- Minimum of biological triplicate
- Minimum of three untreated or mock or vehicle treated controls, processed contemporaneously
- Minimum of three external standard RNA’s, processed contemporaneously
- Minimum three spike-in RNA’s  each sample

**Treated**

**Untreated -**
**Control**

**RNA standard **
**External Control**

**3 Spike RNA’s**

## RNA used in experiments

-  Ratio of 28S to 18S
- Mean
- Standard Deviation
- Range
- Traces for all RNA samples
**Data provided for each the following: **
- Samples in dataset
- Historical for similar tissue/cell prepared in lab
- RNA external standard, contemporaneous with dataset

## Hybridization and array Q.C. 

**Data for experimental samples**
- Array average signal to background ratio
- Array average background
- Array average raw signal
- Log dynamic range
- Average Raw signal intensity for each of three spike-in RNA’s
**Comparative data (Mean, SD and range) **
- Historical for **similar samples** (match for tissue or cell  type)
- Historical average for **RNA standard**
- Historical average for each of **spike-in RNA’s**
- Average for **contemporaneous RNA standard**
- Average for **contemporaneous controls**

## Experiment internal and external consistency

**Experimental samples, correlation coefficient (CC) with **
- each other
- contemporaneous controls
- contemporaneous external standard RNA
- historical external RNA standards
- historical similar tissue or cell line samples

**Mean, SD and range in all cases**

## Sponsor provided “Scientific literature style interpretation”

- Abstract
- Significance relative to application
- Brief methods
- Summary of quality evidence described earlier
- Results
- Discussion
- Conclusion relative the application under consideration
- Conclusions in context with other drugs and toxicants

- Report helps the agency help you

## Summary: What should be provided by sponsor to FDA

**Summary: What should be provided by sponsor to FDA**

**MIAME/MAGE-ML compliant descriptions of experiment(s) and electronic submission of all data**
**Minimum experimental design metrics similar to that required for any other biological experiment**
**“****Youth” of technology requires that additional quality data be submitted to demonstrate competency of experimenters**
**Sponsors interpretation of data in “scientific paper style” format**
**Comparison to community accepted RNA biomarkers and compared to Bench Mark Drugs and Toxicants **
- Interpretation vs. context of current & failed drugs and toxicants

## Internal FDA contextual database

**Micoarray technology requires stepping into the coming age of electronic data submissions**
**Paper submission of microarray data is not useful**
**Used by agency to develop a better understanding of technology**
**Used by the agency to allow a look at the data in the context of other submissions**
**Contextual database is highly useful to provide meaning and balance to interpretation**
- Same biology is present regardless of data preparation technology

## Once the data is available to the agency how should they evaluate?

**Objectives**
- Promote balanced view of the data
- React to truly  significant events
- Ground the analysis in the context of real world effects of drugs, failed drugs, toxicants and standards
**Need a reference database for these purposes**

## What is necessary to provide contextual grounding?   Database containing ...

- Wide diversity of successful drugs, failed relatives, toxicants and standards
- Needed to understand pharmacologic mechanism
- Needed to understand toxicity mechanism
- Multiple tissues with dose and time context
- Linkage of expression data to orthogonal data including:
- Pharmacology including on and off target sites of action
- Histology
- Clinical Chemistry
- Hematology
- Chemical Structure
- In vivo and in vitro data for successful bridging domains

## Benefits of using a reference database to the agency 

- Provides context to interpret events seen with candidates – An example drawn from DrugMatrix

| EGF-Receptor
 (p<0.01) | cKit-Oncogene
(p<0.01) | BCL2 Oncogene
(p<0.001) |
| --- | --- | --- |
| Meloxicam
Torsemide
Nislodipine
Rosigolitazone
Norethindrone | Grenisteron
Torsemide
Clotrimazole
Valproic Acid
Simvastatin
Beta-estradiol
Carboplatin | Busulfan
Clofibric Acid
Melatonin
Sparfloxacin
Withdrawn 
Diethylstilbesterol |

**Oncogenes elevated by approved drugs in liver**

**Context promotes balanced view of data**

## Summary: What should be provided by sponsor to FDA

**MIAME/MAGE-ML compliant descriptions of experiment(s) and electronic submission of all data**
**Minimum experimental design metrics similar to that required for any other biological experiment**
**“****Youth” of technology requires that additional quality data be submitted to demonstrate competency of experimenters**
- Additional quality data beyond that required of established technologies
**Sponsors interpretation of data in “scientific paper style” format**
**Comparison to community accepted RNA biomarkers and compared to Bench Mark Drugs and Toxicants **
- Interpretation vs. context of current drugs and toxicants

## Conclusions Looking forward  

- Micro Array technology is ready to contribute today
- Simple assurances of quality are needed
- Contextual databases to allow meaningful interpretation are available
- Developing understanding and consensus around meaningful RNA biomarkers
- Requirements beyond normal verification of data quality will diminish as community sophistication using technology improves
- Analysis of data collected off several platforms is quite possible
- Same biology is found regardless of the platform
- Clinical applications to accessible human tissues will become common

## Improved predictive power of animal studies making truly predictive animals models a reality

Addresses the Problem: Patients exposure to drugs which are subsequently withdrawn
Addresses the Problem: Only 1 in 10 IND’s become approved drugs  -- Our current methods of candidate characterization are only 10% accurate

**Addresses the Problem**: Patients exposure to drugs which are subsequently withdrawn
**Addresses the Problem:** Only 1 in 10 IND’s become approved drugs  -- Our current methods of candidate characterization are only 10% accurate

**Result**