# Presentation: 526748

## Metrology and Standards Needs for Gene Expression Technologies

- Krishna Ghosh
- Agilent Technologies
- June 10, 2003

## Overview of Presentation

- History of Gene Expression Standards Development
- Microarray/Scanner Fluorescent Standards
- Status update
- Next Steps
- Universal RNA Standards
- Status update
- Next Steps
- NIST Feed back-Next Steps

- June 10, 2003

## Development of Standards for Gene Expression

- Kickoff meeting (NIST, Oct  2002)
- Workshops identified: Microarray /Scanner & Universal RNA
- Collaborative model development and workgroups
- Follow up meeting (NIST, Dec 2002)
- Technical overview of microarray readers and their performance
- Monthly teleconferences of working group to develop/detail artifact(s)
- Universal RNA Standards Workshop (Stanford, March 2003)
- Microarray Fluorescence Standards Working  Group Meeting, (NIST, May 2003)

- June 10, 2003

## NIST Working Group Microarray/Scanner Fluorescence Standards

- _Working Group Objectives_:
- Develop an artifact to measure microarray reader performance using a standardized method. The artifact(s) will be used primarily by manufacturers to standardize the measurement of specifications.
- Identify the appropriate fluorescent material(s) and manufacturing technology to produce a NIST-certified artifact
- Provide the required procedures and tools required to analyze the artifact
- _Accomplishments to Date_:
- Assembled representatives from microarray reader manufacturers (Affymetrix, Agilent, Axon, Perkin Elmer, Biorad, Arrayit and others)
- Created a working draft of the artifact’s features and characteristics
- Reviewed different options for fluorescent materials and manufacturing technologies (organic and inorganic dyes, metal oxides, nanocrystal composites, polymer coating, fluorescent glasses, Sol-gel)

- June 10, 2003

## Artifacts to Measure Scanner Performance Using a Standardized Method*

-    _**Uniformity Artifact:**_  Measures scanner’s uniformity and Signal / Noise for bright features.
-   _**Detection Limit Artifact:**_  Measures scanner’s limit of detection.
-   Artifacts will be manufactured by someone other than NIST, BUT will be qualified by NIST.
-   Artifact(s) intended for use primarily by manufacturers to standardize the measurement of specifications.

- June 10, 2003

## Preliminary Scanner Specification Decisions

- Artifacts will be uniformly coated (except for fiducials and background regions).
- There will be at least two artifacts per dye; one in the middle of the dynamic range meant for uniformity measurement, the other for measuring signal/noise for very dim signals approaching the detection limit.
- The dimmer slide should approximate about 0.5 chromophores per square micron of fluorescent material.
- The glass non-flatness should not exceed +-10 microns.
- The parallelism will be <1 mrad
- The preferred substrate material is glass.
- The artifact will be available as 1.00 mm thick.
- The artifact will be 1x3 inches and can be fitted to other users.
- The dye choice should match Cy3/Cy5 as closely as possible.
- Photostable and environmentally stable

- June 10, 2003

## Coating Requirements*

- Transparent coating on high purity, low background emission slide glass.
- Mimic Cy-3 and Cy-5 dye properties;
- Cy-3 Excitation at 532 nm (frequency doubled Nd-YAG laser)
- Cy-3 Emission at ~562 nm.
- Cy-5 Excitation at 633 nm (Helium-Neon laser)
- Cy-5 Emission at ~660 nm
- Long-Term Stability
- Intensity uniformity over coated area (~1% CV)

- * From presentation of E.Pope

- June 10, 2003

## Present NIST Artifact Slide Layout*

**Present NIST Artifact Slide Layout***

- * From presentation of J.Corson

- June 10, 2003

## Summary of Workshop Presentations14th May 2003

- Metal oxide glasses are less prone to photobleaching than organic dyes and have promising spectral characteristics
- There are a variety of ways to prepare fluorescent standards to meet the Artifact Slide requirements(Matech)
- Fluorescent microspheres offer photostability and compatibility with different attachment chemistries(Molecular Probes)
- Nanocrystals (semi-conductor materials) can mimic emission wavelengths of fluorescent dyes while using common laser and broadband sources (Evident Technologies)

- June 10, 2003

## Microarray/Scanner Fluorescence Standards- Next Steps -

- Select fluorescent material for the artifact
- Silica doped with metal oxides
-  Q –dots/Nanocrystals
- Alternate organic photostable fluorescent dye
- Assemble manufacturers capable of making specified artifact with uniform coating expertise
- Define recommended use and data analysis procedures
- Refine artifact specifications depending on the limitations

- June 10, 2003

## Universal RNA Standards WorkshopStanford University, March 28-29, 2003

- June 10, 2003

## Goals of Universal RNA Standards Workshop

**Educational:** provide participants a forum to share various methods and techniques that are relevant to defining a standard for Gene Expression and RT-PCR technologies
**Awareness:** determine areas of agreement and disagreement on issues of different standards, and where there is a need for additional information
**Guidance:** help define** **how NIST could best help to develop the RNA standard(s) and promote its use
**Requirements:** a reliable, reproducible and manufacturable standard to support the use of gene expression results for IVD and NDA filings (proficiency testing, comparison of submitted data with published data, comparison of different gene expression platforms), AND can be used by manufacturers to certify their products

- June 10, 2003

## Gene Expression Workflow- Standardization  Requirements

- June 10, 2003

## Generalized Workflow

- How many standards are needed to address the workflow differences due to Instrumentation, Reagents, Sample, Operator and Analysis?

- June 10, 2003

## Session 1Standardization of Biological Component of RNA Based Molecular Assays

- _Focus_:  Review the needs for gene expression measurement standards in support of: Safety and efficacy claims of therapeutic products, and Human clinical in vitro diagnostics.
- _Session Summary_:
- Standards can be used for proficiency testing, benchmarking, cross-platform comparisons, inter-laboratory comparisons
- Significant sources of variation are: Array Platform, Lab and Array-to-Array etc..
- Standards, analysis tools & metrics, and training are needed to help assure that platform and sample processor are capable of detecting the “biological truth”
- Reference methods and reference materials (traceable, assignable) are key to defining analytical and clinical performance characteristics
- Consideration should be given to “bioinformational standards” with approved gene lists and indicator patterns; used as training sets for decision rules

- June 10, 2003

## Design for Experiment 

- 1.    Tissue extraction: liver vs. 5-tissue pool from 22 C57 black, adult male mice (NIEHS NTP/OHSU)
- 2.	Standard RNAs: liver, 5-tissue pool (NIEHS)
- Chips
- 		Standard = 18K mouse oligo (Duke)
- 		Resident = cDNA, oligo, Affymetrix, Agilent
- 4. 	Hybridizations (4): liver vs. liver; liver vs. pooled with fluor dye flips
- 5. 	Data quality: Arabidopsis10-gene set in standard RNAs and on chips *(Wang et al, 2002)*
- 6.	OHSU Data Warehouse: Web hosting and data sharing, MIAME sheet for experimental details, gene annotations, resident analysis and stats tools

- June 10, 2003

## Sources of Variation – Preliminary Trends*

- * From presentation of B.Weiss

- June 10, 2003

## FDA Funded collaborative Project for Evaluation of Performance Standards for Toxicogenomic StudiesUsing benchmark genes within mixed tissue samples

- Identify tissue-selective, low variance rat genes from _control_ animal data in large databases (populated using a consistent protocol).
- Select tissues with large difference in number of tissue-selective genes  from control animal data
- Model a pilot set of tissue mixtures for the standard using database info and test on arrays
- Identify probe sets corresponding to benchmark genes on different platforms
- Evaluate the added value of exogenous spike-in standards (platform-dependent).

**Hsiao* *et al*.,  *Physiol. Genomics 7: 97-104, 2001*

- June 10, 2003

## Expected Initial Outcomes

- Identification of probes that can perform similarly across platforms
- Determine normal range of false positive/false negative rates for MTS
- Determine normal range of lab-to-lab variance for MTS
- Determine normal range of cross-platform variance for MTS
- Publication of findings.

- June 10, 2003

## Session 2Metrics for Universal Standard:Expression Arrays

- _Focus_:  What standard parameters/metrics are needed for existing expression array technologies to ensure both intra-laboratory and inter-laboratory comparison? What are the key functions and features of the standard?
- _Session Summary_:
- RNA sample quality can impact usefulness of microarray results (
- Not all RNA is created equal. An RNA quality index, taking into account a number of physical metrics, can be formulated
- RNA spikes can be used to assess and limit variability for this reasonably complex procedure
- Transcript pooling offers an organism specific approach to assessing sensitivity, specificity and reproducibility

- June 10, 2003

## Session 3Appropriate Standards to Meet Metrics

- _Focus_: What are currently implemented intra-laboratory controls and their effectiveness? Are these applicable to inter-laboratory comparisons and cross-platform comparisons?
- _Session Summary_:
- Depending on its purpose, more than one RNA standard is probably required, and will evolve with time
- “Begin with the end in mind”; what are the design specifications to meet the user requirements?
- Complex synthetic sample offers a sample of intermediate manufacturing difficulty and intermediate customer relevance versus oligo-only and complex natural samples
- Universal reference RNA’s are a commercially available blend of total RNA isolated from multiple cell lines designed to maximize gene expression profiling

- June 10, 2003

## Session 4 Metrics for Universal Standard:Quantitative RT-PCR

- _Focus_: What are the key functions and features of standards for RT-PCR? What are currently implemented intra-laboratory controls and their effectiveness?
- _Session Summary_:
- Emerging genetic tests will be evaluated for analytical validity, clinical validity, clinical utility and ethical, legal and social implications
- Internal quantitative standards (e.g., HSK) can demonstrate the impact of sample collection and processing methods on data quality
- Internal calibrators or standard curves are required for accurate absolute quantification; relative quantification is possible with endogenous controls
- Human universal reference RNA can be used for QRT-PCR
- QRT-PCR has applied normalization to control variation in total RNA mass and amplification efficiency, and ratio-ing to “verified” invariant genes

- June 10, 2003

## Themes from the Universal RNA Standard Workshop

**Multiple sources of data variability**
- Different laboratories, platforms, sample types, extraction methods, etc.
- Evolving technologies
- Probe/primer design
**Difficulties sharing data**
- MIAME is a start
- Annotation problems abound
- Is the same analyte being measured?
**Standard methods and metrics for proficiency testing**
- Terminology and definitions are needed
- Differences between analytical and clinical applications

- June 10, 2003

## All RNA is Not Created Equal

- June 10, 2003

## RNA Quality Index

- Sample isolated with minimal degradation
- RNA extracted using the same method
- RNA is proven stable during study
- RNA has a 28S:18S ratio above 1.5
- Free of DNA for Real-time validation

**The procedures are available but not all tools**
** ****are easily used or in simple kit formats.**

- June 10, 2003

## A Good Standard Should*

- Allow performance validation of any _***single platform over***__* *__***time***_***.***
- Facilitate _***comparison between various platforms***_ used to assay gene expression.
- Be constructed in such a manner as to assure _***consistency over***__* *__***time***_***.***
- Include a _***well defined protocol***_ describing how it is made and validated.
- Include two or more samples that allow one to _***make both***__* *__***absolute and relative measurements***_ of the abundance of individual transcripts.
- Not be limited to hybridization-based approaches, but should be _***amenable to use with other assays***_ such as QRT-PCR

- June 10, 2003

## Standards to evaluate platform (and laboratory) performance

- Standard should be relatively ***invariant*** and ***regenerable.***
- Standard should be formed on ***probes**** ****in common between platforms. ***
- Standard should ***resemble test samples*** (e.g., rat tissue standard for toxicogenomics) and also ***query a wide range of**** ****features*** on the arrays
- A set of ***mixed tissue standards*** that have varying ratios of components would contain real and quantifiable gene expression changes between samples and use endpoints measurable on most platforms

- June 10, 2003

## Microarray Performance Characteristics and Controls*

- Characterization of Array
***Design and fabrication****,* e.g. platform type, surface type, composition and spatial layout, number of elements (spot), number of replicates, etc.
***Spot elements****,* e.g.  clone, sequence, PCR  primer pairs, probe length, gene name, etc.
***Built-in controls****,* e.g. housekeeping genes, etc.
- Microarray Controls
- Internal controls (housekeeping genes, synthetic RNA)
- Pooled RNA from cell lines
- Pooled RNA from test samples
- RNA and oligonucleotides from plants and bacteria

- June 10, 2003

## Session 5Proposed Workshop Recommendations

- Standards are required for several purposes in gene expression RNA analysis regardless of whether quantitative RT-PCR or microarray gene expression technologies are employed.
- Periodic laboratory proficiency testing
- Platform performance validation and baseline monitoring
- Cross-platform performance validations
- Inter-laboratory performance validation
- Providing a reference point for regulatory agencies evaluating gene expression profiling data
- A consistent definition of terminology is needed
- The consensus of the attendees was to develop two types of RNA standards, External Synthetic RNA Standard Reference Material & Internal RNA Reference Standard

- June 10, 2003

## Session 5Proposed Workshop Recommendations

- External Synthetic RNA Standard Reference Material
- Moderately complex pool of highly characterized synthetic mRNA targets
- Used across all RNA gene expression profiling platforms, including both RT-PCR based and array based methods.
- Measures the accuracy, dynamic range, sensitivity and specificity of each platform under any given set of conditions.
- Internal RNA Reference Standard
- Spike-in RNA standards used in conjunction with standard probe sequences
- Compatible with all array formats, and available to labs making “in-house” arrays
- Provides an internal measure of the quality of any particular array experiment (sensitivity, dynamic range, and specificity)
- Array manufacturers could also use this material for quality control testing their products
- Reference Method
- Instructions on the correct use of the reference materials are required

- June 10, 2003

## External Synthetic RNA Standard Reference Material

- Modular components
- Individually characterized and pooled
- Extendable and upgradable
- Number of human sequences = 96
- Maximum of 2000 bases beginning with 3’ end
- Selected component characteristics
- Conserved, minimally polymorphic sequence
- Well characterized tissue specificity, splice variants, gene family members, etc.
- Cloned into expression vector, sequenced, high purity and stability
- Represents 106 fold range of absolute quantitative expression

- June 10, 2003

## Internal RNA Reference Standard

- “Spike-in” standard to array probes serves as internal positive control of array performance
- One set of 12 sequences
- Target sequence > 600 nucleotides
- Sequence content acceptable to different array platforms
- Cloned into expression vector, sequenced, high purity
- Covers dynamic range of the platform
- All array platforms have optimized probes with 5-10X redundancy across the array
- Instructions on correct use of spike-in standard to be provided (standard reference method)

- June 10, 2003

## Open Questions and Next steps

- NIST Guidance document published by end of June 2003
- Formation of working group members
- Will NIST take up this project with limited resources and budget?
- Will FDA make a formal request to NIST for specific standards to facilitate acceptance of Tox /Pharmacogenetic data submissions?
- Should the platform manufacturers develop their own standards to establish platform performances?

- June 10, 2003