BACKGROUND
qpcr is widely accepted as the "gold standard" for analysis of gene expression.
recent technological advances have greatly expanded the total number of genes that can be analyzed in a single assay; qpcr experiments now regularly analyze "moderate" numbers of genes, in the range of fifty to a few thousand  <cit> .
however, as the size of qpcr experiments has expanded, the need for effective data normalization techniques has become increasingly apparent.
normalization is the process of adjusting the relative expression measures between samples to compensate for various sources of variability in the assay and so to allow accurate comparisons of the results between different samples and conditions.
nearly all normalization methods are based on the assumption that one or more genes are constitutively expressed at near-constant levels under all experimental conditions and the expression levels of all genes in a sample are adjusted to satisfy that assumption.
the most widely used control genes are those selected from among an assumed set of "housekeeping" genes and typical selections include highly expressed transcripts such as glyceraldehyde-3-phosphate dehydrogenase , β-actin , and 18s ribosomal rna  <cit> .
in most qpcr experiments, a single housekeeping gene is chosen and added to the collection of experimental target genes to be assayed for each sample.
these control genes are then compared between samples, a sample-specific scaling factor is calculated to equalize their expression and applied to all genes in that sample.
this approach has numerous limitations, not least of which is that many of the experimental conditions may alter the expression of the control genes and evidence shows that housekeeping genes may not always perform optimally  <cit> .
more sophisticated normalization methods use multiple housekeeping genes where their respective measures are combined to represent a "virtual" housekeeping gene.
<cit> .
while this approach is more robust than single-gene methods, it too is based on potentially unfounded assumptions about which genes are stably expressed.
these genes still need to be pre-selected and incorporated into the experimental design without any apriori evidence supporting their use.
further, when working with limited quantities of rna, such as from patient samples, this reduces the number of interesting genes whose expression can be assayed.
improvements in qpcr technology have allowed significantly larger numbers of genes to be profiled simultaneously for each sample.
this allows not only more experimental genes to be tested, but also provides an opportunity for a larger number of control genes, spanning a wide range of expression levels, to be used.
however, these broad surveys also offer the possibility of introducing potential biases in normalization by using the data themselves to identify a set of appropriate controls.
in many ways, this parallels widely used normalization methods developed for dna microarray expression analysis, where data-driven methods have become the standard for most experimental designs.
here we present two normalization methods for high-throughput qpcr-based data adapted from those commonly applied in dna microarray analysis: rank-invariant set normalization and quantile normalization  <cit> .
as an example, we apply these methods to a high-throughput qpcr dataset from a time series experiment performed by the riken genome exploration research group to study the temporal transcriptional response of macrophage-like human cells to phorbol myristate acetate  exposure for a set of  <dig>  genes.
RESULTS
quantile normalization algorithm
quantile normalization is one of the most widely used methods in the analysis of microarray experiments  <cit> .
quantile normalization assumes that the on average, the distribution of gene transcript levels within the cell remains nearly constant across samples, so that if the expression of one gene increases, that of another decreases.
a quantile is a measure that lets us assess the degree of spread in a data set.
examples include percentiles, where the data are divided into  <dig> regular intervals, and quartiles which split data into quarters.
for these, the lower quartile represents the 25th percentile which means that 25% of the data is lower than that particular value.
quantile normalization generalizes that approach to an n-fold partition of the data, where n is the number of data points, and assumes that the data for individual samples have the same overall rank-order quantile distribution.
quantile normalization then adjusts the overall expression levels so that the distribution for all samples is equal.
in high-throughput qpcr experiments, the number of genes assayed in each sample can exceed the capacity of a single microtiter plate so assays for one sample are often distributed across multiple pcr plates.
consequently, in normalizing data, one must also consider plate-specific effects that may introduce a bias.
to correct for plate effects in raw qpcr data, one natural solution is to apply a quantile normalization approach where we make the assumption that the distribution of gene expression measures is the same across all plates for the same experimental condition.
this assumption will be reasonable when the gene allocation was based on factors that are unrelated to their expected expression levels; for example, genes are randomly assigned or the assignment is based on an alphabetical ordering by gene name.
by forcing the distribution from each plate to be equal we remove the variability associated with plate-specific effects in the data.
note that alternative solutions to this problem exist, namely one could take a linear model-based approach which incorporates a covariate term to explicitly account for plate-specific effects.
quantile normalization proceeds in two stages.
first, if samples are distributed across multiple plates, normalization is applied to all of the genes assayed for each sample to remove plate-to-plate effects by enforcing the same quantile distribution on each plate.
then, an overall quantile normalization is applied between samples, assuring that each sample has the same distribution of expression values as all of the other samples to be compared.
a similar approach using quantile normalization has been previously described in the context of microarray normalization  <cit> .
briefly, our method entails the following steps:
<dig>  qpcr data from a single rna sample are stored in a matrix m of dimension k  rows by p  columns.
plates with differing numbers of genes are made equivalent by padded plates with missing values to constrain m to a rectangular structure.
<dig>  each column is sorted into ascending order and stored in matrix m'.
the sorted columns correspond to the quantile distribution of each plate.
the missing values are placed at the end of each ordered column.
all calculations in quantile normalization are performed on non-missing values.
<dig>  the average quantile distribution is calculated by taking the average of each row in m'.
each column in m' is replaced by this average quantile distribution and rearranged to have the same ordering as the original row order in m. this gives the within-sample normalized data from one rna sample.
<dig>  steps analogous to  <dig> –  <dig> are repeated for each sample.
between-sample normalization is performed by storing the within-normalized data as a new matrix n of dimension k  rows by n  columns.
steps  <dig> and  <dig> are then applied to this matrix.
rank-invariant set normalization algorithm
the use of "housekeeping genes" in normalization is based on the assumption that one or more genes are expressed at near constant levels across all conditions assayed in a particular experiment.
however, dna microarray and high-throughput qpcr analyses have shown that many genes assumed to be constant in their expression can vary between conditions  <cit> .
rather than making a priori assumptions about which genes are expressed in such a manner, if one has a large enough dataset it is possible to identify invariant genes using the data itself.
this is the premise behind rank-invariant set normalization that was first described by tseng et al.
<cit>  for dna microarray data.
in its original implementation the algorithm selected genes that remained rank-invariant across two conditions, i.e.
when genes are ordered according to their expression levels, rank-invariant genes have the same rank order in both conditions.
here we describe an extension of this method for use on qpcr data with any number of experimental conditions or samples in which we identify a set of stably-expressed genes from within the measured expression data and then use these to adjust expression between samples.
briefly,
<dig>  qpcr data from all samples are stored in matrix r of dimension g  rows by s .
<dig>  we first select gene sets that are rank-invariant across a single sample compared to a common reference.
the reference may be chosen in a variety of ways, depending on the experimental design and aims of the experiment.
as described in tseng et al.
<cit> , the reference may be designated as a particular sample from the experiment , the average or median of all samples, or selecting the sample which is closest to the average or median of all samples.
genes are considered to be rank-invariant if they retain their ordering or rank with respect to expression across the experimental sample versus the common reference sample.
we collect sets of rank-invariant genes for all of the s pairwise comparisons, relative to a common reference.
we take the intersection of all s sets to obtain the final set of rank-invariant genes that is used for normalization.
<dig>  let αj represent the average expression value of the rank-invariant genes in sample j.  then represents the vector of rank-invariant average expression values for all conditions  <dig> to s.
<dig>  we calculate the scale factor βj for sample j where βj represents the ratio of the rank-invariant average expression value in the first sample versus sample j, i.e.
for j =  <dig> to s.
<dig>  finally, we normalize the raw data by multiplying each column j of r by the scale factor βj for j =  <dig> to s.
for our qpcr pma time series data, we identified five rank-invariant genes to be used for normalization .
glyceraldehyde-3-phosphate dehydrogenase ; enolase  <dig>   ; heat shock protein  <dig> kda alpha , class b member  <dig> , actb, eukaryotic translation elongation factor  <dig> alpha  <dig> .
of these, gapdh and actb are oft-used control genes, it is not surprising to find they did not have highly variable expression profiles in this experiment.
the identification of some of the other genes was more unexpected.
hspcb encodes a heat-shock protein hsp 90-beta, whilst eef1a <dig> is involved in translation.
the current version of the gene ontology used was released on 2008-01- <dig>
the go categories are listed for the five genes in the rank-invariant set that were identified as reasonable controls to be used for normalization of the pma dataset.
the presence of gapdh and actb as controls were not surprising for this dataset.
the inclusion of some of the other categories heat-shock protein activity and translation, were more surprising.
the advantage of the rank-invariant approach becomes very clear when you consider that with a standard normalization method, we would only normalize the raw data based on gapdh or actb expression.
however, not only does our approach validate the assumption of near constant expression of gapdh and actb for this experiment, it also provides other stable genes that can be used for a more robust normalization without the need for any a priori selection.
an example of single housekeeping gene normalization using gapdh
in order to compare our approaches with existing methods, we also performed normalization based on a single housekeeping gene, which for the pma data set, was gapdh.
gapdh was the only gene assigned to every plate and to take advantage of this design, we performed normalization in the following way:
<dig>  multiple gapdh expression measures are averaged within each sample.
let δj represent the average expression value of gapdh in sample j.  then represents the vector of average gapdh expression values for all conditions  <dig> to s.
<dig>  we calculate the scale factor λj for sample j where λj represents the ratio of the gapdh expression in the first sample versus sample j, i.e.
for j =  <dig> to s.
<dig>  finally, we normalize the raw data by multiplying the vector of data from sample j with the scale factor λj for j =  <dig> to s.
the approach adopted represents an alternative version from how housekeeping gene normalization is typically performed in qpcr experiments.
the standard approach taken is the delta-delta ct method which involves two subtractions, first between the housekeeping gene expression value  from the gene of interest measured in the control sample and second, the gene of interest measured in the experimental sample and the housekeeping gene in the experimental sample.
we took this approach to exploit the design which involved gapdh measurements being available for all plates in this data set.
in doing so, our implementation of the housekeeping gene more closely parallels the other two methods.
comparison of different normalization approaches
the pma time series experiment described previously provided an opportunity to assess the performance of these methods relative to the use of the single housekeeping gene approach.
the data was normalized using all three methods and the average gene-specific coefficient of variation  was calculated to assess the overall reduction in technical noise associated with each approach.
the cv measures the ratio of the standard deviation to the mean and captures the level of dispersion in the data.
therefore, a normalization method that better reduces technical noise will have a lower average cv.
the results are shown in figure  <dig>  as can be seen, quantile normalization produces the lowest average cv .
both the rank-invariant set method and the gapdh have very similar average cv values , while the cv for the non-normalized data was  <dig> %.
these results suggest that the quantile method has an advantage over the other normalization methods.
we also examined the variance of the raw and normalized data as a function of expression level.
let q <dig> and q <dig> represent two normalization algorithms .
we calculate the variance of each gene expression profile normalized under q <dig> and q <dig> and plot the log2-transformed ratio of these variances as a function of the average expression of each gene for all genes.
the red line represents a smoothed lowess curve that has been fitted to reflect the overall trend of the data  <cit> .
when the curve drops below y =  <dig>  we know that method q <dig> effects a greater reduction in the variation of the data relative to method q <dig>  similarly, when the red curve is above y =  <dig>  method q <dig> is more effective in reducing the variation.
if the data from both methods have similar variances then the red curve should remain at y =  <dig>  bolstad et al.
<cit>  originally used these plots for variance comparisons of different normalization methods for high density oligonucleotide array data.
comparisons of normalized to non-normalized data  also support considerable variance reductions associated with both data-driven approaches across the detectable range of qpcr.
when compared to the non-normalized data, the quantile method has reduced variance across the entire expression spectrum.
both gapdh and the rank-invariant set normalization had regions where these methods resulted in higher variation than the raw data.
these regions however correspond to very low expressed genes  and the data quality at this end of the spectrum is usually considered to be very poor.
these plots demonstrate that the quantile method is more effective in reducing the variances of genes with expression levels that span the detectable range of qpcr compared to the other normalization methods.
as an example of how our normalization strategies perform in action, we highlighted the effects the different normalization methods had on four individual genes: e2f transcription factor  <dig> , early growth response  <dig> , v-myb myeloblastosis viral oncogene homolog  , tumor necrosis factor, alpha-induced protein  <dig> .
these genes were selected as examples because they were typical genes of interest for pma-activated thp- <dig> cells.
comparing the variances of each profile  reveals that in all four cases, the lowest variances are associated with the data-driven approaches.
it is also interesting to note that for the e2f <dig>  egr <dig> and tnfaip <dig> expression profiles, the gapdh normalization method results in profiles with slightly higher variance than the non-normalized profile.
implementation
both quantile and rank-invariant set normalization algorithms for qpcr data are implemented as freely available, open source software using the statistical computing language r. we have adapted the quantile normalization algorithm that originally was included in the limma package  <cit> .
our software is distributed as an r package called qpcrnorm and a short tutorial outlining its use is available from  <cit>  .
the qpcrnorm package is also freely available from bioconductor  <cit> .
discussion
as high-throughput qpcr has become more widespread, it has become clear that we need more effective data methods to ensure the consistent acquisition of reliable, high quality results.
here, we present two data-driven normalization methods for qpcr that offer significant advantages.
rank-invariant normalization eliminates the need for possibly un-founded assumptions about which genes will not be differentially regulated in an experiment while relying on a generally accepted approach.
quantile normalization extends this by allowing for correction of plate effects.
both the rank-invariant set and single housekeeping gene approaches are examples based on scaling.
the raw expression values are transformed by an empirically-derived scale factor and consequently we see an overall reduction in the variability of the normalized data.
on the other hand, the quantile approach replaces the raw data with representative values derived from the average quantile data distribution.
as a result, we see the variability either preserved or increased in the normalized data.
these effects are noticeably obvious by comparing the tightness of the graphs in figure 3d versus 3e .
not only does quantile normalization tend to increase the variability in the distribution, we also notice an increase in the number of genes with more upper extreme ct values.
this is reflected by the tailing effect, as observed in figure 3c, where more genes have values that all fall close to the maximum ct value of  <dig> cycles.
looking at the empirical distributions , we see that the quantile normalized data has the heaviest right tail out of the three normalized data sets.
also of interest is the fact that the quantile normalization method appears to preserve the original distribution of the raw data.
our methods are specifically designed for high-throughput data sets where the number of genes or primer pairs used in the qpcr experiment is moderate to large .
we expect the robustness of both the rank-invariant gene method and the quantile method to break down when the number of genes or primer pairs decreases and drops below this threshold.
we were unable to assess this quantitatively however due to the lack of available data.
the quantile method first focuses on the data within a sample, and applies a normalization correction to ensure the quantile distribution from different plates for the sample have the same quantile distribution.
if there were a small number of genes or primer pairs assigned to each plate, then this normalization would be more susceptible to outliers.
it is assumed that when there is a large number of genes or primer pairs, that the corresponding quantile distribution will be reasonably smooth and cover a realistic range of values expected for qpcr data.
when the number of genes shrinks, the validity of this assumption may become questionable.
CONCLUSIONS
the data-driven normalization alternatives that we have presented have clear advantages when widely-used housekeeping genes are regulated by some experimental factor or condition.
vandesompele et al.
<cit>  use a panel of ten common housekeeping genes and advocate using at least three of these genes for normalization of a given experimental design.
however the appropriateness of this approach still hinges heavily on the assumption that any of these genes is not regulated in the experiment and consequently is suitable as a control.
in the presence of large amounts of data, our data-driven normalization methods represent a robust approach since no a priori assumptions are made regarding which genes might be used as controls and, in general, provides many more genes for normalization than vandesompele et al.
and colleagues suggest.
in the case of rank-invariant set normalization, genes which do satisfy the properties of being a good control gene are easily identified.
quantile normalization corrects for plate-specific effects in the data by requiring samples to have similar distributions, although it does not have the aesthetic advantage of identifying control genes.
overall, our analyses indicate that both methods outperform approaches using a priori sets of housekeeping genes and that quantile normalization gives the best overall performance.
although we used a time course experiment to test these normalization approaches, these methods are applicable to any high-throughput qpcr setup.

 qpcr is widely accepted as the "gold standard" for analysis of gene expression. recent technological advances have greatly expanded the total number of genes that can be analyzed in a single assay; qpcr experiments now regularly analyze "moderate" numbers of genes, in the range of fifty to a few thousand. however, as the size of qpcr experiments has expanded, the need for effective data normalization techniques has become increasingly apparent. normalization is the process of adjusting the relative expression measures between samples to compensate for various sources of variability in the assay and so to allow accurate comparisons of the results between different samples and conditions. nearly all normalization methods are based on the assumption that one or more genes are constitutively expressed at near-constant levels under all experimental conditions and the expression levels of all genes in a sample are adjusted to satisfy that assumption. the most widely used control genes are those selected from among an assumed set of "housekeeping" genes and typical selections include highly expressed transcripts such as glyceraldehyde-3-phosphate dehydrogenase, β-actin, and 18s ribosomal rna. in most qpcr experiments, a single housekeeping gene is chosen and added to the collection of experimental target genes to be assayed for each sample. these control genes are then compared between samples, a sample-specific scaling factor is calculated to equalize their expression and applied to all genes in that sample. this approach has numerous limitations, not least of which is that many of the experimental conditions may alter the expression of the control genes and evidence shows that housekeeping genes may not always perform optimally. more sophisticated normalization methods use multiple housekeeping genes where their respective measures are combined to represent a "virtual" housekeeping gene.. while this approach is more robust than single-gene methods, it too is based on potentially unfounded assumptions about which genes are stably expressed. these genes still need to be pre-selected and incorporated into the experimental design without any apriori evidence supporting their use. further, when working with limited quantities of rna, such as from patient samples, this reduces the number of interesting genes whose expression can be assayed. improvements in qpcr technology have allowed significantly larger numbers of genes to be profiled simultaneously for each sample. this allows not only more experimental genes to be tested, but also provides an opportunity for a larger number of control genes, spanning a wide range of expression levels, to be used. however, these broad surveys also offer the possibility of introducing potential biases in normalization by using the data themselves to identify a set of appropriate controls. in many ways, this parallels widely used normalization methods developed for dna microarray expression analysis, where data-driven methods have become the standard for most experimental designs. here we present two normalization methods for high-throughput qpcr-based data adapted from those commonly applied in dna microarray analysis: rank-invariant set normalization and quantile normalization. as an example, we apply these methods to a high-throughput qpcr dataset from a time series experiment performed by the riken genome exploration research group to study the temporal transcriptional response of macrophage-like human cells to phorbol myristate acetate exposure for a set of genes.  quantile normalization algorithm quantile normalization is one of the most widely used methods in the analysis of microarray experiments. quantile normalization assumes that the on average, the distribution of gene transcript levels within the cell remains nearly constant across samples, so that if the expression of one gene increases, that of another decreases. a quantile is a measure that lets us assess the degree of spread in a data set. examples include percentiles, where the data are divided into regular intervals, and quartiles which split data into quarters. for these, the lower quartile represents the 25th percentile which means that 25% of the data is lower than that particular value. quantile normalization generalizes that approach to an n-fold partition of the data, where n is the number of data points, and assumes that the data for individual samples have the same overall rank-order quantile distribution. quantile normalization then adjusts the overall expression levels so that the distribution for all samples is equal. in high-throughput qpcr experiments, the number of genes assayed in each sample can exceed the capacity of a single microtiter plate so assays for one sample are often distributed across multiple pcr plates. consequently, in normalizing data, one must also consider plate-specific effects that may introduce a bias. to correct for plate effects in raw qpcr data, one natural solution is to apply a quantile normalization approach where we make the assumption that the distribution of gene expression measures is the same across all plates for the same experimental condition. this assumption will be reasonable when the gene allocation was based on factors that are unrelated to their expected expression levels; for example, genes are randomly assigned or the assignment is based on an alphabetical ordering by gene name. by forcing the distribution from each plate to be equal we remove the variability associated with plate-specific effects in the data. note that alternative solutions to this problem exist, namely one could take a linear model-based approach which incorporates a covariate term to explicitly account for plate-specific effects. quantile normalization proceeds in two stages. first, if samples are distributed across multiple plates, normalization is applied to all of the genes assayed for each sample to remove plate-to-plate effects by enforcing the same quantile distribution on each plate. then, an overall quantile normalization is applied between samples, assuring that each sample has the same distribution of expression values as all of the other samples to be compared. a similar approach using quantile normalization has been previously described in the context of microarray normalization. briefly, our method entails the following steps: qpcr data from a single rna sample are stored in a matrix m of dimension k rows by p columns. plates with differing numbers of genes are made equivalent by padded plates with missing values to constrain m to a rectangular structure. each column is sorted into ascending order and stored in matrix m'. the sorted columns correspond to the quantile distribution of each plate. the missing values are placed at the end of each ordered column. all calculations in quantile normalization are performed on non-missing values. the average quantile distribution is calculated by taking the average of each row in m'. each column in m' is replaced by this average quantile distribution and rearranged to have the same ordering as the original row order in m. this gives the within-sample normalized data from one rna sample. steps analogous to – are repeated for each sample. between-sample normalization is performed by storing the within-normalized data as a new matrix n of dimension k rows by n columns. steps and are then applied to this matrix. rank-invariant set normalization algorithm the use of "housekeeping genes" in normalization is based on the assumption that one or more genes are expressed at near constant levels across all conditions assayed in a particular experiment. however, dna microarray and high-throughput qpcr analyses have shown that many genes assumed to be constant in their expression can vary between conditions. rather than making a priori assumptions about which genes are expressed in such a manner, if one has a large enough dataset it is possible to identify invariant genes using the data itself. this is the premise behind rank-invariant set normalization that was first described by tseng et al. for dna microarray data. in its original implementation the algorithm selected genes that remained rank-invariant across two conditions, i.e. when genes are ordered according to their expression levels, rank-invariant genes have the same rank order in both conditions. here we describe an extension of this method for use on qpcr data with any number of experimental conditions or samples in which we identify a set of stably-expressed genes from within the measured expression data and then use these to adjust expression between samples. briefly, qpcr data from all samples are stored in matrix r of dimension g rows by s. we first select gene sets that are rank-invariant across a single sample compared to a common reference. the reference may be chosen in a variety of ways, depending on the experimental design and aims of the experiment. as described in tseng et al., the reference may be designated as a particular sample from the experiment, the average or median of all samples, or selecting the sample which is closest to the average or median of all samples. genes are considered to be rank-invariant if they retain their ordering or rank with respect to expression across the experimental sample versus the common reference sample. we collect sets of rank-invariant genes for all of the s pairwise comparisons, relative to a common reference. we take the intersection of all s sets to obtain the final set of rank-invariant genes that is used for normalization. let αj represent the average expression value of the rank-invariant genes in sample j. then represents the vector of rank-invariant average expression values for all conditions to s. we calculate the scale factor βj for sample j where βj represents the ratio of the rank-invariant average expression value in the first sample versus sample j, i.e. for j = to s. finally, we normalize the raw data by multiplying each column j of r by the scale factor βj for j = to s. for our qpcr pma time series data, we identified five rank-invariant genes to be used for normalization. glyceraldehyde-3-phosphate dehydrogenase; enolase; heat shock protein kda alpha, class b member, actb, eukaryotic translation elongation factor alpha. of these, gapdh and actb are oft-used control genes, it is not surprising to find they did not have highly variable expression profiles in this experiment. the identification of some of the other genes was more unexpected. hspcb encodes a heat-shock protein hsp 90-beta, whilst eef1a is involved in translation. the current version of the gene ontology used was released on 2008-01- the go categories are listed for the five genes in the rank-invariant set that were identified as reasonable controls to be used for normalization of the pma dataset. the presence of gapdh and actb as controls were not surprising for this dataset. the inclusion of some of the other categories heat-shock protein activity and translation, were more surprising. the advantage of the rank-invariant approach becomes very clear when you consider that with a standard normalization method, we would only normalize the raw data based on gapdh or actb expression. however, not only does our approach validate the assumption of near constant expression of gapdh and actb for this experiment, it also provides other stable genes that can be used for a more robust normalization without the need for any a priori selection. an example of single housekeeping gene normalization using gapdh in order to compare our approaches with existing methods, we also performed normalization based on a single housekeeping gene, which for the pma data set, was gapdh. gapdh was the only gene assigned to every plate and to take advantage of this design, we performed normalization in the following way: multiple gapdh expression measures are averaged within each sample. let δj represent the average expression value of gapdh in sample j. then represents the vector of average gapdh expression values for all conditions to s. we calculate the scale factor λj for sample j where λj represents the ratio of the gapdh expression in the first sample versus sample j, i.e. for j = to s. finally, we normalize the raw data by multiplying the vector of data from sample j with the scale factor λj for j = to s. the approach adopted represents an alternative version from how housekeeping gene normalization is typically performed in qpcr experiments. the standard approach taken is the delta-delta ct method which involves two subtractions, first between the housekeeping gene expression value from the gene of interest measured in the control sample and second, the gene of interest measured in the experimental sample and the housekeeping gene in the experimental sample. we took this approach to exploit the design which involved gapdh measurements being available for all plates in this data set. in doing so, our implementation of the housekeeping gene more closely parallels the other two methods. comparison of different normalization approaches the pma time series experiment described previously provided an opportunity to assess the performance of these methods relative to the use of the single housekeeping gene approach. the data was normalized using all three methods and the average gene-specific coefficient of variation was calculated to assess the overall reduction in technical noise associated with each approach. the cv measures the ratio of the standard deviation to the mean and captures the level of dispersion in the data. therefore, a normalization method that better reduces technical noise will have a lower average cv. the results are shown in figure as can be seen, quantile normalization produces the lowest average cv. both the rank-invariant set method and the gapdh have very similar average cv values, while the cv for the non-normalized data was %. these results suggest that the quantile method has an advantage over the other normalization methods. we also examined the variance of the raw and normalized data as a function of expression level. let q and q represent two normalization algorithms. we calculate the variance of each gene expression profile normalized under q and q and plot the log2-transformed ratio of these variances as a function of the average expression of each gene for all genes. the red line represents a smoothed lowess curve that has been fitted to reflect the overall trend of the data. when the curve drops below y = we know that method q effects a greater reduction in the variation of the data relative to method q similarly, when the red curve is above y = method q is more effective in reducing the variation. if the data from both methods have similar variances then the red curve should remain at y = bolstad et al. originally used these plots for variance comparisons of different normalization methods for high density oligonucleotide array data. comparisons of normalized to non-normalized data also support considerable variance reductions associated with both data-driven approaches across the detectable range of qpcr. when compared to the non-normalized data, the quantile method has reduced variance across the entire expression spectrum. both gapdh and the rank-invariant set normalization had regions where these methods resulted in higher variation than the raw data. these regions however correspond to very low expressed genes and the data quality at this end of the spectrum is usually considered to be very poor. these plots demonstrate that the quantile method is more effective in reducing the variances of genes with expression levels that span the detectable range of qpcr compared to the other normalization methods. as an example of how our normalization strategies perform in action, we highlighted the effects the different normalization methods had on four individual genes: e2f transcription factor, early growth response, v-myb myeloblastosis viral oncogene homolog, tumor necrosis factor, alpha-induced protein. these genes were selected as examples because they were typical genes of interest for pma-activated thp- cells. comparing the variances of each profile reveals that in all four cases, the lowest variances are associated with the data-driven approaches. it is also interesting to note that for the e2f egr and tnfaip expression profiles, the gapdh normalization method results in profiles with slightly higher variance than the non-normalized profile. implementation both quantile and rank-invariant set normalization algorithms for qpcr data are implemented as freely available, open source software using the statistical computing language r. we have adapted the quantile normalization algorithm that originally was included in the limma package. our software is distributed as an r package called qpcrnorm and a short tutorial outlining its use is available from. the qpcrnorm package is also freely available from bioconductor. discussion as high-throughput qpcr has become more widespread, it has become clear that we need more effective data methods to ensure the consistent acquisition of reliable, high quality results. here, we present two data-driven normalization methods for qpcr that offer significant advantages. rank-invariant normalization eliminates the need for possibly un-founded assumptions about which genes will not be differentially regulated in an experiment while relying on a generally accepted approach. quantile normalization extends this by allowing for correction of plate effects. both the rank-invariant set and single housekeeping gene approaches are examples based on scaling. the raw expression values are transformed by an empirically-derived scale factor and consequently we see an overall reduction in the variability of the normalized data. on the other hand, the quantile approach replaces the raw data with representative values derived from the average quantile data distribution. as a result, we see the variability either preserved or increased in the normalized data. these effects are noticeably obvious by comparing the tightness of the graphs in figure 3d versus 3e. not only does quantile normalization tend to increase the variability in the distribution, we also notice an increase in the number of genes with more upper extreme ct values. this is reflected by the tailing effect, as observed in figure 3c, where more genes have values that all fall close to the maximum ct value of cycles. looking at the empirical distributions, we see that the quantile normalized data has the heaviest right tail out of the three normalized data sets. also of interest is the fact that the quantile normalization method appears to preserve the original distribution of the raw data. our methods are specifically designed for high-throughput data sets where the number of genes or primer pairs used in the qpcr experiment is moderate to large. we expect the robustness of both the rank-invariant gene method and the quantile method to break down when the number of genes or primer pairs decreases and drops below this threshold. we were unable to assess this quantitatively however due to the lack of available data. the quantile method first focuses on the data within a sample, and applies a normalization correction to ensure the quantile distribution from different plates for the sample have the same quantile distribution. if there were a small number of genes or primer pairs assigned to each plate, then this normalization would be more susceptible to outliers. it is assumed that when there is a large number of genes or primer pairs, that the corresponding quantile distribution will be reasonably smooth and cover a realistic range of values expected for qpcr data. when the number of genes shrinks, the validity of this assumption may become questionable.  the data-driven normalization alternatives that we have presented have clear advantages when widely-used housekeeping genes are regulated by some experimental factor or condition. vandesompele et al. use a panel of ten common housekeeping genes and advocate using at least three of these genes for normalization of a given experimental design. however the appropriateness of this approach still hinges heavily on the assumption that any of these genes is not regulated in the experiment and consequently is suitable as a control. in the presence of large amounts of data, our data-driven normalization methods represent a robust approach since no a priori assumptions are made regarding which genes might be used as controls and, in general, provides many more genes for normalization than vandesompele et al. and colleagues suggest. in the case of rank-invariant set normalization, genes which do satisfy the properties of being a good control gene are easily identified. quantile normalization corrects for plate-specific effects in the data by requiring samples to have similar distributions, although it does not have the aesthetic advantage of identifying control genes. overall, our analyses indicate that both methods outperform approaches using a priori sets of housekeeping genes and that quantile normalization gives the best overall performance. although we used a time course experiment to test these normalization approaches, these methods are applicable to any high-throughput qpcr setup. 
we present and evaluate two data-driven normalization methods that directly correct for technical variation and represent robust alternatives to standard housekeeping gene-based approaches. the utility of the approaches that we describe can be demonstrated most clearly in situations where standard housekeeping genes are regulated by some experimental condition. we evaluated the performance of these methods against a single gene housekeeping gene method and our results suggest that quantile normalization performs best.
 high-throughput real-time quantitative reverse transcriptase polymerase chain reaction is a widely used technique in experiments where expression patterns of genes are to be profiled. current stage technology allows the acquisition of profiles for a moderate number of genes, and this number continues to grow. the use of appropriate normalization algorithms for qpcr-based data is therefore a highly important aspect of the data preprocessing pipeline.  we present and evaluate two data-driven normalization methods that directly correct for technical variation and represent robust alternatives to standard housekeeping gene-based approaches. we evaluated the performance of these methods against a single gene housekeeping gene method and our results suggest that quantile normalization performs best. these methods are implemented in freely-available software as an r package qpcrnorm distributed through the bioconductor project.  the utility of the approaches that we describe can be demonstrated most clearly in situations where standard housekeeping genes are regulated by some experimental condition. for large qpcr-based data sets, our approaches represent robust, data-driven strategies for normalization. 