BACKGROUND
real-time quantitative pcr  represents the current state-of-the-art approach for measuring gene expression; and the method has numerous applications in both biology and biomedicine.
although qpcr is a robust technique results can vary depending on factors such as rna integrity, reverse transcriptase  efficiencies, sample-to-sample variations in amplification efficiency, and variation in cdna sample loading.
using equal sample sizes, assessing rna integrity and equalizing rna concentrations prior to rt are fundamental normalization steps in qpcr  <cit> .
still, normalization to some internal control is essential for accurate qpcr in order to balance sample-to-sample variations within the rt and pcr reactions.
currently, the preferred internal control is achieved by using reference genes  or better a normalization factor based on several reference genes calculated using e.g.
genorm  <cit> .
however, the use of reference genes suffer from a circular argument; i.e.
we normalize target gene expression data to exclude the systematic variation by the means of reference gene expression data obtained by the same method as the data that need normalization.
thus, an assumption for using reference genes is that they are unaffected by the experimental treatment and measured accurately and without error, as we rely on the target gene data to be correctly normalized by the reference gene.
the circularity is partly evened out by the normalization factor approach, based on the expression of several reference genes, following the assumption that the distribution of three or more reference genes is more accurately estimating systematic error than the distribution of only one gene  <cit> .
however, in many studies reference genes are chosen more or less randomly and are not always being validated for the particular experimental conditions.
a further downside is that it can be difficult to find suitable reference genes for certain experimental conditions that affect gene expression broadly  <cit> .
searching for and validating reference genes is thus both time and money consuming and might not always be practical or successful.
heterogeneous samples, however, require a robust normalization strategy.
yet, conventional normalization may introduce unintentional random changes to the variance and mean expression of target genes in lack of good reference genes.
this may cause invalid conclusions, and prevent good target gene data-sets from being accurately analyzed increasing the risk of making type i and ii statistical errors.
thus, the use of reference genes has become the chosen method not because it is extremely good, but because it represents the best available option.
here we present an algorithm, norma-gene, which is applicable to all biological and biomedical studies, especially those that are based on a limited number of genes measured with qpcr.
the method is based on a data-driven normalization of target genes and is valid for as little as five target genes comprising the data-set.
it does not require the use of reference genes allowing researchers to focus their efforts on studying target genes of biological relevance.
methods
we have defined two levels of variance in the qpcr data-sets.
the first level refers to among replicate variation .
this variation includes biological variation and variation in rna extraction and reverse transcription  efficiency.
the second level of variance refers to the variation among the measured genes within a single replicate .
this variance reflects the technical and random variation in the qpcr part of the procedure.
norma-gene reduces systematic and artificial between-replicate bias utilizing the entire data-set of the target genes being studied.
the approach is based on calculating mean expression values for each replicate across the studied target genes and subsequently estimating a normalization factor that estimates and reduces the systematic bias of a replicate across all genes.
for this a least square method is applied to secure the best possible minimization of variability in the data-set based on the bias between replicates within treatment.
least square regression is non-robust to outliers and careful quality control measurement throughout the qpcr process  is thus essential pre-normalization.
the procedure allows the estimation of the remaining variance in the data-set, not explained by experimental bias between the replicates, and the calculation of the precision  of the identified bias.
the procedure is only affected by the number of replicates and genes within a treatment; however, for more than three replicates the number of genes is predominantly of importance for the precision of the normalization.
the procedure is not affected by up-or down-regulation as no between treatment relations are being used.
detailed derivation of the following equations is available in additional file  <dig>  appendix a.
the mathematical theory behind norma-gene
the norma-gene algorithm for a single treatment is shown below and in more detail in additional file  <dig>  appendix a. within each treatment of the non-normalized data-set, n genes are measured for m replicates.
the normalization will take place using log-transformed data.
the normalization factor for each replicate within a treatment is calculated as:
where aj is the bias coefficient for replicate j, i is the index for the genes and nj is the number of genes that are recorded for replicate j,  is the measured gene expression value for sample j and gene i, and  is the estimated mean value for log-transformed gene i data
the standard deviation of aj is found as a global standard deviation within the same treatment assuming that all aj are following the same probability distribution function.
a first order uncertainty assessment for the estimation of aj is given as:
where σlogx is the standard deviation of aj assuming variance homogeneity which is similar to assuming the same relative  error for the data:
eq.
<dig> will tend to take a simple approximately form: aj will tend to be close to unity otherwise the data errors are very large and mi will normally be more than  <dig> and thus the product of nj ·mi will tend to be much larger than nj and thus the ratio  will dominate in eq.
<dig> .
if there are not very many missing data then nj will tend to be constant and equal n and if all these statements are correct then eq.
<dig> can be reduced to:
data handling
target gene normalization has been simplified by automating all calculations in an excel workbook  entitled norma-gene  replicate when relying on reference gene normalization.
norma-gene is very flexible and little affected by missing data, and is able to normalize samples in treatments with missing data.
the calculated norma-gene normalization factor is valid for and can be used on all genes in a corresponding replicate, as the factor is replicate-specific.
the norma-gene normalization improves gradually with the number of genes that are included in the target gene data-set .
thus, as long as a minimum number of data points  is available within a replicate  normalization by norma-gene can be performed.
it is not required that the same five data points are available in all replicates within a treatment, and as such norma-gene can proceed even with quite extensive missing data.
benchmarking norma-gene on artificial data-sets
as the true values of real data-sets are always unknown it can be difficult to evaluate normalization procedures.
thus, we generated artificial data-sets to evaluate the performance of reference gene and norma-gene normalization.
each data-set was comprised of a single treatment with four replicates, with one reference factor  and eight target genes measured in each replicate.
the artificial data-sets were generated in two steps, each with independent variation.
first, we sampled four replicates from a treatment with a true mean of zero and an experiment dependent bias .
the second step was to generate the variation .
the targets genes were in all cases sampled with a fixed variation of 10% while the reference factor was sampled with different levels of variation.
by comparing normalized data to the known true means we were able to evaluate and validate the performance of the norma-gene normalization relative to the reference gene method at different bias-to-variation ratios and at different variation in reference gene-to-variation in target gene ratios.
note that the variation referred to here does not regard the among treatment variation in the reference gene or normalization factor as is usually the concern of this type of normalization.
in the artificial data we assumed no between treatment effects and the reference gene or normalization factor is thus assumed to reflect the optimal situation with no variation among treatments.
rather the sampled variation of both reference and target genes only reflect the precision by which the signal  is measured.
each set of experimental conditions were re-sampled  <dig> times.
data analysis
differences in the efficiency of reducing variances by the two normalization methods  v norma-gene) were tested with wilcoxon's signed-ranked test for two groups of paired observations  <cit> .
RESULTS
the results of the analysis of artificial data-sets are shown in figure  <dig>  to evaluate the performance of the norma-gene normalization on the artificial data we tested; i) the variance reduction at different bias-to-variation ratios using norma-gene.
further, to test the effects of normalization methods we compared performance on two sets of parameter combinations in artificial data; ii) the bias-to-variation ratio, and iii) the ratio of reference-gene variation relative to target-gene variation.
in these two latter tests the quality of the normalization was judged by the deviation of expression level in normalized data from the true expression values.
increasing the bias-to-variation ratio in the artificial data leads to more variation being removed from the data-set by norma-gene .
this is not surprising as norma-gene is designed to identify and reduce bias, and as bias increases as a larger proportion of the total variance is being removed.
the relationship was little affected by number of replicates  and only slightly by number of genes .
the result of the second test showed a superior performance of the norma-gene method with a larger degree of samples closer to the true mean than when reference gene normalized under a large range of conditions .
the bias  to variation  ratio affected the relative performance of the norma-gene normalization method.
with increasing bias-to-variation ratio, the performance of norma-gene decreased until the two methods were equal with the bias being  <dig> times larger than the variation; and the reference factor method was only superior at bias-variation ratios larger than  <dig>  furthermore, re-sampling the data-set with only four genes did not affect the performance of the norma-gene methods confirming the stability of the method even with few genes measured .
as the bias-to-variation ratio affects the effectiveness of norma-gene compared to reference gene normalization it is highly relevant to estimate this ratio in real data-sets before using the norma-gene procedure.
however, the bias-to-variation ratio in real data-sets is difficult to estimate.
one way is to use the relationship between bias-to-variation ratio and average variance reduction as seen in figure  <dig>  top panel.
interestingly, the average variance reduction in the three real data-sets  suggests that the bias-to-variation in these data is between  <dig>  and  <dig>  and thus in a range where norma-gene is performing better than reference gene normalization.
the third and final test on artificial data concerned the level of sample variation when sampling a reference gene/reference factor.
to remind, this level of variance is the sampling variation among genes from the same sample  and not among treatment variance.
thus, for the validation of the method using artificial data we have assumed the perfect reference gene, i.e.
no variation among treatments .
this is the optimal situation and quite unrealistic to achieve in many studies as argued in the text.
the variation we allowed to vary is the technically variation, which determines how well the qpcr procedure measures the true amount of cdna in the sample.
the realistic value in this case most be  <dig> , as reference genes and target genes are measured in the same samples having gone through the same extraction and cdna synthesis procedure, unless more replicates is used for reference genes, or more care is taken by the experimenter while measuring these genes compared to when measuring target genes.
thus, by applying "perfect" reference genes and allowing these to be measured more precisely than target genes we are being very conservative in the test of the norma-gene procedure.
here we found, as expected, that increasing the precision of the reference gene/reference factor increased the performance of this normalization method compared to norma-gene .
the reference gene/reference factor approach needs to be measured four times as precise as the target genes to result in a normalization efficiency equal to the norma-gene method.
as discussed above this will be difficult to achieve using a single reference gene, but possible using a normalization factor based on multiple genes.
note that any effect of treatment on reference genes will add to the level of noise and decrease the precision of reference factor normalization; rendering this situation of four times precision yielding equal normalization a conservative estimate.
in experiments using harsh treatments, e.g.
stress research, those reference genes, totally unaffected by treatment, might prove very difficult to identify.
validation on real data-sets
we applied the norma-gene algorithm to three different data-sets : i) a data-set on springtails with six target and one valid reference gene  <cit> ; ii) a data-set on earthworms with nine target and one valid reference gene ; and iii) a data-set on daphnia with  <dig> target and a genorm based normalization factor using three valid reference genes  <cit> .
in all three data-sets the reported results are for target genes only, but reference genes were included in the norma-gene algorithm for data-set i and ii, to improve the stability of the normalization .
the norma-gene algorithm uses a least square method to secure the best possible minimization of variability in the data-set due to bias between replicates .
the algorithm predicts the relative reduction in variation as a function of the number of genes comprising the data-set.
norma-gene generates a stable normalization with as little as five genes, and addition of further genes to the data-set has little relative effect on the normalization output .
the predicted effect of the number of genes on the normalization output was confirmed in all three data-sets  verifying that using a data-set of five or more genes is a conservative approach for obtaining a stable normalization.
validation of the normalization efficiency was achieved by looking at the distribution of standard deviations for each sample in the raw data after norma-gene normalization, and after normalization to a reference gene  or a normalization factor based on three reference genes .
we plotted the variance reducing effect of reference gene or norma-gene normalization on variances relative to the variance of raw data .
the variance reduction was significantly larger following norma-gene normalization for all three data-sets ; thus significantly reducing the variance compared to the reference gene based normalization.
the relative effect of the different normalization approaches on mean expression values revealed that traditional reference gene normalization affects mean expression values to a greater extend than norma-gene normalized data .
a comparable effect on mean was seen in the artificial data when considering a bias-to-variation ratio of real data-sets of between  <dig>  and  <dig>  .
the mathematics behind norma-gene ensure that mean expression values within treatment are affected minimally by normalization, and thus deviate as little as possible from the raw expression values .
discussion
within microarray analysis a data-driven normalization, e.g.
global lowess, has long been applied  <cit> , but these methodologies rely on very large data-sets in order to deliver meaningful outputs.
recently, two qpcr-based methods using data-driven normalization have been published providing proof of principle.
the first describes a microarray inspired data analysis approach where quantiles are applied to normalize the target genes; yet the approach is only applicable to high-throughput qpcr analysis involving  <dig> to several thousand genes  <cit> .
the other method is developed for normalization of micrornas quantified using qpcr  <cit>  and is according to the authors only applicable to large-scale gene expression studies.
however, no data-driven normalization alternative is currently available for small-scale mrna expression studies using qpcr.
we show here that our method produces normalized data which are closer to the true means under a range of realistic variance parameters.
furthermore, we show that norma-gene produces reduced experimental bias  to a higher extend and thus outperforms the current approach based on reference genes; even when good reference genes are available as in the case of data-set iii  <cit> .
we furthermore show that a reliable normalization may be obtained with as little as a five genes comprising the data-set and verify this on the artificial data-sets.
the results of normalization and the distributions of the normalized data show high comparability between artificial and real data suggesting that the artificial data are behaving in a similar manner to real data and thus that the results of artificial data can be taken as relevant and meaningful for the validation of the method.
norma-gene is designed to reduce within treatment variation only and thus has little effect on the mean expression values between treatments.
contrary, reference gene normalization can have a large effect on the estimated mean expression values across treatments as this approach affects both within and between treatment variation.
this is not a problem if the reference gene have been measured correctly, and thus adjust the raw data accurately.
however, our analysis of the artificial data-sets show that a four-fold reduction in variation in the used reference factor is needed to obtain similarly good results with reference genes.
thus, reference genes, being measured with the same technique as target genes, will realistically be measured with similar error as target genes across treatments; or are affected by the treatment .
normalization factors based on more than one reference genes, i.e.
using the genorm procedure, reduces this problem but does not eliminate it as seen for our real data-set iii.
this will introduce variation across treatments, which was not present in the original data.
furthermore, biological variation may distort the "true" mean expression values when non-validated reference genes are being used for normalization.
in our opinion, the measured raw data provide the best estimate of mean expression values available; and only minor changes to these means following normalization are to be expected as achieved by norma-gene normalization.
norma-gene normalization is virtually non-affected by the above-mentioned issues regarding reference gene related effects on mean expression values.
however, norma-gene requires using a block design to minimize variation across treatments.
if the experimental setup is based on a block design and any handling  of the samples has been conducted block-wise, between treatments variation will be absolute minimal.
hence, a block design is recommended regardless of which type of normalization that is being applied to the raw data.
CONCLUSIONS
the norma-gene algorithm is applicable to small data-sets allowing more target genes to be investigated.
unless reference gene are un-affected by treatment, can be measured much more precisely than target genes or data-sets has bias surpassing variation by more than  <dig> fold  the norma-gene algorithm produces equal or better normalization than reference gene.
thus, no reference gene are needed when five or more target genes are being analyzed ; although it could be an advantage to include a number of valid reference genes to serve as "negative biological controls".
good quality  data are achieved when using norma-gene; and the data are less sensitive to experimental and biological outliers than the current approach based on reference genes.
the statistical assumption of the method is that a relative sample error is randomly distributed across the mean of all genes and replicates within treatment.
this assumption is more likely to be met than the assumptions associated with reference gene normalization.
the norma-gene approach requires applying a block design, i.e.
carefully processing the same replicates from each treatment together.
under these circumstances the design will control for all experimental variability between treatments, leaving norma-gene to normalize the within treatment variation.
applying a block design has the additional advantage of minimizing any potential confounding problems associated with co-regulated genes  that might occur when using reference genes.
norma-gene utilizes a general statistical normalization approach and may thus also be applied on other similar molecular data-sets .
competing interests
the authors declare that they have no competing interests.
abbreviations
aj: bias coefficient for replicate j; i: index for the genes; nj: number of genes that are recorded for replicate j; m: number of replicates; mi: number of replicates for which data exist for gene i; : measured gene expression value for sample j and gene i; : estimated mean value for gene i; rt: reverse transcriptase; qpcr: quantitative pcr.
authors' contributions
lhh and jgs developed the idea and the first version of the norma-gene algorithm, validated its performance and drafted the manuscript.
pbs refined the algorithm and developed the macro-based norma-gene excel workbook.
phk provided statistical input.
all authors read, contributed intellectually and approved the final manuscript.
supplementary material
additional file 1
appendix a. the file is an appendix providing detailed derivation of the equations underlying the norma-gene algorithm.
click here for file
additional file 2
figure s <dig>  the figure shows scatter of the normalized mean expression values in artificial data .
click here for file
additional file 3
table s1-overview of analyzed data-sets.
the table displays a table of experimental information related to the analysed data-sets .
click here for file
additional file 4
figure s <dig>  the figure provides a validation of the effect of number of genes on mean and standard deviation of the fitted a for the real data-sets.
click here for file
additional file 5
figure s <dig>  the figure shows the relative effect of normalization on expression values in each replicate for the real data-sets.
click here for file
additional file 6
figure s <dig>  the figure shows the relative effect of normalization on variation in each treatment for the real data-sets.
click here for file
acknowledgements
we are grateful for the financial support of the carlsberg foundation grant no.
2008-01- <dig> and a sapere aude dff-starting grant from the danish council for independent research  and the danish strategic research council through the nabiit project 2006-06- <dig> "sunano--risk assessment of free nanoparticles" .
we furthermore acknowledge drs.
jesper s. bechsgaard and stefan scholz and two anonymous reviewers for valuable discussions on the norma-gene method and comments on the manuscript.

 real-time quantitative pcr represents the current state-of-the-art approach for measuring gene expression; and the method has numerous applications in both biology and biomedicine. although qpcr is a robust technique results can vary depending on factors such as rna integrity, reverse transcriptase efficiencies, sample-to-sample variations in amplification efficiency, and variation in cdna sample loading. using equal sample sizes, assessing rna integrity and equalizing rna concentrations prior to rt are fundamental normalization steps in qpcr. still, normalization to some internal control is essential for accurate qpcr in order to balance sample-to-sample variations within the rt and pcr reactions. currently, the preferred internal control is achieved by using reference genes or better a normalization factor based on several reference genes calculated using e.g. genorm. however, the use of reference genes suffer from a circular argument; i.e. we normalize target gene expression data to exclude the systematic variation by the means of reference gene expression data obtained by the same method as the data that need normalization. thus, an assumption for using reference genes is that they are unaffected by the experimental treatment and measured accurately and without error, as we rely on the target gene data to be correctly normalized by the reference gene. the circularity is partly evened out by the normalization factor approach, based on the expression of several reference genes, following the assumption that the distribution of three or more reference genes is more accurately estimating systematic error than the distribution of only one gene. however, in many studies reference genes are chosen more or less randomly and are not always being validated for the particular experimental conditions. a further downside is that it can be difficult to find suitable reference genes for certain experimental conditions that affect gene expression broadly. searching for and validating reference genes is thus both time and money consuming and might not always be practical or successful. heterogeneous samples, however, require a robust normalization strategy. yet, conventional normalization may introduce unintentional random changes to the variance and mean expression of target genes in lack of good reference genes. this may cause invalid conclusions, and prevent good target gene data-sets from being accurately analyzed increasing the risk of making type i and ii statistical errors. thus, the use of reference genes has become the chosen method not because it is extremely good, but because it represents the best available option. here we present an algorithm, norma-gene, which is applicable to all biological and biomedical studies, especially those that are based on a limited number of genes measured with qpcr. the method is based on a data-driven normalization of target genes and is valid for as little as five target genes comprising the data-set. it does not require the use of reference genes allowing researchers to focus their efforts on studying target genes of biological relevance. methods we have defined two levels of variance in the qpcr data-sets. the first level refers to among replicate variation. this variation includes biological variation and variation in rna extraction and reverse transcription efficiency. the second level of variance refers to the variation among the measured genes within a single replicate. this variance reflects the technical and random variation in the qpcr part of the procedure. norma-gene reduces systematic and artificial between-replicate bias utilizing the entire data-set of the target genes being studied. the approach is based on calculating mean expression values for each replicate across the studied target genes and subsequently estimating a normalization factor that estimates and reduces the systematic bias of a replicate across all genes. for this a least square method is applied to secure the best possible minimization of variability in the data-set based on the bias between replicates within treatment. least square regression is non-robust to outliers and careful quality control measurement throughout the qpcr process is thus essential pre-normalization. the procedure allows the estimation of the remaining variance in the data-set, not explained by experimental bias between the replicates, and the calculation of the precision of the identified bias. the procedure is only affected by the number of replicates and genes within a treatment; however, for more than three replicates the number of genes is predominantly of importance for the precision of the normalization. the procedure is not affected by up-or down-regulation as no between treatment relations are being used. detailed derivation of the following equations is available in additional file appendix a. the mathematical theory behind norma-gene the norma-gene algorithm for a single treatment is shown below and in more detail in additional file appendix a. within each treatment of the non-normalized data-set, n genes are measured for m replicates. the normalization will take place using log-transformed data. the normalization factor for each replicate within a treatment is calculated as: where aj is the bias coefficient for replicate j, i is the index for the genes and nj is the number of genes that are recorded for replicate j, is the measured gene expression value for sample j and gene i, and is the estimated mean value for log-transformed gene i data the standard deviation of aj is found as a global standard deviation within the same treatment assuming that all aj are following the same probability distribution function. a first order uncertainty assessment for the estimation of aj is given as: where σlogx is the standard deviation of aj assuming variance homogeneity which is similar to assuming the same relative error for the data: eq. will tend to take a simple approximately form: aj will tend to be close to unity otherwise the data errors are very large and mi will normally be more than and thus the product of nj ·mi will tend to be much larger than nj and thus the ratio will dominate in eq.. if there are not very many missing data then nj will tend to be constant and equal n and if all these statements are correct then eq. can be reduced to: data handling target gene normalization has been simplified by automating all calculations in an excel workbook entitled norma-gene replicate when relying on reference gene normalization. norma-gene is very flexible and little affected by missing data, and is able to normalize samples in treatments with missing data. the calculated norma-gene normalization factor is valid for and can be used on all genes in a corresponding replicate, as the factor is replicate-specific. the norma-gene normalization improves gradually with the number of genes that are included in the target gene data-set. thus, as long as a minimum number of data points is available within a replicate normalization by norma-gene can be performed. it is not required that the same five data points are available in all replicates within a treatment, and as such norma-gene can proceed even with quite extensive missing data. benchmarking norma-gene on artificial data-sets as the true values of real data-sets are always unknown it can be difficult to evaluate normalization procedures. thus, we generated artificial data-sets to evaluate the performance of reference gene and norma-gene normalization. each data-set was comprised of a single treatment with four replicates, with one reference factor and eight target genes measured in each replicate. the artificial data-sets were generated in two steps, each with independent variation. first, we sampled four replicates from a treatment with a true mean of zero and an experiment dependent bias. the second step was to generate the variation. the targets genes were in all cases sampled with a fixed variation of 10% while the reference factor was sampled with different levels of variation. by comparing normalized data to the known true means we were able to evaluate and validate the performance of the norma-gene normalization relative to the reference gene method at different bias-to-variation ratios and at different variation in reference gene-to-variation in target gene ratios. note that the variation referred to here does not regard the among treatment variation in the reference gene or normalization factor as is usually the concern of this type of normalization. in the artificial data we assumed no between treatment effects and the reference gene or normalization factor is thus assumed to reflect the optimal situation with no variation among treatments. rather the sampled variation of both reference and target genes only reflect the precision by which the signal is measured. each set of experimental conditions were re-sampled times. data analysis differences in the efficiency of reducing variances by the two normalization methods v norma-gene) were tested with wilcoxon's signed-ranked test for two groups of paired observations.  the results of the analysis of artificial data-sets are shown in figure to evaluate the performance of the norma-gene normalization on the artificial data we tested; i) the variance reduction at different bias-to-variation ratios using norma-gene. further, to test the effects of normalization methods we compared performance on two sets of parameter combinations in artificial data; ii) the bias-to-variation ratio, and iii) the ratio of reference-gene variation relative to target-gene variation. in these two latter tests the quality of the normalization was judged by the deviation of expression level in normalized data from the true expression values. increasing the bias-to-variation ratio in the artificial data leads to more variation being removed from the data-set by norma-gene. this is not surprising as norma-gene is designed to identify and reduce bias, and as bias increases as a larger proportion of the total variance is being removed. the relationship was little affected by number of replicates and only slightly by number of genes. the result of the second test showed a superior performance of the norma-gene method with a larger degree of samples closer to the true mean than when reference gene normalized under a large range of conditions. the bias to variation ratio affected the relative performance of the norma-gene normalization method. with increasing bias-to-variation ratio, the performance of norma-gene decreased until the two methods were equal with the bias being times larger than the variation; and the reference factor method was only superior at bias-variation ratios larger than furthermore, re-sampling the data-set with only four genes did not affect the performance of the norma-gene methods confirming the stability of the method even with few genes measured. as the bias-to-variation ratio affects the effectiveness of norma-gene compared to reference gene normalization it is highly relevant to estimate this ratio in real data-sets before using the norma-gene procedure. however, the bias-to-variation ratio in real data-sets is difficult to estimate. one way is to use the relationship between bias-to-variation ratio and average variance reduction as seen in figure top panel. interestingly, the average variance reduction in the three real data-sets suggests that the bias-to-variation in these data is between and and thus in a range where norma-gene is performing better than reference gene normalization. the third and final test on artificial data concerned the level of sample variation when sampling a reference gene/reference factor. to remind, this level of variance is the sampling variation among genes from the same sample and not among treatment variance. thus, for the validation of the method using artificial data we have assumed the perfect reference gene, i.e. no variation among treatments. this is the optimal situation and quite unrealistic to achieve in many studies as argued in the text. the variation we allowed to vary is the technically variation, which determines how well the qpcr procedure measures the true amount of cdna in the sample. the realistic value in this case most be, as reference genes and target genes are measured in the same samples having gone through the same extraction and cdna synthesis procedure, unless more replicates is used for reference genes, or more care is taken by the experimenter while measuring these genes compared to when measuring target genes. thus, by applying "perfect" reference genes and allowing these to be measured more precisely than target genes we are being very conservative in the test of the norma-gene procedure. here we found, as expected, that increasing the precision of the reference gene/reference factor increased the performance of this normalization method compared to norma-gene. the reference gene/reference factor approach needs to be measured four times as precise as the target genes to result in a normalization efficiency equal to the norma-gene method. as discussed above this will be difficult to achieve using a single reference gene, but possible using a normalization factor based on multiple genes. note that any effect of treatment on reference genes will add to the level of noise and decrease the precision of reference factor normalization; rendering this situation of four times precision yielding equal normalization a conservative estimate. in experiments using harsh treatments, e.g. stress research, those reference genes, totally unaffected by treatment, might prove very difficult to identify. validation on real data-sets we applied the norma-gene algorithm to three different data-sets: i) a data-set on springtails with six target and one valid reference gene; ii) a data-set on earthworms with nine target and one valid reference gene; and iii) a data-set on daphnia with target and a genorm based normalization factor using three valid reference genes. in all three data-sets the reported results are for target genes only, but reference genes were included in the norma-gene algorithm for data-set i and ii, to improve the stability of the normalization. the norma-gene algorithm uses a least square method to secure the best possible minimization of variability in the data-set due to bias between replicates. the algorithm predicts the relative reduction in variation as a function of the number of genes comprising the data-set. norma-gene generates a stable normalization with as little as five genes, and addition of further genes to the data-set has little relative effect on the normalization output. the predicted effect of the number of genes on the normalization output was confirmed in all three data-sets verifying that using a data-set of five or more genes is a conservative approach for obtaining a stable normalization. validation of the normalization efficiency was achieved by looking at the distribution of standard deviations for each sample in the raw data after norma-gene normalization, and after normalization to a reference gene or a normalization factor based on three reference genes. we plotted the variance reducing effect of reference gene or norma-gene normalization on variances relative to the variance of raw data. the variance reduction was significantly larger following norma-gene normalization for all three data-sets; thus significantly reducing the variance compared to the reference gene based normalization. the relative effect of the different normalization approaches on mean expression values revealed that traditional reference gene normalization affects mean expression values to a greater extend than norma-gene normalized data. a comparable effect on mean was seen in the artificial data when considering a bias-to-variation ratio of real data-sets of between and. the mathematics behind norma-gene ensure that mean expression values within treatment are affected minimally by normalization, and thus deviate as little as possible from the raw expression values. discussion within microarray analysis a data-driven normalization, e.g. global lowess, has long been applied, but these methodologies rely on very large data-sets in order to deliver meaningful outputs. recently, two qpcr-based methods using data-driven normalization have been published providing proof of principle. the first describes a microarray inspired data analysis approach where quantiles are applied to normalize the target genes; yet the approach is only applicable to high-throughput qpcr analysis involving to several thousand genes. the other method is developed for normalization of micrornas quantified using qpcr and is according to the authors only applicable to large-scale gene expression studies. however, no data-driven normalization alternative is currently available for small-scale mrna expression studies using qpcr. we show here that our method produces normalized data which are closer to the true means under a range of realistic variance parameters. furthermore, we show that norma-gene produces reduced experimental bias to a higher extend and thus outperforms the current approach based on reference genes; even when good reference genes are available as in the case of data-set iii. we furthermore show that a reliable normalization may be obtained with as little as a five genes comprising the data-set and verify this on the artificial data-sets. the results of normalization and the distributions of the normalized data show high comparability between artificial and real data suggesting that the artificial data are behaving in a similar manner to real data and thus that the results of artificial data can be taken as relevant and meaningful for the validation of the method. norma-gene is designed to reduce within treatment variation only and thus has little effect on the mean expression values between treatments. contrary, reference gene normalization can have a large effect on the estimated mean expression values across treatments as this approach affects both within and between treatment variation. this is not a problem if the reference gene have been measured correctly, and thus adjust the raw data accurately. however, our analysis of the artificial data-sets show that a four-fold reduction in variation in the used reference factor is needed to obtain similarly good results with reference genes. thus, reference genes, being measured with the same technique as target genes, will realistically be measured with similar error as target genes across treatments; or are affected by the treatment. normalization factors based on more than one reference genes, i.e. using the genorm procedure, reduces this problem but does not eliminate it as seen for our real data-set iii. this will introduce variation across treatments, which was not present in the original data. furthermore, biological variation may distort the "true" mean expression values when non-validated reference genes are being used for normalization. in our opinion, the measured raw data provide the best estimate of mean expression values available; and only minor changes to these means following normalization are to be expected as achieved by norma-gene normalization. norma-gene normalization is virtually non-affected by the above-mentioned issues regarding reference gene related effects on mean expression values. however, norma-gene requires using a block design to minimize variation across treatments. if the experimental setup is based on a block design and any handling of the samples has been conducted block-wise, between treatments variation will be absolute minimal. hence, a block design is recommended regardless of which type of normalization that is being applied to the raw data.  the norma-gene algorithm is applicable to small data-sets allowing more target genes to be investigated. unless reference gene are un-affected by treatment, can be measured much more precisely than target genes or data-sets has bias surpassing variation by more than fold the norma-gene algorithm produces equal or better normalization than reference gene. thus, no reference gene are needed when five or more target genes are being analyzed; although it could be an advantage to include a number of valid reference genes to serve as "negative biological controls". good quality data are achieved when using norma-gene; and the data are less sensitive to experimental and biological outliers than the current approach based on reference genes. the statistical assumption of the method is that a relative sample error is randomly distributed across the mean of all genes and replicates within treatment. this assumption is more likely to be met than the assumptions associated with reference gene normalization. the norma-gene approach requires applying a block design, i.e. carefully processing the same replicates from each treatment together. under these circumstances the design will control for all experimental variability between treatments, leaving norma-gene to normalize the within treatment variation. applying a block design has the additional advantage of minimizing any potential confounding problems associated with co-regulated genes that might occur when using reference genes. norma-gene utilizes a general statistical normalization approach and may thus also be applied on other similar molecular data-sets. competing interests the authors declare that they have no competing interests. abbreviations aj: bias coefficient for replicate j; i: index for the genes; nj: number of genes that are recorded for replicate j; m: number of replicates; mi: number of replicates for which data exist for gene i;: measured gene expression value for sample j and gene i;: estimated mean value for gene i; rt: reverse transcriptase; qpcr: quantitative pcr. authors' contributions lhh and jgs developed the idea and the first version of the norma-gene algorithm, validated its performance and drafted the manuscript. pbs refined the algorithm and developed the macro-based norma-gene excel workbook. phk provided statistical input. all authors read, contributed intellectually and approved the final manuscript. supplementary material additional file 1 appendix a. the file is an appendix providing detailed derivation of the equations underlying the norma-gene algorithm. click here for file additional file 2 figure s the figure shows scatter of the normalized mean expression values in artificial data. click here for file additional file 3 table s1-overview of analyzed data-sets. the table displays a table of experimental information related to the analysed data-sets. click here for file additional file 4 figure s the figure provides a validation of the effect of number of genes on mean and standard deviation of the fitted a for the real data-sets. click here for file additional file 5 figure s the figure shows the relative effect of normalization on expression values in each replicate for the real data-sets. click here for file additional file 6 figure s the figure shows the relative effect of normalization on variation in each treatment for the real data-sets. click here for file acknowledgements we are grateful for the financial support of the carlsberg foundation grant no. 2008-01- and a sapere aude dff-starting grant from the danish council for independent research and the danish strategic research council through the nabiit project 2006-06- "sunano--risk assessment of free nanoparticles". we furthermore acknowledge drs. jesper s. bechsgaard and stefan scholz and two anonymous reviewers for valuable discussions on the norma-gene method and comments on the manuscript.
an analysis of the performance of norma-gene compared to reference gene normalization on artificially generated data-sets showed that the norma-gene normalization yielded more precise results under a large range of parameters tested. norma-gene does not require the identification and validation of reference genes allowing researchers to focus their efforts on studying target genes of biological relevance. here we present the norma-gene algorithm that is applicable to all biological and biomedical qpcr studies, especially those that are based on a limited number of assayed genes. the method is based on a data-driven normalization and is useful for as little as five target genes comprising the data-set. normalization of target gene expression, measured by real-time quantitative pcr, is a requirement for reducing experimental bias and thereby improving data quality. furthermore, when tested on three very different real qpcr data-sets norma-gene was shown to be best at reducing variance due to experimental bias in all three data-sets compared to normalization based on the use of reference gene. the currently used normalization approach is based on using one or more reference genes.
 normalization of target gene expression, measured by real-time quantitative pcr, is a requirement for reducing experimental bias and thereby improving data quality. the currently used normalization approach is based on using one or more reference genes. yet, this approach extends the experimental work load and suffers from assumptions that may be difficult to meet and to validate.  we developed a data driven normalization algorithm. an analysis of the performance of norma-gene compared to reference gene normalization on artificially generated data-sets showed that the norma-gene normalization yielded more precise results under a large range of parameters tested. furthermore, when tested on three very different real qpcr data-sets norma-gene was shown to be best at reducing variance due to experimental bias in all three data-sets compared to normalization based on the use of reference gene.  here we present the norma-gene algorithm that is applicable to all biological and biomedical qpcr studies, especially those that are based on a limited number of assayed genes. the method is based on a data-driven normalization and is useful for as little as five target genes comprising the data-set. norma-gene does not require the identification and validation of reference genes allowing researchers to focus their efforts on studying target genes of biological relevance. 