BACKGROUND
real-time pcr is a sensitive method for expression analysis widely used for both cell culture and complex tissues.
relative quantification of mrna levels using real-time pcr data is commonly done using the 2^ method  <cit> .
a central idea of this method is the use of an endogenous control for normalization, a so-called housekeeping gene.
the aim of this normalization is to correct for different amounts of starting material of rna or differences in the cdna synthesis efficiency.
commonly used selection criteria for housekeeping genes are genes with the least amount of variance across all samples and genes that show no trends of change in relation to sample parameters of interest.
however, because of lack of methods to determine low variance - other than real-time pcr itself - the selection of endogenous controls often comes precariously close to circular reasoning.
vandesompele and coworkers have suggested methods to circumvent this, through the iterative calculation of pairwise correlations with other potential endogenous control genes and removal of the most deviating candidates  <cit> .
to investigate the merit of these endogenous control selection methods, we analyzed gene expression using different real-time pcr normalization setups and compared it with gene expression obtained using the fundamentally different approach of expression microarray measurements.
the method of real-time pcr is often used as a gold standard with which to validate findings from expression microarray experiments  <cit> .
this view, that real-time pcr is a gold standard, might be true when looking at individual genes.
however, the specific question of between-sample normalization is usually covered by measuring one or a few supposedly constant endogenous control genes.
with microarrays, on the other hand, the large number of measured genes in microarrays gives a much broader base from which to address sample variation and normalization issues.
we therefore propose to investigate the specific issue of real-time pcr normalization, using correlation to microarray data as our primary metric.
herein, we present an analysis of  <dig> human carotid plaque samples, for which gene expression data have been obtained with affymetrix hg-u <dig> plus  <dig>  arrays and for  <dig> target genes using taqman real-time pcr.
the plaque tissue is typically of a heterogeneous character, containing diverse populations of leukocytes, endothelial cells, and smooth muscle cells in various proportions.
finding and validating a set of control genes that are stable across samples under these conditions is therefore essential for accurate measurement of gene expression levels.
RESULTS
selection of endogenous control genes
we made a definition of established endogenous controls as genes available commercially, such as from applied biosystems.
at the time of the analysis, they were: actb, b <dig> m, gapdh, gusb, hprt <dig>  pgk <dig>  ppia, rplp <dig>  tbp, and tfrc.
from these, gapdh, b <dig> m, ppia, rplp <dig>  and tbp were selected as endogenous control candidates.
they were selected, as described in methods.
the ct value of each of these genes was submitted to the genorm plugin for investigation of the stability index.
the most stable gene pair was gapdh and rplp <dig>  in order of decreasing stability, they were followed by tbp, ppia, and b <dig> m. the exact definition of this method of classification is further described by vandesompele et al.
<cit> .
briefly, the two most stable genes are identified by calculating expression ratios, over all samples, for all pairwise combinations of genes.
for each pair of genes, the standard deviation over all samples is calculated, and for each gene, the mean of all standard deviations is calculated.
genes for which this value is highest are iteratively eliminated--in other words, the algorithm searches for genes that show the same expression profiles across all samples.
comparison of real-time pcr expression data with microarray expression data
real-time pcr data were obtained for  <dig> different genes of interest in addition to the  <dig> candidate endogenous controls.
we compared the measurement of gene expression using these two methods of quantification.
for each gene, this was done by creating a scatter plot, such as the example shown in figure  <dig>  scatter plots for all combinations of genes, probe sets, and endogenous control combinations are found in additional file  <dig>  pearson correlation coefficients were calculated for all of these combinations, and a summary can be seen in figure  <dig>  this figure prompts two discussions: a row-wise discussion of the differences in correlations between different genes and a column-wise discussion of the differences in correlations between different choices of endogenous controls.
differences in correlations between genes
some genes show good correlation between real-time pcr measurements and microarray measurements, and others do not.
these differences can be explained biologically and technically.
it was investigated whether there was any systematic technical bias of the real-time pcr-to-microarray correlation.
this was done by comparing the correlation metric to the mean absolute expression level and the standard deviation, both for microarray values and real-time pcr values.
this is shown as scatter plots in additional file  <dig>  and no patterns could be identified.
technical imprecision in individual measurements can also be a problem.
all microarray scans were subjected to standard quality control measures, as detailed in the methods section, in order to exclude problematic samples.
for real-time pcr, replicate measurements can clarify if deviance is a result of technical imprecision.
a technical variation threshold is described in methods.
for most genes, there were only a few measurements  with technical variation above this threshold.
the exception was alox <dig>  which had  <dig> samples with coefficients of variance above this threshold.
alox <dig> was present in very low quantities, with a mean ct of  <dig>  .
the discrepancy between measurements of this gene is therefore concluded to be due to technical imprecision in real-time pcr, as it is reflected in the lack of correlation between microarray and real-time pcr data.
the data are included in the analysis to show the effects of technical imprecision, but its removal does not change the final conclusion, as specified in the sensitivity analysis in the methods section.
in addition to the threshold values in the bottom of additional file  <dig>  we have included the raw data from the real-time pcr measurements as additional file  <dig>  this threshold could possibly be used as a measure of comparison for use in other experiments that do not have microarray data to compare with.
because of alternative splicing and the difference in probe location for the two measurements, biological variability can also explain differences in measurement techniques for some genes.
it is therefore not necessarily sufficient to talk about measuring a particular gene with two methods; the measurements should also be performed on the same part of the gene.
the locations of real-time pcr probes and primers and of microarray probe sets are shown for the examples of igf <dig> and ednra in figure  <dig>  for all other genes, the same plots are available in additional file  <dig>  endra shows a high degree of correlation, even though different sections of the gene are measured with the two techniques - the opposite is true for igf <dig>  for igf <dig>  alternative splicing is unlikely to be the reason behind the lack of correlation, but unfortunately, for almost all other genes , the real-time pcr primers and microarray probe locations are not in the same exon.
without specific real-time pcr measurements of relevant regions, it is therefore difficult to make a decisive conclusion on the gene-wise differences in correlation between microarray and real-time pcr.
focus shall therefore be on the column-wise discussion, as follows.
differences in correlations between micorarray data and different real-time pcr normalizations
in figure  <dig>  the differences in correlations between different choices of endogenous controls can be read in each column.
as demonstrated, the best correlation between microarray and real-time pcr is, on average, obtained when no endogenous control is used.
the results obtained by using the genorm method of vandesompele et al.
ranks 13th  and 17th  among all  <dig> possible combinations of one or more of  <dig> endogenous control genes.
this is an unexpected result, and it prompts a thorough discussion of the assumptions behind normalizations of pcr gene expression data.
a functional definition of gene expression should be expressed in units of mrna per cell or nucleic mrna concentration or a similar measure, because this is the level at which changes will affect the biology of the cell.
in effect, the methods we compare here all rely on one of three assumptions to get to this functional definition: 1) that there exist one or more genes for which the expression is at a sufficiently constant level to be proportional to the amount of cells across all samples; this is the assumption behind the concept of endogenous control in ΔΔct analysis; 2) that the overall distribution of the expression levels of all genes is the same in all cells, across all samples; this is the assumption behind the quantile normalization, which is the normalization part of the rma pre-processing algorithm used for microarray analysis; and 3) that the total concentration of rna is the same in all cells across all samples; this is the assumption behind the "no endogenous control" calculation of gene expression for real-time pcr data.
this assumption stems from the fact that equal amounts of rna were used in the sample preparation for the real-time pcr measurements, and it could as well be labeled "normalization to total rna mass."
it carries the further assumption that the reverse transcription of the rna to cdna does not introduce a bias.
CONCLUSIONS
we here present gene expression measurements, obtained with different methods, on the same set of rna samples.
we analyze disagreements, which are of biological and technical origin: technical imprecision will surely obstruct data collection, but biological variation can also be thought to interfere through alternative splicing mechanisms.
one finding is that introduction of housekeeping genes in the calculation perturbs the data without improving measurements.
it has previously been speculated that normalizing to total rna mass will produce better results in complex tissue  <cit> , and these results support that notion.
the strength in our study is the high number of samples for which transcript analysis has been made with both real-time pcr and microarray.
this in turns allow us to do the analysis on a detailed level.
we see clear advantages with array analysis, because it has the total gene expression profile to normalize against.
therefore, the obtained data seem more robust normalization-wise, compared with real-time pcr.
the advantages with real-time pcr are the sensitivity and usually a more optimal and up-to-date design of probe-primer pairs.
the data, however, seem to be distorted while going through an endogenous control normalization procedure, either by a single housekeeping gene used out of tradition by the research team or by the geometric mean method, suggested by others  <cit> .
while the bulk of the mass of total rna is ribosomal rna, our results show that it might nonetheless carry fewer perturbations to use this value as is.
this conclusion is of course limited by the circumstances in which we measure.
one factor that is likely to be crucial is the highly heterogeneous tissue in question.
the plaque-derived samples contain varying compositions of different cell types, and this can cause problems when assuming constant expression level of any gene across many different cell types of highly varying transcriptomic composition.
the risk that cell composition changes in a way that affects the rrna-to-mrna ratio is of course also present, but the data show that at least in this data set, this not the case.
another factor that could be of interest to the conclusion is the methods with which the cdna for real-time pcr is prepared.
the rna quality of all samples is carefully controlled, and degraded samples are excluded.
furthermore, the quantification system in use could possibly provide for better precision than previously used spectrophotometers.

 real-time pcr is a sensitive method for expression analysis widely used for both cell culture and complex tissues. relative quantification of mrna levels using real-time pcr data is commonly done using the 2^ method. a central idea of this method is the use of an endogenous control for normalization, a so-called housekeeping gene. the aim of this normalization is to correct for different amounts of starting material of rna or differences in the cdna synthesis efficiency. commonly used selection criteria for housekeeping genes are genes with the least amount of variance across all samples and genes that show no trends of change in relation to sample parameters of interest. however, because of lack of methods to determine low variance - other than real-time pcr itself - the selection of endogenous controls often comes precariously close to circular reasoning. vandesompele and coworkers have suggested methods to circumvent this, through the iterative calculation of pairwise correlations with other potential endogenous control genes and removal of the most deviating candidates. to investigate the merit of these endogenous control selection methods, we analyzed gene expression using different real-time pcr normalization setups and compared it with gene expression obtained using the fundamentally different approach of expression microarray measurements. the method of real-time pcr is often used as a gold standard with which to validate findings from expression microarray experiments. this view, that real-time pcr is a gold standard, might be true when looking at individual genes. however, the specific question of between-sample normalization is usually covered by measuring one or a few supposedly constant endogenous control genes. with microarrays, on the other hand, the large number of measured genes in microarrays gives a much broader base from which to address sample variation and normalization issues. we therefore propose to investigate the specific issue of real-time pcr normalization, using correlation to microarray data as our primary metric. herein, we present an analysis of human carotid plaque samples, for which gene expression data have been obtained with affymetrix hg-u plus arrays and for target genes using taqman real-time pcr. the plaque tissue is typically of a heterogeneous character, containing diverse populations of leukocytes, endothelial cells, and smooth muscle cells in various proportions. finding and validating a set of control genes that are stable across samples under these conditions is therefore essential for accurate measurement of gene expression levels.  selection of endogenous control genes we made a definition of established endogenous controls as genes available commercially, such as from applied biosystems. at the time of the analysis, they were: actb, b m, gapdh, gusb, hprt pgk ppia, rplp tbp, and tfrc. from these, gapdh, b m, ppia, rplp and tbp were selected as endogenous control candidates. they were selected, as described in methods. the ct value of each of these genes was submitted to the genorm plugin for investigation of the stability index. the most stable gene pair was gapdh and rplp in order of decreasing stability, they were followed by tbp, ppia, and b m. the exact definition of this method of classification is further described by vandesompele et al.. briefly, the two most stable genes are identified by calculating expression ratios, over all samples, for all pairwise combinations of genes. for each pair of genes, the standard deviation over all samples is calculated, and for each gene, the mean of all standard deviations is calculated. genes for which this value is highest are iteratively eliminated--in other words, the algorithm searches for genes that show the same expression profiles across all samples. comparison of real-time pcr expression data with microarray expression data real-time pcr data were obtained for different genes of interest in addition to the candidate endogenous controls. we compared the measurement of gene expression using these two methods of quantification. for each gene, this was done by creating a scatter plot, such as the example shown in figure scatter plots for all combinations of genes, probe sets, and endogenous control combinations are found in additional file pearson correlation coefficients were calculated for all of these combinations, and a summary can be seen in figure this figure prompts two discussions: a row-wise discussion of the differences in correlations between different genes and a column-wise discussion of the differences in correlations between different choices of endogenous controls. differences in correlations between genes some genes show good correlation between real-time pcr measurements and microarray measurements, and others do not. these differences can be explained biologically and technically. it was investigated whether there was any systematic technical bias of the real-time pcr-to-microarray correlation. this was done by comparing the correlation metric to the mean absolute expression level and the standard deviation, both for microarray values and real-time pcr values. this is shown as scatter plots in additional file and no patterns could be identified. technical imprecision in individual measurements can also be a problem. all microarray scans were subjected to standard quality control measures, as detailed in the methods section, in order to exclude problematic samples. for real-time pcr, replicate measurements can clarify if deviance is a result of technical imprecision. a technical variation threshold is described in methods. for most genes, there were only a few measurements with technical variation above this threshold. the exception was alox which had samples with coefficients of variance above this threshold. alox was present in very low quantities, with a mean ct of. the discrepancy between measurements of this gene is therefore concluded to be due to technical imprecision in real-time pcr, as it is reflected in the lack of correlation between microarray and real-time pcr data. the data are included in the analysis to show the effects of technical imprecision, but its removal does not change the final conclusion, as specified in the sensitivity analysis in the methods section. in addition to the threshold values in the bottom of additional file we have included the raw data from the real-time pcr measurements as additional file this threshold could possibly be used as a measure of comparison for use in other experiments that do not have microarray data to compare with. because of alternative splicing and the difference in probe location for the two measurements, biological variability can also explain differences in measurement techniques for some genes. it is therefore not necessarily sufficient to talk about measuring a particular gene with two methods; the measurements should also be performed on the same part of the gene. the locations of real-time pcr probes and primers and of microarray probe sets are shown for the examples of igf and ednra in figure for all other genes, the same plots are available in additional file endra shows a high degree of correlation, even though different sections of the gene are measured with the two techniques - the opposite is true for igf for igf alternative splicing is unlikely to be the reason behind the lack of correlation, but unfortunately, for almost all other genes, the real-time pcr primers and microarray probe locations are not in the same exon. without specific real-time pcr measurements of relevant regions, it is therefore difficult to make a decisive conclusion on the gene-wise differences in correlation between microarray and real-time pcr. focus shall therefore be on the column-wise discussion, as follows. differences in correlations between micorarray data and different real-time pcr normalizations in figure the differences in correlations between different choices of endogenous controls can be read in each column. as demonstrated, the best correlation between microarray and real-time pcr is, on average, obtained when no endogenous control is used. the results obtained by using the genorm method of vandesompele et al. ranks 13th and 17th among all possible combinations of one or more of endogenous control genes. this is an unexpected result, and it prompts a thorough discussion of the assumptions behind normalizations of pcr gene expression data. a functional definition of gene expression should be expressed in units of mrna per cell or nucleic mrna concentration or a similar measure, because this is the level at which changes will affect the biology of the cell. in effect, the methods we compare here all rely on one of three assumptions to get to this functional definition: 1) that there exist one or more genes for which the expression is at a sufficiently constant level to be proportional to the amount of cells across all samples; this is the assumption behind the concept of endogenous control in ΔΔct analysis; 2) that the overall distribution of the expression levels of all genes is the same in all cells, across all samples; this is the assumption behind the quantile normalization, which is the normalization part of the rma pre-processing algorithm used for microarray analysis; and 3) that the total concentration of rna is the same in all cells across all samples; this is the assumption behind the "no endogenous control" calculation of gene expression for real-time pcr data. this assumption stems from the fact that equal amounts of rna were used in the sample preparation for the real-time pcr measurements, and it could as well be labeled "normalization to total rna mass." it carries the further assumption that the reverse transcription of the rna to cdna does not introduce a bias.  we here present gene expression measurements, obtained with different methods, on the same set of rna samples. we analyze disagreements, which are of biological and technical origin: technical imprecision will surely obstruct data collection, but biological variation can also be thought to interfere through alternative splicing mechanisms. one finding is that introduction of housekeeping genes in the calculation perturbs the data without improving measurements. it has previously been speculated that normalizing to total rna mass will produce better results in complex tissue, and these results support that notion. the strength in our study is the high number of samples for which transcript analysis has been made with both real-time pcr and microarray. this in turns allow us to do the analysis on a detailed level. we see clear advantages with array analysis, because it has the total gene expression profile to normalize against. therefore, the obtained data seem more robust normalization-wise, compared with real-time pcr. the advantages with real-time pcr are the sensitivity and usually a more optimal and up-to-date design of probe-primer pairs. the data, however, seem to be distorted while going through an endogenous control normalization procedure, either by a single housekeeping gene used out of tradition by the research team or by the geometric mean method, suggested by others. while the bulk of the mass of total rna is ribosomal rna, our results show that it might nonetheless carry fewer perturbations to use this value as is. this conclusion is of course limited by the circumstances in which we measure. one factor that is likely to be crucial is the highly heterogeneous tissue in question. the plaque-derived samples contain varying compositions of different cell types, and this can cause problems when assuming constant expression level of any gene across many different cell types of highly varying transcriptomic composition. the risk that cell composition changes in a way that affects the rrna-to-mrna ratio is of course also present, but the data show that at least in this data set, this not the case. another factor that could be of interest to the conclusion is the methods with which the cdna for real-time pcr is prepared. the rna quality of all samples is carefully controlled, and degraded samples are excluded. furthermore, the quantification system in use could possibly provide for better precision than previously used spectrophotometers. 
in complex tissue, microarray data and real-time pcr data show the best correlation when endogenous control genes are omitted and the normalization is done relative to total rna mass, as measured before reverse transcription. we have found that for real-time pcr in heterogeneous tissue samples, it may be a better choice to normalize real-time pcr ct values to the carefully measured mass of total rna than to use endogenous control genes. we base this conclusion on the fact that total rna mass normalization of real-time pcr data shows better correlation to microarray data. because microarray data use a different normalization approach based on a larger part of the transcriptome, we conclude that omitting endogenous control genes will give measurements more in accordance with actual concentrations. gene expression microarrays and real-time pcr are common methods used to measure mrna levels. each method has a fundamentally different approach of normalization between samples. we here present an analysis of the differences in expression results obtained with microarray and real-time pcr, dependent on different choices of endogenous control genes. relative quantification of gene expression using real-time pcr is often done using the 2^ method, in which the normalization is performed using one or more endogenous control genes.
 gene expression microarrays and real-time pcr are common methods used to measure mrna levels. each method has a fundamentally different approach of normalization between samples. relative quantification of gene expression using real-time pcr is often done using the 2^ method, in which the normalization is performed using one or more endogenous control genes. the choice of endogenous control gene is often arbitrary or bound by tradition. we here present an analysis of the differences in expression results obtained with microarray and real-time pcr, dependent on different choices of endogenous control genes.  in complex tissue, microarray data and real-time pcr data show the best correlation when endogenous control genes are omitted and the normalization is done relative to total rna mass, as measured before reverse transcription.  we have found that for real-time pcr in heterogeneous tissue samples, it may be a better choice to normalize real-time pcr ct values to the carefully measured mass of total rna than to use endogenous control genes. we base this conclusion on the fact that total rna mass normalization of real-time pcr data shows better correlation to microarray data. because microarray data use a different normalization approach based on a larger part of the transcriptome, we conclude that omitting endogenous control genes will give measurements more in accordance with actual concentrations. 