BACKGROUND
by providing a genome-wide view of gene expression, microarrays have become a common exploratory tool in many areas of biological and clinical studies  <cit> .
while there are several different microarray platforms, photolithographically synthesized oligonucleotide arrays from affymetrix have become one of the principal technologies.
these arrays feature multiple 25-mer probes  for each gene, with their measurements summarized into a single number for the estimated expression level of that gene.
because of the important role played by this technology, many methodological studies have focused on improving the extraction of information from these arrays, from image analysis and the proper role of perfect and mismatch probes to distributional properties of the measurements and optimal statistical tests for differential expression  <cit> .
large-scale gene expression data often contain a large amount of noise from various experimental factors.
fortunately, in most cases, the technical variability is relatively small compared to the biological one and its effect can be minimized by using a sufficient number of replicates  <cit> .
however, the high cost of microarray experiments often prevents gathering of enough samples for a reliable analysis in a single laboratory.
in such cases, employing existing microarray datasets from other studies can be an efficient way of improving the reliability of a study.
moreover, as the number of publicly available datasets grows rapidly on public data depositories , it is clear that these datasets should be combined to generate a more comprehensive understanding of underlying biology.
several issues have made this process difficult so far.
first, different datasets have been processed using different procedures due to a lack of uniform standards, e.g., for background correction, normalization, and calculation of expression values.
this makes it difficult to compare them directly.
raw data files are generally unavailable and, even if they are, reprocessing them requires substantial effort.
second, we have lacked datasets with enough controls and replicates, performed under a proper experimental design and with adequate annotations, in order to make proper comparisons.
third, possibly the most troublesome, the experiments have been performed on many different platforms, with significant differences among them.
even within a single platform, technological and algorithmic advances as well as the evolving annotations of the genomes have resulted in succeeding generations of arrays with substantial modification from one generation to the next.
until now, several studies have found varying degrees of disagreement between platforms, sometimes with large discrepancies that call into the question the reliability of certain conclusions reached in microarray studies  <cit> .
a comparison of two affymetrix arrays, hugenefl and hg-u95a, was made previously, but only with the conclusion that the reproducibility is high when the two probe sets share many exact probes and that it is low when they do not  <cit> .
in this work, we carry out a thorough examination of the comparability between the two generations of affymetrix human genechip arrays, hg-u95av <dig> and hg-u133a, both of which have been used extensively for studying human gene expression patterns.
we then propose a method for enhancing their comparability.
the analysis we carry out is made possible by a dataset consisting of the same tissue samples hybridized on both platforms.
the procedure is illustrated in figure  <dig>  using our replicate dataset, we first examine the effectiveness of three schemes for matching the probe sets across different arrays.
we then quantify the surprising amount of difference in analysis results between the platforms, as revealed in correlation analysis, hierarchical clustering, and selection of differentially expressed genes.
we find that comparability can be improved by rescaling expression values or data filtering but that these techniques are limited to few specific analyses.
as a generic method for comparative analysis, we propose selecting a subset of probes that have sequence overlaps with the probes on the other array and recalculating the expression levels based only on this subset.
we demonstrate that this probe filtering significantly improves the reproducibility, without eliminating a significant number of genes from the analysis.
RESULTS
comparison of the methods for probe set matching
the most common method of matching genes in cross-platform studies is to match the unigene ids among genes  <cit> .
one potential problem with this method is that as the unigene database is updated, some tags are retired and new ones are created, and these may not be tracked correctly unless the same version of unigene was used to annotate each platform.
locuslink does not suffer from this problem as much and therefore may be preferable in some cases.
we tested three methods for matching probe sets between u95av <dig> and u133a: unigene ids, locuslink ids, and best match provided by affymetrix  <cit> .
as shown in table  <dig>  there are about  <dig> unique ids shared between u95av <dig> and u133a in all three cases, with slightly more for the best match.
the number of probe sets involved is higher for unigene and locuslink matching, since there are multiple probe sets corresponding to the same id in those cases.
for best match, the sequence mapping is restricted to many-to-one matching.
as a simple way to assess comparability, the pearson correlation coefficient between each array pair from the same sample was calculated and the  <dig> correlation coefficients were averaged.
the results are summarized in table  <dig>  unigene and locuslink matching give practically identical results.
best match, on the other hand, shows somewhat higher reproducibility than other matching methods .
the main reason for the higher reproducibility in best match is most likely that more comparable probes are chosen among multiple matches by considering the sequence information.
the overall reproducibility, however, is surprisingly low.
it has been observed in many replicate studies that expression values from affymetrix arrays show high reproducibility, typically in the range of > <dig>   <cit> .
the low correlation coefficient is already an indication that the cross-generation comparison may not be simple.
we use the best match in the following sections; unigene or locuslink matching performs similarly or slightly worse than best match.
in a similar study  <cit> , the authors report the average correlation of .
<dig> ± .
<dig> between two different generations of affymetrix arabidopsis arrays.
but they conclude that this reproducibility is sufficiently high and that the array generations can be compared without further manipulation of the data.
however, in our experience, this number is much too low.
in the current data set, for instance, the samples in different disease groups give significantly higher correlation coefficients than that.
this is clearly demonstrated later in figure  <dig>  where the arrays in the same generation are shown to be more highly correlated than the arrays in the same disease class.
exactly matched probes between array generations are highly reproducible
there was a possibility that the lack of high correlation between the two versions was caused by a true inconsistency present in the data, perhaps due to rna degradation between the times when the hybridizations on the two platforms were performed.
to make sure that this was not the case, we investigated the quality of our data by examining the subset of probes which have the exactly same sequences between the array generations.
when we examined about 5% of probes that have the same sequence between u95av <dig> and u133a, the mean correlation coefficient of array pairs, calculated by pm intensity, was  <dig>  ±  <dig> .
this is similar to the conclusion in  <cit>  that the probe sets with exactly the same set of probes have a very high correlation.
the high correlation in our dataset confirms that the samples and other experimental factors were nearly identical between the two hybridizations and that any discordant result in comparative analysis is therefore most likely due to the differences in the probe design of the two arrays.
when we compare the expression values between best match and the exactly matched probes, we can easily see the lack of reproducibility for the best match case .
it is clear that the probe-level sequence information has a large impact on the relationship between the abundance of transcript and the reported intensity  <cit>  and that the use of probe sequences would be necessary in order to choose a subset of relatively consistent probes between u95av <dig> and u133a for enhanced reproducibility.
standard probe set matching produces discordant results in analyses
to determine the extent to which the analysis results from the two versions of the arrays agree, we employ the two most frequent tools for exploratory analysis: cluster analysis and identification of differentially expressed genes.
for evaluating the compatibility in terms of cluster analysis, we combined the datasets from u95av <dig> and u133a by best match.
then, the  <dig> samples were clustered by agglomerative hierarchical clustering method with the pearson correlation coefficient as the distance measure.
figure  <dig> shows the dendrogram of  <dig> samples.
unexpectedly, instead of each array pair from the same biopsy specimen clustering together, the two array types form the two main clusters.
in other words, the most distinguishing feature of the data is the array version, rather than the actual characteristics of the samples.
to examine the reason for this incongruent result, correlation coefficients of all the possible sample pairings of the combined dataset were calculated.
figure  <dig> shows the correlation coefficients as a color map.
the two red parts of the map  represent the high correlation coefficients among samples from the same array version.
compared to these, the correlation coefficients across u95av <dig> and u133a are relatively low .
next, we identified differentially expressed genes between the dms and other myopathies from each dataset , using the two-sample t-test with unequal variances .
if the two generations of arrays were comparable, the lists of differentially expressed genes should contain many overlapping genes.
to increase the possibility of overlaps, we filtered out non-expressed genes by deleting those in which more than 75% of the samples received absent calls in both u95av <dig> and u133a arrays.
when we examine the list of genes identified in common in the two cases, however, its length is disappointingly small.
when we look at the list of length  <dig> or smaller, the percentage of overlap does not exceed 25%.
the plot of the percentage of genes common in both lists as a function of the list size is virtually identical to the dashed line in figure 7b .
this low overlap indicates that the two array types give highly inconsistent results and brings into question the reliability of the highly ranked genes in either platform.
we do note, however, that this result must be interpreted in terms of the sample size and other characteristics of the specific dataset.
a low percentage is often partially due to the presence of a large number of genes that are differentially expressed to a similar extent in a particular dataset, in which case a ranking of the genes would be expected to be somewhat unstable.
gene scaling and data filtering can enhance comparability in specific situations
to understand the reason for the discordance observed in figure  <dig>  we have examined a large number of probes.
the underlying problem, we have discovered, is due to a large number of probe sets that exhibit similar relative expression patterns but at different absolute levels.
as an illustration, we plot the expression pattern of one such probe set pair, 35828_at of u95av <dig> and 208978_at of u133a, in figure  <dig>  clearly, although the expression patterns of these genes are similar in terms of a correlation coefficient, their scales are very different.
this behavior is not simple to explain, but we believe it may be related to a large amount of cross-hybridization by a subset of badly designed probes in a probe set, especially for u95av <dig>  that would have the effect of amplifying the overall expression values.
a simple solution to this problem is to scale expression values for each gene across samples, for instance, making the mean to be  <dig> and the standard deviation to be  <dig>  the effect of this gene scaling on the gene pair from figure  <dig> is illustrated in figure  <dig>  the similarity in the expression pattern is more clearly visible and the measurements for this gene are now more comparable.
while the pearson correlations for the genes are not impacted by this linear scaling for genes, the correlations do change for the arrays.
figures  <dig> and  <dig> show the effect of gene scaling on the clustering result and the correlation coefficient of sample pairs, respectively.
in figure  <dig>  the arrays from each platform corresponding to the same sample are now clustered together in every case.
in figure  <dig>  the high correlation among the arrays of same type ) is diminished and the correlation between specimen samples across array types is highlighted .
for comparing datasets in a cluster analysis, gene scaling appears to work very well.
while gene scaling was effective in cluster analysis, it is limited to evening the influence of different genes in a global analysis by focusing on their patterns.
it does not enhance the comparability, for instance, in terms of identifying differentially expressed genes in most algorithms.
for that case, some simple filtering schemes could enhance reproducibility instead.
one way is to consider only the genes that exhibit strong correlations between the two versions.
to see the impact of this on the selection of differentially expressed genes, we calculated the overlap for the  <dig>  genes whose profiles on the two array versions were highly correlated.
the result is plotted in figure  <dig> .
to make sure that the increase in the overlap percentage is not due to the smaller number of genes, we also calculated the overlap for bootstrap samples of same size and averaged the result in figure  <dig> .
as expected, data filtering by correlation coefficients greatly improved the comparability, more than doubling the percentage of genes in common.
with more datasets such as the one we examine here, it is in theory possible to catalog a comprehensive list of genes that are reproducible across arrays, and use only these genes in subsequent comparative studies.
instead of choosing highly-correlated gene pairs, we can also filter data by expression values.
figure  <dig> shows the distribution of correlation coefficients for genes between the versions stratified by their average expression values.
we first note that the distribution for all genes is very wide, with the pearson correlation coefficient of .
<dig> ±  <dig>  reflecting the poor concordance for the probe set values on the two platforms.
with the stratification, it is clear that highly expressed genes tend to give more reproducible expression patterns across the two versions, although there still is a fraction of genes with low or even negative correlation.
the disadvantage of this type of filtering is that, as in the filtering by correlation, it inevitably reduces the number of probe sets for the analysis significantly.
probe filtering by overlapping length highly improves reproducibility with enough probe sets for comparison
we now describe a more general method for improving comparability by filtering at the probe level, instead of at the probe set level.
we have already observed that the probes with exactly the same sequences on the two generations give highly reproducible values  but that the probe sets do not.
this implies that specific probe sequences within the same target region can produce strikingly different results, and suggests that comparability would improve if we select only those probes that have sequence similarities on the two arrays.
to carry this out, we mapped the location of all probes using blat, as described in methods.
when we select a subset of probes, we mask the rest in the raw data  and then recompute the expression values using the same algorithm used in mas  <dig> .
an optimal selection scheme requires a balance.
on the one hand, we would like to require as large a sequence overlap as possible between the probes to ensure high reproducibility.
on the other hand, a stringent restriction means that the number of usable probe sets in an array is reduced and also that each probe set value will be less robust because it is derived from fewer probes.
figure  <dig> shows the correlation coefficient of array pairs from the same sample according to two criteria: the minimum overlapping length  and the minimum fraction of used probes per probe set .
the latter refers to the fraction for each probe set, e.g., 30% minimum means that at least  <dig> out of  <dig> probes for u133a and  <dig> out of  <dig> for u95av <dig> must satisfy the sequence overlap requirement.
if there are too few probes left in a probe set, we discard the probe set as unreliable.
in figure  <dig>  we plot the average of the correlations for the pairs of u95av <dig> and u <dig> chips on which the same sample is hybridized.
we see that the average correlation improves substantially with the greater amount of sequence overlap at all ranges.
it also improves with the minimum percentage of probes used but only slightly.
figure  <dig> shows the number of usable probe set pairs according to the same two criteria.
it appears, for example, that we can obtain highly comparable results  with a large number of probe sets  for comparative analysis.
for a given value of minimum overlap length, we can also calculate the average number of probes per probe set  in addition to the number of retained probe sets.
with  <dig> bp minimum overlap, more than 90% of probe sets can be used, with the expression levels calculated from an average of 30% of the original probes per probe set.
to emphasize the improvement, we again show in figure  <dig> the increase in the mean correlation coefficient of array pairs, without any criterion on the fraction of used probes per probe set.
as a baseline, the mean correlation coefficient of array pairs using best match is also represented .
enhancement in the mean correlation coefficient of array pairs is roughly proportional to the minimum overlapping length.
it appears that the mean correlation coefficient can be worse than in the case of best match when the minimum overlapping length is less than  <dig> bp.
it is possibly because such a small overlap constitutes enough dissimilarity as to confer no functional relationship between the probes and instead other good probes that do not have overlaps are thrown away.
based on figures  <dig> and  <dig>  we suggest that the minimum overlapping length of more than  <dig> bp is necessary for obtaining significantly improved results in terms of correlation coefficient of array pairs .
next, we show the improvement of comparability in terms of selecting differentially expressed genes.
figure  <dig> shows the percentage of commonly identified differentially expressed genes between u95av <dig> data and u133a data when the probes are filtered with minimum overlapping length of  <dig> bp.
the number of usable probe set pairs in this case is more than  <dig> .
for comparison, the result for the best match  case is also drawn .
from figure  <dig>  it is clear that the improvement in comparability is significant, especially when the number of selected genes is small.
for example, without the probe filtering, the lists of top  <dig> genes in the two data sets have no genes in common; with filtering,  <dig> ~ 50% of the genes are shared.
these results demonstrate that the filtered and recomputed data sets are more comparable with only a small reduction in the number of usable probe sets.
deviation from the original expression profile after probe filtering can be controlled by criterion on the overlapping length
a reduction in the number of usable probes inevitably results in the deviation of the recomputed expression values from the original values calculated using all probes.
figure  <dig> shows the mean spearman correlation coefficients between the expression values using all probes and those using only the selected probes by our criteria.
we use the spearman correlation here to capture the changes in the ranks of genes.
as expected, the correlation decreases, as more stringent criteria are applied and a smaller subset of probes is chosen.
interestingly, the deviation in u95av <dig> arrays is much larger than in u133a arrays, although the average fraction of used probes per probe set in each case is similar .
for example, the mean correlation coefficient is greater than  <dig>  in u133a when the criterion on the minimum overlapping length is less than  <dig> bp.
for the same criterion, the mean correlation coefficient is about  <dig>  in u95av <dig>  this appears to indicate that, in the process of making the two versions more similar, the larger changes occur to the expression levels in u95av <dig> arrays.
this result is consistent with the fact that probe design for u133a was performed in a more principled way than for u95av <dig> and that u133a values are closer to the true values  <cit> .
in addition to recalculating the expression values, the affymetrix present or absent calls can also be calculated.
figure  <dig> shows the percentage of present calls for each reduced group of probe sets.
the probe filtering appears to reduce the percentage of present calls, possibly because having fewer probes per probe set increases the likelihood of absent calls.
the usefulness of these calls can be debated; we simply present it here for those who find the calls helpful.
in any case, we note that the percentage sharply drops down as the minimum overlapping length increases past  <dig> bp.
both figures  <dig> and  <dig> indicate that 18– <dig> bp may be a reasonable cut-off values for the overlap length.
we note that in filtering the probes, our goal is to simply make the expression profiles from u95av <dig> and u133a more comparable.
in the process, it is possible that this procedure sometimes results in less accurate expression values in absolute terms.
by requiring that the probes in u133a have a sequence overlap with the less reliable set in u95av <dig>  we may be discarding some useful probes and, as a result, may be producing less accurate expression values.
this is a trade-off that we make in order to utilize other data sets for a comparative study, but we should be aware of this fact in subsequent analysis.
CONCLUSIONS
comparative analysis of different microarray types has a potential to generate more comprehensive and reliable results by fully exploiting available data.
understanding and resolving both the inter-platform and inter-generation data remain an important and challenging practical issue.
so far, attempts at such comparisons have been few, and many were limited to simple observations of low correlations in expression values.
in this work, we provided a more quantitative and comprehensive description of the issues and inconsistencies through the analysis of a unique dataset consisting of hg-u95av <dig> and hg-u133a hybridizations for each of the sample biopsies, and then we described a general method for resolving some of the problems.
we first observed in cluster analysis that with a standard matching of genes, the dominant feature of the dataset is not the sample characteristics but the array type.
but we found that for clustering, this problem can be mitigated by rescaling each gene.
we note, however, that this method is effective under certain assumptions, e.g., that there are enough samples for each array type and that each dataset does not contain unrelated experiments.
if two groups of patients under study are measured on two different arrays, for example, a gene scaling will simply make the samples more homogeneous and reduce the differences between the groups.
we also examined the inconsistencies in the list of differentially expressed genes obtained in the two cases.
the overlap was very low, indicating that such a list may be platform-dependent and must be interpreted with caution.
some data filtering steps, either by selecting a subset of genes that are empirically shown to be well-correlated between platforms or by focusing only on highly-expressed genes, can be helpful at times, but they do not resolve the underlying problem.
our approach based on the probe-level sequence information resulted in a significant improvement in the reproducibility in terms of correlation coefficients and selection of differentially expressed genes.
as the probes aligned to multiple regions in the genome are eliminated and the probes that share larger segments are selected, the expression values become more consistent.
this result is promising because it does not use data-dependent information such as the empirical correlation for each gene between different versions of arrays, which can only be obtained through special datasets such as ours.
we examined the effect of the minimal sequence overlap length and the minimum number of probes per gene on the reproducibility, and found that, when the parameters are chosen properly, higher correlation can be attained while retaining a large number of probes for further analysis.
we also examined the deviation from the original data when new expression values are calculated after probe filtering.
in general, we recommend the minimum overlapping length of  <dig> ~  <dig> bp and that at least  <dig> ~ 20% of probes in a probe set be present in the filtering step for a comparative analysis between u95av <dig> and u133a.
combining data across multiple platforms remains a formidable challenge.
as a first step, we have studied the issues associated with combining data from multiple generations of a single platform and proposed one method.
from our analysis, it is clear that technological issues can have significant effect and that one should be aware of the potential pitfalls in studies involving more than a single array type.
in principle, the approach of selecting probes with sequence overlaps can be applied to other array types as well as to different versions of oligonucleotide arrays.
for example, to study expression profiles of conserved regions across species using a different array for each species, more accurate results may be obtained by using only a subset of probes with sequence similarity.
in each case, appropriate criteria for the length of overlap and the number of probes needed for a robust estimate of a probe set value need to be investigated for different contexts, but the results we provide in this work can serve as a guide.

 by providing a genome-wide view of gene expression, microarrays have become a common exploratory tool in many areas of biological and clinical studies. while there are several different microarray platforms, photolithographically synthesized oligonucleotide arrays from affymetrix have become one of the principal technologies. these arrays feature multiple 25-mer probes for each gene, with their measurements summarized into a single number for the estimated expression level of that gene. because of the important role played by this technology, many methodological studies have focused on improving the extraction of information from these arrays, from image analysis and the proper role of perfect and mismatch probes to distributional properties of the measurements and optimal statistical tests for differential expression. large-scale gene expression data often contain a large amount of noise from various experimental factors. fortunately, in most cases, the technical variability is relatively small compared to the biological one and its effect can be minimized by using a sufficient number of replicates. however, the high cost of microarray experiments often prevents gathering of enough samples for a reliable analysis in a single laboratory. in such cases, employing existing microarray datasets from other studies can be an efficient way of improving the reliability of a study. moreover, as the number of publicly available datasets grows rapidly on public data depositories, it is clear that these datasets should be combined to generate a more comprehensive understanding of underlying biology. several issues have made this process difficult so far. first, different datasets have been processed using different procedures due to a lack of uniform standards, e.g., for background correction, normalization, and calculation of expression values. this makes it difficult to compare them directly. raw data files are generally unavailable and, even if they are, reprocessing them requires substantial effort. second, we have lacked datasets with enough controls and replicates, performed under a proper experimental design and with adequate annotations, in order to make proper comparisons. third, possibly the most troublesome, the experiments have been performed on many different platforms, with significant differences among them. even within a single platform, technological and algorithmic advances as well as the evolving annotations of the genomes have resulted in succeeding generations of arrays with substantial modification from one generation to the next. until now, several studies have found varying degrees of disagreement between platforms, sometimes with large discrepancies that call into the question the reliability of certain conclusions reached in microarray studies. a comparison of two affymetrix arrays, hugenefl and hg-u95a, was made previously, but only with the conclusion that the reproducibility is high when the two probe sets share many exact probes and that it is low when they do not. in this work, we carry out a thorough examination of the comparability between the two generations of affymetrix human genechip arrays, hg-u95av and hg-u133a, both of which have been used extensively for studying human gene expression patterns. we then propose a method for enhancing their comparability. the analysis we carry out is made possible by a dataset consisting of the same tissue samples hybridized on both platforms. the procedure is illustrated in figure using our replicate dataset, we first examine the effectiveness of three schemes for matching the probe sets across different arrays. we then quantify the surprising amount of difference in analysis results between the platforms, as revealed in correlation analysis, hierarchical clustering, and selection of differentially expressed genes. we find that comparability can be improved by rescaling expression values or data filtering but that these techniques are limited to few specific analyses. as a generic method for comparative analysis, we propose selecting a subset of probes that have sequence overlaps with the probes on the other array and recalculating the expression levels based only on this subset. we demonstrate that this probe filtering significantly improves the reproducibility, without eliminating a significant number of genes from the analysis.  comparison of the methods for probe set matching the most common method of matching genes in cross-platform studies is to match the unigene ids among genes. one potential problem with this method is that as the unigene database is updated, some tags are retired and new ones are created, and these may not be tracked correctly unless the same version of unigene was used to annotate each platform. locuslink does not suffer from this problem as much and therefore may be preferable in some cases. we tested three methods for matching probe sets between u95av and u133a: unigene ids, locuslink ids, and best match provided by affymetrix. as shown in table there are about unique ids shared between u95av and u133a in all three cases, with slightly more for the best match. the number of probe sets involved is higher for unigene and locuslink matching, since there are multiple probe sets corresponding to the same id in those cases. for best match, the sequence mapping is restricted to many-to-one matching. as a simple way to assess comparability, the pearson correlation coefficient between each array pair from the same sample was calculated and the correlation coefficients were averaged. the results are summarized in table unigene and locuslink matching give practically identical results. best match, on the other hand, shows somewhat higher reproducibility than other matching methods. the main reason for the higher reproducibility in best match is most likely that more comparable probes are chosen among multiple matches by considering the sequence information. the overall reproducibility, however, is surprisingly low. it has been observed in many replicate studies that expression values from affymetrix arrays show high reproducibility, typically in the range of >. the low correlation coefficient is already an indication that the cross-generation comparison may not be simple. we use the best match in the following sections; unigene or locuslink matching performs similarly or slightly worse than best match. in a similar study, the authors report the average correlation of. ±. between two different generations of affymetrix arabidopsis arrays. but they conclude that this reproducibility is sufficiently high and that the array generations can be compared without further manipulation of the data. however, in our experience, this number is much too low. in the current data set, for instance, the samples in different disease groups give significantly higher correlation coefficients than that. this is clearly demonstrated later in figure where the arrays in the same generation are shown to be more highly correlated than the arrays in the same disease class. exactly matched probes between array generations are highly reproducible there was a possibility that the lack of high correlation between the two versions was caused by a true inconsistency present in the data, perhaps due to rna degradation between the times when the hybridizations on the two platforms were performed. to make sure that this was not the case, we investigated the quality of our data by examining the subset of probes which have the exactly same sequences between the array generations. when we examined about 5% of probes that have the same sequence between u95av and u133a, the mean correlation coefficient of array pairs, calculated by pm intensity, was ±. this is similar to the conclusion in that the probe sets with exactly the same set of probes have a very high correlation. the high correlation in our dataset confirms that the samples and other experimental factors were nearly identical between the two hybridizations and that any discordant result in comparative analysis is therefore most likely due to the differences in the probe design of the two arrays. when we compare the expression values between best match and the exactly matched probes, we can easily see the lack of reproducibility for the best match case. it is clear that the probe-level sequence information has a large impact on the relationship between the abundance of transcript and the reported intensity and that the use of probe sequences would be necessary in order to choose a subset of relatively consistent probes between u95av and u133a for enhanced reproducibility. standard probe set matching produces discordant results in analyses to determine the extent to which the analysis results from the two versions of the arrays agree, we employ the two most frequent tools for exploratory analysis: cluster analysis and identification of differentially expressed genes. for evaluating the compatibility in terms of cluster analysis, we combined the datasets from u95av and u133a by best match. then, the samples were clustered by agglomerative hierarchical clustering method with the pearson correlation coefficient as the distance measure. figure shows the dendrogram of samples. unexpectedly, instead of each array pair from the same biopsy specimen clustering together, the two array types form the two main clusters. in other words, the most distinguishing feature of the data is the array version, rather than the actual characteristics of the samples. to examine the reason for this incongruent result, correlation coefficients of all the possible sample pairings of the combined dataset were calculated. figure shows the correlation coefficients as a color map. the two red parts of the map represent the high correlation coefficients among samples from the same array version. compared to these, the correlation coefficients across u95av and u133a are relatively low. next, we identified differentially expressed genes between the dms and other myopathies from each dataset, using the two-sample t-test with unequal variances. if the two generations of arrays were comparable, the lists of differentially expressed genes should contain many overlapping genes. to increase the possibility of overlaps, we filtered out non-expressed genes by deleting those in which more than 75% of the samples received absent calls in both u95av and u133a arrays. when we examine the list of genes identified in common in the two cases, however, its length is disappointingly small. when we look at the list of length or smaller, the percentage of overlap does not exceed 25%. the plot of the percentage of genes common in both lists as a function of the list size is virtually identical to the dashed line in figure 7b. this low overlap indicates that the two array types give highly inconsistent results and brings into question the reliability of the highly ranked genes in either platform. we do note, however, that this result must be interpreted in terms of the sample size and other characteristics of the specific dataset. a low percentage is often partially due to the presence of a large number of genes that are differentially expressed to a similar extent in a particular dataset, in which case a ranking of the genes would be expected to be somewhat unstable. gene scaling and data filtering can enhance comparability in specific situations to understand the reason for the discordance observed in figure we have examined a large number of probes. the underlying problem, we have discovered, is due to a large number of probe sets that exhibit similar relative expression patterns but at different absolute levels. as an illustration, we plot the expression pattern of one such probe set pair, 35828_at of u95av and 208978_at of u133a, in figure clearly, although the expression patterns of these genes are similar in terms of a correlation coefficient, their scales are very different. this behavior is not simple to explain, but we believe it may be related to a large amount of cross-hybridization by a subset of badly designed probes in a probe set, especially for u95av that would have the effect of amplifying the overall expression values. a simple solution to this problem is to scale expression values for each gene across samples, for instance, making the mean to be and the standard deviation to be the effect of this gene scaling on the gene pair from figure is illustrated in figure the similarity in the expression pattern is more clearly visible and the measurements for this gene are now more comparable. while the pearson correlations for the genes are not impacted by this linear scaling for genes, the correlations do change for the arrays. figures and show the effect of gene scaling on the clustering result and the correlation coefficient of sample pairs, respectively. in figure the arrays from each platform corresponding to the same sample are now clustered together in every case. in figure the high correlation among the arrays of same type ) is diminished and the correlation between specimen samples across array types is highlighted. for comparing datasets in a cluster analysis, gene scaling appears to work very well. while gene scaling was effective in cluster analysis, it is limited to evening the influence of different genes in a global analysis by focusing on their patterns. it does not enhance the comparability, for instance, in terms of identifying differentially expressed genes in most algorithms. for that case, some simple filtering schemes could enhance reproducibility instead. one way is to consider only the genes that exhibit strong correlations between the two versions. to see the impact of this on the selection of differentially expressed genes, we calculated the overlap for the genes whose profiles on the two array versions were highly correlated. the result is plotted in figure. to make sure that the increase in the overlap percentage is not due to the smaller number of genes, we also calculated the overlap for bootstrap samples of same size and averaged the result in figure. as expected, data filtering by correlation coefficients greatly improved the comparability, more than doubling the percentage of genes in common. with more datasets such as the one we examine here, it is in theory possible to catalog a comprehensive list of genes that are reproducible across arrays, and use only these genes in subsequent comparative studies. instead of choosing highly-correlated gene pairs, we can also filter data by expression values. figure shows the distribution of correlation coefficients for genes between the versions stratified by their average expression values. we first note that the distribution for all genes is very wide, with the pearson correlation coefficient of. ± reflecting the poor concordance for the probe set values on the two platforms. with the stratification, it is clear that highly expressed genes tend to give more reproducible expression patterns across the two versions, although there still is a fraction of genes with low or even negative correlation. the disadvantage of this type of filtering is that, as in the filtering by correlation, it inevitably reduces the number of probe sets for the analysis significantly. probe filtering by overlapping length highly improves reproducibility with enough probe sets for comparison we now describe a more general method for improving comparability by filtering at the probe level, instead of at the probe set level. we have already observed that the probes with exactly the same sequences on the two generations give highly reproducible values but that the probe sets do not. this implies that specific probe sequences within the same target region can produce strikingly different results, and suggests that comparability would improve if we select only those probes that have sequence similarities on the two arrays. to carry this out, we mapped the location of all probes using blat, as described in methods. when we select a subset of probes, we mask the rest in the raw data and then recompute the expression values using the same algorithm used in mas. an optimal selection scheme requires a balance. on the one hand, we would like to require as large a sequence overlap as possible between the probes to ensure high reproducibility. on the other hand, a stringent restriction means that the number of usable probe sets in an array is reduced and also that each probe set value will be less robust because it is derived from fewer probes. figure shows the correlation coefficient of array pairs from the same sample according to two criteria: the minimum overlapping length and the minimum fraction of used probes per probe set. the latter refers to the fraction for each probe set, e.g., 30% minimum means that at least out of probes for u133a and out of for u95av must satisfy the sequence overlap requirement. if there are too few probes left in a probe set, we discard the probe set as unreliable. in figure we plot the average of the correlations for the pairs of u95av and u chips on which the same sample is hybridized. we see that the average correlation improves substantially with the greater amount of sequence overlap at all ranges. it also improves with the minimum percentage of probes used but only slightly. figure shows the number of usable probe set pairs according to the same two criteria. it appears, for example, that we can obtain highly comparable results with a large number of probe sets for comparative analysis. for a given value of minimum overlap length, we can also calculate the average number of probes per probe set in addition to the number of retained probe sets. with bp minimum overlap, more than 90% of probe sets can be used, with the expression levels calculated from an average of 30% of the original probes per probe set. to emphasize the improvement, we again show in figure the increase in the mean correlation coefficient of array pairs, without any criterion on the fraction of used probes per probe set. as a baseline, the mean correlation coefficient of array pairs using best match is also represented. enhancement in the mean correlation coefficient of array pairs is roughly proportional to the minimum overlapping length. it appears that the mean correlation coefficient can be worse than in the case of best match when the minimum overlapping length is less than bp. it is possibly because such a small overlap constitutes enough dissimilarity as to confer no functional relationship between the probes and instead other good probes that do not have overlaps are thrown away. based on figures and we suggest that the minimum overlapping length of more than bp is necessary for obtaining significantly improved results in terms of correlation coefficient of array pairs. next, we show the improvement of comparability in terms of selecting differentially expressed genes. figure shows the percentage of commonly identified differentially expressed genes between u95av data and u133a data when the probes are filtered with minimum overlapping length of bp. the number of usable probe set pairs in this case is more than. for comparison, the result for the best match case is also drawn. from figure it is clear that the improvement in comparability is significant, especially when the number of selected genes is small. for example, without the probe filtering, the lists of top genes in the two data sets have no genes in common; with filtering, ~ 50% of the genes are shared. these results demonstrate that the filtered and recomputed data sets are more comparable with only a small reduction in the number of usable probe sets. deviation from the original expression profile after probe filtering can be controlled by criterion on the overlapping length a reduction in the number of usable probes inevitably results in the deviation of the recomputed expression values from the original values calculated using all probes. figure shows the mean spearman correlation coefficients between the expression values using all probes and those using only the selected probes by our criteria. we use the spearman correlation here to capture the changes in the ranks of genes. as expected, the correlation decreases, as more stringent criteria are applied and a smaller subset of probes is chosen. interestingly, the deviation in u95av arrays is much larger than in u133a arrays, although the average fraction of used probes per probe set in each case is similar. for example, the mean correlation coefficient is greater than in u133a when the criterion on the minimum overlapping length is less than bp. for the same criterion, the mean correlation coefficient is about in u95av this appears to indicate that, in the process of making the two versions more similar, the larger changes occur to the expression levels in u95av arrays. this result is consistent with the fact that probe design for u133a was performed in a more principled way than for u95av and that u133a values are closer to the true values. in addition to recalculating the expression values, the affymetrix present or absent calls can also be calculated. figure shows the percentage of present calls for each reduced group of probe sets. the probe filtering appears to reduce the percentage of present calls, possibly because having fewer probes per probe set increases the likelihood of absent calls. the usefulness of these calls can be debated; we simply present it here for those who find the calls helpful. in any case, we note that the percentage sharply drops down as the minimum overlapping length increases past bp. both figures and indicate that 18– bp may be a reasonable cut-off values for the overlap length. we note that in filtering the probes, our goal is to simply make the expression profiles from u95av and u133a more comparable. in the process, it is possible that this procedure sometimes results in less accurate expression values in absolute terms. by requiring that the probes in u133a have a sequence overlap with the less reliable set in u95av we may be discarding some useful probes and, as a result, may be producing less accurate expression values. this is a trade-off that we make in order to utilize other data sets for a comparative study, but we should be aware of this fact in subsequent analysis.  comparative analysis of different microarray types has a potential to generate more comprehensive and reliable results by fully exploiting available data. understanding and resolving both the inter-platform and inter-generation data remain an important and challenging practical issue. so far, attempts at such comparisons have been few, and many were limited to simple observations of low correlations in expression values. in this work, we provided a more quantitative and comprehensive description of the issues and inconsistencies through the analysis of a unique dataset consisting of hg-u95av and hg-u133a hybridizations for each of the sample biopsies, and then we described a general method for resolving some of the problems. we first observed in cluster analysis that with a standard matching of genes, the dominant feature of the dataset is not the sample characteristics but the array type. but we found that for clustering, this problem can be mitigated by rescaling each gene. we note, however, that this method is effective under certain assumptions, e.g., that there are enough samples for each array type and that each dataset does not contain unrelated experiments. if two groups of patients under study are measured on two different arrays, for example, a gene scaling will simply make the samples more homogeneous and reduce the differences between the groups. we also examined the inconsistencies in the list of differentially expressed genes obtained in the two cases. the overlap was very low, indicating that such a list may be platform-dependent and must be interpreted with caution. some data filtering steps, either by selecting a subset of genes that are empirically shown to be well-correlated between platforms or by focusing only on highly-expressed genes, can be helpful at times, but they do not resolve the underlying problem. our approach based on the probe-level sequence information resulted in a significant improvement in the reproducibility in terms of correlation coefficients and selection of differentially expressed genes. as the probes aligned to multiple regions in the genome are eliminated and the probes that share larger segments are selected, the expression values become more consistent. this result is promising because it does not use data-dependent information such as the empirical correlation for each gene between different versions of arrays, which can only be obtained through special datasets such as ours. we examined the effect of the minimal sequence overlap length and the minimum number of probes per gene on the reproducibility, and found that, when the parameters are chosen properly, higher correlation can be attained while retaining a large number of probes for further analysis. we also examined the deviation from the original data when new expression values are calculated after probe filtering. in general, we recommend the minimum overlapping length of ~ bp and that at least ~ 20% of probes in a probe set be present in the filtering step for a comparative analysis between u95av and u133a. combining data across multiple platforms remains a formidable challenge. as a first step, we have studied the issues associated with combining data from multiple generations of a single platform and proposed one method. from our analysis, it is clear that technological issues can have significant effect and that one should be aware of the potential pitfalls in studies involving more than a single array type. in principle, the approach of selecting probes with sequence overlaps can be applied to other array types as well as to different versions of oligonucleotide arrays. for example, to study expression profiles of conserved regions across species using a different array for each species, more accurate results may be obtained by using only a subset of probes with sequence similarity. in each case, appropriate criteria for the length of overlap and the number of probes needed for a robust estimate of a probe set value need to be investigated for different contexts, but the results we provide in this work can serve as a guide. 
through a comparative analysis on a variety of datasets, a more comprehensive view of the underlying mechanism or structure can be obtained. we find that the use of the probe set matching table for comparative analysis provided by affymetrix produces better results than matching by unigene or locuslink identifiers but still remains inadequate. as a generic method for improving comparability, we select a subset of probes with overlapping sequence segments in the two array types and recalculate expression values based only on the selected probes. however, as we discover in this work, continual changes in genomic sequence annotations and probe design criteria make it difficult to compare gene expression data even from different generations of the same microarray platform. with a careful filtering of the probes based on their sequence overlaps, data from different generations of microarrays can be combined more effectively. we first describe the extent of discordance between the results derived from two generations of affymetrix oligonucleotide arrays, as revealed in cluster analysis and in identification of differentially expressed genes. rescaling of expression values for each gene across samples and data filtering by expression values enhance comparability but only for few specific analyses. compatibility between high-density oligonucleotide arrays is significantly affected by probe-level sequence information. we show that this filtering of probes significantly improves the comparability while retaining a sufficient number of probe sets for further analysis.
 one of the important challenges in microarray analysis is to take full advantage of previously accumulated data, both from one's own laboratory and from public repositories. through a comparative analysis on a variety of datasets, a more comprehensive view of the underlying mechanism or structure can be obtained. however, as we discover in this work, continual changes in genomic sequence annotations and probe design criteria make it difficult to compare gene expression data even from different generations of the same microarray platform.  we first describe the extent of discordance between the results derived from two generations of affymetrix oligonucleotide arrays, as revealed in cluster analysis and in identification of differentially expressed genes. we then propose a method for increasing comparability. the dataset we use consists of a set of human muscle biopsy samples from patients with inflammatory myopathies that were hybridized on both hg-u95av and hg-u133a human arrays. we find that the use of the probe set matching table for comparative analysis provided by affymetrix produces better results than matching by unigene or locuslink identifiers but still remains inadequate. rescaling of expression values for each gene across samples and data filtering by expression values enhance comparability but only for few specific analyses. as a generic method for improving comparability, we select a subset of probes with overlapping sequence segments in the two array types and recalculate expression values based only on the selected probes. we show that this filtering of probes significantly improves the comparability while retaining a sufficient number of probe sets for further analysis.  compatibility between high-density oligonucleotide arrays is significantly affected by probe-level sequence information. with a careful filtering of the probes based on their sequence overlaps, data from different generations of microarrays can be combined more effectively. 