BACKGROUND
with the development of the dna microarray and other technologies that probe gene expression on an "omic" scale, we are now able to discover associations between biological conditions based on their molecular underpinnings.
seminal work by golub et al.
<cit>  classified leukemia samples by their global gene expression profiles, demonstrating that transcriptomic signatures can aid in functional prediction and improve our molecular understanding of disease.
hughes et al.
<cit>  predicted the effects of novel gene deletions and chemical treatments by profiling yeast mutants and comparing new arrays to this reference.
more recent studies examined cellular transcriptional response to drug treatment  <cit>  and disease  <cit>  in order to identify novel relationships between apparently unrelated conditions and compounds.
this work not only demonstrated the utility of expression-based discovery, but also suggested that functional studies about drugs and diseases can utilize data from different platforms and cell types.
this general approach to hypothesis generation - namely, finding associations between diverse conditions based on gene expression - has great potential to further biological and biomedical research if implemented on a large scale.
here we develop methods for content-based gene expression search using an entire experiment as a query.
that is, given an input experiment comparing case to control, we aim to identify other experiments that show similar patterns of differential expression.
this concept is exemplified by the connectivity map  <cit> , which searches for relationships between treatment-control comparisons for small molecules.
while the connectivity map focused on drug treatment and disease, a similar approach across a sufficiently large data source would allow for the identification of associations between gene knockdowns, diseases, drugs, and myriad other perturbations and phenotypes.
public repositories provide a wealth of data amenable to this task.
the largest of these repositories, the national center for biotechnology information  gene expression omnibus   <cit> , now contains over  <dig>  individual samples from more than  <dig>  experiments detailing the molecular characteristics of diverse cell types, diseases, and drug treatments.
the european bioinformatics institute  arrayexpress repository  <cit>  and stanford microarray database  <cit>  host additional data.
while geo supports searches of its content based on free-text and controlled-vocabulary annotations, there is increasing interest in methods for querying microarray databases based on the molecular measurements themselves  <cit> .
the power of this approach would grow with the size of the repository.
current methods for content-based search typically involve a two-step process: they identify a gene set of interest and then search for experiments in which this gene set is important.
several groups have introduced methods for identifying experiments that co-express  <cit>  or differentially express  <cit>  a given gene set.
recently, ebi implemented the gene expression atlas, which provides this latter functionality over their curated array archive  <cit> .
these methods, however, require that both the query and target experiments differentially express genes above some hard threshold, and thus may miss more subtle or noisy relationships  <cit> .
other approaches, typified by gene set enrichment analysis   <cit> , partially bypass this requirement by comparing a subset of genes to ranked profiles, using a hard threshold for the query experiment and a soft threshold for the queried experiments  <cit> .
while previous approaches require designating a group of differentially expressed genes, we explore the possibility of using as a query a differential expression  profile, consisting of a complete list of features and associated expression scores.
by examining all genes shared between query and queried experiments, we aim to identify experimental conditions and perturbations that exhibit similar transcriptional responses.
a successful strategy in this effort should reconcile differences between species, platform types, and normalization methods, as well as overcome the confounding effects of noise and technical replicability.
to achieve this, we consider combinations of methods for three tasks: data representation, dimension reduction, and search algorithm.
first, we consider the problem of data representation.
typical microarray analysis methods represent differential expression as a fold-change, comparing the expression in one set of samples to that in another  <cit> .
however, because public expression databases consist of a broad range of data types and experimental modalities, rank-based representations are often employed to account for the disparities in the distributions of observed data  <cit> .
here we compare both parametric and nonparametric data representations to determine the best approach for comparing de profiles.
we also consider an alternate representation of gene expression data, and construct de profiles based on the p-value of differential expression.
a second challenge is that gene expression profiles from high-throughput technologies consist of up to tens of thousands of measurements per sample.
in addition to the computational complexity involved in handling these large datasets, high dimensionality often confounds data mining techniques  <cit> .
in particular, high-dimensional, multimodal data lends itself to over-fitting and reduced performance  <cit> .
many solutions to this problem have been proposed, of which dimensionality reduction is the foremost.
matrix decomposition  <cit> , feature selection  <cit> , and module or gene-set based approaches  <cit>  attempt to capture the most relevant data while removing redundant or noisy features  <cit> .
given an appropriate data representation for differential expression, the final challenge is how best to calculate the similarity between two experiments.
while fujibuchi et al.
use spearman rank correlation to compare individual microarrays  <cit> , it is not clear whether a similar approach is appropriate for de profiles.
several recent studies use a modified pearson correlation measure on rank-normalized profiles  <cit> .
other work suggests that weighting expression values by each gene's variance may improve classification and analysis  <cit> .
to begin to address these challenges, we test several search schema representing combinations of data representation, dimension reduction, and correlation measures in a curated collection of  <dig> disease-related geo experiments.
we create de profiles to represent the changes in transcription between normal and disease samples , and evaluate the performance of our schema in retrieving experiments that measure the same disease as a query experiment .
we find that a projection method for dimension reduction performs as well or better than search in gene-space, and introduce an intuitive p-value weighted correlation coefficient that performs the best in our test compendium.
using the most successful parameters, we exhaustively index geo datasets  totaling  <dig>  arrays and  <dig>  experiments.
we demonstrate the utility of our method by querying our database of de profiles with several experiments examining transcription factor knockdown in embryonic and neural stem cells.
this work demonstrates the feasibility of content-based microarray search for the large-scale discovery of functional links between gene expression experiments.
RESULTS
evaluation of data representation and similarity measures
to develop a differential-expression search utility for geo, we first evaluated various data processing pipelines in a compendium of  <dig> microarray experiments comparing normal to diseased tissue.
this collection included three diseases with differing genetic origin: duchenne muscular dystrophy, huntington's disease, and breast cancer.
the studies originated from different laboratories and measured primary human disease samples as well as animal disease models.
although these experiments represented various combinations of species, platform, and normalization techniques, they clustered primarily by disease and tissue .
to search this collection of experiments based on differential expression , we created a de profile for each experiment , consisting of a list of features  each with an associated score .
we permuted various processing and ranking techniques to search for the combination of parameters that was best able to identify other experiments of the same disease given a query experiment.
we evaluated the sensitivity and specificity of these processing pipelines with leave-one-out cross-validation: we used each experiment to query the remaining  <dig> experiments with the goal of identifying other experiments that measure the same disease.
first we compared the effects of data representation on our ability to retrieve relevant experiments.
using both pearson and spearman correlation, we found that representing differential expression as a log fold-change nominally outperformed rank- and p-value-based representations .
for subsequent tests we focused on the fold-change representation.
next we evaluated all combinations of four dimension reduction methods and six similarity measures .
for dimension reduction, projection onto features identified by independent component analysis   <cit>  outperformed module-based representations.
while none of the dimension reduction methods made convincing improvements over the gene-level analysis, the ica projection method did not result in a loss of information, successfully recapitulating performance in gene-space using a significantly reduced number of features.
for similarity measures, unweighted and p-value weighted pearson correlations nominally outperformed spearman correlation for analysis in gene- and ica feature-space, resulting in the highest overall areas under the receiver operating curve.
p-weighted spearman correlation performed the worst for all dimension reduction methods.
the receiver operating curves for the best-performing search methods indicated that, on average, about 50% of the true positives could be recovered with greater than 90% specificity.
this high specificity is important for search because typically the first few results, rather than a complete list, are examined.
to evaluate the performance of our search over the top results in each search, we calculated the "precision at 4" for each of the  <dig> experiments, permuting labels to create a null model .
the average precision for duchenne muscular dystrophy and huntington's disease exceeded the random model at a 95% confidence interval for 13/ <dig> and 8/ <dig> experiments, respectively.
the "precision at 4" for breast cancer, a genetically complex disease, was also high, but it significantly surpassed the random model in only 4/ <dig> experiments.
constructing a network of geo differential expression experiments
our comparisons of data processing methods and similarity measures suggested that the p-weighted pearson correlation in gene- or ica-space is most effective at retrieving biologically relevant de profiles.
because the ica-based method reduced the number of features by a factor of  <dig>  we used this approach to systematically index geo datasets.
we created a total of  <dig>  de profiles, one for each combination of ncbi-curated experimental conditions within a dataset.
for example, if a dataset had " <dig> hr", " <dig> hr", and " <dig> hr" groups, we generated a comparison for each of " <dig> hr vs  <dig> hr", " <dig> hr vs  <dig> hr", and " <dig> hr vs  <dig> hr."
we excluded  <dig> comparisons that failed to successfully map to human genes through homologene; these experiments measured primarily bacterial and plant species.
to visualize this set of profiles, we calculated pairwise similarities using p-weighted pearson correlation and created a network of differential expression experiments .
random comparisons were used to build a null distribution of similarity scores.
with a strict cutoff , highly-connected subnetworks consisting of multiple profiles from the same dataset emerged.
clusters of profiles from multiple experiments also were apparent, linking datasets that examined related biological processes and perturbations.
figure 4b shows a multi-experiment cluster examining gonad development in mouse, consisting of differential expression profiles from gds <dig>  gds <dig>  and gds <dig>  each profile compares gonad tissue at two developmental stages, between  <dig> and  <dig> days post coitum.
the highly significant associations between the testis  and ovary  reflect known molecular similarities between male and female gonad development, especially before gestation day  <dig>   <cit> .
profiles comparing later stages in development are not linked between the sexes .
application to nanog knockdown in embryonic stem cells
this search method allowed us to simultaneously investigate a wide range of perturbations, conditions, and comparisons using the hypothesis that experiments that differentially express similar genes and pathways would also share functional phenotypic relationships.
to assess the utility of this approach, we used data from gds <dig> to investigate the effects of nanog knockdown in embryonic stem cells   <cit> .
we created a de profile comparing nanog knockdown to control in mouse escs, and queried all geo datasets to identify other experiments that have similar differential expression patterns.
because the transcription factor nanog is required for the maintenance of pluripotency in escs  <cit> , we hypothesized that this search would find profiles comparing embryonic stem cells to differentiated cells.
indeed, ten of the top fifteen matching profiles consisted of experiments comparing less differentiated to more differentiated mouse embryoid bodies of various genetic lineages .
in matching experiments, differentiation was induced by removal of lif   <cit> , a cytokine necessary to maintain the undifferentiated state of escs  <cit> .
the nanog knockdown search also identified comparisons from gds <dig>  also from loh et al.
<cit> , where esc differentiation was induced by drug treatment with retinoic acid  or hexa-methylene-bis-acetamide .
in addition to mouse esc datasets, this search produced interesting comparisons with different experimental systems.
result  <dig> supports a similarity between nanog knockdown and the comparison of non-small cell lung carcinoma  to small cell lung cancer .
sclc, the more aggressive disease, has been linked with expression of stem cell factor  <cit>  and the hedgehog signaling pathway  <cit> .
these relationships suggest that, in a broad sense, sclc compared to nsclc may have a more stem-like transcriptional program.
our method also identifies the genes that drive the correlation between two profiles.
these genes have the most significant coordinated changes in the two experiments.
when we examined the genes driving the correlation for result  <dig>  we found cytokeratin krt <dig> overexpressed in both nanog knockdown compared to control and in nsclc compared to sclc , fitting previous examinations of krt <dig> by immunohistochemistry  <cit> .
on the other end of the spectrum, we found the relatively uncharacterized gene fxyd <dig>  a regulator of n, k-atpase  <cit> .
fxyd <dig> is down-regulated during nanog knockdown  and up-regulated in several sclc cell lines .
this suggests that fxyd <dig> plays a role in the transcriptional programs in common between embryonic stem cells and sclc.
application to foxo <dig> knockout in neural stem cells
to evaluate the predictive potential of our search method in another system, we examined the effects of foxo <dig> knockout in neural stem cells .
foxo <dig> regulates nsc homeostasis by preventing premature differentiation and controlling oxygen metabolism  <cit> .
throughout the body, the foxo family of transcription factors regulate a wide variety of cellular processes including glucose metabolism, cell cycle arrest, differentiation, and detoxification of reactive oxygen species   <cit> .
we created a de profile comparing wild type to foxo3-/- adult mice using normalized data from gse <dig>  a query of geo de profiles yielded numerous significant results, the most significant of which are shown in figure  <dig>  several matching profiles  implicate foxo <dig> in hypoxia response: data from gds <dig> and gds <dig> compare mcf- <dig> breast cancer cells under hypoxic and normoxic conditions as well as with sirnas targeting hypoxia-inducible factor  <dig>  and hif-2α.
bakker et al.
found that foxo <dig> is activated in response to hypoxic stress in mouse embryonic broblasts , and furthermore that this activation requires functional hif-1α  <cit> .
renault et al.
also found that foxo <dig> is required for the expression of hypoxia-dependent genes in nscs  <cit> .
gds <dig>  compares p <dig> and cbp null mefs in response to dipyridyl  or control .
dp, a hypoxia mimetic, induces hif-1α  <cit>  and thus potentially foxo <dig>  in all four hypoxia-related profile matches, therefore, the direction of the comparisons accurately predicts known foxo <dig> biology.
to further probe the relationship between foxo <dig> and hypoxia, we examined the genes responsible for the high correlation between our foxo <dig> query and result  <dig>  predictably, we found genes associated with both hypoxia and foxo <dig> signaling .
for instance, ddit <dig> and ndrg <dig>  both of which have been found previously to be activated during hypoxia  <cit>  also contain foxo binding motifs in their regulatory regions  <cit> .
other matches from the foxo <dig> search  point to a role for foxo <dig> in cellular response to cytokine interleukin- <dig>  stimulation.
all three of these matching profiles compare cytotoxic t cell line  at  <dig> hour after il- <dig> stimulation to a later time point .
from the direction of these comparisons, we would predict that il- <dig> stimulates a transcriptional program that is similar to that of foxo <dig> knockout.
indeed, il- <dig> signaling leads to phosphorylation and inactivation of foxo <dig> in ctll- <dig> cells  <cit> , confirming this hypothesis.
discussion
in optimizing our data processing and search pipeline, we found that linear combinations of gene expression features derived in a separate compendium benefited our analysis.
the most effective dimension reduction technique involved projecting each de profile into a feature-space identified by independent component analysis.
we previously used ica to identify fundamental components of human gene expression from a large compendium of  <dig>  arrays, of which only a small subset overlap with the experiments examined here  <cit> .
the ica projection method reduced the set of features from on the order of  <dig>  to less than  <dig>  allowing for rapid indexing and searching of large libraries of differential expression profiles.
furthermore, this approach outperformed module-based methods, possibly because the linear model incorporated data from all of the genes rather than only those that participate in discrete gene sets.
despite the fact that these ica features were derived in human data, they proved robust in identifying and ranking experiments in closely related species as well.
thus, our results support previous findings that gene expression features derived in one compendium can be useful for interpreting data from new datasets  <cit> .
to calculate similarities between differential expression profiles, we introduced a novel weighting scheme that incorporates information about a feature's significance of differential expression.
this approach provides an intuitive means for emphasizing the contributions of features that are significantly differentially expressed in both experiments, which may represent the most relevant common biology.
at the same time, the weighted correlation incorporates even genes that are not significantly differentially expressed, potentially capturing the effects of broader transcriptional changes.
we observed that this scheme worked well with pearson correlation, but did not perform as well when combined with rank-based correlation.
future work will characterize the behavior of this similarity measure on a larger scale.
we used the most successful data processing pipeline to index all geo datasets.
our results with transcription factor experiments suggest that this approach can provide predictions for genes, phenotypes and perturbations that share functional similarities with a query experiment.
analysis of nanog knockdown in escs successfully identified other esc differentiation time courses, induced by a variety of factors, from amongst almost  <dig>  other profiles .
the same search predicted a link between small lung cell carcinoma and esc transcriptional programs.
for a less well characterized transcription factor, foxo <dig>  our method also succeeded in recapitulating known biology across species and experimental systems.
although it is clear that foxo <dig> has lineage-specific effects  <cit> , we identified a role for foxo <dig> in hypoxia response that appears to transcend tissue type  <cit> .
for uncharacterized comparisons, this information has the potential to provide useful hypotheses for phenotypes and pathways to investigate.
as in more traditional microarray analyses, however, interpretation of the most significant genes identified by our weighting scheme remains difficult.
our analysis of the foxo <dig> search revealed a number of genes involved in both hypoxia and foxo <dig> signaling, linking these two pathways.
however, the top genes in the nanog knockdown search failed to reveal convincing pathways that might explain the relationship between small lung carcinoma and escs.
while we focus on the interpretation of several individual genes in this study, future efforts may benefit from the use of gene set enrichment tools to find pathways that are significantly represented in the top gene list.
as experimentalists continue to explore and deposit information about cellular processes and perturbations, the utility of content-based search approaches will increase.
with a larger bank of transcriptomic data and a high chance of identifying overlapping and functionally related biology, an "experiment-omic" screen might be the first step in characterizing a novel dataset.
to realize this, further ontological indexing of expression databases may also be necessary  <cit> .
several groups have already begun to integrate expression with textual phenotype data to enable gene function prediction  <cit>  and automatic disease diagnosis  <cit>  from large databases.
even for expression-driven methods, controlled annotations for experimental variables, tissue types, and culture systems would allow for more accurate assessments of functional relevance.
finally, ontological indexing of textual annotations will enable the creation of more sophisticated connectivity maps linking not just diseases and drugs, but also gene knockdowns, over-expression studies, and genotype comparisons.
these ontology-informed studies may not only search public repositories based on gene expression, but also provide meta-analysis across phenotypic categories.
CONCLUSIONS
we have explored computational methods needed to search large repositories for relevant experiments based on differential expression, using an experiment as a query.
while previous studies use hard thresholding to select gene sets of interest  <cit> , we propose a data-driven approach that uses information from all shared genes to compare two experiments.
differential expression profiles containing scores for each gene or feature were generated and compared using correlation metrics, following the hypothesis that this direct and intuitive method would perform well across diverse datasets.
in a collection of  <dig> experiments comparing normal to diseased tissue, we achieved an average auc of  <dig>  for retrieving experiments that measure the same disease.
we further demonstrated the ability of our method to identify functionally relevant experiments from a large database of studies.
future work will include implementing the principles learned here into a web-based application.
public deployment of these methods will enable discoveries in drug repurposing, disease classification, and systems repositioning as we explore the molecular underpinnings of diverse biological processes and phenotypes.

 with the development of the dna microarray and other technologies that probe gene expression on an "omic" scale, we are now able to discover associations between biological conditions based on their molecular underpinnings. seminal work by golub et al. classified leukemia samples by their global gene expression profiles, demonstrating that transcriptomic signatures can aid in functional prediction and improve our molecular understanding of disease. hughes et al. predicted the effects of novel gene deletions and chemical treatments by profiling yeast mutants and comparing new arrays to this reference. more recent studies examined cellular transcriptional response to drug treatment and disease in order to identify novel relationships between apparently unrelated conditions and compounds. this work not only demonstrated the utility of expression-based discovery, but also suggested that functional studies about drugs and diseases can utilize data from different platforms and cell types. this general approach to hypothesis generation - namely, finding associations between diverse conditions based on gene expression - has great potential to further biological and biomedical research if implemented on a large scale. here we develop methods for content-based gene expression search using an entire experiment as a query. that is, given an input experiment comparing case to control, we aim to identify other experiments that show similar patterns of differential expression. this concept is exemplified by the connectivity map, which searches for relationships between treatment-control comparisons for small molecules. while the connectivity map focused on drug treatment and disease, a similar approach across a sufficiently large data source would allow for the identification of associations between gene knockdowns, diseases, drugs, and myriad other perturbations and phenotypes. public repositories provide a wealth of data amenable to this task. the largest of these repositories, the national center for biotechnology information gene expression omnibus, now contains over individual samples from more than experiments detailing the molecular characteristics of diverse cell types, diseases, and drug treatments. the european bioinformatics institute arrayexpress repository and stanford microarray database host additional data. while geo supports searches of its content based on free-text and controlled-vocabulary annotations, there is increasing interest in methods for querying microarray databases based on the molecular measurements themselves. the power of this approach would grow with the size of the repository. current methods for content-based search typically involve a two-step process: they identify a gene set of interest and then search for experiments in which this gene set is important. several groups have introduced methods for identifying experiments that co-express or differentially express a given gene set. recently, ebi implemented the gene expression atlas, which provides this latter functionality over their curated array archive. these methods, however, require that both the query and target experiments differentially express genes above some hard threshold, and thus may miss more subtle or noisy relationships. other approaches, typified by gene set enrichment analysis, partially bypass this requirement by comparing a subset of genes to ranked profiles, using a hard threshold for the query experiment and a soft threshold for the queried experiments. while previous approaches require designating a group of differentially expressed genes, we explore the possibility of using as a query a differential expression profile, consisting of a complete list of features and associated expression scores. by examining all genes shared between query and queried experiments, we aim to identify experimental conditions and perturbations that exhibit similar transcriptional responses. a successful strategy in this effort should reconcile differences between species, platform types, and normalization methods, as well as overcome the confounding effects of noise and technical replicability. to achieve this, we consider combinations of methods for three tasks: data representation, dimension reduction, and search algorithm. first, we consider the problem of data representation. typical microarray analysis methods represent differential expression as a fold-change, comparing the expression in one set of samples to that in another. however, because public expression databases consist of a broad range of data types and experimental modalities, rank-based representations are often employed to account for the disparities in the distributions of observed data. here we compare both parametric and nonparametric data representations to determine the best approach for comparing de profiles. we also consider an alternate representation of gene expression data, and construct de profiles based on the p-value of differential expression. a second challenge is that gene expression profiles from high-throughput technologies consist of up to tens of thousands of measurements per sample. in addition to the computational complexity involved in handling these large datasets, high dimensionality often confounds data mining techniques. in particular, high-dimensional, multimodal data lends itself to over-fitting and reduced performance. many solutions to this problem have been proposed, of which dimensionality reduction is the foremost. matrix decomposition, feature selection, and module or gene-set based approaches attempt to capture the most relevant data while removing redundant or noisy features. given an appropriate data representation for differential expression, the final challenge is how best to calculate the similarity between two experiments. while fujibuchi et al. use spearman rank correlation to compare individual microarrays, it is not clear whether a similar approach is appropriate for de profiles. several recent studies use a modified pearson correlation measure on rank-normalized profiles. other work suggests that weighting expression values by each gene's variance may improve classification and analysis. to begin to address these challenges, we test several search schema representing combinations of data representation, dimension reduction, and correlation measures in a curated collection of disease-related geo experiments. we create de profiles to represent the changes in transcription between normal and disease samples, and evaluate the performance of our schema in retrieving experiments that measure the same disease as a query experiment. we find that a projection method for dimension reduction performs as well or better than search in gene-space, and introduce an intuitive p-value weighted correlation coefficient that performs the best in our test compendium. using the most successful parameters, we exhaustively index geo datasets totaling arrays and experiments. we demonstrate the utility of our method by querying our database of de profiles with several experiments examining transcription factor knockdown in embryonic and neural stem cells. this work demonstrates the feasibility of content-based microarray search for the large-scale discovery of functional links between gene expression experiments.  evaluation of data representation and similarity measures to develop a differential-expression search utility for geo, we first evaluated various data processing pipelines in a compendium of microarray experiments comparing normal to diseased tissue. this collection included three diseases with differing genetic origin: duchenne muscular dystrophy, huntington's disease, and breast cancer. the studies originated from different laboratories and measured primary human disease samples as well as animal disease models. although these experiments represented various combinations of species, platform, and normalization techniques, they clustered primarily by disease and tissue. to search this collection of experiments based on differential expression, we created a de profile for each experiment, consisting of a list of features each with an associated score. we permuted various processing and ranking techniques to search for the combination of parameters that was best able to identify other experiments of the same disease given a query experiment. we evaluated the sensitivity and specificity of these processing pipelines with leave-one-out cross-validation: we used each experiment to query the remaining experiments with the goal of identifying other experiments that measure the same disease. first we compared the effects of data representation on our ability to retrieve relevant experiments. using both pearson and spearman correlation, we found that representing differential expression as a log fold-change nominally outperformed rank- and p-value-based representations. for subsequent tests we focused on the fold-change representation. next we evaluated all combinations of four dimension reduction methods and six similarity measures. for dimension reduction, projection onto features identified by independent component analysis outperformed module-based representations. while none of the dimension reduction methods made convincing improvements over the gene-level analysis, the ica projection method did not result in a loss of information, successfully recapitulating performance in gene-space using a significantly reduced number of features. for similarity measures, unweighted and p-value weighted pearson correlations nominally outperformed spearman correlation for analysis in gene- and ica feature-space, resulting in the highest overall areas under the receiver operating curve. p-weighted spearman correlation performed the worst for all dimension reduction methods. the receiver operating curves for the best-performing search methods indicated that, on average, about 50% of the true positives could be recovered with greater than 90% specificity. this high specificity is important for search because typically the first few results, rather than a complete list, are examined. to evaluate the performance of our search over the top results in each search, we calculated the "precision at 4" for each of the experiments, permuting labels to create a null model. the average precision for duchenne muscular dystrophy and huntington's disease exceeded the random model at a 95% confidence interval for 13/ and 8/ experiments, respectively. the "precision at 4" for breast cancer, a genetically complex disease, was also high, but it significantly surpassed the random model in only 4/ experiments. constructing a network of geo differential expression experiments our comparisons of data processing methods and similarity measures suggested that the p-weighted pearson correlation in gene- or ica-space is most effective at retrieving biologically relevant de profiles. because the ica-based method reduced the number of features by a factor of we used this approach to systematically index geo datasets. we created a total of de profiles, one for each combination of ncbi-curated experimental conditions within a dataset. for example, if a dataset had " hr", " hr", and " hr" groups, we generated a comparison for each of " hr vs hr", " hr vs hr", and " hr vs hr." we excluded comparisons that failed to successfully map to human genes through homologene; these experiments measured primarily bacterial and plant species. to visualize this set of profiles, we calculated pairwise similarities using p-weighted pearson correlation and created a network of differential expression experiments. random comparisons were used to build a null distribution of similarity scores. with a strict cutoff, highly-connected subnetworks consisting of multiple profiles from the same dataset emerged. clusters of profiles from multiple experiments also were apparent, linking datasets that examined related biological processes and perturbations. figure 4b shows a multi-experiment cluster examining gonad development in mouse, consisting of differential expression profiles from gds gds and gds each profile compares gonad tissue at two developmental stages, between and days post coitum. the highly significant associations between the testis and ovary reflect known molecular similarities between male and female gonad development, especially before gestation day. profiles comparing later stages in development are not linked between the sexes. application to nanog knockdown in embryonic stem cells this search method allowed us to simultaneously investigate a wide range of perturbations, conditions, and comparisons using the hypothesis that experiments that differentially express similar genes and pathways would also share functional phenotypic relationships. to assess the utility of this approach, we used data from gds to investigate the effects of nanog knockdown in embryonic stem cells. we created a de profile comparing nanog knockdown to control in mouse escs, and queried all geo datasets to identify other experiments that have similar differential expression patterns. because the transcription factor nanog is required for the maintenance of pluripotency in escs, we hypothesized that this search would find profiles comparing embryonic stem cells to differentiated cells. indeed, ten of the top fifteen matching profiles consisted of experiments comparing less differentiated to more differentiated mouse embryoid bodies of various genetic lineages. in matching experiments, differentiation was induced by removal of lif, a cytokine necessary to maintain the undifferentiated state of escs. the nanog knockdown search also identified comparisons from gds also from loh et al., where esc differentiation was induced by drug treatment with retinoic acid or hexa-methylene-bis-acetamide. in addition to mouse esc datasets, this search produced interesting comparisons with different experimental systems. result supports a similarity between nanog knockdown and the comparison of non-small cell lung carcinoma to small cell lung cancer. sclc, the more aggressive disease, has been linked with expression of stem cell factor and the hedgehog signaling pathway. these relationships suggest that, in a broad sense, sclc compared to nsclc may have a more stem-like transcriptional program. our method also identifies the genes that drive the correlation between two profiles. these genes have the most significant coordinated changes in the two experiments. when we examined the genes driving the correlation for result we found cytokeratin krt overexpressed in both nanog knockdown compared to control and in nsclc compared to sclc, fitting previous examinations of krt by immunohistochemistry. on the other end of the spectrum, we found the relatively uncharacterized gene fxyd a regulator of n, k-atpase. fxyd is down-regulated during nanog knockdown and up-regulated in several sclc cell lines. this suggests that fxyd plays a role in the transcriptional programs in common between embryonic stem cells and sclc. application to foxo knockout in neural stem cells to evaluate the predictive potential of our search method in another system, we examined the effects of foxo knockout in neural stem cells. foxo regulates nsc homeostasis by preventing premature differentiation and controlling oxygen metabolism. throughout the body, the foxo family of transcription factors regulate a wide variety of cellular processes including glucose metabolism, cell cycle arrest, differentiation, and detoxification of reactive oxygen species. we created a de profile comparing wild type to foxo3-/- adult mice using normalized data from gse a query of geo de profiles yielded numerous significant results, the most significant of which are shown in figure several matching profiles implicate foxo in hypoxia response: data from gds and gds compare mcf- breast cancer cells under hypoxic and normoxic conditions as well as with sirnas targeting hypoxia-inducible factor and hif-2α. bakker et al. found that foxo is activated in response to hypoxic stress in mouse embryonic broblasts, and furthermore that this activation requires functional hif-1α. renault et al. also found that foxo is required for the expression of hypoxia-dependent genes in nscs. gds compares p and cbp null mefs in response to dipyridyl or control. dp, a hypoxia mimetic, induces hif-1α and thus potentially foxo in all four hypoxia-related profile matches, therefore, the direction of the comparisons accurately predicts known foxo biology. to further probe the relationship between foxo and hypoxia, we examined the genes responsible for the high correlation between our foxo query and result predictably, we found genes associated with both hypoxia and foxo signaling. for instance, ddit and ndrg both of which have been found previously to be activated during hypoxia also contain foxo binding motifs in their regulatory regions. other matches from the foxo search point to a role for foxo in cellular response to cytokine interleukin- stimulation. all three of these matching profiles compare cytotoxic t cell line at hour after il- stimulation to a later time point. from the direction of these comparisons, we would predict that il- stimulates a transcriptional program that is similar to that of foxo knockout. indeed, il- signaling leads to phosphorylation and inactivation of foxo in ctll- cells, confirming this hypothesis. discussion in optimizing our data processing and search pipeline, we found that linear combinations of gene expression features derived in a separate compendium benefited our analysis. the most effective dimension reduction technique involved projecting each de profile into a feature-space identified by independent component analysis. we previously used ica to identify fundamental components of human gene expression from a large compendium of arrays, of which only a small subset overlap with the experiments examined here. the ica projection method reduced the set of features from on the order of to less than allowing for rapid indexing and searching of large libraries of differential expression profiles. furthermore, this approach outperformed module-based methods, possibly because the linear model incorporated data from all of the genes rather than only those that participate in discrete gene sets. despite the fact that these ica features were derived in human data, they proved robust in identifying and ranking experiments in closely related species as well. thus, our results support previous findings that gene expression features derived in one compendium can be useful for interpreting data from new datasets. to calculate similarities between differential expression profiles, we introduced a novel weighting scheme that incorporates information about a feature's significance of differential expression. this approach provides an intuitive means for emphasizing the contributions of features that are significantly differentially expressed in both experiments, which may represent the most relevant common biology. at the same time, the weighted correlation incorporates even genes that are not significantly differentially expressed, potentially capturing the effects of broader transcriptional changes. we observed that this scheme worked well with pearson correlation, but did not perform as well when combined with rank-based correlation. future work will characterize the behavior of this similarity measure on a larger scale. we used the most successful data processing pipeline to index all geo datasets. our results with transcription factor experiments suggest that this approach can provide predictions for genes, phenotypes and perturbations that share functional similarities with a query experiment. analysis of nanog knockdown in escs successfully identified other esc differentiation time courses, induced by a variety of factors, from amongst almost other profiles. the same search predicted a link between small lung cell carcinoma and esc transcriptional programs. for a less well characterized transcription factor, foxo our method also succeeded in recapitulating known biology across species and experimental systems. although it is clear that foxo has lineage-specific effects, we identified a role for foxo in hypoxia response that appears to transcend tissue type. for uncharacterized comparisons, this information has the potential to provide useful hypotheses for phenotypes and pathways to investigate. as in more traditional microarray analyses, however, interpretation of the most significant genes identified by our weighting scheme remains difficult. our analysis of the foxo search revealed a number of genes involved in both hypoxia and foxo signaling, linking these two pathways. however, the top genes in the nanog knockdown search failed to reveal convincing pathways that might explain the relationship between small lung carcinoma and escs. while we focus on the interpretation of several individual genes in this study, future efforts may benefit from the use of gene set enrichment tools to find pathways that are significantly represented in the top gene list. as experimentalists continue to explore and deposit information about cellular processes and perturbations, the utility of content-based search approaches will increase. with a larger bank of transcriptomic data and a high chance of identifying overlapping and functionally related biology, an "experiment-omic" screen might be the first step in characterizing a novel dataset. to realize this, further ontological indexing of expression databases may also be necessary. several groups have already begun to integrate expression with textual phenotype data to enable gene function prediction and automatic disease diagnosis from large databases. even for expression-driven methods, controlled annotations for experimental variables, tissue types, and culture systems would allow for more accurate assessments of functional relevance. finally, ontological indexing of textual annotations will enable the creation of more sophisticated connectivity maps linking not just diseases and drugs, but also gene knockdowns, over-expression studies, and genotype comparisons. these ontology-informed studies may not only search public repositories based on gene expression, but also provide meta-analysis across phenotypic categories.  we have explored computational methods needed to search large repositories for relevant experiments based on differential expression, using an experiment as a query. while previous studies use hard thresholding to select gene sets of interest, we propose a data-driven approach that uses information from all shared genes to compare two experiments. differential expression profiles containing scores for each gene or feature were generated and compared using correlation metrics, following the hypothesis that this direct and intuitive method would perform well across diverse datasets. in a collection of experiments comparing normal to diseased tissue, we achieved an average auc of for retrieving experiments that measure the same disease. we further demonstrated the ability of our method to identify functionally relevant experiments from a large database of studies. future work will include implementing the principles learned here into a web-based application. public deployment of these methods will enable discoveries in drug repurposing, disease classification, and systems repositioning as we explore the molecular underpinnings of diverse biological processes and phenotypes. 
we use existing and novel dimension reduction and correlation measures to rank relevant experiments in an entirely data-driven manner, allowing emergent features of the data to drive the results. experiments across platforms, tissue types, and protocols inform the analysis of new datasets. we develop methods to retrieve gene expression experiments that differentially express the same transcriptional programs as a query experiment. with the expansion of public repositories such as the gene expression omnibus, we are rapidly cataloging cellular transcriptional responses to diverse experimental conditions. we apply this method to index all geo datasets, and demonstrate the utility of our approach by identifying pathways and conditions relevant to transcription factors nanog and foxo methods that query these repositories based on gene expression content, rather than textual annotations, may enable more effective experiment retrieval as well as the discovery of novel associations between drugs, diseases, and other perturbations. avoiding thresholds, we generate differential expression profiles that include a score for each gene measured in an experiment. a combination of matrix decomposition and p-weighted pearson correlation proves the most suitable for comparing differential expression profiles.
 with the expansion of public repositories such as the gene expression omnibus, we are rapidly cataloging cellular transcriptional responses to diverse experimental conditions. methods that query these repositories based on gene expression content, rather than textual annotations, may enable more effective experiment retrieval as well as the discovery of novel associations between drugs, diseases, and other perturbations.  we develop methods to retrieve gene expression experiments that differentially express the same transcriptional programs as a query experiment. avoiding thresholds, we generate differential expression profiles that include a score for each gene measured in an experiment. we use existing and novel dimension reduction and correlation measures to rank relevant experiments in an entirely data-driven manner, allowing emergent features of the data to drive the results. a combination of matrix decomposition and p-weighted pearson correlation proves the most suitable for comparing differential expression profiles. we apply this method to index all geo datasets, and demonstrate the utility of our approach by identifying pathways and conditions relevant to transcription factors nanog and foxo  content-based gene expression search generates relevant hypotheses for biological inquiry. experiments across platforms, tissue types, and protocols inform the analysis of new datasets. 