BACKGROUND
endogenous biomarkers are molecules whose levels in a tissue or fluid of an organism correlate with the presence of a given disease.
for instance, the prostate specific antigen  is an fda-approved endogenous biomarker for prostate cancer  <cit> .
identifying biomarkers for specific cancer types is expected to lead to the development of early diagnostic methods that can reduce morbidity and mortality  <cit> .
here, we show that cancer biomarkers can be discovered by mining human and mouse expressed sequence tags .
furthermore, we show that biomarkers identified in this way perform well over the wide range of tumor types and classifications used in microarray studies.
our results demonstrate the existence of a small group of genes whose expression levels are affected in a wide array of cancers.
various approaches have been used to search for cancer biomarkers.
methods that mine gene expression include serial analysis of gene expression   <cit> , microarray analysis involving the comparison of tumor samples versus normal tissues , large scale meta analysis of cancer microarray data  <cit> , or massively parallel signature sequencing   <cit> .
expressed sequence tags such as found in dbest  <cit>  are another abundant source of gene expression data.
we have previously described tissueinfo, an approach to determine whole genome tissue expression profiles using data in dbest  <cit> .
in this article, we asked whether est data could be mined to identify cancer biomarkers.
RESULTS
mining dbest
we extended tissueinfo to mine ests for genes that are differentially expressed in non-tumor versus tumor tissues .
figure  <dig> presents how many times human transcripts appear in tumor versus non-tumor tissues.
the slope of the line that best fit the points is a consequence of how many est libraries are available in tumor and non-tumor samples.
the variation around the regression line, however, reflects both the random sampling effect of est sequencing  and the differential regulation of the genes between the tumor and non tumor conditions.
the scatter observed when considering all transcripts  can be considerably reduced when focusing on transcripts preferentially expressed in a given tissue.
this noise reduction technique is central to the est mining approach presented in this manuscript.
after filtering, we used a two-tailed fisher exact test to assign a p-value to each transcript and quantify the likelihood of differential expression between tumor and non-tumor .
human-mouse comparison
we mined mouse ests in the same way and compared the results to those obtained with the human data.
ortholog information was obtained from ensembl .
plotting the p-values obtained for human and mouse orthologs, we confirmed that the tissueinfo cancer p-value is uniformly distributed in regions where the p-value is not significant .
further, we asked if selecting genes in human by p-value provided more overlap with mouse than would be expected by chance if p-values were distributed randomly in the human/mouse ortholog gene set.
we selected  <dig> human genes  and  <dig> mouse genes  and observed an overlap of  <dig> genes.
randomly selecting  <dig> genes and  <dig> genes in the human/mouse ortholog gene set yielded overlaps of ~ <dig> .
the difference is significant , indicating that the overlap observed cannot be explained by chance alone.
these data are a strong indication that our approach works consistently in the two organisms, despite important differences in tissue and est availability or cancer biology.
the list of  <dig> genes is annotated and provided in supplementary table  <dig> .
the references cited in the table show that most genes on this list have been studied and found to be differentially regulated in cancer.
interestingly, of the  <dig> genes that both our mouse and human screens detect at high stringency, there is only one gene in common with the perou-brown-botstein proliferation cluster  <cit> .
this indicates that these  <dig> genes common between human and mouse are not the usual markers of proliferation and poor differentiation.
false discovery rate
to confirm that small p-values indicate transcripts that are differentially expressed between tumor and non-tumor tissues, we estimated the false discovery rate, a statistical measure used to estimate the bias introduced by multiple hypothesis testing  <cit> .
the false discovery rate  is the ratio fdr = fp/, where fp is the number of false positives detected below a threshold and tp is the number of true positives .
to calculate fdr, fp and tp need to be known.
we estimated fp as the number of genes selected in a control experiment .
we estimated tp by comparing the control to the tumor vs. non-tumor data .
figure  <dig> compares the number of transcripts found below various thresholds of the p-value in the data and in the control experiment, and plots 1-fdr as a function of the exponent of the p-value threshold.
figure  <dig> indicates that combining p-values for the human and mouse screens lowers the false discovery rate of our approach, from about 30% with the human data only to about 20% when human and mouse data are combined.
hm200
we focused on human/mouse orthologous genes and selected the  <dig> genes that were ranked best by the combined human/mouse p-value phm, where phm = ph × pm .
we called this gene set hm <dig>  each gene which is part of hm <dig> has an orthologous counterpart in human or mouse, and has been selected because of differential expression in human cancer, mouse cancer, or both, as measured by our approach.
from figure  <dig> , we estimate that over 78% of the genes in hm <dig> are truly differentially expressed between tumor and non-tumor tissues.
in contrast, if one were to randomly select genes from the genome to identify  <dig> genes truly significantly differentially expressed between tumor and non-tumors, the expected precision of the task would be  <dig> % .
using our approach with the combined human and mouse dbest data is therefore  <dig> times  more precise than a random selection.
hm <dig> is enriched in cancer-associated genes
we asked if genes in hm <dig> were more likely to be oncogenes or tumor suppressor genes.
to address this question, we counted how many genes in hm <dig> overlap with a set of  <dig>  genes previously studied for their role in oncogenesis .
we found that  <dig> % of genes in hm <dig> overlap with this list , a number to be compared to  <dig> %  in the whole human/mouse ortholog gene set.
the difference is significant , and comparable to the enrichment recently reported by aouacheria and colleagues when identifying genes bearing cancer-associated nonsynonymous snps in dbest  <cit> .
hm <dig> as predictive biomarkers
although hm <dig> genes were identified in est data, we reasoned that they should be accurate predictors in cancer microarray data sets.
testing hm <dig> on microarray data is an independent test of their biomarker abilities.
this is a strong test because the only common point between microarray and expressed sequence tags is that these technologies measure gene expression.
therefore, to determine whether hm <dig> genes could be used as cancer biomarkers in human, we asked how well hm <dig> compared to other gene lists in a variety of microarray cancer classification tasks.
lists of gene markers have been derived in many microarray studies for different types of cancer and we collected  <dig> such gene lists from published studies .
because gene lists vary in the number of genes that they contain, in addition to the hm <dig> gene list, we evaluated hm <dig>  hm <dig> and hm <dig> gene lists with the top  <dig>   <dig> and  <dig> genes from hm <dig>  respectively .
we also used a negative control gene list nc01- <dig>  of  <dig> genes with high tissueinfo cancer p-values .
although the complete list of genes on an array is not a practical biomarker list , we included the full set of genes found on the array used for each study as a positive control.
in total, our evaluation compared the predictive performance of  <dig> gene lists over  <dig> different microarray datasets .
we included in this study all the microarray datasets that we could obtain.
the number of gene lists included in the study was limited only by the computational cost of performing the evaluation .
microarray studies of cancer frequently compare two different conditions.
in some studies, one condition is a tumor and the other the normal tissue.
in other studies, conditions are derived from cytology, and tumor grades are used as conditions.
the most interesting studies define conditions from data obtained about the patient after the tumor sample has been taken .
the microarray datasets collected for this study include at least one representative of each of these types of classifications.
some microarray datasets are annotated such that multiple different classification tasks can be performed with them.
for instance, in the pomeroy <dig> dataset, five classifications can be performed: normal cerebellum/medulloblastoma, normal/malignant glioma, normal/atypical teratoid or rhabdoid tumor, normal/primitive neuroectodermal tumor, and medulloblastoma/malignant glioma.
in such cases, we included each possible classification if this classification had at least three patients in each class.
supplementary table  <dig> lists the classifications considered in this study .
in our evaluation, the number of gene lists, datasets and classification conditions combine to produce  <dig>  different cancer classification tasks.
we evaluate the leave-one-out accuracy of each gene list on each classification task, filter out classification tasks for which the p-value of the label permutation test exceeds  <dig>  , and then rank gene lists in each dataset from highest to lowest accuracy .
briefly, a gene list obtains rank  <dig> on a classification task if it yielded the highest accuracy for that task among the gene lists included in the evaluation.
figure  <dig> presents a tally of the rank for each gene list in our evaluation.
.
supplementary table  <dig> shows the statistical data used to build figure  <dig> .
the  <dig>  classification tasks measure two properties of each gene list: robustness and generality.
we define robustness as the property of a biomarker set to perform well on independent datasets obtained for the same tumor type.
robustness should not be confused with leave-one-out performance, which estimates how well a predictor can generalize using information from a single training set.
robustness is a more important property than leave-one-out performance for clinical applications because any training set is unlikely to sample most of the variability found in the patient population.
thus, a good performance with one training set may not translate to another training set.
for example, when we classify the pomeroy <dig> dataset using the gds <dig> gene list , we test the robustness of the gds <dig> gene list because both datasets include data from patients with medulloblastoma, and patients for these studies were recruited independently.
these tests are performed to measure robustness.
in contrast to robustness, we define generality as the property of a biomarker set to perform well on a tissue, tumor type or classification task other than the one it was trained for.
a perfect biomarker set is one that would be both robust and general.
in the test presented in figure  <dig>  such a biomarker set would appear with one peak only, at rank  <dig>  and the height of the peak would coincide with the total number of classifications performed in the evaluation.
in contrast, the peaks of sets of genes unrelated to cancer should be randomly distributed across ranks and show no preference for the better ranks.
our negative control gene list nc01- <dig>  shows precisely this behavior .
gene lists obtained from published studies generally show a skewed distribution towards the best ranks .
we observe that hm <dig> and hm <dig> present the most skewed distributions of the cancer marker gene lists.
given the range of classification tasks, tissue and tumor types sampled in our evaluation, figure  <dig> indicates that the hm <dig> and hm <dig> marker sets are both robust and general cancer marker sets over the range of cancer classifications evaluated.
prior comparisons of est data and microarray data have confirmed that these two approaches to measuring gene expression have different strengths and drawbacks .
it is therefore notable to see that both est and microarray data indicate that the genes in the hm <dig> gene list are differentially expressed in cancer and can predict cancer-related conditions .
this clearly shows that the hm <dig> gene list is not random and that our approach to mining expressed sequence tags is effective.
hm <dig> are mostly novel biomarkers
we asked if genes in hm <dig> had been recognized before as biomarkers.
we calculated the overlap between genes in hm <dig> and the  <dig> lists used in our evaluation .
numbers of genes overlapping between hm <dig> and these gene lists are shown in table  <dig> with a gene list similarity measure .
see methods for calculations and a discussion of the validity of the pseudo p-values shown in table  <dig>  table  <dig> shows that three of the previously reported cancer marker gene lists overlap more significantly with hm <dig> than others.
these lists are pomeroy-400-classic-desmoplastic , pomeroy-class- <dig> , and gds <dig> .
interestingly, these three lists also showed better generality when tested across datasets than most other gene lists  in the rank evaluation test shown in figure  <dig>  this shows that although  <dig> hm <dig> genes have been identified by pomeroy and colleagues as central nervous system tumor markers, most genes in hm <dig> had not been previously recognized as cancer markers as a group.
table  <dig> also includes a comparison between hm <dig> and the proliferation cluster described by perou, brown, botstein and colleagues in  <cit> .
the comparison identifies only two genes in common between hm <dig> and the proliferation cluster the main contribution of our study is to show that a small group of genes  are predictive cancer markers that outperform other cancer marker lists over different tumor types and classification tasks.
these results demonstrate the existence of a set of genes whose transcription levels are perturbed in a variety of tumors types, irrespective of the tissue of origin of the tumor.
hm <dig> and cancer pathways
since the products of genes in hm <dig> are differentially expressed in tumor and non-tumor  and tumor grade or tumor evolution , we hypothesized that hm <dig> would be likely to interact with genes shown to be altered in tumorigenesis .
to test this hypothesis, we queried the ingenuity pathways knowledge base with accession codes for protein products of hm <dig>  the ingenuity pathway database is a curated database that provides access to  <dig>  million protein-protein and gene-protein interactions and offers tools that cluster genes into interaction networks.
supplementary table  <dig>  and supplementary figures 1– <dig>  present gene networks identified when hm <dig> genes are used as focus genes .
documentation available on the ingenuity web site does not describe how the score is calculated, but indicates that the p-value of observing a network by chance is estimated to be 110-score.
the networks shown on supplementary table  <dig>  are thus unlikely to have been generated by chance .
genes shown in bold  or in gray  are members of hm <dig>  notably, network  <dig> contains the cancer-related genes fos, fosb, fosl <dig>  and junb; network  <dig> contains mycn, bcl <dig>  egr <dig>  <cit> ; network  <dig> contains kinesin 5a, 5b, and 5c; network  <dig> contains myc, network  <dig> contains the v-akt murine thymoma viral oncogene homolog  <dig> ; network  <dig> contains the retinoblastoma protein  and network  <dig> contains the p <dig> protein ; network  <dig> contains the peroxisome proliferator-activated receptor γ  and the granulocyte differentiation factor c/ebpα   <cit> .
following this observation, we tested if genes found in these networks that are not hm <dig> biomarkers are enriched in cancer-related genes.
of the  <dig> genes in the  <dig> best networks shown in supplementary table  <dig>  that are not part of hm <dig>   <dig> %  overlap with cancer-related genes .
in contrast, the cancer-related genes represent  <dig> %  of human genes with a hugo gene name .
the difference is significant .
interestingly, the overlap with cancer-related genes is significantly higher for genes known to interact with genes in hm <dig> than for genes in hm <dig> themselves  .
this supports our hypothesis that hm <dig> genes are interacting directly with genes involved in tumorigenesis and that their transcripts are markers of cancer-related signaling alterations.
the cancer-related genes themselves may not be transcriptionally regulated, but expression of downstream genes such as those found in hm <dig> is.
hm <dig> is enriched in mrna binding proteins
we performed a gene ontology classification screen and found genes in hm <dig> to be significantly enriched in mrna binding proteins   <cit> .
this is not surprising because there is growing evidence that deregulation of protein translation may participate in malignant transformation  <cit> .
the mechanisms by which tumor cells selectively deregulate protein synthesis are still unclear  <cit> .
therefore, the identification of genes that may participate in selective protein synthesis deregulation is of great interest.
the networks provided as supplementary figures 1– <dig>  provide candidates for such genes.
for instance, supplementary figure  <dig>  shows that seven genes in hm <dig> are connected to jerky .
the number of connections between jrk and hm <dig> genes is comparable to the number of connections between p <dig> and hm <dig>  this is surprising because jrk was first studied as a temporal lobe epilepsy gene candidate  <cit> .
more recent studies indicate that jerky is an mrna binding protein that binds mrna through a cenp-b domain.
jerky shows selectivity in the set of mrna molecules that it binds, and may regulate the availability of mrnas to the translational machinery  <cit> .
the cenp-b domain is found in nine other human proteins .
the conservation of this domain suggests that some of these other proteins may also bind mrna.
however, the ability of these other proteins to bind mrna selectively remains to be determined.
we predict that jerky  is involved in tumorigenesis by selectively deregulating protein translation.
this hypothesis is supported by the presence of the fragile mental retardation  <dig> protein  on network  <dig>   <cit> .
fmr <dig> binds mrnas selectively and is believed to regulate their translation, thereby regulating the protein expression of specific genes  <cit> .
fmr <dig> binds mrnas through kh motifs.
these motifs are also found in nova- <dig>  a biomarker for paraneoplastic opsoclonus-myoclonus ataxia disorder   <cit> .
this discussion illustrates how interaction networks containing hm <dig> genes can be used to develop new molecular hypotheses for cancer biology.
potential issues when mining ests
we present and validate a new approach to identifying cancer biomarkers by mining expressed sequence tags.
because expressed sequence tags are a vastly heterogeneous type of data, any approaches that mine this source of data need to account for several issues.
many of these issues were discussed in our previous publication and will not be repeated here  <cit> .
however, it is worth noting that the evaluation that we conducted in  <dig> found that tissueinfo was accurate in identifying if a gene was expressed in a tissue in 80–89% of cases  <cit> .
a new issue specific to mining ests for cancer markers is that dbest contains different proportions of est libraries from tumor and non-tumor tissues.
this is clearly illustrated in figure  <dig>  panel "all tissues", where most transcripts are found to match double the ests in non-tumor tissues than in tumor tissues.
in our new approach, we have addressed this issue in two ways.
first, we identify groups of genes that are mostly expressed in a given tissue/organ or tissue type .
second, we calculate a fisher exact test p-value which takes into account  the number of times a given transcript matches an est in tumor, and in non-tumor, and  the overall proportion of tumor/non-tumor ests for the tissue the transcript is mostly expressed in.
this test adequately corrects for the tissues with varied proportions of ests in tumor and non-tumor tissues, and at the same time assigns significance as a function of the number of est matches for the transcript.
the second factor is important because we have shown that tissueinfo identifications are more accurate when a transcript matches more ests  <cit> .
the false discovery rate analysis that we present in figure  <dig> indicates that despite these various potential issues, our approach was able to identify genes differentially expressed in tumor/non-tumor tissues with about 80% precision .
additional independent validations, such as the overlap with cancer gene lists, the overlap between human and mouse predictions, and most importantly the large microarray evaluation study that we conducted, support these conclusions.
microarray validation confirms est mining
the microarray evaluation study that we present in this manuscript is a large study by the current standards of the field.
indeed, this study considers a test set of  <dig> microarray datasets, when most marker discovery studies test a set of markers in one or two independently collected datasets.
an exception is the study of rhodes et al, where  <dig> datasets were used for independent validation  <cit>   furthermore, our validation study considers the set of markers that we identified in competition with  <dig> previously published genes lists, one negative control and a positive control .
this contrasts with most published studies where single lists of markers are typically evaluated in isolation  <cit> .
we used a stringent permutation test to evaluate the impact of the number of samples in each condition and intra-dataset correlations to the performance observed in a classification .
the negative control gene list is not expected to perform at a statistically significant level and indeed, most of its classifications are filtered out by the permutation test .
applying this stringent protocol to the comparison of multiple marker lists on different datasets and classification tasks revealed that some marker lists are more general than others.
to the best of our knowledge, the generality of cancer marker lists has not been studied before.
hm <dig> achieves the best or second best classification in 79% of the classifications considered .
further, we report that one previously published list  also shows good generality across tumor types.
our observation that some marker lists are predictive over a wide range of tumor types and classification types strongly suggests the existence of a subset of genes under the transcriptional control of pathways consistently deregulated in different cancer types.
.
our results also suggest that new cancer marker studies should report on the generality of the markers that they identify because results obtained with general markers are more likely to be reproducible.
hm <dig> applications
we anticipate that the discovery of the hm <dig> cancer marker gene set and the characterization of its predictive utility across tumor types will open new strategies for the identification of genes whose alterations are required steps in tumorigenesis .
also, since hm <dig> markers are predictive across tumor types, a subset of these markers can be selected a priori for gene selection in microarray data, and complemented with genes selected from the specific study.
this strategy will substantially reduce the amount of over-learning that occurs when genes/features are selected from the same dataset that is used for classification.
further, patient diagnostics will benefit from the knowledge that a small set of genes are predictive for a large array of cancers, because molecular tools that are more sensitive and accurate than microarrays  can be developed and used with small gene lists.
CONCLUSIONS
we have presented a new approach to mining expressed sequence tags to discover cancer transcriptional markers.
we validated this new approach with a series of tests, including false discovery rate analysis.
we used this new approach to identify a set of  <dig> cancer biomarker genes  from human and mouse est data.
we further validated hm <dig> with data from  <dig> publicly available microarray datasets and showed that, as a set, hm <dig> genes were effective in predicting conditions of clinical interest.
the microarray validation protocol which we describe in this manuscript helps compare the predictive ability of a given gene list against previously published gene lists and helps assess the level of generality and robustness of the gene list under study across a variety of conditions.
with this protocol, we showed that hm <dig> is a general cancer biomarker set which is predictive across a variety of tumor types and clinical conditions.
the manuscript also presents a pathway analysis of genes in hm <dig> which suggests that genes in hm <dig> are under transcriptional control of pathways known to be deregulated in tumorigenesis.

 endogenous biomarkers are molecules whose levels in a tissue or fluid of an organism correlate with the presence of a given disease. for instance, the prostate specific antigen is an fda-approved endogenous biomarker for prostate cancer. identifying biomarkers for specific cancer types is expected to lead to the development of early diagnostic methods that can reduce morbidity and mortality. here, we show that cancer biomarkers can be discovered by mining human and mouse expressed sequence tags. furthermore, we show that biomarkers identified in this way perform well over the wide range of tumor types and classifications used in microarray studies. our results demonstrate the existence of a small group of genes whose expression levels are affected in a wide array of cancers. various approaches have been used to search for cancer biomarkers. methods that mine gene expression include serial analysis of gene expression, microarray analysis involving the comparison of tumor samples versus normal tissues, large scale meta analysis of cancer microarray data, or massively parallel signature sequencing. expressed sequence tags such as found in dbest are another abundant source of gene expression data. we have previously described tissueinfo, an approach to determine whole genome tissue expression profiles using data in dbest. in this article, we asked whether est data could be mined to identify cancer biomarkers.  mining dbest we extended tissueinfo to mine ests for genes that are differentially expressed in non-tumor versus tumor tissues. figure presents how many times human transcripts appear in tumor versus non-tumor tissues. the slope of the line that best fit the points is a consequence of how many est libraries are available in tumor and non-tumor samples. the variation around the regression line, however, reflects both the random sampling effect of est sequencing and the differential regulation of the genes between the tumor and non tumor conditions. the scatter observed when considering all transcripts can be considerably reduced when focusing on transcripts preferentially expressed in a given tissue. this noise reduction technique is central to the est mining approach presented in this manuscript. after filtering, we used a two-tailed fisher exact test to assign a p-value to each transcript and quantify the likelihood of differential expression between tumor and non-tumor. human-mouse comparison we mined mouse ests in the same way and compared the results to those obtained with the human data. ortholog information was obtained from ensembl. plotting the p-values obtained for human and mouse orthologs, we confirmed that the tissueinfo cancer p-value is uniformly distributed in regions where the p-value is not significant. further, we asked if selecting genes in human by p-value provided more overlap with mouse than would be expected by chance if p-values were distributed randomly in the human/mouse ortholog gene set. we selected human genes and mouse genes and observed an overlap of genes. randomly selecting genes and genes in the human/mouse ortholog gene set yielded overlaps of ~. the difference is significant, indicating that the overlap observed cannot be explained by chance alone. these data are a strong indication that our approach works consistently in the two organisms, despite important differences in tissue and est availability or cancer biology. the list of genes is annotated and provided in supplementary table. the references cited in the table show that most genes on this list have been studied and found to be differentially regulated in cancer. interestingly, of the genes that both our mouse and human screens detect at high stringency, there is only one gene in common with the perou-brown-botstein proliferation cluster. this indicates that these genes common between human and mouse are not the usual markers of proliferation and poor differentiation. false discovery rate to confirm that small p-values indicate transcripts that are differentially expressed between tumor and non-tumor tissues, we estimated the false discovery rate, a statistical measure used to estimate the bias introduced by multiple hypothesis testing. the false discovery rate is the ratio fdr = fp/, where fp is the number of false positives detected below a threshold and tp is the number of true positives. to calculate fdr, fp and tp need to be known. we estimated fp as the number of genes selected in a control experiment. we estimated tp by comparing the control to the tumor vs. non-tumor data. figure compares the number of transcripts found below various thresholds of the p-value in the data and in the control experiment, and plots 1-fdr as a function of the exponent of the p-value threshold. figure indicates that combining p-values for the human and mouse screens lowers the false discovery rate of our approach, from about 30% with the human data only to about 20% when human and mouse data are combined. hm200 we focused on human/mouse orthologous genes and selected the genes that were ranked best by the combined human/mouse p-value phm, where phm = ph × pm. we called this gene set hm each gene which is part of hm has an orthologous counterpart in human or mouse, and has been selected because of differential expression in human cancer, mouse cancer, or both, as measured by our approach. from figure, we estimate that over 78% of the genes in hm are truly differentially expressed between tumor and non-tumor tissues. in contrast, if one were to randomly select genes from the genome to identify genes truly significantly differentially expressed between tumor and non-tumors, the expected precision of the task would be %. using our approach with the combined human and mouse dbest data is therefore times more precise than a random selection. hm is enriched in cancer-associated genes we asked if genes in hm were more likely to be oncogenes or tumor suppressor genes. to address this question, we counted how many genes in hm overlap with a set of genes previously studied for their role in oncogenesis. we found that % of genes in hm overlap with this list, a number to be compared to % in the whole human/mouse ortholog gene set. the difference is significant, and comparable to the enrichment recently reported by aouacheria and colleagues when identifying genes bearing cancer-associated nonsynonymous snps in dbest. hm as predictive biomarkers although hm genes were identified in est data, we reasoned that they should be accurate predictors in cancer microarray data sets. testing hm on microarray data is an independent test of their biomarker abilities. this is a strong test because the only common point between microarray and expressed sequence tags is that these technologies measure gene expression. therefore, to determine whether hm genes could be used as cancer biomarkers in human, we asked how well hm compared to other gene lists in a variety of microarray cancer classification tasks. lists of gene markers have been derived in many microarray studies for different types of cancer and we collected such gene lists from published studies. because gene lists vary in the number of genes that they contain, in addition to the hm gene list, we evaluated hm hm and hm gene lists with the top and genes from hm respectively. we also used a negative control gene list nc01- of genes with high tissueinfo cancer p-values. although the complete list of genes on an array is not a practical biomarker list, we included the full set of genes found on the array used for each study as a positive control. in total, our evaluation compared the predictive performance of gene lists over different microarray datasets. we included in this study all the microarray datasets that we could obtain. the number of gene lists included in the study was limited only by the computational cost of performing the evaluation. microarray studies of cancer frequently compare two different conditions. in some studies, one condition is a tumor and the other the normal tissue. in other studies, conditions are derived from cytology, and tumor grades are used as conditions. the most interesting studies define conditions from data obtained about the patient after the tumor sample has been taken. the microarray datasets collected for this study include at least one representative of each of these types of classifications. some microarray datasets are annotated such that multiple different classification tasks can be performed with them. for instance, in the pomeroy dataset, five classifications can be performed: normal cerebellum/medulloblastoma, normal/malignant glioma, normal/atypical teratoid or rhabdoid tumor, normal/primitive neuroectodermal tumor, and medulloblastoma/malignant glioma. in such cases, we included each possible classification if this classification had at least three patients in each class. supplementary table lists the classifications considered in this study. in our evaluation, the number of gene lists, datasets and classification conditions combine to produce different cancer classification tasks. we evaluate the leave-one-out accuracy of each gene list on each classification task, filter out classification tasks for which the p-value of the label permutation test exceeds, and then rank gene lists in each dataset from highest to lowest accuracy. briefly, a gene list obtains rank on a classification task if it yielded the highest accuracy for that task among the gene lists included in the evaluation. figure presents a tally of the rank for each gene list in our evaluation.. supplementary table shows the statistical data used to build figure. the classification tasks measure two properties of each gene list: robustness and generality. we define robustness as the property of a biomarker set to perform well on independent datasets obtained for the same tumor type. robustness should not be confused with leave-one-out performance, which estimates how well a predictor can generalize using information from a single training set. robustness is a more important property than leave-one-out performance for clinical applications because any training set is unlikely to sample most of the variability found in the patient population. thus, a good performance with one training set may not translate to another training set. for example, when we classify the pomeroy dataset using the gds gene list, we test the robustness of the gds gene list because both datasets include data from patients with medulloblastoma, and patients for these studies were recruited independently. these tests are performed to measure robustness. in contrast to robustness, we define generality as the property of a biomarker set to perform well on a tissue, tumor type or classification task other than the one it was trained for. a perfect biomarker set is one that would be both robust and general. in the test presented in figure such a biomarker set would appear with one peak only, at rank and the height of the peak would coincide with the total number of classifications performed in the evaluation. in contrast, the peaks of sets of genes unrelated to cancer should be randomly distributed across ranks and show no preference for the better ranks. our negative control gene list nc01- shows precisely this behavior. gene lists obtained from published studies generally show a skewed distribution towards the best ranks. we observe that hm and hm present the most skewed distributions of the cancer marker gene lists. given the range of classification tasks, tissue and tumor types sampled in our evaluation, figure indicates that the hm and hm marker sets are both robust and general cancer marker sets over the range of cancer classifications evaluated. prior comparisons of est data and microarray data have confirmed that these two approaches to measuring gene expression have different strengths and drawbacks. it is therefore notable to see that both est and microarray data indicate that the genes in the hm gene list are differentially expressed in cancer and can predict cancer-related conditions. this clearly shows that the hm gene list is not random and that our approach to mining expressed sequence tags is effective. hm are mostly novel biomarkers we asked if genes in hm had been recognized before as biomarkers. we calculated the overlap between genes in hm and the lists used in our evaluation. numbers of genes overlapping between hm and these gene lists are shown in table with a gene list similarity measure. see methods for calculations and a discussion of the validity of the pseudo p-values shown in table table shows that three of the previously reported cancer marker gene lists overlap more significantly with hm than others. these lists are pomeroy-400-classic-desmoplastic, pomeroy-class-, and gds. interestingly, these three lists also showed better generality when tested across datasets than most other gene lists in the rank evaluation test shown in figure this shows that although hm genes have been identified by pomeroy and colleagues as central nervous system tumor markers, most genes in hm had not been previously recognized as cancer markers as a group. table also includes a comparison between hm and the proliferation cluster described by perou, brown, botstein and colleagues in. the comparison identifies only two genes in common between hm and the proliferation cluster the main contribution of our study is to show that a small group of genes are predictive cancer markers that outperform other cancer marker lists over different tumor types and classification tasks. these results demonstrate the existence of a set of genes whose transcription levels are perturbed in a variety of tumors types, irrespective of the tissue of origin of the tumor. hm and cancer pathways since the products of genes in hm are differentially expressed in tumor and non-tumor and tumor grade or tumor evolution, we hypothesized that hm would be likely to interact with genes shown to be altered in tumorigenesis. to test this hypothesis, we queried the ingenuity pathways knowledge base with accession codes for protein products of hm the ingenuity pathway database is a curated database that provides access to million protein-protein and gene-protein interactions and offers tools that cluster genes into interaction networks. supplementary table and supplementary figures 1– present gene networks identified when hm genes are used as focus genes. documentation available on the ingenuity web site does not describe how the score is calculated, but indicates that the p-value of observing a network by chance is estimated to be 110-score. the networks shown on supplementary table are thus unlikely to have been generated by chance. genes shown in bold or in gray are members of hm notably, network contains the cancer-related genes fos, fosb, fosl and junb; network contains mycn, bcl egr; network contains kinesin 5a, 5b, and 5c; network contains myc, network contains the v-akt murine thymoma viral oncogene homolog; network contains the retinoblastoma protein and network contains the p protein; network contains the peroxisome proliferator-activated receptor γ and the granulocyte differentiation factor c/ebpα. following this observation, we tested if genes found in these networks that are not hm biomarkers are enriched in cancer-related genes. of the genes in the best networks shown in supplementary table that are not part of hm % overlap with cancer-related genes. in contrast, the cancer-related genes represent % of human genes with a hugo gene name. the difference is significant. interestingly, the overlap with cancer-related genes is significantly higher for genes known to interact with genes in hm than for genes in hm themselves. this supports our hypothesis that hm genes are interacting directly with genes involved in tumorigenesis and that their transcripts are markers of cancer-related signaling alterations. the cancer-related genes themselves may not be transcriptionally regulated, but expression of downstream genes such as those found in hm is. hm is enriched in mrna binding proteins we performed a gene ontology classification screen and found genes in hm to be significantly enriched in mrna binding proteins. this is not surprising because there is growing evidence that deregulation of protein translation may participate in malignant transformation. the mechanisms by which tumor cells selectively deregulate protein synthesis are still unclear. therefore, the identification of genes that may participate in selective protein synthesis deregulation is of great interest. the networks provided as supplementary figures 1– provide candidates for such genes. for instance, supplementary figure shows that seven genes in hm are connected to jerky. the number of connections between jrk and hm genes is comparable to the number of connections between p and hm this is surprising because jrk was first studied as a temporal lobe epilepsy gene candidate. more recent studies indicate that jerky is an mrna binding protein that binds mrna through a cenp-b domain. jerky shows selectivity in the set of mrna molecules that it binds, and may regulate the availability of mrnas to the translational machinery. the cenp-b domain is found in nine other human proteins. the conservation of this domain suggests that some of these other proteins may also bind mrna. however, the ability of these other proteins to bind mrna selectively remains to be determined. we predict that jerky is involved in tumorigenesis by selectively deregulating protein translation. this hypothesis is supported by the presence of the fragile mental retardation protein on network. fmr binds mrnas selectively and is believed to regulate their translation, thereby regulating the protein expression of specific genes. fmr binds mrnas through kh motifs. these motifs are also found in nova- a biomarker for paraneoplastic opsoclonus-myoclonus ataxia disorder. this discussion illustrates how interaction networks containing hm genes can be used to develop new molecular hypotheses for cancer biology. potential issues when mining ests we present and validate a new approach to identifying cancer biomarkers by mining expressed sequence tags. because expressed sequence tags are a vastly heterogeneous type of data, any approaches that mine this source of data need to account for several issues. many of these issues were discussed in our previous publication and will not be repeated here. however, it is worth noting that the evaluation that we conducted in found that tissueinfo was accurate in identifying if a gene was expressed in a tissue in 80–89% of cases. a new issue specific to mining ests for cancer markers is that dbest contains different proportions of est libraries from tumor and non-tumor tissues. this is clearly illustrated in figure panel "all tissues", where most transcripts are found to match double the ests in non-tumor tissues than in tumor tissues. in our new approach, we have addressed this issue in two ways. first, we identify groups of genes that are mostly expressed in a given tissue/organ or tissue type. second, we calculate a fisher exact test p-value which takes into account the number of times a given transcript matches an est in tumor, and in non-tumor, and the overall proportion of tumor/non-tumor ests for the tissue the transcript is mostly expressed in. this test adequately corrects for the tissues with varied proportions of ests in tumor and non-tumor tissues, and at the same time assigns significance as a function of the number of est matches for the transcript. the second factor is important because we have shown that tissueinfo identifications are more accurate when a transcript matches more ests. the false discovery rate analysis that we present in figure indicates that despite these various potential issues, our approach was able to identify genes differentially expressed in tumor/non-tumor tissues with about 80% precision. additional independent validations, such as the overlap with cancer gene lists, the overlap between human and mouse predictions, and most importantly the large microarray evaluation study that we conducted, support these conclusions. microarray validation confirms est mining the microarray evaluation study that we present in this manuscript is a large study by the current standards of the field. indeed, this study considers a test set of microarray datasets, when most marker discovery studies test a set of markers in one or two independently collected datasets. an exception is the study of rhodes et al, where datasets were used for independent validation furthermore, our validation study considers the set of markers that we identified in competition with previously published genes lists, one negative control and a positive control. this contrasts with most published studies where single lists of markers are typically evaluated in isolation. we used a stringent permutation test to evaluate the impact of the number of samples in each condition and intra-dataset correlations to the performance observed in a classification. the negative control gene list is not expected to perform at a statistically significant level and indeed, most of its classifications are filtered out by the permutation test. applying this stringent protocol to the comparison of multiple marker lists on different datasets and classification tasks revealed that some marker lists are more general than others. to the best of our knowledge, the generality of cancer marker lists has not been studied before. hm achieves the best or second best classification in 79% of the classifications considered. further, we report that one previously published list also shows good generality across tumor types. our observation that some marker lists are predictive over a wide range of tumor types and classification types strongly suggests the existence of a subset of genes under the transcriptional control of pathways consistently deregulated in different cancer types.. our results also suggest that new cancer marker studies should report on the generality of the markers that they identify because results obtained with general markers are more likely to be reproducible. hm applications we anticipate that the discovery of the hm cancer marker gene set and the characterization of its predictive utility across tumor types will open new strategies for the identification of genes whose alterations are required steps in tumorigenesis. also, since hm markers are predictive across tumor types, a subset of these markers can be selected a priori for gene selection in microarray data, and complemented with genes selected from the specific study. this strategy will substantially reduce the amount of over-learning that occurs when genes/features are selected from the same dataset that is used for classification. further, patient diagnostics will benefit from the knowledge that a small set of genes are predictive for a large array of cancers, because molecular tools that are more sensitive and accurate than microarrays can be developed and used with small gene lists.  we have presented a new approach to mining expressed sequence tags to discover cancer transcriptional markers. we validated this new approach with a series of tests, including false discovery rate analysis. we used this new approach to identify a set of cancer biomarker genes from human and mouse est data. we further validated hm with data from publicly available microarray datasets and showed that, as a set, hm genes were effective in predicting conditions of clinical interest. the microarray validation protocol which we describe in this manuscript helps compare the predictive ability of a given gene list against previously published gene lists and helps assess the level of generality and robustness of the gene list under study across a variety of conditions. with this protocol, we showed that hm is a general cancer biomarker set which is predictive across a variety of tumor types and clinical conditions. the manuscript also presents a pathway analysis of genes in hm which suggests that genes in hm are under transcriptional control of pathways known to be deregulated in tumorigenesis. 
these results indicate the existence of at least one general cancer marker set whose predictive value spans several tumor types and classification types. we also identify the previously published pomeroy- list as another general cancer marker set. gene expression data are a rich source of information about the transcriptional dis-regulation of genes in cancer. strikingly, pomeroy- has genes in common with hm our data suggest that a core set of genes are responsive to the deregulation of pathways involved in tumorigenesis in a variety of tumor types and that these genes could serve as transcriptional cancer markers in applications of clinical interest. finally, our study suggests new strategies to select and evaluate cancer biomarkers in microarray studies. indeed, when compared to published cancer marker gene lists, hm achieves the best or second best classification performance in 79% of the classifications considered. genes that display differential regulation in cancer are a subtype of cancer biomarkers. we present an approach to mine expressed sequence tags to discover cancer biomarkers.
 gene expression data are a rich source of information about the transcriptional dis-regulation of genes in cancer. genes that display differential regulation in cancer are a subtype of cancer biomarkers.  we present an approach to mine expressed sequence tags to discover cancer biomarkers. a false discovery rate analysis suggests that the approach generates less than 22% false discoveries when applied to combined human and mouse whole genome screens. with this approach, we identify the genes most consistently differentially expressed in cancer and proceed to characterize these genes. when used for prediction in a variety of cancer classification tasks, we show that hm and the shorter gene list hm are very competitive cancer biomarker sets. indeed, when compared to published cancer marker gene lists, hm achieves the best or second best classification performance in 79% of the classifications considered.  these results indicate the existence of at least one general cancer marker set whose predictive value spans several tumor types and classification types. our comparison with other marker gene lists shows that hm markers are mostly novel cancer markers. we also identify the previously published pomeroy- list as another general cancer marker set. strikingly, pomeroy- has genes in common with hm our data suggest that a core set of genes are responsive to the deregulation of pathways involved in tumorigenesis in a variety of tumor types and that these genes could serve as transcriptional cancer markers in applications of clinical interest. finally, our study suggests new strategies to select and evaluate cancer biomarkers in microarray studies. 