BACKGROUND
high throughput genomics technologies generate a vast amount of data.
determining the biologically and clinically significant findings of an experiment can be a daunting task.
applying functional knowledge to genomic data is one method that has been used to reduce data complexity and establish biologically plausible arguments.
these methods rely on a priori definition of gene sets, and the results necessarily depend on the strength of the annotations  <cit> .
genome-wide annotation of gene function has garnered much attention and the comprehensive gene ontology  consortium annotations are widely used  <cit> .
few tools based on ontology are available for annotating genome-wide data with disease associations.
the lack of ontology based disease annotation prevents the application of disease knowledge to genomic data, therefore hindering the discovery of gene-disease associations from high throughput genomics technologies.
online mendelian inheritance in man , curated by the ncbi and johns hopkins university, is arguably the most widely used disease gene annotation database.
although the curation process provides highly detailed annotation and minimizes errors, there is a noticeable delay in updating.
furthermore, the vocabulary of omim is predominately text based, far from comprehensive, and is difficult to use  <cit> .
it is simply not possible to download a list of diseases from omim and users have resorted to mining the clinical synopsis free text section of omim for disease discovery  <cit> .
it also focuses on genetic diseases with classic mendelian inheritance, thus eliminating the wide range of diseases resulting from more complicated environmental and genetic interactions.
another source of gene-disease mappings from linkage studies is the "genetic association database"   <cit>  which aims to "collect, standardize and archive genetic association study data".
the structure of the classification system gad used to classify its diseases is not apparent.
diseases are classified in  <dig> broad classes including an "other" class , an unknown number of broad phenotype classes below those and a further number of narrow phenotype classes.
this lack of apparent ontology makes it hard to determine the number and types of diseases gad contains.
for example, searching for "crohn's disease" returns  <dig> results but searching for "regional enteritis" returns no results.
researchers have also used abstracts and titles from medline as a data source for inferring gene-disease associations  <cit> .
although a current and rich source of information, the free text form of medline abstracts presents difficulties for determining the context of the association between gene and disease  <cit> .
this is particularly true when genes are identified by semantically ambiguous gene symbols which may or may not apply to a disease recognized in free text.
for instance cat can refer to the catalase gene or a feline animal, depending on the context.
a generif  is a brief  annotation to a gene in the ncbi database and contains gene specific information including disease associations.
these entries are modifiable by ncbi users willing to provide their email address.
such a wiki-type of resource offers low mapping error of gene symbols and allows a rapid update by the research community  <cit> .
despite this utility, generif has been infrequently considered as a data source for text mining, evidenced by the fact that only six papers indexed in pubmed contain the term "generif".
one of these describes a data mining tool called milano, which counts occurrences of each generif annotated gene with user-defined terms selected from medical subject headings  and includes some disease terms  <cit> .
the authors found generif superior to medline, pubmatrix, bear geneinfo, and microgenie, for identifying p <dig> affected genes.
a more recent approach using conditional random fields to map a test set of generifs to mesh terms further validates generifs as a comprehensive data source  <cit>  for the human genome annotation.
to provide a comprehensive disease to gene annotation, we used the disease ontology   <cit>  to identify relevant diseases in generifs.
the disease ontology is a manually inspected subset of unified medical language system  and includes concepts from outside the umls disease/disorder semantic network including various cancers, congenital abnormalities, deformities and mental disorders.
while many researchers have mapped diseases to mesh terms  <cit>  or omim  <cit>  the disease ontology is larger and should therefore provide greater disease coverage.
the hierarchal structure also allows more general disease terms to be distinguished from subclasses, in order to account for "over-mapping" of disease terms to a textually larger database.
we used a thesaurus based approach  for analyzing generifs with demonstrated success in studying clinically relevant terms  <cit> .
RESULTS
mapping genes to disease ontology
the process for mapping disease ontology  terms to generifs is illustrated in figure 1a.
the disease ontology is a disease-focused comprehensive subset of unified medical language system  and outside terms structured as a directed acyclic graph, similar to the structure of the gene ontology  from the go consortium.
metamap transfer tool  was used to map the do to generifs.
these mappings are stored in the open biomedical ontologies  format and can be manually edited using the open source graph editor dagedit .
an example gene-disease association is shown in figure 1b.
the generif entry for tgfb <dig> links the gene to the do term.
the related disease ontology terms are also provided for the mapped do term.
in cases where multiple mappings of a gene are possible along a branch of the disease ontology tree, genes are mapped to the most specific disease term.
disease ontology annotations of a human gene describe unique roles for genes in the context of disease, and are complementary to gene ontology annotations.
the gene-disease mapping for atp7b is provided as an example in figure  <dig>  the gene description is provided, along with do, omim, and go annotations.
this example demonstrates that generif results in more disease associations than omim for this gene.
also note that the do annotation uses a formal vocabulary of "hepatolenticular degeneration" instead of "wilson disease".
to assess the gene-disease associations we identify, graphs illustrating the mapping of single genes to diseases as well as single diseases to genes are presented in figure  <dig>  similar results from omim are reported on the graph as a point of reference.
plots of the number of diseases per gene  and the number of genes per disease  suggest the depth and coverage of do annotation is higher than omim.
in addition, both the number of diseases per gene and the number of genes per disease demonstrate scale-free properties, which have been observed in both biological networks and citation networks  <cit> .
as such, there is no 'mode' or 'scale' as observed in a gaussian distribution; the number of diseases per gene spreads from  <dig> to  <dig>  the gene with the greatest number of disease associations is interleukin  <dig> .
moreover, the result from the do annotation suggests a 'rich get richer' phenomenon: the top  <dig>  genes are implicated in  <dig>  diseases.
not surprisingly, several cancers including breast, prostate, liver, colon, and metastatic lesions have a larger number of genes associated with them .
other non-cancer diseases implicated with many genes include other complex diseases including diabetes and rheumatoid arthritis.
when we look at the genes associated with many diseases, the interleukins including il1b, il <dig>  il <dig>  and il <dig> are near the top of the list .
biologically, this is likely due to the fact that inflammation is a common pathological consequence.
in addition, matrix metalloproteinases mmp <dig> and mmp <dig>  the cell growth and cell cycle regulators cdkn2a, bcl <dig>  and egfr, and methylentetrahydrofolate reductase  genes are among the top genes associated with numerous disease conditions.
performance evaluation
for evaluation, we use two commonly used performance metrics in textual data retrieval, which are defined as follows:
from these formulas, we can see they are closely related to false positive and false negative rates that are used in other fields.
a recall rate of 100% and a precision rate of 100% are of ideal situations.
for disease annotation, we constructed a truth table of the homayouni gene collection  <cit>  manually using generif and omim text as a source.
for the homayouni gene collection, there are  <dig> generifs, with an average of  <dig>  generifs per gene and a median of  <dig>  on average our algorithm maps  <dig> % of generifs to a disease, with recall and precision rates of  <dig> % and  <dig> % respectively .
mapping the homayouni gene collection to omim, however, results in a recall rate of  <dig> % and precision of  <dig> %.
since omim is a curated database, the slightly higher precision rate for omim compared to generif is not surprising.
however, this small loss in precision by using omim is accompanied by a dramatically reduced recall rate.
the homayouni gene collection was used to estimation of recall and precision of gene mappings to the disease ontology.
a network visualization based on the do annotation of the human genome
to illustrate the connected nature of gene-disease mappings, we plot the genes associated with any of the four well-studied cancers  and show the gene-disease relationships from our analysis .
each dark grey dot represents one gene, and  <dig> genes are annotated to ovarian cancer,  <dig> genes are annotated to breast cancer,  <dig> genes are annotated to neuroblastoma, and  <dig> genes are annotated to multiple myeloma.
the diseases are denoted using large light grey dots and the size of the dot is proportional to the number of genes connecting to it.
the shaded circle at the center of the figure highlights the  <dig> genes related to all of four diseases .
the functions of these genes include cell cycle regulation, apoptosis, growth factor signaling, and extracellular matrix remodeling.
although these pathways are well known to play a role in cancer initiation or progression, the identification of this specific set of genes may be useful for researcher interested in identifying targets common to these four cancers in particular.
discussion
the disease ontology consists of a manually inspected subset of umls and terms outside the umls disease and disorder semantic network including various cancers, congenital abnormalities, deformities and mental disorders that are important to researchers trying to understand the genetic and molecular basis of a particular disease.
therefore, compared to umls, the disease ontology is much larger in size and more specific to the disease of interest.
it therefore offers greater disease coverage with improved accuracy.
in addition, its hierarchal structure allows a more specific disease term to be binned to a more general disease term at different levels which is especially useful for disease ontology enrichment analysis analogous to gene ontology enrichment analysis in experiments applying high throughput technologies.
our results indicate that generifs are an excellent data source for discovering disease-gene relationships.
this is primarily due to the large number of generifs relative to omim entries, and the surprisingly high  frequency of disease references.
the disease coverage of omim would be improved if the free text had been mined, but only  <dig> genes in omim have a clinical synopsis section and limiting our analysis to these entries would bias our results.
using the clinical synopsis section in addition to other omim free text would increase the number of false positives since omim free text frequently includes diseases without a direct relationship to the gene, usually for comparative purposes or in reference to experiments in model organisms.
errors in our method may arise from a variety of sources including problems with mmtx, many of which have already been elucidated  <cit> .
the problem of having disease terms present in omim or generif, but missing in do or umls was infrequent but did include some cases such craniofacial-deafness-hand syndrome.
a more significant problem contributing to the majority of false positives was the discovery of disease terms in generif that indicated only a partial, ambiguous or no association to the gene in question.
fortunately, the succinctness of generif means that this occurs less frequently than in abstracts  which may contain diseases not directly related to the gene.
we found only one incorrectly assigned generif in the  <dig> generifs examined, indicating that this is a minor source of error.
one result from our analysis is that omim performs poorly relative to generif with newly discovered mappings.
a fairly typical case is the alpha-2-marcroglobuin gene.
while omim includes mappings for alzheimer's disease and pulmonary emphysema , it excludes potential links to benign prostatic hyperplasia, multiple sclerosis and argyrophilic grain disease.
this may be a result of omim's stronger requirement for evidence, but failure to keep pace with current research may also contribute.
we annotated the human genome with disease ontology and reported its performance.
such an annotation will enable many graphical and statistical applications similar to previously what has been done with gene ontology annotations.
an example of this is presented in figure  <dig>  where all the genes in the human genome with established links to four cancers led to the identification of eleven genes in common between these four cancers.
this analysis facilitates identification of relevant targets or markers for diseases with common etiology or pathology, and has implications for biological plausibility as well as therapeutic potential.
although previous studies have demonstrated the utility of this approach, the improved coverage and accuracy of our analysis provide even greater potential  <cit> .
our future plans include developing a web-enabled database application of the disease ontology for the research community .
CONCLUSIONS
similar to the go annotation, we provide a do annotation of the human genome; each annotation is supported by a peer-reviewed publication as required by generif.
it enables researchers to study gene-disease relationships computationally.
the do annotation of the human genome is available in both tab-delimited format and relational database format , which allows them to be easily adapted for other applications.

 high throughput genomics technologies generate a vast amount of data. determining the biologically and clinically significant findings of an experiment can be a daunting task. applying functional knowledge to genomic data is one method that has been used to reduce data complexity and establish biologically plausible arguments. these methods rely on a priori definition of gene sets, and the results necessarily depend on the strength of the annotations. genome-wide annotation of gene function has garnered much attention and the comprehensive gene ontology consortium annotations are widely used. few tools based on ontology are available for annotating genome-wide data with disease associations. the lack of ontology based disease annotation prevents the application of disease knowledge to genomic data, therefore hindering the discovery of gene-disease associations from high throughput genomics technologies. online mendelian inheritance in man, curated by the ncbi and johns hopkins university, is arguably the most widely used disease gene annotation database. although the curation process provides highly detailed annotation and minimizes errors, there is a noticeable delay in updating. furthermore, the vocabulary of omim is predominately text based, far from comprehensive, and is difficult to use. it is simply not possible to download a list of diseases from omim and users have resorted to mining the clinical synopsis free text section of omim for disease discovery. it also focuses on genetic diseases with classic mendelian inheritance, thus eliminating the wide range of diseases resulting from more complicated environmental and genetic interactions. another source of gene-disease mappings from linkage studies is the "genetic association database" which aims to "collect, standardize and archive genetic association study data". the structure of the classification system gad used to classify its diseases is not apparent. diseases are classified in broad classes including an "other" class, an unknown number of broad phenotype classes below those and a further number of narrow phenotype classes. this lack of apparent ontology makes it hard to determine the number and types of diseases gad contains. for example, searching for "crohn's disease" returns results but searching for "regional enteritis" returns no results. researchers have also used abstracts and titles from medline as a data source for inferring gene-disease associations. although a current and rich source of information, the free text form of medline abstracts presents difficulties for determining the context of the association between gene and disease. this is particularly true when genes are identified by semantically ambiguous gene symbols which may or may not apply to a disease recognized in free text. for instance cat can refer to the catalase gene or a feline animal, depending on the context. a generif is a brief annotation to a gene in the ncbi database and contains gene specific information including disease associations. these entries are modifiable by ncbi users willing to provide their email address. such a wiki-type of resource offers low mapping error of gene symbols and allows a rapid update by the research community. despite this utility, generif has been infrequently considered as a data source for text mining, evidenced by the fact that only six papers indexed in pubmed contain the term "generif". one of these describes a data mining tool called milano, which counts occurrences of each generif annotated gene with user-defined terms selected from medical subject headings and includes some disease terms. the authors found generif superior to medline, pubmatrix, bear geneinfo, and microgenie, for identifying p affected genes. a more recent approach using conditional random fields to map a test set of generifs to mesh terms further validates generifs as a comprehensive data source for the human genome annotation. to provide a comprehensive disease to gene annotation, we used the disease ontology to identify relevant diseases in generifs. the disease ontology is a manually inspected subset of unified medical language system and includes concepts from outside the umls disease/disorder semantic network including various cancers, congenital abnormalities, deformities and mental disorders. while many researchers have mapped diseases to mesh terms or omim the disease ontology is larger and should therefore provide greater disease coverage. the hierarchal structure also allows more general disease terms to be distinguished from subclasses, in order to account for "over-mapping" of disease terms to a textually larger database. we used a thesaurus based approach for analyzing generifs with demonstrated success in studying clinically relevant terms.  mapping genes to disease ontology the process for mapping disease ontology terms to generifs is illustrated in figure 1a. the disease ontology is a disease-focused comprehensive subset of unified medical language system and outside terms structured as a directed acyclic graph, similar to the structure of the gene ontology from the go consortium. metamap transfer tool was used to map the do to generifs. these mappings are stored in the open biomedical ontologies format and can be manually edited using the open source graph editor dagedit. an example gene-disease association is shown in figure 1b. the generif entry for tgfb links the gene to the do term. the related disease ontology terms are also provided for the mapped do term. in cases where multiple mappings of a gene are possible along a branch of the disease ontology tree, genes are mapped to the most specific disease term. disease ontology annotations of a human gene describe unique roles for genes in the context of disease, and are complementary to gene ontology annotations. the gene-disease mapping for atp7b is provided as an example in figure the gene description is provided, along with do, omim, and go annotations. this example demonstrates that generif results in more disease associations than omim for this gene. also note that the do annotation uses a formal vocabulary of "hepatolenticular degeneration" instead of "wilson disease". to assess the gene-disease associations we identify, graphs illustrating the mapping of single genes to diseases as well as single diseases to genes are presented in figure similar results from omim are reported on the graph as a point of reference. plots of the number of diseases per gene and the number of genes per disease suggest the depth and coverage of do annotation is higher than omim. in addition, both the number of diseases per gene and the number of genes per disease demonstrate scale-free properties, which have been observed in both biological networks and citation networks. as such, there is no'mode' or'scale' as observed in a gaussian distribution; the number of diseases per gene spreads from to the gene with the greatest number of disease associations is interleukin. moreover, the result from the do annotation suggests a'rich get richer' phenomenon: the top genes are implicated in diseases. not surprisingly, several cancers including breast, prostate, liver, colon, and metastatic lesions have a larger number of genes associated with them. other non-cancer diseases implicated with many genes include other complex diseases including diabetes and rheumatoid arthritis. when we look at the genes associated with many diseases, the interleukins including il1b, il il and il are near the top of the list. biologically, this is likely due to the fact that inflammation is a common pathological consequence. in addition, matrix metalloproteinases mmp and mmp the cell growth and cell cycle regulators cdkn2a, bcl and egfr, and methylentetrahydrofolate reductase genes are among the top genes associated with numerous disease conditions. performance evaluation for evaluation, we use two commonly used performance metrics in textual data retrieval, which are defined as follows: from these formulas, we can see they are closely related to false positive and false negative rates that are used in other fields. a recall rate of 100% and a precision rate of 100% are of ideal situations. for disease annotation, we constructed a truth table of the homayouni gene collection manually using generif and omim text as a source. for the homayouni gene collection, there are generifs, with an average of generifs per gene and a median of on average our algorithm maps % of generifs to a disease, with recall and precision rates of % and % respectively. mapping the homayouni gene collection to omim, however, results in a recall rate of % and precision of %. since omim is a curated database, the slightly higher precision rate for omim compared to generif is not surprising. however, this small loss in precision by using omim is accompanied by a dramatically reduced recall rate. the homayouni gene collection was used to estimation of recall and precision of gene mappings to the disease ontology. a network visualization based on the do annotation of the human genome to illustrate the connected nature of gene-disease mappings, we plot the genes associated with any of the four well-studied cancers and show the gene-disease relationships from our analysis. each dark grey dot represents one gene, and genes are annotated to ovarian cancer, genes are annotated to breast cancer, genes are annotated to neuroblastoma, and genes are annotated to multiple myeloma. the diseases are denoted using large light grey dots and the size of the dot is proportional to the number of genes connecting to it. the shaded circle at the center of the figure highlights the genes related to all of four diseases. the functions of these genes include cell cycle regulation, apoptosis, growth factor signaling, and extracellular matrix remodeling. although these pathways are well known to play a role in cancer initiation or progression, the identification of this specific set of genes may be useful for researcher interested in identifying targets common to these four cancers in particular. discussion the disease ontology consists of a manually inspected subset of umls and terms outside the umls disease and disorder semantic network including various cancers, congenital abnormalities, deformities and mental disorders that are important to researchers trying to understand the genetic and molecular basis of a particular disease. therefore, compared to umls, the disease ontology is much larger in size and more specific to the disease of interest. it therefore offers greater disease coverage with improved accuracy. in addition, its hierarchal structure allows a more specific disease term to be binned to a more general disease term at different levels which is especially useful for disease ontology enrichment analysis analogous to gene ontology enrichment analysis in experiments applying high throughput technologies. our results indicate that generifs are an excellent data source for discovering disease-gene relationships. this is primarily due to the large number of generifs relative to omim entries, and the surprisingly high frequency of disease references. the disease coverage of omim would be improved if the free text had been mined, but only genes in omim have a clinical synopsis section and limiting our analysis to these entries would bias our results. using the clinical synopsis section in addition to other omim free text would increase the number of false positives since omim free text frequently includes diseases without a direct relationship to the gene, usually for comparative purposes or in reference to experiments in model organisms. errors in our method may arise from a variety of sources including problems with mmtx, many of which have already been elucidated. the problem of having disease terms present in omim or generif, but missing in do or umls was infrequent but did include some cases such craniofacial-deafness-hand syndrome. a more significant problem contributing to the majority of false positives was the discovery of disease terms in generif that indicated only a partial, ambiguous or no association to the gene in question. fortunately, the succinctness of generif means that this occurs less frequently than in abstracts which may contain diseases not directly related to the gene. we found only one incorrectly assigned generif in the generifs examined, indicating that this is a minor source of error. one result from our analysis is that omim performs poorly relative to generif with newly discovered mappings. a fairly typical case is the alpha-2-marcroglobuin gene. while omim includes mappings for alzheimer's disease and pulmonary emphysema, it excludes potential links to benign prostatic hyperplasia, multiple sclerosis and argyrophilic grain disease. this may be a result of omim's stronger requirement for evidence, but failure to keep pace with current research may also contribute. we annotated the human genome with disease ontology and reported its performance. such an annotation will enable many graphical and statistical applications similar to previously what has been done with gene ontology annotations. an example of this is presented in figure where all the genes in the human genome with established links to four cancers led to the identification of eleven genes in common between these four cancers. this analysis facilitates identification of relevant targets or markers for diseases with common etiology or pathology, and has implications for biological plausibility as well as therapeutic potential. although previous studies have demonstrated the utility of this approach, the improved coverage and accuracy of our analysis provide even greater potential. our future plans include developing a web-enabled database application of the disease ontology for the research community.  similar to the go annotation, we provide a do annotation of the human genome; each annotation is supported by a peer-reviewed publication as required by generif. it enables researchers to study gene-disease relationships computationally. the do annotation of the human genome is available in both tab-delimited format and relational database format, which allows them to be easily adapted for other applications. 
the human genome has been extensively annotated with gene ontology for biological functions, but minimally computationally annotated for diseases. the validation data set suggests a 91% recall rate and 97% precision rate of disease annotation using generif, in contrast with a 22% recall and 98% precision using omim. the much higher recall rate of our approach demonstrates that annotating human genome with disease ontology and generif for diseases dramatically increases the coverage of the disease annotation of human genome. the results were validated against the homayouni gene collection using recall and precision measurements. we used the unified medical language system metamap transfer tool to discover gene-disease relationships from the generif database. we utilized a comprehensive subset of umls, which is disease-focused and structured as a directed acyclic graph, to filter and interpret results from mmtx. we compared our results with the widely used online mendelian inheritance in man annotations.
 the human genome has been extensively annotated with gene ontology for biological functions, but minimally computationally annotated for diseases.  we used the unified medical language system metamap transfer tool to discover gene-disease relationships from the generif database. we utilized a comprehensive subset of umls, which is disease-focused and structured as a directed acyclic graph, to filter and interpret results from mmtx. the results were validated against the homayouni gene collection using recall and precision measurements. we compared our results with the widely used online mendelian inheritance in man annotations.  the validation data set suggests a 91% recall rate and 97% precision rate of disease annotation using generif, in contrast with a 22% recall and 98% precision using omim. our thesaurus-based approach allows for comparisons to be made between disease containing databases and allows for increased accuracy in disease identification through synonym matching. the much higher recall rate of our approach demonstrates that annotating human genome with disease ontology and generif for diseases dramatically increases the coverage of the disease annotation of human genome. 14– july the international conference on bioinformatics & computational biology las vegas, nv, usa 