BACKGROUND
the gene ontology  provides a structured, controlled vocabulary of terms, which are interrelated forming a directed acyclic graph  for describing and categorizing  the attributes for genes, gene products and sequences  <cit> .
the "molecular function"  category describes fundamental biochemical activities  at the molecular level  <cit> .
as a popular resource used for functional annotation, mf provides rich information and a convenient way to study gene functional similarity by comparing terms with which the genes are annotated  <cit> , which subsequently supports a wide variety of applications, such as assessing target gene functions  <cit> , predicting gene functional associations  <cit> , inferring protein nomenclature  <cit> , predicting sub-cellular localization  <cit> , discovering new pathways  <cit> , etc.
in order to compute gene-to-gene functional similarities using go, various computational approaches have been developed.
these approaches can be classified into two distinct categories: 1) group-wise, meaning calculating gene-to-gene similarity directly based on a statistical framework considering all the terms annotated to the target genes  <cit> , and 2) pair-wise, i.e., indirectly computing gene-to-gene similarity using term-to-term similarities computed with go semantic measures  <cit> .
each of the aforementioned measurements adopts one or a few kinds of knowledge in the go efficiently.
however, they do not rely on all of the rich information available in the go databases.
in this paper, we propose a new rank-based gene semantic similarity measure called intego , which can integrate the state-of-the-art gene-to-gene measures  <cit>   to bring the performance of the go-based functional similarity studies to a higher level.
in the first go-based measure category , by combining elements of the topology and annotation information, the yu measure calculates a probabilistic level of similarity from go, in order to directly compute gene similarity  <cit> .
the main idea of the yu measure is that a pair of genes should be very similar if they are included in a functional group with a few proteins, whereas the similarity is lower if the gene pair belongs to a large gene group.
mathematically, given two gene g <dig> and g <dig>  the gene-to-gene similarity can be calculated with:
genesimyu=-lnng <dig> g2n
where ng <dig> g2is the total number of gene pairs that have the same set of lowest common ancestors  as g <dig> and g2; n is the total number of gene pairs in the selected go category.
a lca is the common ancestor with the highest information content .
in the illustrative example in figure  <dig>  there are in total  <dig> gene pairs possible among the ten genes; the lca of gene pair g <dig> and g <dig> is t <dig>  and the number of gene pairs  is  <dig>  therefore, the similarity of g <dig> and g <dig> based on the yu measure is −ln =  <dig> .
the yu measure considers both the elements of topological distance and the lca distance.
however, it simplifies the computation of shared information of both genes without using all of the common parents of the go terms annotated to g <dig> or g <dig>  which neglects the locations of lcas and the aggregate semantic contributions from the parents of the target terms .
alternatively, the sora  <cit>  measure computes the ic of a term set by means of combining inherited and extended information content of the terms based on the structure of go.
gene functional similarity is estimated using the ic overlap ratio of term sets.
however, like the yu measure, it ignores valuable information implicit in the semantics, i.e., the common parents of the go terms, when calculating the shared ic and relationships among involved terms.
in the measures in the second category , the pair-wise term comparisons originally developed for natural language processing  <cit>  are utilized, and are strongly dependent on the specific taxonomy.
among the earlier developed methods, an ic based measure called the resnik measure has showed strong correlations between its results and gene expression similarities on yeast  <cit> .
mathematically, given a go term t, its ic is defined as a negative log likelihood ic = − log, where gt and groot are the sets of genes annotated to term t and the root term  respectively.
in the resnik measure, the similarity between term t <dig> and t <dig> is defined as the ic of lca: termsimresnik  = ic.
although the resnik measure strongly correlated with the gene expression data  <cit> , terms sharing the same lca have the same semantic similarity, even if they are at very different levels of go.
consequently, it cannot differentiate term pairs that are far from lca with term pairs close to the same lca.
in the illustrative example in figure  <dig>  the common parent of t <dig> and t <dig> is t <dig>  which is the same as the lca of t <dig> and t <dig>  according to the resnik measure, simresnik  = simresnik  =  <dig> , but clearly the distance from t <dig> and t <dig> to the lca is shorter.
to take both the distance from lca to the target terms and the distance from lca to root into account  <cit> , a later-developed measure called the schlicker measure was proposed:
termsimschliker=2×icic+ic×1-|glca12||groot|
where glca <dig> is the set of genes annotated to the lca of t <dig> and t <dig>
in eq  <dig>  the first part on the right side of the equation quantifies the distance from terms t <dig> and t <dig> to their lca, and the second part measures the distance from the lca to the root, where a short former distance and a long later distance indicate a higher similarity.
experimental results revealed that the schlicker measure agrees with sequence similarity  <cit> .
in the same example in figure  <dig>  the schlicker measure is able to differentiate term pair  and  with termsimschlicker =  <dig>  and termsimschlicker =  <dig> .
however, the common problem of the schlicker measure and the resnik measure is that they only consider a single common ancestor, neglecting the fact that two go terms may have multiple common ancestors in the go structure  <cit> .
recently, the wang measure was proposed to consider all of the parent terms of the target terms  <cit> .
given a term t <dig> and its parent term p, the semantic contribution of p to t <dig>  denoted as st <dig> p, is defined as the maximal semantic contribution of the paths from t <dig> to p. the go term similarity in the wang measure is defined in eq  <dig>  where p <dig>  are the sets of all of the parents of t <dig> .
termsimwang=∑p∈p1∩p2∑t∈p1st <dig> p+ ∑t∈p2st <dig> p
the experiment result shows that this measure performs significantly better than resnik measure on yeast protein functional similarities  <cit> .
however, the wang measure ignores both the topological distances among the lcas and the statistics of gene annotations that the yu measure has taken into consideration.
for the same example in figure  <dig>  to compare the similarity of term t <dig> and t <dig>  all of the common parents of the two terms, p <dig> = {t <dig>  t <dig>  t <dig>  t <dig>  t <dig>  root} and p <dig> = {t <dig>  t <dig>  t <dig>  t <dig>  t <dig>  root}, are considered by the wang measure.
for the resnik, schlicker and wang measures, gene-to-gene similarity is computed based on the go term similarities that annotate to the target genes.
in wang et al  <cit> , let g <dig> and g <dig> be two genes and t <dig> and t <dig> be the sets of go terms annotated to g <dig> and g <dig>  the gene-to-gene similarity is calculated by eq 4:
genesim=∑t∈t1termsim+ ∑t∈t2termsim|t1| + |t2|
where t is a go term, termsim = maxti∈tx sim, which represents the highest similarity between t and term set tx.
note that, for both |t1| and |t2|, only the terms with t ermsim ≠  <dig> are counted.
to the best of our knowledge, the existing measures emphasize on only one or few types of relationships between genes but ignores the others.
one of these measures may be better than the others on one specific set of terms and genes, but may perform worse than the other measures on another gene set.
since none of the existing measures takes into account all of the aspects of go , which is of course a challenging task, it is hypothesized that the integration of multiple measures can improve the performance, since integration of multiple methods has been widely applied for performance boosting  <cit> .
in this paper, we proposed a rank-based gene semantic similarity measure called intego by synergistically integrating the state-of-the-art gene-to-gene similarity measures.
the integrated measures are called seed measures in the rest of paper.
the major contributions of our work are:
• while the existing measures only consider one or few aspects of the problem, intego is an integrative approach, which conceptually considers all of the information in go to reduce incorrect score assignments.
in addition, intego employs an adaptive approach for the optimization of the seed measure integration.
• a rank-based approach is used to integrate multiple seed measures.
since the values from different seed measures have different scales and distributions, a direct integration of the values may lead to biased results.
with our rank-based approach, intego unifies the scale and distribution among different seed measures, ensuring fair comparison.
• intego is an open framework, which adds the flexibility to integrate more go similarity measures, more advanced evaluation and integration methods in the future.
intego was systematically tested on three species with different levels of complexity of go annotations, i.e., yeast, arabidopsis and human.
the experimental results on all of the three species show that intego performs consistently better than the other measures in all of the tests.
method
in order to integrate multiple seed measures in intego, two key problems need to be solved: first, how to select the most appropriate seed measures for integration; second, how to integrate all of the scores from the different seed measures.
to solve these problems, intego is divided into two steps: 1) to compute similarity scores with every seed measure individually and rank the scores, and 2) to evaluate and integrate the ranks of multiple seed measures.
rank-based similarity
the outputs of the different gene-to-gene similarity measures have different scales and distributions.
therefore, a direct integration of the values may lead to biased results.
in intego, we unify the scale and distribution among different seed measures with a rank-based approach.
one common problem of rankbased approaches though is the data size dependence, i.e., while a rank-based approach can work well on a relative large dataset, it is often inadequate on a small set of data.
for example, if only two genes are provided by a user, the similarity rank of the two genes is always one, regardless how high or how low the actual similarity score is.
therefore, instead of requiring users to always provide a large set of genes to compare , intego maintains a background set of genes  for every species of interest to unify the similarity scores from the multiple seed measures.
bg must satisfy two requirements: 1) it is large enough; 2) it unbiasedly includes the full spectrum of gene similarity scores, ranging from the lowest to the highest.
the framework of intego is shown in figure  <dig>  in the steps with grey background, the similarity scores in bg are pre-calculated with all of the seed measures and saved in a database called genesimdb.
when a user inputs a gene set g, the similarity scores of all of the gene pairs in g and all of the gene pairs between g and bg will be calculated with all of the seed measures, and be merged into genesimdb.
if g is a subset of bg, intego will output the results directly.
finally, all of the gene pairs in genesimdb are sorted incrementally based on their gene similarity scores and are ranked.
the ranked gene similarity score ranksim for genes g <dig> and g <dig> in g is calculated as:
ranksim=2×rg <dig> g2m <dig>
where rg <dig> g2m is the rank of gene pair g <dig> and g <dig> using seed measure m, and bg is the predefined background gene set, and g is the user provided gene set.
the ranked similarity indicates how similar a given gene pair is in the background of all of the gene pairs.
one advantage to use the rank-based measure is to unify different scales and distributions among the seed measures.
therefore, the agreement among the ranks could indicate the functional similarities appropriately.
an illustrative example is shown in table  <dig>  given ten gene pairs, three measures  are used to calculate the gene-to-gene semantic similarities based on the go.
the first column of the values show that the similarity scores of measure ma, mb and mc have different scales and different distributions.
for example, the semantic similarity of gene pair  <dig> is  <dig>  for measure ma and  <dig>  for measure mb, although they both mean the highest functional similarity in their own datasets.
the second column of the values show the ranks of the gene pairs under each seed measure in assenting order.
ma
mb
mc
ma
mb
mc
illustrative example for integration similarity, where ma, mb and mc are three seed gene-to-gene functional similarity measures.
adaptive integration approach
the rank-based semantic similarities of gene pairs from every seed measure provide an unique opportunity to compute the gene-to-gene similarities with all the information of go utilized by the seed measures.
a key problem here is how to select the most appropriate integration approach.
although there are many integration approaches all working well on certain domains, there does not exist one method that is always better than the others.
in fact, to choose an appropriate integration method is largely dependent on the content of the study.
therefore, we propose an adaptive approach to automatically select the most appropriate integration method from a set of candidates.
the main idea of the adaptive approach is to score all of the methods in the pool of the candidate integration approaches with the background set bg, and then select the best one.
intego provides four integration methods: max, min, mean and median.
as an open system, intego also allows users to use their own integration methods.
mathematically, let ranksim be rank-based similarity of gene g <dig> and g <dig> using seed measure m, intego is defined as:
intego=maxm∈mranksimifi=maxminm∈mranksimifi=minmeanm∈mranksimifi=meanmedianm∈mranksimifi=medianintegrationm∈mranksimifi=other_integration
where m is a set of seed measures and i is an integration method which is max, min, mean, median of all of the ranks, or any other integration method that is defined by the user.
for an illustrative example in table  <dig>  the results based on the four different integration methods are shown in the third column.
to automatically determine which integration method is the best, all of the gene pair similarities in bg are calculated based on each candidate integration method and are evaluated systematically with biological data.
recent studies used the correlation coefficient of gene expression correlations or gene sequence similarities to evaluate the mf based gene similarities  <cit> .
however, it is not always correlated between gene functional similarities and gene expression correlation or sequence similarities  <cit> .
furthermore, previous studies show that enzymes are usually categorized biochemically with ec  numbers but not their nucleotide or amino acid sequences  <cit> , which indicates that it could be a better way for using ec numbers to explain molecular function with the criteria that the molecular functions of a group of genes are similar if they have the same ec numbers  <cit> .
to systematically use ec to choose an integration method, all of the genes in bg are grouped based on their ec numbers , and then the differences between the inter- and intra-ec gene-to-gene similarities are tested.
with an integration method, the higher the ratio between intra-ec gene similarities and inter-ec gene similarities, the better the integration method is.
quantitatively, we utilize the logged fold change  measure which has been widely used in the gene expression studies  <cit> .
the logfc score of ec ei is defined in eq 7:
logfc=1|e|×∑ej∈e;g∩g=θ∑g∈gdiffg|g|
where g is set of all of genes which ec number is ei; e is a set of ecs which do not have overlapped genes with ei  ∩ g = ∅); diffg is computed as:
diffg=ln|g|×∑g′∈g+c)|g|×∑g*∈g+c)
where c is a laplacian smoothing parameter which is a constant small positive number; g is the set of all of the genes assigned to ec ei except gene g; g is the set of all of the genes assigned to ec ej; g is a gene assigned to ei.
in eq  <dig>  the numerator represents the inter-ec distance and the denominator represents the intra-ec distance.
the higher the diffg is, the more obvious the positive difference between inter-ec difference and intra-ec difference is.
for example, given nine genes in bg, four of which have the same ec number, labeled as e <dig>  and the other five genes belong to another ec number, labeled as e <dig>  to calculate the logfc score for e <dig>  we first compute diffg with eq  <dig>  meaning that every gene in e <dig> is compared with every other gene in e <dig> for the average intra-ec difference, and then every gene in e <dig> is compared with every gene in e <dig> to get the inter-ec differences.
logfc is the average of all of the diffg scores for the genes assigned to e <dig>
the method that has the highest logfc scores for all of the ecs are considered as the most appropriate integration method for bg.
if a user input set g is much smaller than bg , we assume the selected method is also the most suitable for g ∪ bg.
if the size of g is comparable to bg, it is not necessary to use bg, then the integration method shall be selected based on the evaluation on g.
RESULTS
to systematically evaluate the performance of intego, we tested it on three model organisms with different levels of go annotation scale and complexity.
for each of them, we adopted ec numbers and protein sequences as independent biological evidences.
data preparation
the go annotation and structure data were downloaded from the go website .
to systematically evaluate different go-based gene-to-gene similarity measures on mf, the pathway and ec number information of yeast, arabidopsis were downloaded from the saccharomyces genome database , plantcyc  and humancyc  respectively.
note that our ec based evaluation method requires that an ec has at least two genes.
in yeast, arabidopsis and human,  <dig>   <dig> and  <dig> ecs satisfy the criteria.
the protein sequences were downloaded from the saccharomyces genome database , tair  and uniprot  respectively.
let e be the set of all of the ecs that have at least one gene assignment, we define bg as the set of all of the genes that has at least one ec assignments in e. this definition of bg ensures that for any gene in bg the intra-ec similarity is valid.
the sizes of bg are  <dig>   <dig>  and  <dig>  for yeast, arabidopsis and human respectively.
an experiment on the variation of the background set  reveals that the use of a relatively smaller background set may affect performance significantly.
additional file  <dig>   <dig> and  <dig> show that the distribution of the gene-to-gene similarities with yu, schlicker and wang measures, where the similarity scores are spread in the full spectrum of the range.
in summary, the background gene sets are well prepared.
intego was implemented with java jdk  <dig>  and jung library  <cit> .
the experiments were run on a windows  <dig> computer with intel i <dig> cpu and  <dig> gb ram.
selecting seed measures
in order to select the most appropriate seed measures for intego, we screened four existing measures  using the ec based evaluation method.
figure  <dig> shows that for the yu, schlicker and wang measures, it is not distinguishable that one measure is significant better than another.
the yu, schlicker and wang measures all performed the best on yeast with the highest median value.
the schlicker measure performs best on arabidopsis, while the yu measure is best on human.
therefore, we chose all of the three as the seed measures in intego.
we did not choose the resnik measure, because it is clearly not as good as the other measures in all of the three species.
note that the upper-bound and the lower-bound of the logfc scores in figure  <dig> were set to  <dig> and - <dig>  respectively to eliminate outliers.
in addition, figure  <dig> shows that although all of the three seed measures perform equally well in some ecs, each measure has its own favorable ec groups.
for example, the schlicker and wang measures perform the best in  <dig> and  <dig> out of the total  <dig> arabidopsis ecs respectively ), which is greater than the yu measure .
however, the yu measure performs the best in  <dig> out of the total  <dig> human ecs, which dominant the ec group distribution in human ).
therefore, an appropriate integration of these measures may combine the advantages of different measures and improve the overall performance.
note that although only four measures were screened in the experiment, more measures can be evaluated and added later since intego has an open framework.
selecting integration method
in order to select the most appropriate integration method, four different approaches  were tested and compared.
figure  <dig> shows that max performs the best among the four integration methods.
in yeast, although almost all of the measures have the same median value, the 25th percentile of max is  <dig>  significantly higher than the yu, schlicker and wang measure  and the other integration methods.
in arabidopsis and human, the median of max are both  <dig>  which is also significantly higher than that of all of the other integration methods.
it indicates that the performance of max, a simple integration approach, has been increased to around 2-fold.
this is because the integration considers all of the aspects of go, while an individual seed measure, although nicely designed, is compromised in that it focuses on only one of few kinds of knowledge in go.
the other integration measures, especially min, however, cannot distinguishably improve the gene similarity performance.
as shown in figure  <dig>  the result of min is even worse than the seed measures.
it indicates that the performance of gene-to-gene similarity could be significantly improved only by the appropriate integration.
as mentioned in the previous section, the seed measures have their own favorable ec groups.
to test whether max take advantage of all of the strength of the seed measures, we compared max with the yu, schlicker and wang measure on all of the ecs.
figure  <dig> ,  and  show that max dominant the ec groups, clearly different to the results in figure  <dig>  in detail, max performs the best in  <dig> and  <dig> out of  <dig> and  <dig> ecs in arabidopsis and human respectively, while the numbers are only  <dig>   <dig>   <dig> in arabidopsis and  <dig>   <dig>   <dig> in human for the yu, wang and schlicker measures respectively.
in summary, the experiment indicates that integrating multiple measures could improve the performance of gene similarity measurement and max is the best integration method.
statistics analysis was carried out to test whether the results of the best integration measure  of intego is statistically the best.
we compared intego with the three seed measures using tukeyhsd test  <cit> .
the p-values shown in table  <dig> and the 95% family-wise confidence level  indicate that the results of max are significant better than the results of all of the seed measures in yeast, arabidopsis and human, with the only exception that the schlicker measure's results are comparable in yeast, in that the schlicker measure performs very well in yeast, so there is little room for intego to improve.
arabidopsis
adjusted p-values for comparing max with yu, schlicker and wang measure using tukeyhsd.
significant p-values are in bold fonts.
protein sequence based performance evaluation
in addition to use ec as the evaluation criteria, protein sequence similarities were employed as independent evidence for further performance study.
although the correlation coefficient between semantic similarity and sequence similarity is not as strong as ec, it is generally accepted that as sequence similarity increases, so does the chance that these proteins are homologues, in which case they are likely to have identically annotated molecular functions  <cit> .
in our test, sequence similarity scores ) of all genes in the bg of the three species were calculated with blast, resulting in  <dig>  yeast,  <dig>  arabidopsis and  <dig>  human gene pairs.
as shown in figure  <dig>  the semantic similarity measurements show a correlation with sequence similarity.
the covariance scores  on all of the three species reveal that intego is overall the best measure.
CONCLUSIONS
comparing gene at the functional level is vital for various of applications  <cit> .
the existing go semantic based measures either calculate gene-to-gene similarities directly  <cit> , or indirectly compute gene-to-gene similarities with term-to-term similarities  <cit> .
unfortunately, none of them takes into account all of the respects of rich information in go .
in this paper, we proposed a new measure called intego to appropriately integrate the seed measures with the following advantages: 1) intego employs an adaptive approach which enables the optimization of seed measure integration; 2) it applies a rank-based integration approach, which unifies the scale and distribution differences among different seed measures; 3) intego is an open-platform measure that allows users to add/delete seed measures, redefine the background gene set and change the rank-based integration method.
to demonstrate the advantages of intego, we compared its ec-assigned gene similarities and sequence similarities with three existing measures  in yeast, arabidopsis and human.
comparing with these state-of-the-art measures, the experimental results show that intego increases the logfc scores to about two-fold.
it indicates that integrating multiple measures appropriately can improve the performance of the functional similarity measure.
especially, we found that taking the maximal ranks from all of the seed measures performs the best.
the covariances between semantic similarities and protein sequence similarities shows intego is clear the best out of all the tested measures.
in intego, to maintain a large background gene set is expensive.
therefore, extending intego from mf to bp or even other biological or medical ontologies is not a trivial problem.
in the future, we will continue to improve intego to be more efficient and to be applicable on more ontologies.
as an open framework, the performance of intego may be further improved by synergistically integrating more seed measurements.
we will continue to integrate and compare intego with more recent gene-to-gene measurements in the future.
we will continue to explore better integration methods, such as using em algorithm to optimize the weight for each seed measure, to achieve better performance.
competing interests
the authors declare that they have no competing interests.
authors' contributions
jc conceived the project.
jp, jc and yw designed the algorithm and experiments.
jp implemented the algorithm and finished the experiments.
supplementary material
additional file 1
average logfc scores for different sizes of background set.
to test whether the selection of bg will affect the integration performance, we compared the results for different background set on yeast.
first, given the full set of bg, a subset of gene pairs were randomly selected with the percentage varying from 10% to 100%.
this process was repeated for  <dig> times.
second, as shown in additional file  <dig>  the logfc scores for each subset size were calculated based on the randomly selected gene pairs.
since we do not use the full set, the computable ecs are also a subset of all of the computable ecs.
in additional file  <dig>  the logfc score increases linearly from  <dig> to  <dig> when the coverage increases from 10% to 90%, then suddenly jumps to a high score  when all of the background genes were used, indicating that first, the size of the background set affects the integration measure significantly, second, to use the full background set is the best, although it slightly increases the computational time.
click here for file
additional file 2
distribution of the gene-to-gene similarities with yu measure.
distribution of the gene-to-gene similarities with yu measure for all of the genes in the background gene set  on yeast.
click here for file
additional file 3
distribution of the gene-to-gene similarities with schlicker measure.
distribution of the gene-to-gene similarities with schlicker measure for all of the genes in the background gene set  on yeast.
click here for file
additional file 4
distribution of the gene-to-gene similarities with wang measure.
distribution of the gene-to-gene similarities with wang measure for all of the genes in the background gene set  on yeast.
click here for file
additional file 5
the 95% family-wise confidence level of tukeyhsd test on yeast.
the 95% family-wise confidence level of tukeyhsd test on yeast, which compared max with all the three seed measures .
click here for file
additional file 6
the 95% family-wise confidence level of tukeyhsd test on arabidopsis.
the 95% family-wise confidence level of tukeyhsd test on arabidopsis, which compared max with all the three seed measures .
click here for file
additional file 7
the 95% family-wise confidence level of tukeyhsd test on human.
the 95% family-wise confidence level of tukeyhsd test on human, which compared max with all the three seed measures .
click here for file
additional file 8
the covariance sores comparing with sequence similarity.
the covariance sores comparing with sequence similarity on yeast, arabidopsis and human for max, yu, schlicker and wang measure.
click here for file
acknowledgements
this project has been funded by the u.s. department of energy  and the national natural science foundation of china grant  to y. w.
declarations
the publication costs for this article were funded by the corresponding author's institution.
this article has been published as part of bmc bioinformatics volume  <dig> supplement  <dig>  2014: selected articles from the twelfth asia pacific bioinformatics conference : bioinformatics.
the full contents of the supplement are available online at http://www.biomedcentral.com/bmcbioinformatics/supplements/15/s <dig>

 the gene ontology provides a structured, controlled vocabulary of terms, which are interrelated forming a directed acyclic graph for describing and categorizing the attributes for genes, gene products and sequences. the "molecular function" category describes fundamental biochemical activities at the molecular level. as a popular resource used for functional annotation, mf provides rich information and a convenient way to study gene functional similarity by comparing terms with which the genes are annotated, which subsequently supports a wide variety of applications, such as assessing target gene functions, predicting gene functional associations, inferring protein nomenclature, predicting sub-cellular localization, discovering new pathways, etc. in order to compute gene-to-gene functional similarities using go, various computational approaches have been developed. these approaches can be classified into two distinct categories: 1) group-wise, meaning calculating gene-to-gene similarity directly based on a statistical framework considering all the terms annotated to the target genes, and 2) pair-wise, i.e., indirectly computing gene-to-gene similarity using term-to-term similarities computed with go semantic measures. each of the aforementioned measurements adopts one or a few kinds of knowledge in the go efficiently. however, they do not rely on all of the rich information available in the go databases. in this paper, we propose a new rank-based gene semantic similarity measure called intego, which can integrate the state-of-the-art gene-to-gene measures to bring the performance of the go-based functional similarity studies to a higher level. in the first go-based measure category, by combining elements of the topology and annotation information, the yu measure calculates a probabilistic level of similarity from go, in order to directly compute gene similarity. the main idea of the yu measure is that a pair of genes should be very similar if they are included in a functional group with a few proteins, whereas the similarity is lower if the gene pair belongs to a large gene group. mathematically, given two gene g and g the gene-to-gene similarity can be calculated with: genesimyu=-lnng g2n where ng g2is the total number of gene pairs that have the same set of lowest common ancestors as g and g2; n is the total number of gene pairs in the selected go category. a lca is the common ancestor with the highest information content. in the illustrative example in figure there are in total gene pairs possible among the ten genes; the lca of gene pair g and g is t and the number of gene pairs is therefore, the similarity of g and g based on the yu measure is −ln =. the yu measure considers both the elements of topological distance and the lca distance. however, it simplifies the computation of shared information of both genes without using all of the common parents of the go terms annotated to g or g which neglects the locations of lcas and the aggregate semantic contributions from the parents of the target terms. alternatively, the sora measure computes the ic of a term set by means of combining inherited and extended information content of the terms based on the structure of go. gene functional similarity is estimated using the ic overlap ratio of term sets. however, like the yu measure, it ignores valuable information implicit in the semantics, i.e., the common parents of the go terms, when calculating the shared ic and relationships among involved terms. in the measures in the second category, the pair-wise term comparisons originally developed for natural language processing are utilized, and are strongly dependent on the specific taxonomy. among the earlier developed methods, an ic based measure called the resnik measure has showed strong correlations between its results and gene expression similarities on yeast. mathematically, given a go term t, its ic is defined as a negative log likelihood ic = − log, where gt and groot are the sets of genes annotated to term t and the root term respectively. in the resnik measure, the similarity between term t and t is defined as the ic of lca: termsimresnik = ic. although the resnik measure strongly correlated with the gene expression data, terms sharing the same lca have the same semantic similarity, even if they are at very different levels of go. consequently, it cannot differentiate term pairs that are far from lca with term pairs close to the same lca. in the illustrative example in figure the common parent of t and t is t which is the same as the lca of t and t according to the resnik measure, simresnik = simresnik =, but clearly the distance from t and t to the lca is shorter. to take both the distance from lca to the target terms and the distance from lca to root into account, a later-developed measure called the schlicker measure was proposed: termsimschliker=2×icic+ic×1-|glca12||groot| where glca is the set of genes annotated to the lca of t and t in eq the first part on the right side of the equation quantifies the distance from terms t and t to their lca, and the second part measures the distance from the lca to the root, where a short former distance and a long later distance indicate a higher similarity. experimental results revealed that the schlicker measure agrees with sequence similarity. in the same example in figure the schlicker measure is able to differentiate term pair and with termsimschlicker = and termsimschlicker =. however, the common problem of the schlicker measure and the resnik measure is that they only consider a single common ancestor, neglecting the fact that two go terms may have multiple common ancestors in the go structure. recently, the wang measure was proposed to consider all of the parent terms of the target terms. given a term t and its parent term p, the semantic contribution of p to t denoted as st p, is defined as the maximal semantic contribution of the paths from t to p. the go term similarity in the wang measure is defined in eq where p are the sets of all of the parents of t. termsimwang=∑p∈p1∩p2∑t∈p1st p+ ∑t∈p2st p the experiment result shows that this measure performs significantly better than resnik measure on yeast protein functional similarities. however, the wang measure ignores both the topological distances among the lcas and the statistics of gene annotations that the yu measure has taken into consideration. for the same example in figure to compare the similarity of term t and t all of the common parents of the two terms, p = {t t t t t root} and p = {t t t t t root}, are considered by the wang measure. for the resnik, schlicker and wang measures, gene-to-gene similarity is computed based on the go term similarities that annotate to the target genes. in wang et al, let g and g be two genes and t and t be the sets of go terms annotated to g and g the gene-to-gene similarity is calculated by eq 4: genesim=∑t∈t1termsim+ ∑t∈t2termsim|t1| + |t2| where t is a go term, termsim = maxti∈tx sim, which represents the highest similarity between t and term set tx. note that, for both |t1| and |t2|, only the terms with t ermsim ≠ are counted. to the best of our knowledge, the existing measures emphasize on only one or few types of relationships between genes but ignores the others. one of these measures may be better than the others on one specific set of terms and genes, but may perform worse than the other measures on another gene set. since none of the existing measures takes into account all of the aspects of go, which is of course a challenging task, it is hypothesized that the integration of multiple measures can improve the performance, since integration of multiple methods has been widely applied for performance boosting. in this paper, we proposed a rank-based gene semantic similarity measure called intego by synergistically integrating the state-of-the-art gene-to-gene similarity measures. the integrated measures are called seed measures in the rest of paper. the major contributions of our work are: • while the existing measures only consider one or few aspects of the problem, intego is an integrative approach, which conceptually considers all of the information in go to reduce incorrect score assignments. in addition, intego employs an adaptive approach for the optimization of the seed measure integration. • a rank-based approach is used to integrate multiple seed measures. since the values from different seed measures have different scales and distributions, a direct integration of the values may lead to biased results. with our rank-based approach, intego unifies the scale and distribution among different seed measures, ensuring fair comparison. • intego is an open framework, which adds the flexibility to integrate more go similarity measures, more advanced evaluation and integration methods in the future. intego was systematically tested on three species with different levels of complexity of go annotations, i.e., yeast, arabidopsis and human. the experimental results on all of the three species show that intego performs consistently better than the other measures in all of the tests. method in order to integrate multiple seed measures in intego, two key problems need to be solved: first, how to select the most appropriate seed measures for integration; second, how to integrate all of the scores from the different seed measures. to solve these problems, intego is divided into two steps: 1) to compute similarity scores with every seed measure individually and rank the scores, and 2) to evaluate and integrate the ranks of multiple seed measures. rank-based similarity the outputs of the different gene-to-gene similarity measures have different scales and distributions. therefore, a direct integration of the values may lead to biased results. in intego, we unify the scale and distribution among different seed measures with a rank-based approach. one common problem of rankbased approaches though is the data size dependence, i.e., while a rank-based approach can work well on a relative large dataset, it is often inadequate on a small set of data. for example, if only two genes are provided by a user, the similarity rank of the two genes is always one, regardless how high or how low the actual similarity score is. therefore, instead of requiring users to always provide a large set of genes to compare, intego maintains a background set of genes for every species of interest to unify the similarity scores from the multiple seed measures. bg must satisfy two requirements: 1) it is large enough; 2) it unbiasedly includes the full spectrum of gene similarity scores, ranging from the lowest to the highest. the framework of intego is shown in figure in the steps with grey background, the similarity scores in bg are pre-calculated with all of the seed measures and saved in a database called genesimdb. when a user inputs a gene set g, the similarity scores of all of the gene pairs in g and all of the gene pairs between g and bg will be calculated with all of the seed measures, and be merged into genesimdb. if g is a subset of bg, intego will output the results directly. finally, all of the gene pairs in genesimdb are sorted incrementally based on their gene similarity scores and are ranked. the ranked gene similarity score ranksim for genes g and g in g is calculated as: ranksim=2×rg g2m where rg g2m is the rank of gene pair g and g using seed measure m, and bg is the predefined background gene set, and g is the user provided gene set. the ranked similarity indicates how similar a given gene pair is in the background of all of the gene pairs. one advantage to use the rank-based measure is to unify different scales and distributions among the seed measures. therefore, the agreement among the ranks could indicate the functional similarities appropriately. an illustrative example is shown in table given ten gene pairs, three measures are used to calculate the gene-to-gene semantic similarities based on the go. the first column of the values show that the similarity scores of measure ma, mb and mc have different scales and different distributions. for example, the semantic similarity of gene pair is for measure ma and for measure mb, although they both mean the highest functional similarity in their own datasets. the second column of the values show the ranks of the gene pairs under each seed measure in assenting order. ma mb mc ma mb mc illustrative example for integration similarity, where ma, mb and mc are three seed gene-to-gene functional similarity measures. adaptive integration approach the rank-based semantic similarities of gene pairs from every seed measure provide an unique opportunity to compute the gene-to-gene similarities with all the information of go utilized by the seed measures. a key problem here is how to select the most appropriate integration approach. although there are many integration approaches all working well on certain domains, there does not exist one method that is always better than the others. in fact, to choose an appropriate integration method is largely dependent on the content of the study. therefore, we propose an adaptive approach to automatically select the most appropriate integration method from a set of candidates. the main idea of the adaptive approach is to score all of the methods in the pool of the candidate integration approaches with the background set bg, and then select the best one. intego provides four integration methods: max, min, mean and median. as an open system, intego also allows users to use their own integration methods. mathematically, let ranksim be rank-based similarity of gene g and g using seed measure m, intego is defined as: intego=maxm∈mranksimifi=maxminm∈mranksimifi=minmeanm∈mranksimifi=meanmedianm∈mranksimifi=medianintegrationm∈mranksimifi=other_integration where m is a set of seed measures and i is an integration method which is max, min, mean, median of all of the ranks, or any other integration method that is defined by the user. for an illustrative example in table the results based on the four different integration methods are shown in the third column. to automatically determine which integration method is the best, all of the gene pair similarities in bg are calculated based on each candidate integration method and are evaluated systematically with biological data. recent studies used the correlation coefficient of gene expression correlations or gene sequence similarities to evaluate the mf based gene similarities. however, it is not always correlated between gene functional similarities and gene expression correlation or sequence similarities. furthermore, previous studies show that enzymes are usually categorized biochemically with ec numbers but not their nucleotide or amino acid sequences, which indicates that it could be a better way for using ec numbers to explain molecular function with the criteria that the molecular functions of a group of genes are similar if they have the same ec numbers. to systematically use ec to choose an integration method, all of the genes in bg are grouped based on their ec numbers, and then the differences between the inter- and intra-ec gene-to-gene similarities are tested. with an integration method, the higher the ratio between intra-ec gene similarities and inter-ec gene similarities, the better the integration method is. quantitatively, we utilize the logged fold change measure which has been widely used in the gene expression studies. the logfc score of ec ei is defined in eq 7: logfc=1|e|×∑ej∈e;g∩g=θ∑g∈gdiffg|g| where g is set of all of genes which ec number is ei; e is a set of ecs which do not have overlapped genes with ei ∩ g = ∅); diffg is computed as: diffg=ln|g|×∑g′∈g+c)|g|×∑g*∈g+c) where c is a laplacian smoothing parameter which is a constant small positive number; g is the set of all of the genes assigned to ec ei except gene g; g is the set of all of the genes assigned to ec ej; g is a gene assigned to ei. in eq the numerator represents the inter-ec distance and the denominator represents the intra-ec distance. the higher the diffg is, the more obvious the positive difference between inter-ec difference and intra-ec difference is. for example, given nine genes in bg, four of which have the same ec number, labeled as e and the other five genes belong to another ec number, labeled as e to calculate the logfc score for e we first compute diffg with eq meaning that every gene in e is compared with every other gene in e for the average intra-ec difference, and then every gene in e is compared with every gene in e to get the inter-ec differences. logfc is the average of all of the diffg scores for the genes assigned to e the method that has the highest logfc scores for all of the ecs are considered as the most appropriate integration method for bg. if a user input set g is much smaller than bg, we assume the selected method is also the most suitable for g ∪ bg. if the size of g is comparable to bg, it is not necessary to use bg, then the integration method shall be selected based on the evaluation on g.  to systematically evaluate the performance of intego, we tested it on three model organisms with different levels of go annotation scale and complexity. for each of them, we adopted ec numbers and protein sequences as independent biological evidences. data preparation the go annotation and structure data were downloaded from the go website. to systematically evaluate different go-based gene-to-gene similarity measures on mf, the pathway and ec number information of yeast, arabidopsis were downloaded from the saccharomyces genome database, plantcyc and humancyc respectively. note that our ec based evaluation method requires that an ec has at least two genes. in yeast, arabidopsis and human, and ecs satisfy the criteria. the protein sequences were downloaded from the saccharomyces genome database, tair and uniprot respectively. let e be the set of all of the ecs that have at least one gene assignment, we define bg as the set of all of the genes that has at least one ec assignments in e. this definition of bg ensures that for any gene in bg the intra-ec similarity is valid. the sizes of bg are and for yeast, arabidopsis and human respectively. an experiment on the variation of the background set reveals that the use of a relatively smaller background set may affect performance significantly. additional file and show that the distribution of the gene-to-gene similarities with yu, schlicker and wang measures, where the similarity scores are spread in the full spectrum of the range. in summary, the background gene sets are well prepared. intego was implemented with java jdk and jung library. the experiments were run on a windows computer with intel i cpu and gb ram. selecting seed measures in order to select the most appropriate seed measures for intego, we screened four existing measures using the ec based evaluation method. figure shows that for the yu, schlicker and wang measures, it is not distinguishable that one measure is significant better than another. the yu, schlicker and wang measures all performed the best on yeast with the highest median value. the schlicker measure performs best on arabidopsis, while the yu measure is best on human. therefore, we chose all of the three as the seed measures in intego. we did not choose the resnik measure, because it is clearly not as good as the other measures in all of the three species. note that the upper-bound and the lower-bound of the logfc scores in figure were set to and - respectively to eliminate outliers. in addition, figure shows that although all of the three seed measures perform equally well in some ecs, each measure has its own favorable ec groups. for example, the schlicker and wang measures perform the best in and out of the total arabidopsis ecs respectively ), which is greater than the yu measure. however, the yu measure performs the best in out of the total human ecs, which dominant the ec group distribution in human ). therefore, an appropriate integration of these measures may combine the advantages of different measures and improve the overall performance. note that although only four measures were screened in the experiment, more measures can be evaluated and added later since intego has an open framework. selecting integration method in order to select the most appropriate integration method, four different approaches were tested and compared. figure shows that max performs the best among the four integration methods. in yeast, although almost all of the measures have the same median value, the 25th percentile of max is significantly higher than the yu, schlicker and wang measure and the other integration methods. in arabidopsis and human, the median of max are both which is also significantly higher than that of all of the other integration methods. it indicates that the performance of max, a simple integration approach, has been increased to around 2-fold. this is because the integration considers all of the aspects of go, while an individual seed measure, although nicely designed, is compromised in that it focuses on only one of few kinds of knowledge in go. the other integration measures, especially min, however, cannot distinguishably improve the gene similarity performance. as shown in figure the result of min is even worse than the seed measures. it indicates that the performance of gene-to-gene similarity could be significantly improved only by the appropriate integration. as mentioned in the previous section, the seed measures have their own favorable ec groups. to test whether max take advantage of all of the strength of the seed measures, we compared max with the yu, schlicker and wang measure on all of the ecs. figure, and show that max dominant the ec groups, clearly different to the results in figure in detail, max performs the best in and out of and ecs in arabidopsis and human respectively, while the numbers are only in arabidopsis and in human for the yu, wang and schlicker measures respectively. in summary, the experiment indicates that integrating multiple measures could improve the performance of gene similarity measurement and max is the best integration method. statistics analysis was carried out to test whether the results of the best integration measure of intego is statistically the best. we compared intego with the three seed measures using tukeyhsd test. the p-values shown in table and the 95% family-wise confidence level indicate that the results of max are significant better than the results of all of the seed measures in yeast, arabidopsis and human, with the only exception that the schlicker measure's results are comparable in yeast, in that the schlicker measure performs very well in yeast, so there is little room for intego to improve. arabidopsis adjusted p-values for comparing max with yu, schlicker and wang measure using tukeyhsd. significant p-values are in bold fonts. protein sequence based performance evaluation in addition to use ec as the evaluation criteria, protein sequence similarities were employed as independent evidence for further performance study. although the correlation coefficient between semantic similarity and sequence similarity is not as strong as ec, it is generally accepted that as sequence similarity increases, so does the chance that these proteins are homologues, in which case they are likely to have identically annotated molecular functions. in our test, sequence similarity scores ) of all genes in the bg of the three species were calculated with blast, resulting in yeast, arabidopsis and human gene pairs. as shown in figure the semantic similarity measurements show a correlation with sequence similarity. the covariance scores on all of the three species reveal that intego is overall the best measure.  comparing gene at the functional level is vital for various of applications. the existing go semantic based measures either calculate gene-to-gene similarities directly, or indirectly compute gene-to-gene similarities with term-to-term similarities. unfortunately, none of them takes into account all of the respects of rich information in go. in this paper, we proposed a new measure called intego to appropriately integrate the seed measures with the following advantages: 1) intego employs an adaptive approach which enables the optimization of seed measure integration; 2) it applies a rank-based integration approach, which unifies the scale and distribution differences among different seed measures; 3) intego is an open-platform measure that allows users to add/delete seed measures, redefine the background gene set and change the rank-based integration method. to demonstrate the advantages of intego, we compared its ec-assigned gene similarities and sequence similarities with three existing measures in yeast, arabidopsis and human. comparing with these state-of-the-art measures, the experimental results show that intego increases the logfc scores to about two-fold. it indicates that integrating multiple measures appropriately can improve the performance of the functional similarity measure. especially, we found that taking the maximal ranks from all of the seed measures performs the best. the covariances between semantic similarities and protein sequence similarities shows intego is clear the best out of all the tested measures. in intego, to maintain a large background gene set is expensive. therefore, extending intego from mf to bp or even other biological or medical ontologies is not a trivial problem. in the future, we will continue to improve intego to be more efficient and to be applicable on more ontologies. as an open framework, the performance of intego may be further improved by synergistically integrating more seed measurements. we will continue to integrate and compare intego with more recent gene-to-gene measurements in the future. we will continue to explore better integration methods, such as using em algorithm to optimize the weight for each seed measure, to achieve better performance. competing interests the authors declare that they have no competing interests. authors' contributions jc conceived the project. jp, jc and yw designed the algorithm and experiments. jp implemented the algorithm and finished the experiments. supplementary material additional file 1 average logfc scores for different sizes of background set. to test whether the selection of bg will affect the integration performance, we compared the results for different background set on yeast. first, given the full set of bg, a subset of gene pairs were randomly selected with the percentage varying from 10% to 100%. this process was repeated for times. second, as shown in additional file the logfc scores for each subset size were calculated based on the randomly selected gene pairs. since we do not use the full set, the computable ecs are also a subset of all of the computable ecs. in additional file the logfc score increases linearly from to when the coverage increases from 10% to 90%, then suddenly jumps to a high score when all of the background genes were used, indicating that first, the size of the background set affects the integration measure significantly, second, to use the full background set is the best, although it slightly increases the computational time. click here for file additional file 2 distribution of the gene-to-gene similarities with yu measure. distribution of the gene-to-gene similarities with yu measure for all of the genes in the background gene set on yeast. click here for file additional file 3 distribution of the gene-to-gene similarities with schlicker measure. distribution of the gene-to-gene similarities with schlicker measure for all of the genes in the background gene set on yeast. click here for file additional file 4 distribution of the gene-to-gene similarities with wang measure. distribution of the gene-to-gene similarities with wang measure for all of the genes in the background gene set on yeast. click here for file additional file 5 the 95% family-wise confidence level of tukeyhsd test on yeast. the 95% family-wise confidence level of tukeyhsd test on yeast, which compared max with all the three seed measures. click here for file additional file 6 the 95% family-wise confidence level of tukeyhsd test on arabidopsis. the 95% family-wise confidence level of tukeyhsd test on arabidopsis, which compared max with all the three seed measures. click here for file additional file 7 the 95% family-wise confidence level of tukeyhsd test on human. the 95% family-wise confidence level of tukeyhsd test on human, which compared max with all the three seed measures. click here for file additional file 8 the covariance sores comparing with sequence similarity. the covariance sores comparing with sequence similarity on yeast, arabidopsis and human for max, yu, schlicker and wang measure. click here for file acknowledgements this project has been funded by the u.s. department of energy and the national natural science foundation of china grant to y. w. declarations the publication costs for this article were funded by the corresponding author's institution. this article has been published as part of bmc bioinformatics volume supplement 2014: selected articles from the twelfth asia pacific bioinformatics conference: bioinformatics. the full contents of the supplement are available online at http://www.biomedcentral.com/bmcbioinformatics/supplements/15/s 
in gene ontology, the "molecular function" categorization is a widely used knowledge framework for gene function comparison and prediction. gene ontologysemantic similarityintegrative measure17- january the twelfth asia pacific bioinformatics conference shanghai, china we introduce a rank-based gene semantic similarity measure called intego by synergistically integrating the state-of-the-art gene-to-gene similarity measures. its structure and annotation provide a convenient way to compare gene functional similarities at the molecular level. by integrating three go based seed measures, intego significantly improves the performance by about two-fold in all the three species studied. the existing gene similarity measures, however, solely rely on one or few aspects of mf without utilizing all the rich information available including structure, annotation, common terms, lowest common parents.
 in gene ontology, the "molecular function" categorization is a widely used knowledge framework for gene function comparison and prediction. its structure and annotation provide a convenient way to compare gene functional similarities at the molecular level. the existing gene similarity measures, however, solely rely on one or few aspects of mf without utilizing all the rich information available including structure, annotation, common terms, lowest common parents.  we introduce a rank-based gene semantic similarity measure called intego by synergistically integrating the state-of-the-art gene-to-gene similarity measures. by integrating three go based seed measures, intego significantly improves the performance by about two-fold in all the three species studied.  intego is a systematic and novel method to study gene functional associations. the software and description are available at http://www.msu.edu/~jinchen/intego. gene ontologysemantic similarityintegrative measure17- january the twelfth asia pacific bioinformatics conference shanghai, china 