with the number of sequenced genomes at various stages of completion being in the tens of thousands   and the number of genomic features ( genes , rnas , etc . ) identified across these genomes in the millions , the need for accurate and consistent genomic annotation is paramount .
the gene ontology ( go )   was created in 1998 by researchers at flybase , the saccharomyces genome database ( sgd ) , and the mouse genome database as a collaborative effort to address the need for consistent descriptions of gene products across different databases .
this group has since grown to include 26 consortium members and associates   and the go is a key member of the open biological and biomedical ontologies ( obo ) community . in this context
ontologies provide a controlled vocabulary for representing and communicating knowledge about a topic and a set of relationships that hold among the terms of the vocabulary .
the topic for the go is genes and gene products such as transcripts , proteins , or rnas that are described in three related subontologies ( also called namespaces ) , biological process , the broad biological system in which a gene product is involved ; molecular function , the specific role a gene product has or potentially has within a biological process ; and cellular component , the location in a cell where the gene product performs its molecular function .
each ontology is composed of nodes ( terms ) and edges ( relationships ) and is structured as a directed acyclic graph ( dag ) . as one moves down the nodes of the graph from a parent to a child , terms become increasingly more specific .
a dag allows for child terms to have more than one parent and this enables complex relationships to exist between them .
the go currently contains approximately 38 thousand terms that have been used by consortium members to annotate almost 25 million gene products .
each annotation is accompanied by an evidence code to denote the method by which the annotation was made .
computationally sourced annotations , such as the evidence code iea ( inferred from electronic annotation ) , are regarded as less confident annotations than those that are assigned on the strength of experimental evidence .
all data is publicly available for download and the go team provides a number of entry points to query the data including an application programming interface ( api ) to permit custom queries to either a go database or an ontology flat file .
the results of gene expression microarrays can be enriched with terms pertaining to biological process , which helps to discover if entire pathways are upregulated as opposed to simply individual genes .
subcellular location can be predicted for an unknown gene sequence by performing a similarity search such as blast and inferring from the cellular component go terms of top hits .
one such limitation is that , by design , there is no link between the three subontologies .
this means one can not directly answer questions such as  what are all the biological processes occurring in the cellular component 
mungall   exploited the high degree of regularity in phrase structure of obo term definitions and converted tokenized definitions from the go , biochemical ontology , and the cell ontology into a language ( obol ) that can be parsed computationally with a reasoner .
this was used , amongst other things , to find missing relationships within the go and between ontologies .
similarly , wroe et al .   converted the go to a daml + oil framework to enable reasoners to parse the ontology .
bada and hunter   used regular expressions to find patterns in go definitions and create an assertional model to integrate the three go subontologies , the chemical entities of biological interest ontology ( chebi ) , and the cell type ontology ( cto ) .
interrogated the go , compiling groups of terms cooccurring in annotations of gene products , in order to suggest potential biologically relevant terms to annotators .
describe functionality within the quickgo tool from the go annotation ( goa ) group   that provides information on how many times a query term cooccurs with other go terms specifically across the uniprot knowledgebase ( uniprotkb ) database .
this allows researchers to input a query term such as  nucleus  and view all terms that are frequently annotated alongside this term .
however , there are limitations to this method , such as not being able to explore custom subsets of data within the uniprotkb database and a hard limit on the number of terms returned to the user .
this paper presents golink , an alternative tool for finding go terms from across the three go namespaces that cooccur with a given query term .
golink differs from existing tools by using the go api to mine the full complement of the go database , using methods that take into account the namespace of a query term when assessing cooccurrence .
the key advantages of using this particular method and source data are described along with other practical uses for the tool .
these include predicting the most likely biological process for genes by using just their assigned molecular function terms , where golink achieves levels of specificity , sensitivity , and accuracy above 80% .
golink is written in perl and requires the go database api ( go - db - perl ) module ( go::apphandle ) , available from the comprehensive perl archive network ( cpan ) .
golink requires access to a go database ( ideally locally installed ) . for the analyses in this paper , the go database v201212 was downloaded from http://www.geneontology.org/go.downloads.database.shtml and installed using the instructions at http://archive.geneontology.org/latest-full/readme .
all analyses were run under 64-bit linux on a single 2.3  ghz core of a multicore amd server ; however , golink can be run on both windows and macos platforms provided perl , the required cpan modules , and a go database are installed .
golink is made freely available from the project website ( http://bioinformatics.childhealthresearch.org.au/software/golink/ )   under the gnu general public license ( gpl ) .
all usage instructions are detailed in the software manual also available from the project website .
a user provides golink with a go term such as go:0005634 representing  nucleus ,  which is referred to as the  query  term , along with any desired filters ( evidence codes or annotation source databases  see below ) .
golink will then retrieve all genes that have been annotated with the query term or any of its child terms ( figure 1(a ) ) .
each gene is assessed in turn ( figure 1(b ) ) and its full complement of go annotation is assigned to one or more terms list ( figure 1(c ) ) based on whether or not the annotation fulfills the list 's specific criteria as follows.all_list : stores all the gene 's annotated go terms.query_list : stores all the gene 's annotated go terms only if the query term is one of those terms .
this method is similar to quickgo.parchild_list : stores all the gene 's annotated go terms only if all terms within the same namespace as the query term are either one of the query term 's parents or children .
query_list : stores all the gene 's annotated go terms only if the query term is one of those terms .
parchild_list : stores all the gene 's annotated go terms only if all terms within the same namespace as the query term are either one of the query term 's parents or children . 
 in an iterative process
, each list is gradually formed of  cooccurring  terms from all namespaces that fit the criterion above and that can subsequently be confidently associated with the initial query term . once the lists are compiled , scores are calculated for terms in each list to allow them to be ranked .
pr  empirical probability ratio : 
 ( 1)(t / q)(c / a ) . 

s%empirical probability similarity ratio : 
 ( 2)(t(q+ct))100 , 
 where , given term x in the context of a single list , q = the number of gene products fulfilling the criteria of the list ; t = the number of gene products fulfilling the criteria of the list that contain term x ; c = the number of gene products annotated with term x in the entire go database , given any filters ; and a = the number of gene products in the entire go database , given any filters .
the empirical probability ratio is a measure of how similar t is with c. while being a useful score this penalises interesting candidates that have high c but a slightly lower value of t. this discrepancy is addressed in the empirical probability similarity ratio , which not only accounts for the relationship between t and c but gives higher scores where t and q are also similar in value .
users can explore subsets of go annotation data by optionally applying a choice of three filters .
the namespace filter controls the nature of the terms output to the final terms list .
unfiltered , golink will return terms from all namespaces that cooccur with the query term .
however , users can request that only terms from a limited set of the three namespaces are returned . when a go term is assigned to a gene product by a go consortium member ( e.g. , flybase , sgd ) an evidence code
golink provides a database filter to target annotations made by a particular consortium member and an evidence code filter to allow fine - grained control on the level of quality of terms returned . for convenience a 
 can be specified to not include terms from a particular database or evidence code .
golink will only consider terms that are not attributed with the evidence code iea ( inferred from electronic annotation ) .
given that the iea evidence code suggests a less confident term assignment , using  !
golink terms lists can be ordered as required ; however , it is recommended that results are sorted by descending empirical probability similarity ratio ( s% ) and then descending empirical probability ratio ( pr ) .
this serves to place the terms most strongly linked to the query term at the top of the list and also mirrors the ordering method used by quickgo .
golink outputs three terms lists , each with related yet contrasting results due to the differing criteria used to compile each list .
when reviewing these lists it is important to take the initial query term used for the analysis into account .
if the query term originates from towards the top of go dag , it will be more likely to have a large number of child terms , which inevitably results in many more terms being returned to all the golink terms lists than for more specific query terms at lower levels of the go graph . in this case
conversely , lower level terms are more likely to not return any results to the more stringent golink terms lists due to there being fewer annotations where specific criteria are met . here
golink terms lists were generated using the biological process query term  regulation of gene expression 
comparisons of golink terms lists were performed using a three - way venn diagram of all terms in each list using venny .
a perl script was created that , given an initial go term , pulls all genes annotated with that term from the go database along with their other annotated go terms . using this script a  positive  and a  negative  list of genes and their go annotation
( go:0010468 ) for the positive list and a combination of  lipid metabolic process  ( go:0006629 ) and  cell motility  ( go:0048870 ) for the negative list .
genes were only included in each list if they had at least one molecular function term as part of their annotation .
golink terms lists were then generated using the biological process query term  regulation of gene expression 
( go:0010468 ) , opting to only output molecular function terms . in all cases the evidence code filter  !
a gene in the positive or negative list was deemed as having a role in the  regulation of gene expression  if any of its molecular function terms matched either the first 10 , 20 , 30 , 40 , or 50 ( separately ) molecular function terms in each of the three golink terms lists .
those genes in the negative list providing positive results were manually checked in uniprotkb   to ensure that they did not also have a role in the  regulation of gene expression  as well as either 
lipid metabolic process  or  cell motility .  a statistical assessment of performance was calculated as follows : 
 ( 3)sensitivity = tp(tp+fn),specificity = tn(fp+tn),positive  predictive  value  ( ppv)=tp(tp+fp),negative  predictive  value  ( npv)=tn(tn+fn),accuracy=(tp+tn)(tp+tn+fp+fn ) , 
 where tp = number of genes correctly predicted to be involved with the regulation of gene expression ; tn = number of genes correctly predicted to be not involved with the regulation of gene expression ; fp = number of genes incorrectly predicted to be involved with the regulation of gene expression ; and fn = number of genes incorrectly predicted to not be involved with the regulation of gene expression . in this
context , sensitivity refers to how likely golink predicts a role in the regulation of gene expression for those genes in the positive list ( tp ) , specificity measures how well golink predicts no role in the regulation of gene expression for genes in the negative list ( tn ) , ppv assesses the chance that a gene is involved with regulation of gene expression given a prediction that it is involved , and npv calculates the chance that a gene is not involved with regulation of gene expression given a prediction that it is not involved in that process .
mixed bar and line graphs displaying the results of this assessment were produced using r .
( saccharomyces genome database ) ,  pombase  ( main resource for the fission yeast schizosaccharomyces pombe ) ,  mgi  ( mouse genome informatics ) ,  zfin  ( the zebrafish model organism database ) , and  uniprotkb  , separate golink terms lists were generated using the biological process query term  regulation of gene expression 
 using data from each respective all_list , pairwise comparisons were made between terms from uniprotkb and each of the other source databases using venny .
a quickgo ( released on friday , 04 january 2013 ) terms list was exported by using the query term  regulation of gene expression  and selecting from the  ! iea
an equivalent golink dataset was generated by using the query term  regulation of gene expression  and applying the  !
the position at which each of the 100 quickgo terms fell in the sorted golink list was then obtained .
terms list was used to compare against the quickgo terms list as the term assignment criteria for both lists are the most comparable .
the exported quickgo terms list was also subjected to a separate similar statistical assessment to that described above , whereby a gene in the positive or negative list was deemed as having a role in the  regulation of gene expression  if any of its molecular function terms matched any of the molecular function terms in the quickgo terms list .
tables 1(a)1(c ) show the top 10 golink cooccurring terms and their scores in each terms list generated using the query term  regulation of gene expression  ( go:0010468 ) and applying the evidence code filter  !
 the complete lists can be found in supplementary file 1 ( see supplementary material available online at http://dx.doi.org/10.1155/2013/594528 ) .
overall the all_list , query_list and parchild_list , returned 6513 , 419 , and 322 terms , respectively .  
as expected there are a large number of unique terms in the all_list given the comparatively large number of terms returned and there are no unique terms found within either the query_list or parchild_list .
the 360 terms overlapping the all_list and query_list in the venn diagram ( figure 2 ) represent terms from gene products that are annotated with the query term but have other terms in the biological process namespace that are not either a parent or a child of the query .
the 263 terms ( figure 2 ) overlapping the all_list and parchild_list represent terms from gene products that were not directly annotated with the query term but with one of its child terms .
overall , there were 59 consensus terms across the three lists , which are listed in supplementary table 1
. a useful application of the golink term lists is , for example , to use a gene 's existing molecular function go terms to establish whether it has a role in a particular biological process , such as the  regulation of gene expression . 
if a gene has been annotated with only molecular function terms , one can compare these terms to a golink terms list for a biological process query term of interest .
if the molecular function terms are found in the golink terms list , one can infer that the gene 's function may be associated with the initial biological process query term . 

figure 3(a ) and supplementary file 2 show the sensitivity , specificity , ppv , npv , and accuracy of such an assessment matching only the molecular function terms from the three golink terms lists with a positive and negative list of genes and their annotated go terms .
they also show the proportion of genes found in each of the positive and negative lists by the three golink terms lists .
the positive list contained 41 genes ( 1163 terms , 154 mf terms ) annotated as having a role in the regulation of gene expression , while the negative list contained 154 individual genes ( 2723 terms , 503 molecular function terms ) .
the genes contained in the positive and negative lists and their associated go annotation can be found in supplementary file 2 .
the top 50 molecular function terms from each of the three golink terms lists that were used in the matching process are also shown in supplementary file 2 and a venn diagram showing their overlap can be found in supplementary figure 1 .
it should be noted that the rationale behind using only up to the first 50 molecular function terms from the golink terms lists in the matching process was simply that each list contained at least 50 terms .
overall the query_list outperformed the other two golink terms lists in predicting genes with a role in the regulation of gene expression using only their assigned molecular function terms .
for this list , sensitivity , specificity and accuracy all averaged between ~80% and 90% with the highest values occurring with comparisons using the top 30 golink molecular function terms . using these top 30 terms in
the query_list yielded 33/41 true positives and only 12/154 false positives giving a ppv of 73% and a npv of 94% .
the all_list and parchild_list showed increasingly higher sensitivity as more terms were added in to the comparison , but this was at the expense of specificity , which remained above 50% but much lower than that of the query_list .
eleven of the 154 negative genes were deemed to be actually true positives ( supplementary table 2 ) with a role in the regulation of gene expression as well as either  cell motility  or  lipid metabolic process  despite all but one not being directly annotated with
 these genes ( such as sterol regulatory element - binding proteins 1 and 2 and coup transcription factor 2 ) were removed from their respective negative lists and the statistics recalculated , which served to increase values marginally across the board ( table ( a ) in supplementary file 3 ) .
most of the false positive genes predicted by the golink terms lists were annotated with the go term  protein binding  ( go:0005515 ) .
this fairly generic term has relevance for both the negative and positive lists so users of the software are cautioned to consider this when performing analyses of this nature and may opt to remove terms from the golink lists where appropriate . indeed removing both the false positives from the negative list and removing the go term
 protein binding  from the golink terms lists have a substantial impact on the statistical assessment across the board as can be seen in figure 3(b ) . 

figure 4 shows the overlap of golink terms obtained using four species specific annotation source databases ( mgi , pombase , zfin , and sgd ) in comparison with terms obtained from the more broad uniprotkb database , using the same query term (  regulation of gene expression  ) and evidence code filter (  ! iea  ) .
for each of the species specific databases figure 4 shows that , as expected , the majority of terms ( between 53% and 87% ) in their respective lists were also in the uniprotkb list .
this group represents those terms that cooccur with the query term in annotation within both the species specific database and in annotation by uniprotkb for other species .
the remaining terms are either unique to the species specific database or to uniprotkb . in the context of the species specific
database these unique terms are either highly relevant to the species and only ever likely to have been used in annotation by the curators of the species specific database ( e.g. ,  ascospore formation  within pombase and sgd ,  fin development  within zfin ) or represent terms that only cooccur with the query term in this database compared to uniprotkb .
conversely , those terms unique to uniprotkb represent terms highly irrelevant to the species specific database ( e.g. ,  viral reproduction  in all cases ) or similarly only cooccur with the query term within uniprotkb assigned annotation . tables ( a)(c ) in supplementary file 5 show the full golink query_list and quickgo terms lists used in the comparison ( ( b ) and ( c ) ) and a combined summary detailing the positions of the top 100 quickgo terms as found in the golink query_list ( a ) .
overall the vast majority ( 72% ) of quickgo terms are confined to the top 200 golink terms and figure 5 shows that there is a 64% overlap between the top 100 quickgo and golink terms .
while all 100 quickgo terms are represented somewhere in the golink query list , table ( a ) in supplementary file 5 shows that there are distinct differences in the ordering of the results within the more significant first 100 terms .
this can be explained by the way protein identifiers are mapped within the database underlying quickgo .
the database used by golink is maintained by the go consortium . within this database ,
the database for quickgo is maintained by the goa team and provides high quality annotation for all proteins within uniprot . here
, single protein identifiers are first mapped to uniprot accessions , and in many cases one identifier can map to several uniprot accessions ( since there is likely to be several matches in trembl , the nonreviewed section of uniprot ) . in these cases
the same or similar annotation is assigned to each of the uniprot accessions . while appropriate in the context of the goa project this leads to more redundancy in annotation in the database used by quickgo than within that used by golink .
consequently , when considering cooccurring terms in quickgo , terms from those genes that have multiple entries in trembl will appear to cooccur more often than is the case within the go database , which may not be desired .
for example , in table ( a ) in supplementary file 5 , quickgo reports that the term  forebrain development ,  which appears second in the quickgo terms list , has been used to annotate 308 proteins within the goa database .
this term appears tenth in the golink list , where golink reports its assignment to 200 gene products within the go database .
furthermore quickgo and golink report that this term cooccurs in annotation with the query term on 56 and 26 occasions respectively . on closer examination ,
multiple uniprot accession numbers exist that map to single mgi identifiers annotated with the term 
forebrain development  and this has the effect of inflating the number of cooccurrences with the query term , thus influencing the s% calculation and positioning it higher in the quickgo terms list .
a more extreme example can be seen for the term  spermatid differentiation  which appears at position 11 in the quickgo list but position 531 in the golink list .
in addition , the result of the pr calculation is altered as a consequence of the increased number of proteins within the goa database . as both quickgo and golink order their results by s% and then by pr and quickgo only provides the first 100 hits
, this explains why equivalent search parameters on these similar but different source databases lead to similar but differing results for golink compared to quickgo . in order to test quickgo 's ability to predict a gene 's biological process from its molecular function a similar matching process to that undertaken for golink was performed with the top 100 cooccurring terms from quickgo .
only 9 of the 41 positive genes were found using these terms , which consequently gave lower comparative performance scores to golink ( data not shown ) , however it should be noted that as seen in table ( c ) in supplementary file 5 this list of terms only contained 4 molecular function terms to use in the comparison process .
the gene ontology is a dynamic , growing resource for the annotation of genes and gene products .
it is organized into three subontologies describing the cellular location of action of a gene or gene product , the main process it is involved in , and its role in this process . in order for the three subontologies to remain orthogonal , there are no designed links between them
. however , this prevents legitimate attempts to answer queries involving data from more than one of the subontologies .
this paper discusses a new tool , golink , which uses the go perl api to explore the go database and implement three increasingly stringent methods to compile lists of go terms cooccurring in gene product annotations with a provided query term . when given an initial query term , the three methods store either all terms cooccurring with the query term or any of its child terms ( all_list ) , only those terms cooccurring with query term ( query_list ) or only those terms cooccurring in annotations with the query term or its child terms where other annotated terms in the same namespace ( cellular component , molecular function , and biological process ) as the query term are limited to the query term and any of its parent or child terms ( parchild_list ) .
however , the key difference between the remaining golink lists and quickgo is that the latter uses a pool of terms from genes directly annotated with the query term , whereas the golink all_list and parchild_list use a pool derived from genes annotated with the query term and/or its child terms .
the main reason golink uses this extended pool is to not penalize the partonomic / taxonomic relationship between a term and its child terms .
furthermore , this allows golink to apply further logical stringency in the parchild_list to specifically exclude occurrences where the query term is annotated together with terms in the same namespace that are neither a parent nor a child of the query term .
for example , if a gene performs a particular function and operates both in the cytoplasm and the nucleus it may well be annotated with 2 cellular component terms ( nucleus and cytoplasm ) and at least 1 molecular function term . in this case
the molecular function term can not be exclusively linked with either of the cellular component terms as the molecular function term may only be relevant for the gene 's function in the cytoplasm .
if this distinction is required then the parchild_list will take this into account , whereas this functionality is not possible using quickgo .
the other main advantages of golink over quickgo lie in the distinct differences between the underlying databases of the two software and the availability of flexible methods to filter the terms assessed and those returned .
the process of identifier mapping in the goa database that provides the annotation used by quickgo leads to an increased redundancy of annotation across this database as single gene product identifiers commonly map to multiple uniprot accessions .
this results in a potentially inflated cooccurrence of go terms as demonstrated in this paper .
quickgo provides two results sets ; one unfiltered list of cooccurring terms and one with a  !
filter applied to discount associations from purely computationally assigned terms . for both , only the first 100 cooccurring terms are returned .
golink is much more flexible in that not only does it produce multiple terms lists of varying stringency , but it also provides fine - grained customizable filtering .
the database filter allows a user to identify cooccurring terms within the context of a particular species that could be diluted if searching within a broader context such as the full uniprot database .
the evidence code filter gives precise control over the quality of annotation considered during an analysis using the wide variety of evidence codes provided within the go database .
finally , applying the golink namespace filter will only return results from a defined set of the three available namespaces .
if an analysis only called for examining cooccurring molecular function terms then this assessment can only be confidently made using golink as the full complement of cooccurring molecular function terms may not be available within the static top 100 results gleaned from quickgo .
this limitation of quickgo also hinders its use in the types of analyses undertaken in the statistical assessment of golink to predict , for example , a gene 's biological process from its assigned molecular function terms , as there may not be sufficient molecular function terms returned .
quickgo cooccurring terms are immediately available via the quickgo web interface , whereas the golink algorithm can take a number of hours to run for some analyses .
there are a number of sections of the golink algorithm ; however , that lend themselves to parallel processing
. this will be available in future versions of the software ; however , its absence in the current version does not hinder the tool 's utility .
the golink method performs very well in predicting the biological process from a gene 's assigned molecular function terms , showing both high sensitivity and specificity .
this application of golink is equivalent to first predicting molecular function terms for an unknown gene using , for example , interproscan   then using golink to generate terms lists for a biological process query term of interest and then predicting the likelihood of the unknown gene being involved in that particular biological process . similarly using a cellular component query term and comparing the golink terms with interproscan or otherwise derived go terms , a researcher can predict the cellular component of a gene . in cases
where a more direct prediction of , for example , biological process is required , multiple molecular function query terms could be provided to golink with the most likely associated biological process being the highest scoring consensus term found in all terms lists .
in addition , as cellular component , molecular function , and biological process terms can be incorporated into the golink terms lists , in a similar method to bada et al .
, golink can be used to help annotators choose potential terms to assign to a gene .
furthermore , it was discovered that a number of false positives found in the statistical analysis were not specifically assigned the term  regulation of gene expression  but clearly have a role in this process .
golink is a perl based tool that finds terms cooccurring with a given query term in annotation across the full complement of the gene ontology database .
it has advantages over other existing tools and can be used in a variety of applications .
 the gene ontology ( go ) provides a resource for consistent annotation of genes and gene products that is extensively used by numerous large public repositories . 
 the go is constructed of three subontologies describing the cellular component of action , molecular function , and overall biological process of a gene or gene product . 
 querying across the subontologies is problematic and no standard method exists to , for example , find all molecular functions occurring in a particular cellular component . 
 golink addresses this problem by finding terms from all subontologies cooccurring with a term of interest in annotation across the entire go database . 
 genes annotated with this term are exported and their go annotation is assigned to three separate golink terms lists based on specific criteria . 
 the software was used to predict the most likely biological process for a group of genes using just their molecular function terms giving sensitivity , specificity , and accuracy between 80 and 90% across all the terms lists . 
 golink is made freely available for noncommercial use and can be downloaded from the project website . 