Software

DAVID Gene Functional Classification Tool: A novel biological
module-centric algorithm to functionally analyze large gene list

Da Wei Huang^1¶^, Brad T. Sherman^1¶^, Qina Tan^1^, Jack R. Collins^3^,
W. Gregory Alvord^4^, Robert Stephens^3^, Michael W. Baseler^5^, H.
Clifford Lane^2^, Richard A. Lempicki^1\*^

^1^ Laboratory of Immunopathogenesis and Bioinformatics, ^5^Clinical
Services Program, SAIC-Frederick, Inc., National Cancer Institute at
Frederick, MD 21702; ^2^Laboratory of Immunoregulation, National
Institute of Allergy and Infectious Diseases, National Institutes of
Health, Bethesda, MD, 20892; ^3^ Advanced Biomedical Computing Center,
SAIC-Frederick, Inc., National Cancer Institute at Frederick, MD 21702;
^4^Computer & Statistical Services, Data Management Services, National
Cancer Institute at Frederick, Frederick, MD 21702.

^¶^ These authors equally contributed to this study.

\* Correspondence: Richard A Lempicki. E-mail:
[rlempicki@mail.nih.gov](mailto:rlempicki@niaid.nih.gov)

E-mail addresses for authors:

Da Wei Huang [huangdawei@mail.nih.gov](mailto:huangdawei@niaid.nih.gov)

Brad T. Sherman [bsherman@mail.nih.gov](mailto:bsherman@niaid.nih.gov)

Qina Tan [tanq@mail.nih.gov](../tanq@mail.nih.gov)

Jack R. Collins <collinsj@ncifcrf.gov>

W. Gregory Alvord <gwa@css.ncifcrf.gov>

Robert Stephens <bobs@ncifcrf.gov>

> Michael W. Baseler [mbaseler@mail.nih.gov](../mbaseler@mail.nih.gov)

H. Clifford Lane <clane@niaid.nih.gov>

Richard A Lempicki
[rlempicki@mail.nih.gov](mailto:rlempicki@niaid.nih.gov)

**ABSTRACT**

Numerous high-throughput genomic technologies, including microarray and
proteomics platforms, generate large lists of genes. Major effort has
been focused on algorithms to rapidly summarize enriched biological
processes associated with gene lists. However, these algorithms usually
do not take advantage of the rich inter-relationships between genes and
biological terms. To address this issue, we developed the DAVID Gene
Functional Classification Tool, which uses a novel fuzzy clustering
algorithm to quickly condense a list of genes or associated biological
terms into organized classes of related genes or biology, called
biological modules. This organization is accomplished by mining the
complex relationships found in 14 sources of functional annotation.
Classes are ordered based on overall enrichment scores of each member,
and tools are provided to better understand gene-to-term relationships
within a class.

The DAVID Gene Functional Classification Tool provides a powerful method
for investigators to group functionally related genes and terms into a
manageable number of functional biological modules to more efficiently
interpret genome-scale datasets. The tool is a component of the freely
available DAVID Bioinformatics Resources
([http://david.abcc.ncifcrf.gov](http://david.abcc.ncifcrf.gov/)).

**RATIONALE**

Biological interpretation of large gene lists derived from
high-throughput genomic or proteomic studies can be a challenging and
daunting process. Some of the difficulties include: 1) acquiring large
amounts of functional annotation for every gene; 2) the distributed
nature of annotation across numerous sources, i.e., not centralized; 3)
summarizing which genes are associated with specific biological
processes and ranking these processes by over-representation analysis;
4) condensing repetitive or redundant annotation data; 5) identifying
functional biological modules consisting of related genes and terms, and
6) viewing inter-relationships between groups of genes and groups of
biological terms. A number of publicly available bioinformatics tools
have addressed points 1-3 above, including, but not limited to, GoMiner,
DAVID, EASE, GOstat, Onto-express, GoToolBox, FatiGO, GOSSIP, GFINDer,
GOBar, etc. \[1-25\]. The power of many of these applications is to
systematically highlight the most over-represented biological terms, out
of a list of hundreds or thousands of terms, to increase the likelihood
of investigators identifying biological processes most pertinent to the
biological phenomena under study \[17\]. While these tools are extremely
useful, they are still weak in mining the many-to-many gene-to-term
relationships found in functional annotation databases, as well as in
condensing redundant content.

Individual genes can clearly be associated with multiple biological
terms and, conversely, individual biological terms can be associated
with multiple genes. These associations form a complex relationship
network of "many-genes-to-many-terms" which represents the true complex
nature of biological processes. Data-mining tools that can extract these
complex and redundant relationships should be able to identify
functional gene-term biological modules. This identification can be
accomplished by using exploratory statistical methods that identify
groups of genes sharing similar biological terms or, alternatively,
identifying groups of biological terms sharing similar genes. For
example, if a subset of genes in a list is sodium transporters, then one
can expect that they will have major functional annotations in common. A
method that can group these genes based on the strength of overlap of
the functional annotation should identify modules of related genes and
terms. Similarly, terms that have many genes in common can also be
grouped into a module of related terms and genes; for example, the terms
"apoptosis", "cell death", "death", and "regulation of cell death" will
be grouped together because these terms share a large number of common
genes. The advantages of this method of classifying groups of genes and
terms into biological modules are: 1) it largely reduces redundant
results into a manageable size; 2) it is much easier to understand and
visualize gene-to-gene, term-to-term, and gene-to-term relationships,
since related genes and terms are brought together in one place; and 3)
it is much easier to relate biological modules of interest to a study
than it is to relate hundreds of individual terms.

To this end, we developed the DAVID Gene Functional Classification Tool
and the DAVID Functional Annotation Clustering Tool to provide a
module-centric approach for functional analysis of large gene lists.
First, we developed a new method to measure gene-gene similarity, based
on the assumption that genes that share global functional annotation
profiles are functionally related to each other. Conversely, we measure
term-term similarity based on the assumption that terms that share
global gene profiles are functionally related to each other. Then, a
novel fuzzy heuristic partitioning method was developed to group related
genes or terms into functional groups (biological modules) based on the
similarity distances measure. The fuzziness feature of the partitioning
method allows a gene or term to participate in more than one functional
group, better reflecting the true "multiple-roles" nature of genes that
can be lost if exclusive methods, such as Hierarchical, K-mean, or SOM
clustering are used. Functional groups are ranked based on all group
members' overall participation in the enriched biological processes
associated with the total gene list. A global view of group-to-group
relationships is also provided through a unique fuzzy heat map
visualization. A subset of "drill-down" functions associated with each
biological module allows investigators to explore and visualize
relationships between genes and terms. In this paper, we will mainly
describe the key algorithms associated with the DAVID Gene Functional
Classification Tool, illustrate the usefulness of several of the
functionalities, and demonstrate how quickly investigators can apply the
information in a biological module to their study.

**IMPLEMENTATION**

The DAVID Gene Functional Classification Tool
(<http://david.abcc.ncifcrf.gov/gene2gene.jsp>) and DAVID Functional
Annotation Clustering Tool (<http://david.abcc.ncifcrf.gov/summary.jsp>)
are two new components integrated in DAVID Bioinformatics Resources
([http://david.abcc.ncifcrf.gov](http://david.abcc.ncifcrf.gov/)). They
were designed as a server-client application on a UNIX server, with the
Tomcat web server as the serving engine. Java is the primary language
used for calculations as well as the user interface which utilizes Java
Server Page (JSP) technology. In-memory Java data objects containing all
mappings between genes and annotation were developed to advance the
calculation speed. The DAVID Functional Annotation Clustering Tool uses
the same algorithm as DAVID Gene Functional Classification Tool, but
conversely. Therefore, to illustrate the key scientific concepts, we
only describe the major procedures of the DAVID Gene Functional
Classification Tool; these procedures consist of three major steps:
measurement of functional relationship of gene pairs, heuristic fuzzy
partitioning of genes into functional gene groups, and visualization of
results in text and graphic modes (figure 1).

**A novel approach to measure functional relationship of gene pairs
based on the similarity of global annotation profiles.**

The traditional ways of grouping related genes are based on either
sequence similarity (sequence homologs), functional categories (protein
domain families), or co-expression clusters (microarray clusters). In
fact, the majority of co-functioning genes are neither sequence-related
nor in the same protein families, such as genes in the same pathway.
Therefore, the traditional phylogenetic grouping methods are powerful
for evolution-based studies, but too specific and strict to be of much
use in classifying genes for the purpose of functional annotation. We
propose a novel method to identify related genes by measuring the
similarity of their global annotation profiles based on the hypothesis
that if two genes have similar annotation profiles, they should be
functionally related. This method is able to identify much broader gene
groups in which genes share major common biological features as well as
tolerate some differences. For example, many different types of genes,
with or without too much sequence similarity, could be grouped into a
transcription regulation class. We believe that the broader functional
groups are more useful for functional annotation purposes and hence,
biological interpretation.

Firstly, a gene-term annotation matrix (figure 2 A) was compiled in a
binary mode using thousands of annotation terms in 14 annotation
categories (including Gene Ontology \[GO\], Biological Process, GO
Molecular Function, GO Cellular Component, KEGG Pathways, BioCarta
Pathways, Swiss-Prot Keywords, BBID Pathways, SMART Domains, NIH Genetic
Association DB, UniProt Sequence Features, COG/KOG Ontology, NCBI OMIM,
InterPro Domains, and PIR SuperFamily Names) collected in the DAVID
knowledgebase \[26\] (additional file 7). Then*, Kappa* statistics, a
chance-corrected measure of co-occurrence between two sets of
categorized data, is adopted to statistically measure the annotation
co-occurrence of any given gene pairs \[27\] (additional file 11). Since
the annotation profile is in a binary categorical scale, *kappa*
statistics are more suitable than the Pearson correlation, which is
typically used for continuous, non-categorical data.

For given annotation profiles of genes *m* and *n*, *O~mn\ ~*represents
the observed co-occurrence; *A~mn\ ~*represents chance co-occurrence;
and *K~mn\ ~*is the *kappa* value representing the degree of annotation
co-occurrence between genes *m* and *n.*

Where *K~mn\ ~*is 1 for perfect co-occurrence and 0 for co-occurrence no
better than random chance (figure 2 B).

While building the binary annotation matrix, the annotation terms could
be redundant or in a structured relationship because many terms from
different sources may have the same biological meaning; in addition, GO
terms fall into a parent-child relationship in the GO hierarchical
structure. We broke all redundant and structured terms into
"independent" terms in a flat, linear collection (figure 2 A). We
believe that an equally weighted, linear, all-inclusive strategy can
greatly simplify the situation, as well as maximally leverage the
heterogeneous annotations in the similarity measurement (additional file
12) . To answer the question, "Can this strategy specifically detect the
real relationship of gene-gene?", we conducted three studies. The first
was to compare the *kappa* score distribution of every possible pair of
human genes (\~300 million pairs) to that of reported human
protein-protein interaction pairs \[28\]. If the similarity measurement
can specifically detect gene-gene relationships rather than random
noise, we would expect to see the score distribution of the
protein-protein interaction pairs to shift to the higher value end
(figure 3 B). The second study was to detect the *kappa* score
distribution of genes specifically selected because their names
contained the word *chemokine*. Since we selected genes with an extreme
bias of similarity, we expected the *kappa* scores to give much higher
values (figure 3 B). The third study was to compare the *kappa* score
distribution of all human gene pairs to that of artificial gene pairs
with annotation profiles randomly generated, based on the true human
annotation frequency. It was expected that the simulated *kappa* scores
would only be located in the lower value end (figure 3 B). The three
independent studies, combined with the extensive test analysis on
microarray datasets, strongly supported the strategy that functional
similarity measurement is able to specifically detect gene-gene
relationships, particularly for the pairs with a *kappa* score 0.35 or
above, as suggested by our randomization study (figure 3B). However,
since the measurement relies on known annotation profiles, this method
will obviously not work for the genes that lack annotation (figure 3 A)
(additional file 10).

**A novel fuzzy heuristic partitioning approach to classify a gene list
into functionally related groups based on the functional similarity
scores.**

After the *kappa* score matrix of all possible pair-wide genes is
calculated, it is possible to classify the highly related genes. We
examined the typical clustering methods, including hierarchical tree,
K-mean, hierarchical, FANNY, and SOM. All of them produced weaker
clustering results (additional file 5) with our test datasets. The poor
clustering results stem from one or more of the following weaknesses
associated with the aforementioned clustering algorithms: 1) Genes must
be assigned to one cluster, even though their absolute relationship is
weak to all clusters. This results in higher contamination of clusters
with noise by forcing membership of weakly related genes. 2) Genes can
only belong to one cluster, which does not align well with the basic
biological nature that of genes, in that one gene could participate in
multiple, different roles. 3) Outliers and uneven cluster sizes can
greatly affect clustering quality. 4) It is difficult to know the
optimal K (number of clusters) for K-means, FANNY, or SOM.

This situation motivated us to develop a novel partitioning approach,
heuristic fuzzy multiple-linkage partitioning, to better reflect the
structure of functional annotation data. It can be described as three
major steps (figure 4, additional file 13):

-   Multiple initial seeds: Each gene is selected to serve as a medoid,
    or center of an initial cluster, as long as it meets minimum
    relatedness (user input parameters, such as genes, are related to
    more than 3 other genes with *kappa* \>0.3) to other genes in the
    list.

-   Merge seeds by a minimum, multiple linkage (i.e., merge two seeds
    when they share 50% of their group members).

-   Repeat Step 2 until no more merging can occur.

This novel method works better than others for this particular type of
analysis (additional file 5). This method: 1) eliminates overall
irrelevant/weak elements, as orphan genes, to significantly push the
signal out of noise; 2) allows for fuzziness by allowing genes to be
assigned to more than one cluster which aligns with the biological
nature; 3) automatically determines the number of clusters (K value); 4)
generates grand groups for easy interpretation; 5) tolerates outliers
extremely well by excluding them in step 1.

There is no gold standard or null hypothesis to evaluate clustering
methods and hence no right or wrong answers for any given clustering
algorithm. One method may work better than others in the sense that it
is more sensitive to the natural structure of a particular problem.
However, this method, like any other heuristic approach, has the common
weakness that an improper running criteria setting can lead to distorted
results. In order to aid less advanced users with the setting of these
criteria, we preset 5 general levels representing combinations of the
detailed settings from very low to very high stringencies; and based on
our extensive tests on multiple datasets, the default stringency level
(medium) (additional file 10) should be optimal for most of the cases.

To further examine the sensitivity (type I error) and specificity (type
II error) of the heuristic fuzzy partitioning method, we tested the
clustering method with two independent gene lists from our microarray
studies. For the sensitivity test, each orphan gene eliminated by step 1
was post-examined to determine if it should be included in any of the
final groups by satisfying the minimum *kappa* threshold (\>=0.35) with
a majority of group members (\>=50%). Any orphan genes qualifying to be
post-assigned to one or more of the final groups would be considered as
"leaks" (false negative). We observed the leaking rate (type II error)
to be between 1 and 2%, which falls into the generally accepted rate of
less than 5%. For the specificity test, *kappa* distributions of all
possible gene pairs within each group show 1-3% of *kappa* scores below
the minimum *kappa* threshold. We found that a typical classification
job usually results in an \~1-5% false positive rate, an \~1-5% false
negative rate and with \~20% of genes being eliminated as "orphan
genes." Most importantly, the method is able to identify key members of
groups so that the major biology of each group can quickly be
determined. Since the analytic approach is biological module-centric,
the major biology associated with each gene group is determined by the
majority of gene members rather than by individual genes. Thus, the
biology of each group should be very stable, even though there is a
small chance that a few members could be missing (type II error). In
summary, this clustering method shows reasonable performance by
eliminating irrelevant, "noisy" genes and by bringing together strongly
related functional groups, while maintaining the fuzzy nature of
biology.

The last question is, "Which final functional gene groups are more
significant for the experiment?" We extended the traditional enrichment
analysis logic so that a gene group is more important if a majority of
its gene members is associated with highly enriched annotation terms as
found in the traditional enrichment analysis of the total gene list.
Thus, the enrichment score of each group is measured by the geometric
mean of EASE Scores (modified Fisher Exact) \[2\] of the enriched
annotation terms related to the group. In order to emphasize that the
geometric mean is a relative score instead of an absolute p-value, minus
log transformation is applied on the geometric mean (additional file 6).
Therefore, the group enrichment scores are intended to order the
relative importance of the gene groups instead of as absolute decision
values. A higher score for a group indicates that the group members are
involved in more important (enriched) roles.

**Visualization of results in a very simple text format and a novel
fuzzy heat map view**

We implemented both a very simple text format (figure 5) and a
comprehensive novel fuzzy heat map graphic view (additional file 4) to
present the functional groups derived from the above procedure. The text
format simply lists all functional gene groups identified by the
algorithm. Although it looks like linear format, the view allows the
user to visualize the multidimensional data of the groups, i.e., group
members consisting of multiple related genes and terms. Users are able
to easily explore the major functional groups by viewing many related
genes and annotation terms brought together by the tool. Some accessory,
"drill-down" functionalities (e.g., Enriched Terms Report, 2-D View,
etc.) (figure 5) are available for each functional group for users to
rapidly explore the associated biology in detail. For example, the
"Enriched Term Report" button lists the major annotation terms
associated within the functional groups based on the DAVID enrichment
engine; the "2-D View" button gives the detailed relationship of
genes-to-terms in a 2-dimensional heat map view so that the user is able
to examine the rich relationship of related genes and annotations
in-depth (figure 6); the "Related Genes" button allows users to refine
the group gene members in different scopes, which can extend the
membership of interest and also correct potential type I & II errors in
the clustering algorithm (additional file 9 C). Furthermore, the text
format provides links to the list of orphan genes not classified into
any functional groups. These genes are orphaned because they do not meet
one or more of the partitioning criteria (i.e., group membership
thresholds, etc.) The list is provided since they may be important genes
for the user to examine.

**RESULTS AND DISCUSSION**

We examined the newly developed biological module-centric tools (see
additional file 8 for graphic tutorial of using the tools) on a
published microarray dataset of which the gene list was available as
demo list 2 on our tool entry page \[29\]. It is important to mention
that, to avoid potential bias, the dataset of this case study is
different than those used during algorithm development. In this example
dataset, authors treated freshly isolated peripheral blood mononuclear
cells (PBMCs) with an HIV envelope protein (gp120) and further measured
genome-wide gene expression changes using Affymetrix U95A chips \[29,
30\]. This study provides a global view of the complex interaction
between viral and cellular factors, which is an essential mechanism for
HIV replication in resting or suboptimally activated PBMCs. A
functionally significant annotation of \~400 genes (additional file 1)
derived from the microarray experiment, was classified by the authors
into five major functional categories: Cytokines, Chemokines,
Transcription Factors, Kinases, and Membrane Fusion \[29\]. While the
Cytokine and Chemokine categories were systematically highlighted by
EASE (a Gene Ontology enrichment analysis based on the Fisher Exact
Test) \[2\], other annotation categories reported in the publication
were discovered through semi-manual analysis by bioinformatics experts
with an advanced level of knowledge of both biology and computer tools.

**The same data re-analyzed by typical Functional Annotation Tools**

After the continuous additions of annotation for genes as well as the
refinement of gene-term enrichment algorithms during the years since
this study was published, it is interesting to see how the systematic
results from current functional annotation tools compare to those
reported in this publication. Some of the popular functional annotation
tools, such as DAVID Gene Functional Annotation Tool, GOStat, and
GoMiner \[1, 3, 16\], were chosen to identify major biological terms
with the same gene list, respectively. In order to maximally reflect the
design spirit of each tool and also make the results more comparable, we
kept all default parameters of the tools unchanged, except for
synchronizing the data coverage scope within all GO levels (DAVID covers
multiple data sources and GOstat covers GO level 3 or above by default).
Although all of the testing tools are based on similar gene-term
enrichment algorithms, the sensitivity and specificity could be
different due to different updates of GO data content, different
background gene lists, different ranking score algorithms, different
gene ID mapping schemes, etc. After obtaining hundreds of annotation
terms reported by each of the above tools, the top 20 terms were
compared with each other (table 1). Approximately 30% of the top terms
overlapped between at least two of the tools, e.g., cytokine/chemokine
activity, inflammatory response, etc. Even though the results from the
tools all point in the same biological direction, there are four obvious
problems: 1) Redundant/similar terms appear in different (significance)
levels within the reports, (e.g., response to stress, response to
wounding, response to pathogenic bacteria, response to other organisms,
response to external biotic stimulus, inflammatory response, etc.),
which makes it difficult for the user to gain or maintain a clear focus
of the whole biological picture. It is not easy for users to
comprehensively pool all genes related to the same key biology without
manually summarizing all related redundant terms. 2) The redundancy of
terms could largely dilute the focus on other key biology which has few
or no redundancies (e.g., only one term is for establishment of cellular
localization). If several redundant terms are represented in the top of
the list, less redundant terms may be pushed down the list, possibly
decreasing the chance of discovery; for example, a transcription
regulation term, reported in an original publication, was not listed in
the top 20 by any of the tools). 3) In contrast, due to differences of
the annotation levels of different sources, redundant terms may
themselves be diluted. While alone a single term may not be at the top
of the list, in combination with redundant/similar terms, the biological
function may be very significant. 4) Current tools do not emphasize the
inter-relationships between key biological terms (e.g., relationships
between chemokine/cytokine and signal transduction).

In conclusion, the recent improvement of functional annotation tools
provides a powerful means for users to systematically identify key
biological functions associated with a gene list. However, due to the
weaknesses discussed above, refinement of current gene-term enrichment
algorithms and improvement of software usability alone may not address
all the issues. Therefore, the development of novel alternative
algorithms as a complement is still very necessary.

**The same data analyzed by the DAVID Gene Functional Classification
Tool**

The same gene list (additional file 1) was submitted to our newly
developed DAVID Gene Functional Classification Tool described previously
(additional file 8). The tool is able to efficiently handle up to 3000
genes at a time, within a few seconds. The tool classified the \~400
genes into 16 functional groups (table 2 and additional file 2). The
result is much more focused, simplified, and in a manageable size for
investigators' interpretation comparing to a few hundred terms, of which
many are redundant from the traditional tools. More importantly, all
five reported annotation categories are covered by the 16 functional
groups (table 2). In addition, the tool also lists another 11
interesting gene groups not reported in the original publication. For
example, group 13 (tubulin genes) plays a critical role in the
nucleation of microtubule assembly. Some studies suggest that HIV
infection leads to enteric microtubule depolymerization, resulting in
increases of HIV permeability \[31\]. This tool focuses on the overall
major common annotation terms associated with a gene group rather than
one term or one gene at a time, thereby producing clearer, more concise
results that can better allow for focus on the major biology of the
experiment. The tool simplifies the results by condensing the redundant
terms and summarizing inter-relationships. This analytic logic and
presentation format closely mimics how the human brain works and the
results better represent the nature of biology.

The DAVID Gene Functional Classification Tool allows users to further
explore a given biological module/gene group in-depth. For example, the
*enriched terms* button "2D View" is able to list all related terms and
genes for these kinase group. Thus, a user who is not familiar with
kinases can explore the terms of kinase activity, transferase activity,
ATP-binding, nucleotide binding, protein metabolism, tyrosine
specificity, serine/threonine specificity, regulation of G protein
signaling, and signal transduction, etc. in one view at the same time
(figure 6). Therefore, we can quickly learn the biology for the kinase
group with the above related terms in a single view and also identify
the fine differences among them. For example, there are two G-protein
coupling receptor kinases, three protein tyrosine kinases and 6 kinases
involved in cell surface receptor-linked signal transduction among the
23 kinases within the group (figure 6). The fine details may be very
important to pinpoint the key biology associated with the study.

Furthermore, the DAVID Gene Functional Classification Tool allows one
gene to be present in more than one functional group, which closely
reflects the nature of biology whereby one gene could play multiple
roles in different processes. This fuzziness feature improves the
chances of discovery by maximally preserving all of the true
relationships. For example, *general transcription factor II H
(GTF2H4/TFIIH, 41371_at)* was assigned to group 2 (transcription
regulation group) and group 5 (DNA damage/repair group), respectively
(additional file 2). Some studies suggest *TFIIH* increases polymerase
processivity in HIV infection \[32\]. Currently, there are few reports
about the *TFIIH* DNA repair mechanism being involved in HIV infection,
although this DNA repair mechanism could be essential in HIV
integration. Hence, the fuzzy capability allows users not only to focus
on the *TFIIH* transcription regulation role but also to consider the
possible role in HIV integration through the DNA repair mechanism. For
another example, *ring finger protein 40 (RNF40)* is in group 2
(transcription regulation group) and group 10 (chromosome assembly)
(additional file 2). Although the biological significance of ring finger
protein in HIV infection is still largely unclear, the annotation result
points out two potential areas for further exploration: 1. Ring finger
protein regulates the TNF-related transcriptional pathway, which is
critical to many aspects of HIV transcription. 2. It plays some role in
DNA packaging and chromosome integration. Thus, the fuzziness capability
is a powerful feature to maximally preserve the biological patterns and
to discover fine differences of the same gene as compared to the
exclusive methods.

The sensitivity of the Functional Classification Tool can vary with
different datasets and stringency criteria. If the running criteria are
not suitable to a particular dataset, the output can be distorted. In
such cases, some exploration of different running stringencies is
necessary in order to obtain the optimal results to meet the expectation
of the study.

**The same data analyzed by the Functional Annotation Clustering Tool**

Due to the redundancy problems in the results obtained from traditional
annotation tools (table 1), a Functional Annotation Clustering Tool was
also developed to organize the highly redundant annotation term results
into a simplified and clustered format. This new format allows
investigators to focus on an annotation group level by quickly skipping
many redundant terms within the group. Compared to 222 individual terms
reported by the DAVID Functional Annotation Tool, a traditional
term-centric enrichment method, the new tool was able to organize them
into 65 annotation clusters (additional file 3). For example, the
Annotation Cluster 3 (immune-response group) consists of 11 redundant
terms, i.e., response to stress; inflammatory response; response to
external stimulus; response to pest, pathogen or parasite; etc. These
similar terms are spread throughout the traditional term-centric
enrichment report list of 222 terms. Most importantly, the top 20
annotation clusters (table 3 and additional file 3) contain all
annotation categories reported by the original publication, as well as
interesting groups not identified. The highly organized and simplified
annotation results allow users to quickly focus on the major biology at
an annotation cluster level instead of trying to come to the same
conclusions by putting together pieces which are scattered throughout a
list of hundreds of terms. In addition, the annotation cluster is
helpful in comprehensively pooling all related genes associated with an
annotation cluster consisting of many related terms. For example, each
of the 11 terms within Cluster 3 (immune-response cluster) associates
with different genes. A pooled gene list brought together by cluster 3
regarding immune-response could be much more comprehensive, compared to
the genes selected from one or a few individual terms. Moreover, the
tool could possibly bring up the terms not passing the minimum
enrichment threshold but highly related to other terms with significant
enrichment scores. In conclusion, the clustered result condenses the
data into smaller, much more organized biological term modules, which
allows investigators to quickly and comprehensively focus on the key
biology of interest.

**CONCLUSIONS**

The DAVID Gene Functional Classification Tool is able to organize and
condense large gene lists into biologically meaningful modules. It
changes functional annotation analysis from term- or gene-centric to
biological module-centric. This method takes into account the redundant
and network nature of biological annotation contents in order to
concentrate on the larger biological picture rather than an individual
term or gene. The DAVID Gene Functional Classification Tool is
complementary to other functional annotation tools. The unique
clustering and visualization techniques developed for this project can
be extended to other applications, such as microarray expression
analysis.

**LIST OF ABBREVIATIONS**

**DAVID** Database for Annotation, Visualization and Integrated
Discovery

**GO** Gene Ontology

**PIR** Protein Information Resource

**NCBI** National Center for Biotechnology Information

**NIAID** National Institute of Allergy and Infectious Diseases

**ABCC** Advanced Biomedical Computing Center

**SOM** Self-Organizing Map

**PBMC** peripheral blood mononuclear cells

**LIB** Laboratory of Pathogenesis and Bioinformatics

**ACKNOWLEDGEMENTS**

> The authors are grateful to the reviewers for their constructive
> comments. We would like to thank David Liu and David Bryant in the
> ABCC group for database and web server support. We thank Yongjian Guo
> in BSSP/OTIS/NIAID for his comments on the manuscript. We would like
> to thank Wei Gao, Melaku Gedil, Ping Ren, and Jun Yang in the LIB
> group for helpful works and discussions, and Doug Powell in the CS&S
> group for helpful statistical discussions. We also thank Bill Wilton
> and Mike Tartakovsky for information technology and network support.
> The project has been funded in whole with federal funds from the
> National Institute of Allergy and Infectious Diseases (NIAID),
> National Institutes of Health (NIH) through the National Cancer
> Institute, National Institutes of Health, under contract N01-CO-12400.
> The content of this publication does not necessarily reflect the views
> or policies of the Department of Health and Human Services, nor does
> mention of trade names, commercial products, or organizations imply
> endorsement by the U.S. Government.

**REFERENCES**

**FIGURE LEGENDS**

![](media/image1.png){width="5.624305555555556in" height="5.375in"}

**Figure 1.** Flow chart of the procedures for the DAVID Gene Functional
Classification Tool and the DAVID Functional Annotation Clustering Tool.

A

  ---------------- --------- ----------- --------- --------- ----------- ----------
                   Cell      Apoptosis   Ph domain Sh2       Apoptosis   Membrane
                   death                           domain    pathway     

  Gene a           1         1           0         0         1           0

  Gene b           1         1           0         1         1           0

  Gene c           1         0           0         1         1           1

  Gene d           1         1           0         0         1           1

  Gene e           0         1           1         1         1           1

  Gene f           0         0           1         1         0           1

  Gene g           0         0           1         1         0           1

                                                                         
  ---------------- --------- ----------- --------- --------- ----------- ----------

B

**Figure 2.** A hypothetical example to detect gene-gene functional
relationship by *kappa* *statistics*. A. The all-redundant and
structured terms are broken into 'independent' terms in a flat linear
collection. Each gene associates with some of the annotation term
collection so that a gene-annotation matrix can be built in a binary
format, where 1 represents a positive match for the particular gene-term
and 0 represents the unknown. Thus, each gene has a unique profile of
annotation terms represented by a combination of 1s and 0s. B. For a
particular example of genes *a* and *b*, a contingency table was
constructed for *kappa statistics* calculation. The higher *kappa* score
(0.66) indicates that genes *a* and *b* are in considerable agreement,
more so than by random chance. To flip the table 90 degrees, the *kappa*
score of term-term can be achieved, based on the agreement of common
genes (not shown).

Figure 3

A

> ![](media/image2.png){width="5.620833333333334in"
> height="3.015972222222222in"}

B.

![](media/image3.png){width="5.252777777777778in"
height="3.5694444444444446in"}

**Figure 3.** The gene-gene functional relationship can be specifically
detected by *kappa* *statistics*. A. *Kappa* scores were calculated for
all possible combination of human gene-gene pairs (\~300 million). Only
gene-gene pairs with a higher number of annotation terms in common
possibly have good *kappa* values. B. *Kappa* scores were calculated for
all possible human gene-gene pairs, gene-gene pairs with randomized
annotation terms, all collected protein-protein interacting pairs, and
all *chemokine* gene pairs respectively. The distributions of those
*kappa* scores from protein-protein interacting pairs (pink) and
*chemokine* gene pairs (light blue) significantly shift to the high
value end compared to human total (blue); conversely, the *kappa* score
distribution (yellow) of gene pairs with randomized annotation terms
remains in the lower value end below 0.35. Interestingly, for the human
genome (blue), over 50% of the *kappa* scores equal 0 (no detectable
relationships) and \>95% are lower than 0.35. All together, this
indicates that *kappa* statistics can specifically detect the functional
relationship of gene-gene.

![](media/image4.png){width="5.844444444444444in" height="4.8125in"}

**Figure 4.** Graphic illustration of the heuristic fuzzy partition
algorithm. A. Hypothetically, each element (gene) can be positioned in a
virtual two-dimensional space, based on its characters (annotation
terms). The distance represents the degree of relationship (*kappa
score*) among the genes. B. Any gene has a chance as a medoid to form an
initial seeding group. Only the initial groups with enough closely
related members (e.g., members \>3 and *kappa* score \>= 0.4) are
qualified (solid-line circle). Conversely, unqualified ones are in
dash-line circles. Importantly, the genes not covered by any qualified
initial seeding group are considered as outliers (in gray), which are
carried along, but do not participate in the next steps. C. Every
qualified initial seeding group is iteratively merged with each other to
form a larger group based on the multi-linkage rule, i.e., sharing 50%
or more of memberships, until all secondary clusters (thicker oval) are
stable. D. Finally, three final groups (thicker oval) are formed because
they can no longer be merged with any other group. One gene (in red)
belonging to two groups represents the fuzziness capability of the
algorithm. And outliers (in gray) are removed for clearer presentation.

![](media/image5.png){width="6.49375in" height="6.131944444444445in"}

**Figure 5.** A text format report from the Gene Functional
Classification Tool. The example shows the output of 16 genes
(additional file 1) analyzed by the tool with default setting. Without
prior knowledge, the tool is able to classify genes into three gene
functional groups. On each group header, a set of buttons is provided
for in-depth exploration of the annotation for the group. "T" reports
the major enriched annotation terms associated with the group. The "Heat
Map" symbol provides a detailed graphic view of gene-term relationships.
"RG" searches other related genes in the genome but not in the list.

![](media/image6.wmf){width="6.495833333333334in"
height="3.4319444444444445in"}

**Figure 6.** An example of genes-to-terms 2-D view. All the related 23
kinase genes and their associated annotation terms from gene group 3
(kinase group) for demo list 2 are displayed in a 2-D heat map-like
interactive graphic view. Green represents the positive association
between the gene-term; conversely, black means the unknown relationship.
The annotation terms are ordered based on their enrichment scores
associated with the group. The kinase commonly related annotations (big
green block) are shown on the left side, and the scattered pattern
(green and black) on the right side shows the functional difference.

**TABLES**

**Table 1**

  ---------- ------------------- --------------------- ----------------------
  **Rank**   **GOMiner**         **DAVID**             **GOstat**

  1          inflammatory        response to           cell-cell signaling
             response            pathogenic bacteria   

  2          clathrin coat of    chemokine activity    response to pest,
             coated pit                                pathogen or parasite

  3          viral genome        cell migration        response to stress
             replication                               

  4          morphogenesis       clathrin-coated       response to external
                                 vesicle               biotic stimulus

  5          cytokine activity   clathrin vesicle coat response to wounding

  6          establishment of    clathrin coated       negative regulation of
             spindle             vesicle membrane      biological process
             localization                              

  7          cell communication  receptor binding      negative regulation of
                                                       physiological process

  8          establishment of    response to other     cytoplasmic vesicle
             mitotic spindle     organism              membrane
             localization                              

  9          regulation of       kinase activity       cytoplasmic vesicle
             cellular process                          membrane

  10         regulation of       RNA polymerase II     negative regulation of
             biological process  transcription factor  cellular process
                                 activity              

  11         development         clathrin coat         regulation of
                                                       biological process

  12         signal transduction establishment of      cell proliferation
                                 cellular localization 

  13         viral infectious    cell differentiation  phagocytic vesicle
             cycle                                     

  14         positive regulation cell death            calpain inhibitor
             of protein                                activity
             metabolism                                

  15         regulation of       regulation of isotype cell adhesion
             protein-nucleus     switching             
             import                                    

  16         immune cell         membrane-bound        negative regulation of
             migration           vesicle               cellular physiological
                                                       process

  17         organ development   cell cycle            vesicle membrane

  18         organogenesis       membrane fraction     inflammatory response

  19         chemotaxis          angiogenesis          cell communication

  20         taxis               cell communication    cell differentiation
  ---------- ------------------- --------------------- ----------------------

Table 1. The top 20 enriched terms for demo list 2 by various functional
annotation tools. The example gene list was analyzed by GoMiner, DAVID,
and Gostat, respectively. The annotation data coverage was set to GO
terms of all levels, and all other parameters are used by the tool's
default setting. Only the top 20 terms from each tool are selected in
the table. Many of the terms are in redundancy.

Table 2.

  ----------------- -------------------------------------- ---------------
  **Functional      **Associated Biology**                 **Group
  Group \#**                                               Enrichment
                                                           Score**

  Gene Functional   Chemokine/cytokine                     3.37
  Group 1                                                  

  Gene Functional   Transcription regulation               2.89
  Group 2                                                  

  Gene Functional   Signal transduction/membrane receptors 2.68
  Group 3                                                  

  Gene Functional   Kinase activity                        2.54
  Group 4                                                  

  Gene Functional   DNA damage/repair                      2.23
  Group 5                                                  

  Gene Functional   Iron binding                           2.05
  Group 6                                                  

  Gene Functional   RNA processing/splicing factors        1.81
  Group7                                                   

  Gene Functional   Organic acid transport                 1.71
  Group 8                                                  

  Gene Functional   Cation/ion transport                   1.69
  Group 9                                                  

  Gene Functional   DNA metabolism/chromosome organization 1.53
  Group 10                                                 

  Gene Functional   Cellular macromolecule catabolism      1.41
  Group 11                                                 

  Gene Functional   Metalloprotease                        1.34
  Group 12                                                 

  Gene Functional   Macrotubule                            1.24
  Group 13                                                 

  Gene Functional   Protein localization/fusion            1.17
  Group 14                                                 

  Gene Functional   Amine metabolism                       1.1
  Group 15                                                 

  Gene Functional   RAS small GTPase                       1.03
  Group 16                                                 
  ----------------- -------------------------------------- ---------------

Table 2. Gene functional groups identified by the Functional
Classification Tool. The genes of demo list 2 were analyzed by the
Functional Classification Tool. The major biology terms associated with
each group are manually summarized based on gene-term enrichment buttons
provided for each functional group.

Table 3

  ------------------- -------------------------------------- --------------
  **Annotation        **Representative Annotation Terms**    **Enrichment
  Cluster**                                                  Score**

  Annotation Cluster  negative regulation of biological      5.38
  1                   process                                

  Annotation Cluster  signal transduction                    4.36
  2                                                          

  Annotation Cluster  inflammatory response                  3.75
  3                                                          

  Annotation Cluster  extracellular region                   3.69
  4                                                          

  Annotation Cluster  cytokine/chemokine activity            3.12
  5                                                          

  Annotation Cluster  viral genome replication               2.23
  6                                                          

  Annotation Cluster  cell death/apoptosis                   2.19
  7                                                          

  Annotation Cluster  regulation of biological process       2.18
  8                                                          

  Annotation Cluster  organ morphogenesis                    2.06
  9                                                          

  Annotation Cluster  regulation of cell cycle               2.01
  10                                                         

  Annotation Cluster  positive regulation of biological      1.87
  11                  process                                

  Annotation Cluster  biological process unknown             1.76
  12                                                         

  Annotation Cluster  physiological interaction between      1.69
  13                  organisms                              

  Annotation Cluster  antimicrobial humoral response         1.52
  14                                                         

  Annotation Cluster  transcription cofactor activity        1.46
  15                                                         

  Annotation Cluster  integral to plasma membrane            1.44
  16                                                         

  Annotation Cluster  coated vesicle membrane                1.42
  17                                                         

  Annotation Cluster  DNA repair/DNA metabolism              1.38
  18                                                         

  Annotation Cluster  kinase activity                        1.3
  19                                                         

  Annotation Cluster  myoblast differentiation               1.27
  20                                                         
  ------------------- -------------------------------------- --------------

Table 3. Annotation clusters identified by the DAVID Functional
Annotation Clustering Tool. The genes of demo list 2 were analyzed by
the Functional Annotation Clustering Tool. The representative biology
terms associated with the top 20 annotation clusters are manually
selected, showing a much clearer and non-redundant view of the
annotation terms associated with the study.

**ADDITIONAL FILES**

**Additional Files can be found at
[http://david.abcc.ncifcrf.gov/manuscripts/fuzzy_cluster](http://david.abcc.ncifcrf.gov/manuscripts/fuzzy_cluster/)/**

**Additional File 1** (in Excel format; \~800 KB): Genes used in the
paper: 409 Affymetrix IDs of demo list 2; 84 chemokine genes; \~17,000
pairs of protein-protein interactions; and 16 Affy IDs.

**Additional File 2** (in Excel format; \~40 KB): Complete output in
text format for demo list 2 analyzed by the DAVID Gene Functional
Classification Tool.

**Additional File 3** (in Excel format; \~80 KB): Complete output in
text format for demo list 2 analyzed by the DAVID Functional Annotation
Clustering Tool.

**Additional File 4** (in PowerPoint format; \~150 KB): Fuzzy Heat Map
visualization of biological modules. The genes in demo list 2 were
analyzed by DAVID Gene Functional Classification Tool. The identified
biological groups/modules were displayed by the Fuzzy Heat Map.

**Additional File 5** (in PowerPoint Format; \~300 KB)**:** The novel
fuzzy heuristic partitioning method compared with other clustering
methods. The binary gene-term matrix (like figure 2 A) was compiled and
submitted to different clustering engines, including SOM, Hierarchical
clustering, and K-mean. The results were visualized by the traditional
heat map view. The same data was also analyzed by the DAVID Gene
Functional Annotation Tool and visualized by the Fuzzy Heat Map
Visualization.

**Additional File 6** (in PowerPoint format; \~30 KB): An example of the
group enrichment score calculation used for the Functional Annotation
Clustering Tool.

**Additional File 7** (in MS Word format; \~40 KB): 14 annotation
categories used in the DAVID Functional Classification Tool.

**Additional File 8** (in MS Word format; \~1.4 M): Graphic instruction
and tutorial how to use the DAVID Functional Classification Tool and the
DAVID Functional Annotation Clustering Tool.

**Additional File 9** (in MS Word format; \~0.5 M): The output examples
for the Related Gene Search and Related Term Search. A. Related Gene
Search for "Interleukin 8" in the scope of demo list 2. B. Related Term
Search for "Inflammatory Response" in the scope of all annotations. C.
Related Gene Search for a group of genes, group 1 for demo list 2,
identified by the DAVID Gene Functional Classification Tool.

**Additional File 10** (in MS Word format; \~0.6 M): The default setting
for minimum overlapped annotation in *kappa* score calculation. A.
Significant *kappa* scores (\>= 0.35 based on randomization study on
figure 3) only can be obtained for gene-gene pairs with higher
overlapped annotation terms (\>= 10). Thus, there is no reason to
calculate *kappa* scores, in an attempt to save the calculating time for
DAVID Functional Classification, for the large number of those gene-gene
pairs with fewer annotation terms overlapped. A conservative default
filter is 4. B. Such default filer (blue curve) has somewhat greater
impact on the significant *kappa* scores in the higher end, compared to
that in the lower end. However, it will skip a significant amount of
*kappa* calculation of gene-gene pairs.

**Additional File 11** (in MS Word format; \~0.4 M): The effect of
*Kappa* statistics on biased annotation data.

**Additional File 12** (in MS Word format; \~0.2 M): A hypothetical
example to measure the relationships of gene-gene pairs by *Kappa*
statistics with annotations organized in a \'flat\' matrix.

**Additional File 13** (in MS Word format; \~0.2 M): A hypothetical
example to show the general procedure of heuristic multiple-linkage
fuzzy clustering.

**Additional Files can be found at
[http://david.abcc.ncifcrf.gov/manuscripts/fuzzy_cluster](http://david.abcc.ncifcrf.gov/manuscripts/fuzzy_cluster/)/**