BACKGROUND
annotation of gene products for newly sequenced genomes is usually done electronically by transfer of functional information from proteins that have very similar amino acid sequences.
however, for many of the proteins in a newly sequenced genome, a database search will not reveal a sequence which shares a high degree of sequence identity of known function and therefore no functional information can reliably be transferred.
as a result many sequences are annotated as 'hypothetical protein' or 'protein of unknown function'.
typically some 30–40% of proteins in genomes sequenced so far have no annotation and this is an impediment to the exploitation of genome sequence data.
part of the difficult in inferring function from sequence is that sequence similarity is in general a sufficient but not necessary condition for functional or structural similarity and many proteins that have little discernible similarity at the sequence level have similar structures and functions.
a major challenge for in silico annotation methods is to identify these remote relationships.
accurate identification would enable a larger proportion of the currently sequenced genomes to have putative functional annotation.
early database searching methods compared the unknown query sequence with each database sequence in turn.
more sensitive methods exploited patterns of conservation that are revealed through multiple sequence alignments by performing sequence-profile comparisons.
this is, in effect the approach of intermediate searching methods and also programs such as psi-blast  <cit> .
more recently this approach has been extended to profile-profile comparisons  <cit> .
one of the problems with constructing profiles is how to include a large number of diverse sequences: ideally one would like to include a large amount of diversity, but as more diverse sequences are included the profile is likely to be corrupted due to alignment errors.
high throughput structural determination projects are generating large numbers of protein 3-dimensional structures  <cit> .
structure based multiple alignments of proteins are likely to be considerably more accurate than sequence based alignments and we would expect the corresponding profiles to be of higher quality  <cit> .
building a profile for a query sequence of unknown structure is generally done through iterative database search, as implemented in psi-blast.
for such sequences of unknown structure there is little choice of method since there is no structural information available.
however, it is not clear what the best method is for building profiles of those proteins of known structure and different groups have therefore used differing strategies.
one approach is to build one profile representing an entire group of related proteins .
this can be done by either using a sequence alignment of the proteins, or using a structure-based alignment of the proteins depending on the availability of sufficient number of 3-d structures for members of the superfamily.
the superfamily model can be enriched with close hits from sequence databases to the proteins being modelled, and hybrid profiles with secondary structural information included have shown added value  <cit> .
the alternative strategy is to build individual sequence profiles for each protein in the family, the strategy we refer to as domain models.
this is the strategy used by gough  <cit> .
does a single superfamily model of a large number of diverse sequences perform better at the detection of remote superfamily members than using multiple domain models built for each individual member of the superfamily?
gough et al concluded that multiple models were more effective  <cit> .
we feel that the question of whether to build domain or superfamily models to represent a superfamily is worth revisiting for a number of reasons.
firstly, recent years have seen the development of profile/profile comparison methods.
secondly, gough et al only tested how many hits were returned beneath a threshold score.
in this paper, we use roc analysis to examine how many hits are returned from all possible true relationships, where true relationships are defined by scop superfamilies.
the scop database uses structure to group related proteins, and therefore some of these relationships would not be apparent from sequence considerations alone  <cit> .
finally, we also examine the alignment accuracy produced by the differing models, a question not addressed by gough et al.
RESULTS
remote homolog detection
however, in practice, when annotating, one only wishes to consider the region of reliable matches.
there are approximately  <dig>  potential false hits in the database.
an error rate  of  <dig> % corresponds to  <dig> hits.
figure  <dig> shows the same roc curves, but in this region of much lower probability of error.
in this plot, the superfamily models have a slightly larger area under the curve.
they also detect up to 5% more true hits for the same number of errors as the domain models.
superfamily specific truncated roc analysis
the sixteen superfamilies in our dataset with more than  <dig> domains are also shown in the figure  <dig>  these represent large and sequence diverse superfamilies .
a number of these large, diverse superfamilies such as the s-adenosyl methyltransferases, alpha-beta hydrolases, cytochrome c, thioredoxin and immunoglobulin perform well with both domain and superfamily models, with roc <dig> values greater than  <dig> .
similarly the 'winged-helix' dna binding domain, the 4-helical cytokines, the nucleic-acid binding domain and the e-set domain perform poorly with both models.
for a few superfamilies there exists a large difference in performance between the single and multiple models: the fad/nad superfamily performs better with the superfamily model than with the domain models.
conversely, the nad superfamily performs better with the domain models.
alignment accuracy
discussion
does a single profile of a protein superfamily built from structure-based alignments perform better at recognition than multiple domain models?
the comparison is not straightforward and this analysis identifies some of the factors that are important in a comparison of single and multiple models using profile-profile methods.
the structure-based multiple alignments used to build the profiles for single models may be poor for some of the superfamilies, although in the absence of suitable expert reference alignments this is difficult to assess.
a detailed assessment of the validity of the alignment method is described in  <cit> .
in addition, the definition of a superfamily is not without limitations and may change.
globally, the use of a set of models representing domains is preferable to using superfamily models.
this is in line with previous results .
however, for low error thresholds, both types of models perform similarly in terms of the number of homolgous relationships detected.
in terms of particular superfamilies, the situation is different.
over 20% of the superfamilies tested there was a large difference in performance of domain or superfamily models, evenly distributed to favour either model.
in addition to the ability to detect homologs, producing an accurate alignment is also important.
we have investigated the accuracy of the alignments produced by both types of model.
for many superfamilies, the superfamily model correctly aligns more positions.
this suggests that examination of the scoring scheme used for superfamily models could be improved, thereby increasing the accuracy of homology detection.
CONCLUSIONS
using a sensitive profile-profile method we have investigated the performance of single structure-based models and multiple sequence models  in detecting remote superfamily members.
we find that overall, multiple models perform better in recognition although single structure-based models display better alignment accuracy.

 annotation of gene products for newly sequenced genomes is usually done electronically by transfer of functional information from proteins that have very similar amino acid sequences. however, for many of the proteins in a newly sequenced genome, a database search will not reveal a sequence which shares a high degree of sequence identity of known function and therefore no functional information can reliably be transferred. as a result many sequences are annotated as'hypothetical protein' or'protein of unknown function'. typically some 30–40% of proteins in genomes sequenced so far have no annotation and this is an impediment to the exploitation of genome sequence data. part of the difficult in inferring function from sequence is that sequence similarity is in general a sufficient but not necessary condition for functional or structural similarity and many proteins that have little discernible similarity at the sequence level have similar structures and functions. a major challenge for in silico annotation methods is to identify these remote relationships. accurate identification would enable a larger proportion of the currently sequenced genomes to have putative functional annotation. early database searching methods compared the unknown query sequence with each database sequence in turn. more sensitive methods exploited patterns of conservation that are revealed through multiple sequence alignments by performing sequence-profile comparisons. this is, in effect the approach of intermediate searching methods and also programs such as psi-blast. more recently this approach has been extended to profile-profile comparisons. one of the problems with constructing profiles is how to include a large number of diverse sequences: ideally one would like to include a large amount of diversity, but as more diverse sequences are included the profile is likely to be corrupted due to alignment errors. high throughput structural determination projects are generating large numbers of protein 3-dimensional structures. structure based multiple alignments of proteins are likely to be considerably more accurate than sequence based alignments and we would expect the corresponding profiles to be of higher quality. building a profile for a query sequence of unknown structure is generally done through iterative database search, as implemented in psi-blast. for such sequences of unknown structure there is little choice of method since there is no structural information available. however, it is not clear what the best method is for building profiles of those proteins of known structure and different groups have therefore used differing strategies. one approach is to build one profile representing an entire group of related proteins. this can be done by either using a sequence alignment of the proteins, or using a structure-based alignment of the proteins depending on the availability of sufficient number of 3-d structures for members of the superfamily. the superfamily model can be enriched with close hits from sequence databases to the proteins being modelled, and hybrid profiles with secondary structural information included have shown added value. the alternative strategy is to build individual sequence profiles for each protein in the family, the strategy we refer to as domain models. this is the strategy used by gough. does a single superfamily model of a large number of diverse sequences perform better at the detection of remote superfamily members than using multiple domain models built for each individual member of the superfamily? gough et al concluded that multiple models were more effective. we feel that the question of whether to build domain or superfamily models to represent a superfamily is worth revisiting for a number of reasons. firstly, recent years have seen the development of profile/profile comparison methods. secondly, gough et al only tested how many hits were returned beneath a threshold score. in this paper, we use roc analysis to examine how many hits are returned from all possible true relationships, where true relationships are defined by scop superfamilies. the scop database uses structure to group related proteins, and therefore some of these relationships would not be apparent from sequence considerations alone. finally, we also examine the alignment accuracy produced by the differing models, a question not addressed by gough et al.  remote homolog detection however, in practice, when annotating, one only wishes to consider the region of reliable matches. there are approximately potential false hits in the database. an error rate of % corresponds to hits. figure shows the same roc curves, but in this region of much lower probability of error. in this plot, the superfamily models have a slightly larger area under the curve. they also detect up to 5% more true hits for the same number of errors as the domain models. superfamily specific truncated roc analysis the sixteen superfamilies in our dataset with more than domains are also shown in the figure these represent large and sequence diverse superfamilies. a number of these large, diverse superfamilies such as the s-adenosyl methyltransferases, alpha-beta hydrolases, cytochrome c, thioredoxin and immunoglobulin perform well with both domain and superfamily models, with roc values greater than. similarly the'winged-helix' dna binding domain, the 4-helical cytokines, the nucleic-acid binding domain and the e-set domain perform poorly with both models. for a few superfamilies there exists a large difference in performance between the single and multiple models: the fad/nad superfamily performs better with the superfamily model than with the domain models. conversely, the nad superfamily performs better with the domain models. alignment accuracy discussion does a single profile of a protein superfamily built from structure-based alignments perform better at recognition than multiple domain models? the comparison is not straightforward and this analysis identifies some of the factors that are important in a comparison of single and multiple models using profile-profile methods. the structure-based multiple alignments used to build the profiles for single models may be poor for some of the superfamilies, although in the absence of suitable expert reference alignments this is difficult to assess. a detailed assessment of the validity of the alignment method is described in. in addition, the definition of a superfamily is not without limitations and may change. globally, the use of a set of models representing domains is preferable to using superfamily models. this is in line with previous results. however, for low error thresholds, both types of models perform similarly in terms of the number of homolgous relationships detected. in terms of particular superfamilies, the situation is different. over 20% of the superfamilies tested there was a large difference in performance of domain or superfamily models, evenly distributed to favour either model. in addition to the ability to detect homologs, producing an accurate alignment is also important. we have investigated the accuracy of the alignments produced by both types of model. for many superfamilies, the superfamily model correctly aligns more positions. this suggests that examination of the scoring scheme used for superfamily models could be improved, thereby increasing the accuracy of homology detection.  using a sensitive profile-profile method we have investigated the performance of single structure-based models and multiple sequence models in detecting remote superfamily members. we find that overall, multiple models perform better in recognition although single structure-based models display better alignment accuracy. 
using profile-profile methods for remote homolog detection we benchmark the performance of single structure-based superfamily models and multiple domain models. on average, over all superfamilies, using a truncated receiver operator characteristic we find that multiple domain models outperform single superfamily models, except at low error rates where the two models behave in a similar way. using a sensitive profile-profile method we have investigated the performance of single structure-based models and multiple sequence models in detecting remote superfamily members. we find that overall, multiple models perform better in recognition although single structure-based models display better alignment accuracy. the detection of relationships between a protein sequence of unknown function and a sequence whose function has been characterised enables the transfer of functional annotation. here we examine how the type of profile built affects its performance, both in detecting remote homologs and in the resulting alignment accuracy. in particular, we consider whether it is better to model a protein superfamily using a single structure-based alignment that is representative of all known cases of the superfamily, or to use multiple sequence-based profiles each representing an individual member of the superfamily. for 12% of all superfamilies the roc value for superfamily models is greater than above the domain models and for 10% of superfamilies the domain models show a similar improvement in performance over the superfamily models.
 the detection of relationships between a protein sequence of unknown function and a sequence whose function has been characterised enables the transfer of functional annotation. however in many cases these relationships can not be identified easily from direct comparison of the two sequences. methods which compare sequence profiles have been shown to improve the detection of these remote sequence relationships. however, the best method for building a profile of a known set of sequences has not been established. here we examine how the type of profile built affects its performance, both in detecting remote homologs and in the resulting alignment accuracy. in particular, we consider whether it is better to model a protein superfamily using a single structure-based alignment that is representative of all known cases of the superfamily, or to use multiple sequence-based profiles each representing an individual member of the superfamily.  using profile-profile methods for remote homolog detection we benchmark the performance of single structure-based superfamily models and multiple domain models. on average, over all superfamilies, using a truncated receiver operator characteristic we find that multiple domain models outperform single superfamily models, except at low error rates where the two models behave in a similar way. however there is a wide range of performance depending on the superfamily. for 12% of all superfamilies the roc value for superfamily models is greater than above the domain models and for 10% of superfamilies the domain models show a similar improvement in performance over the superfamily models.  using a sensitive profile-profile method we have investigated the performance of single structure-based models and multiple sequence models in detecting remote superfamily members. we find that overall, multiple models perform better in recognition although single structure-based models display better alignment accuracy. 