BACKGROUND
impressive progress in protein structure modelling has been achieved over the last decade; however, improvement between subsequent rounds of the critical assessment of techniques for protein structure prediction  is often considered to be modest  <cit> .
given the current accuracy, protein models are useful for qualitative analysis and decision-making in support of a wide range of experimental work.
high accuracy modelling is essential for important applications such as, molecular replacement experiments  <cit> , function predictions  <cit>  and virtual drug screening  <cit> .
modelling techniques, however, are still not accurate enough to close the gap between known protein sequences  and solved protein structures .
regardless of the current limitations of modelling, two very encouraging observations have been made from the casp <dig> results  <cit> .
first, the gap between the quality of fully automated and manual modelling techniques has narrowed and second, improvement beyond the best template is achieved more frequently.
modern template-based modelling pipelines can be divided into a number of common steps.
a typical pipeline starts with template identification and alignment construction.
in the next step, models are built using single templates, multiple templates or template fragments.
the resulting models are then often refined, and finally the models are ranked and the "best" model selected.
template search and alignment algorithms are showing significant improvements in accuracy and are becoming increasingly efficient.
well established sequence alignment algorithms such as fasta  <cit> , blast  <cit>  and psi-blast  <cit>  are often replaced, or enhanced by more sensitive algorithms.
these sensitive algorithms are based on multiple sequence alignments, sequence profiles or hidden markov models and take other information such as secondary structure prediction into account  <cit> .
the impact of better alignments on the final model's quality is substantial, as errors made at this stage are not likely to be recovered during the subsequent modelling process.
once a single template or several templates have been selected and the alignments constructed, models can be built.
it is common practice to search the conformational space in order to further refine the structures  <cit> .
several different approaches have been developed for this task, using conserved constraints  <cit> , genetic algorithms  <cit> , monte carlo sampling  <cit> , molecular dynamics  <cit> , principle components analysis  <cit>  or a combination of techniques  <cit> .
previous studies have shown that techniques combining several approaches perform best, when the different steps are carefully balanced  <cit> .
for quality control and to reduce computational costs, protein model ranking and filtering can be applied at almost any stage of a modelling pipeline.
energy functions or statistical potentials are used to select a final model and usually form an integral part of the refinement method itself.
given their importance, the ability to select the best model based on energy alone is still relatively poor  <cit> .
moreover, most energy scoring methods are optimised in the context of specific modelling approaches, and applying them in a different environment may produce less reliable results.
model selection has become an important field of protein modelling, and a separate category has been introduced in the 4th critical assessment of fully automated structure prediction  named model quality assessment programs .
the importance of this field was further underlined by a category called quality assessment  introduced in casp <dig>  <cit> .
several independent methods have been established and widely used to differentiate between models of high and poor quality  <cit> .
two different approaches can be distinguished; mqaps scoring models in the context of model ensembles and mqaps scoring single models independently.
however, most of the top ranking mqaps are dependent on the information of model ensembles  <cit> .
despite all of the above efforts and improvement in protein model construction, ranking and selection, it is still not possible to consistently produce models of high quality.
to further progress template-based modelling, it is necessary to carefully evaluate each step and minimise the accumulated errors.
here, we describe a hierarchical modelling approach , where each step has been carefully evaluated, giving new insights into generating and selecting better models.
a known limitation of genetic algorithm  approaches in protein modelling is that models tend to end up in local minima, not exploring the conformational landscape enough to be able to find the global energy minima.
as a way to alleviate this situation we have implemented the novel concept of alternating evolutionary pressure  into our genetic algorithm  search engine.
in aep intermediate rounds of unrestricted linear growth are introduced, enabling the models to overcome small energy barriers.
the aep approach is shown to promote greater sampling of the conformational landscape thereby enabling better structures to emerge, thus facilitating final model selection.
methods
in the following, the dataset, the algorithms applied and the pipeline of the modelling approach are described.
the core of the method is an optimization protocol based on a genetic algorithm .
this approach mimics the principles of evolution, combining and mutating protein model ensembles.
details of the ga approach used to model and refine protein structures can be found in a previous publication  <cit> .
to assure maximum yield, each step in the modelling pipeline has been evaluated separately.
for an overview of the pipeline see figure  <dig>
dataset
as the main objective of this work is to highlight problems in protein modelling and to extract potential solutions, the performance of the approach was benchmarked against well established modelling methods.
we considered all sequences from the seventh round of casp  <cit>  which were downloaded from the protein structure prediction center webpage .
the  <dig> protein sequences comprise  <dig> single and  <dig> multi-domain proteins.
the final dataset consists of the  <dig> targets out of the  <dig> targets, for which reasonable templates could be identified.
template identification, sequence alignments and initial model building
templates were identified and sequence alignments constructed using the hidden markov model based algorithm hhpred  <cit>  in conjunction with psi-blast  <cit>  results and the pdb <dig> database, downloaded from the hhsearch webpage .
standard values were used for hhsearch, as provided by the software distribution.
in order to allow a fair and unbiased comparison with all template-based casp <dig> servers, the data set was restricted to the information available at the time of casp <dig>  therefore, the psi-blast results, psipred  <cit>  secondary structure predictions and selected templates were created using time-stamped data.
for the initial model building, all side chains were stripped off the templates, and the query sequence assigned to the backbone, according to the hhpred alignment.
at this stage, neither insertions nor deletions were modelled or side chains added.
for several targets, we were not able to identify the substantially better templates used by the hhpred servers.
this might be due to the fact that only the pdb <dig> database is available for download; the hhpred server uses a combination of the pdb <dig> and pdb <dig> database.
it seems reasonable that better templates can only be found once both databases are used in conjunction, especially in cases where single, isolated, templates of higher quality are available.
repair algorithm
due to deletions and insertions in the alignment, initial models are likely to be fractured.
missing residues within β-strands are especially hard to insert since it is very likely that any closing process will disrupt the precise hydrogen-bond network.
on the other hand, substantial progress can be made focusing on coil and helical regions.
we have developed a novel protein model repair protocol, which enables the modelling of incomplete coil and helical secondary structure elements with the correct length, thereby helping to further break away from the initial templates.
the new loop conformations are restricted to highly populated Φ/Ψ angles of the ramachandran plot  <cit>  and backbone clashes are not allowed.
the ga conformational search engine is applied to all initial models after backbone completion; it is, therefore, not necessary to extensively sample conformational space at this stage.
during processing, backbone bonds with non-standard length are first identified and fixed.
for models with missing backbone elements, fragments are then created according to the psipred secondary structure prediction, using internal coordinates with standard bond and angle values .
these fragments are spliced into the backbone of the incomplete model and subsequently adjusted using a mechanism for closure, based on the cyclic coordinate descent algorithm  <cit> .
the procedure is fully automated and only requires a protein model and the psipred prediction.
all parameters used in the closing algorithm's procedure were derived from our ga algorithm, which was trained on the casp <dig> and cafasp <dig> datasets.
for algorithmic details .
model ranking
several energy functions are used throughout the modelling pipeline.
preliminary investigations indicated that poor quality input models are not selected in the ga optimization process.
using model pre-ranking to remove these models at an early stage, allows more computational time to be spent on the better models.
in the present approach, the best models are selected after optimization based on their energy scores.
to quantify the ranking ability of our energy-scoring scheme for models, we calculate the pearson correlation-coefficient between the energy and sc scores, defined below.
structure comparison  score
in order to assess the quality of the models generated, a measure describing the conformational similarity between models and the known native structure  was required.
we used a structural comparison scoring scheme, defined as the mean of the scaled tm  <cit> , gdt  <cit>  and maxsub  <cit>  scores.
since all three scores are scaled to the range  <cit> , the final sc score also ranges from  <dig> to  <dig>
sc= <dig>
energy scoring schemes
for each target all models were ranked using several different scoring schemes.
a novel fast scoring function  was used in the pre-ranking step.
the fsf scoring scheme is composed of the following terms:
sfsf = w1spp + w2scl + w3sss
where spp, the residue-residue pair potentials, is a score of the internal packing according to an empirically derived mixed backbone atom-centroid potential, as described previously  <cit> .
scl, the clash penalty, clashes are counted between two residues if any two backbone-atoms from any pair of non-consecutive residues are closer than  <dig> Å. sss, the secondary structure score, is a sum of psipred confidence scores for matches between predicted  <cit>  and assigned secondary structure.
the weights for the fsf were selected using the simplex algorithm  <cit>  on the casp <dig> and cafasp <dig> datasets and are given by: w <dig> =  <dig> , w <dig> =  <dig>  and w <dig> = - <dig> .
for scoring of the model populations during the ga optimization, the coarse energy score is used to preselect the models and reduce the population size to  <dig> models.
subsequently the fine energy score is used to further reduce the population to  <dig> models that are used for the next round.
the coarse scoring scheme includes the following terms:
scoarse = w1spp + w2scl + w3sss + w4shb + w3Δscomp
where spp, scl and sss are defined as for the fsf.
shb, the number of hydrogen bonds calculated using the software stride  <cit> .
Δscomp is the compactness reference score  <cit> .
the weights for the coarse scoring scheme are: w <dig> =  <dig> , w <dig> =  <dig> , w <dig> = - <dig> , w <dig> = - <dig>  and w <dig> =  <dig> .
the fine scoring scheme includes the following terms:
sfine = w1seef + w2sse + w3Δscomp
where seef is calculated in the following way.
scwrl  <dig>  is used to replace all the side chains  <cit>  for the energy calculation .
all scored models are minimised and then scored using the effective energy function  <cit>   in charmm.
sse, the solvent accessibility is calculated using the software pops_a  <cit>  and the solvation free energy  <cit> .
Δscomp, as previously defined  <cit> .
the weights for the fine scoring scheme are: w <dig> =  <dig> , w <dig> =  <dig>  and w <dig> =  <dig> .
after optimization with the ga protocol, the final models are ranked with a combination of the fine and coarse energy scores, and an all atom pair-potential score, dfire  <cit> .
in the combined energy function the coarse and fine energy components are weighted according to the best template's sequence identity.
sequence identities were binned into three ranges:  <dig> –  <dig> ,  <dig>  –  <dig> ,  <dig>  –  <dig> .
for each of these ranges, the weights were optimised using the simplex algorithm on the casp <dig> dataset, see table  <dig>
weights for the final combined energy score assigned using the simplex algorithm, calculated from the casp <dig> and cafasp <dig> datasets.
the higher the sequence identity, the less the coarse score is weighted.
the most representative structure of the final ensemble, which is taken from the middle of the largest cluster, was used as a control.
this was done to investigate how consistently the top ranked solutions are selected, compared to the most representative conformation of the final ensemble.
energy minimization
to investigate whether full cartesian space minimization facilitates protein model selection, all models, repaired and un-repaired, were minimised.
the steepest descent and adopted basis newton-raphson methods were applied, as implemented in charmm  <cit>  until the value of the gradient dropped below  <dig>  kcal mol- <dig> Å - <dig>
ga recombination
we previously described an efficient move-set used to search conformational space of protein models  <cit> .
this move-set includes three global operators: the single and double crossover and the protein mutation operator; and two local operators: the helix and the coil mutation operators.
a quick protein health check is performed during and after the application of the operators: Φ/Ψ angles must lie within the highly populated areas of the ramachandran plot and the change in energy-score is subject to a pseudo metropolis criterion.
the protocol was optimised using the casp <dig> and cafasp <dig> datasets.
the following modifications have been made to the protocol.
firstly, the input models are clustered using the nearest neighbour method.
the metric for this clustering approach is based upon overall protein model similarity weighted with the secondary structure scores.
only the largest two clusters are used for further optimization, thereby removing a few, poor outliers.
secondly, the range of movement for mutations has been changed, to allow finer movements.
this was achieved by allowing all values within the highly populated areas of the ramachandran plot.
after applying the closing algorithm to all selected models, the models were submitted to five parallel runs of the ga protocol.
the optimization is run for at least five, and a maximum of  <dig> rounds, dependant on the population convergence.
running the ga for longer was found to increase the probability of ending in incorrect local minima conformations .
alternating evolutionary pressure
in a ga where alternating evolutionary pressure  is applied, a number of non-scored rounds are allowed between each scored sampling round.
in these non-scored rounds, the population grows linearly and the structures in the ensemble are allowed to sample energetically unfavourable states.
although energy evaluation is not applied, to ensure reasonable sampling, basic protein health checks associated with the operators are still in place.
four different setups were applied: a normal ga and a ga with one to three non-scored intermediate rounds.
in each setup,  <dig> fully ranked rounds were performed where the population was reduced to the top  <dig> models.
accessibility
the complete modelling procedure can be accessed via a web server interface at:  <cit> .
the average running time for a protein model of  <dig> residues is 6– <dig> hours for the standard ga and 15– <dig> hours if two intermediate non-scored rounds are used .
for details of this server .
RESULTS
for this study we modelled  <dig> diverse protein sequences from the casp <dig> dataset of the category template-based modelling.
first, a novel backbone repair algorithm is introduced and compared to the performance of modeller.
in the next step an optimal setup for pre-ranking is investigated.
the resulting models are recombined using a ga and the improvement in model selection due to the introduction of aep is shown.
finally, a summary is given showing the performance of several possible modelling pipelines.
structural comparison of models before and after repair
we have developed a novel algorithm for completing and closing protein backbones.
coil and helical regions are completed and the length of incomplete helical secondary structure elements is adjusted to agree with the predicted secondary structure.
in contrast to other loop modelling methods, only a single conformation is created for each added structural element.
these conformations are further sampled once the model undergoes recombination using the ga.
the distribution of improvement in sc score due to this repair process is shown in figure  <dig>  in 72% of all cases, completed structures show improvement in comparison to their initial score.
the models improved by the repair algorithm show an average improvement of  <dig>  sc score and the best 25% of the population  shows an improvement greater than  <dig>  sc score.
analysing the sc score in terms of the assigned secondary structure, we found that approximately 80% of the improvement made for all models lies within helical secondary structure elements .
in contrast to this, only approximately 20% improvement is gained completing coil regions.
approximately 60% of the improvement is situated in the core region, defined here as the region between the n and c terminal secondary structure elements.
the rest of the improvement is located within the termini.
comparing the models derived from the top ten alignments of each target, with the equivalent models constructed with the automodel function in modeller  <cit> , the repairing algorithm scores on average  <dig>  sc score compared to  <dig>  sc score for modeller.
although the score for the closing algorithm is not significantly better, this method allows repair of models without the need for the alignment and/or template.
initial ranking
for modelling pipelines with extensive conformational search algorithms it is not obvious when to rank models.
ranking can be applied at several stages, such as before insertions and deletions are dealt with, after backbone completion, after minimization or after refinement.
intuitively, one might think that backbone completion is a minimum criterion to be fulfilled before further consideration on model quality can be made.
to address this question, we analysed the effectiveness of the fsf and dfire scoring schemes before and after repair.
on the other hand, ranking according to the alignments scores given by the alignments algorithms is normally not sufficient for model selection either.
ranking purely based on the coverage dependant sequence identities of the alignments, produced a correlation score of  <dig> , 12% smaller than the best fsf ranking.
in figure  <dig> it can be seen that the best ranking is obtained using the fsf on unrepaired, minimised structures, improving the correlation by 6% compared to the best dfire configuration and by 13% compared to ranking using the weighted sid.
non-repaired models are generally easier to rank, this is valid both for dfire and the fsf.
ranking unrepaired models results in an improved correlation coefficient of  <dig> % for dfire,  <dig> % for the fsf.
minimization further facilitates ranking ability for unrepaired models by a further 1% using the fsf.
recombination
to further sample the conformational space and select a good final model, all repaired models were recombined and optimised using the ga. figure  <dig> shows the median, first and third quantile of the different modelling populations pre/post ga. all ga runs only have ten fine energy scored rounds.
applying more than ten fine energy rounds increases the probability of convergence of the model ensemble into a local minimum .
running the ga optimization using the sc score to the native protein structure as the fitness function shows how much improvement can potentially be .
after the application of the ga using the native structure as guidance, the model ensemble is very narrow and on average the final population is improved by 51% to an sc score of  <dig>  for non-repaired and  <dig>  for repaired models.
interestingly even in this ideal scenario the move set is unable to produce better structure due to a lack of good quality templates, absence of secondary structure elements in the model population or insufficient sampling.
it can also be seen that repaired models clearly improve the overall population, due to the models not missing secondary structure elements.
longer sampling further increases the improvement, but for the purpose of comparison we limited the sampling to  <dig> rounds.
in practice we do not have access to the reference structure and have to rely on energy scores to drive the ga. figure  <dig> also shows the results using different energy scores for final model selection.
it can be seen that repairing structures does improve the top model, for all energy functions used.
the best results can be obtained using the dfire energy function producing a  <dig> % improvement compared to the fine score, a  <dig> % improvement compared to the fsf score and a  <dig> % improvement compared to the combined energy function.
the difference between the average sc score for the best models created using the energy and sc score to drive the ga is only 7%.
however, a further 4% improvement is lost when selecting the model with the lowest energy out of the final energy driven ensemble.
alternating evolutionary pressure
gas and other similar conformational search algorithms suffer from the problem that they tend to stay within local minima instead of exploring further afield and potentially finding a deeper minimum.
we investigated whether alternating evolutionary pressure  could facilitate energy based model selection, by gently pushing models over small energy barriers.
this idea is illustrated in figure  <dig>  showing that small changes in protein structure, although insignificant in terms of the sc score, produce significantly better energies, hence facilitating protein model selection.
two main elements dictate the success of ga approaches.
one is the set of operators  and the other is the fitness function .
classically, gas operate for several generations, iteratively applying the conformational search engine and the fitness function  <cit>  until convergence is obtained.
however, interesting results can be observed, once a series of conformational changes are applied, without intermediate population scoring and reduction.
within these non-scored intermediate rounds the population grows linearly.
the finer energy evaluation and the reduction of the model ensemble to the best members are not applied; however, during these rounds the basic protein health checks of the operators are still applied .
a similar approach to aep was introduced by qian et al., where the refinement of protein structures was achieved using an iterative alternation of diversification and intensification steps  <cit> .
this approach combines ideas from tabu search and conformational space annealing, however, it is only applied if the lowest energy refined structures have not converged and show several variable and less reliable regions.
in general, this methodology is different from classical gas where the optimization process is more variable, less directed and, therefore, convergence is only achieved after intensive sampling.
for these classical gas, the principle of aep has only been used before to provide theoretical predictions of algorithm performance  <cit> .
here, we take this idea one step further by removing the ranking step for a number of intermediate rounds.
we used four different setups: the standard ga and the ga with one, two and three non-scored intermediate rounds.
the results presented in figure  <dig> show that our ability to identify the better conformations varies strongly depending on the number of aep rounds used.
for each ga setup, we calculated the percentage of targets for which the best model based on sc score also had the lowest energy score.
using the standard ga protocol, the best model was identified in 25% of all targets using the dfire energy scores.
similar results are produced with a single  intermediate round .
for the runs with two intermediate rounds  the best model was identified in 40% of all targets.
however, for three intermediate rounds  the selectivity dropped to 30%.
allowing two intermediate, un-scored ga rounds yield the best results for model selection in this analysis.
in this setup, small energy barriers can be overcome, producing some very good individual models with low energies.
however, once the evolutionary pressure is too low, as seen for the aep with three intermediate rounds, the whole population drifts away and the quality of the lowest energy model decreases.
in figure  <dig> we present the coarse energy distributions for two representative remote homology targets, t <dig> and t <dig>  and fine energy distribution for two high homology targets, t <dig> and t <dig>  energies for the normal ga and aep1- <dig> are shown for each distribution.
generally it can be seen that the energy funnel is less well defined for lower homology modelling, t <dig> and t <dig>  a known observation for energy-based model ranking.
the advantage of the optimization process with aep <dig>  is illustrated in t <dig>  in this energy plot it can be seen that more sampling of higher quality models is found for aep <dig> .
indeed, for 80% of all cases, including t <dig>  t <dig> and t <dig>  aep <dig> produced the best results in the final selection.
t <dig> and t <dig> are examples where sampling with the standard ga is sufficient.
in the cases of t <dig>  t <dig> and t <dig> the problem of local minima for aep <dig>  can be seen, where some individual models of poorer quality have very low energies compared to all other sampled models.
sampling of the normal ga is often not as thorough as for aep, which can be seen in t <dig> and especially well in t <dig>  two harder modelling targets.
aep <dig> is sampling more space than the standard ga, but still less than aep <dig> and produces inferior results.
in order to further understand the effects of aep we applied several "normal ga" runs with adapted parameters.
first standard gas were run with the same number of rounds as given for aep1-3; these runs produced significantly inferior results compared to the standard ga. this effect can be explained by the oversampling of local minima, which could not be prevented even using statistically derived constraints .
additionally, we increased the population size for normal gas to  <dig>   <dig> and  <dig> models per round, which increased the computational costs but did not show any improvement in model quality for the lowest energy ranked models.
in general, it seems that the aep <dig> protocol gives the better balance between sampling the variable regions without drifting too far away in regions that are more structurally conserved; models that undergo consecutive multiple mutations in the structurally conserved regions are less likely to survive.
below, we present two cases; the first where there is no improvement using aep; the second where significant improvement could be achieved having two intermediate, un-scored, sampling rounds.
case i: t0380
for t <dig> a β-strand mainly protein with  <dig> residues had to be modelled.
with the template search  <dig> different templates were identified and, after backbone completion, the sc scores ranged from  <dig>  –  <dig> .
only one high quality template was identified for the starting population.
the best member in all final ensembles had a sc score of approximately  <dig> .
for none of the four recombination-setups were we able to select a model close to the best input.
the worst results were created using the aep <dig>  all selected models ranged between a sc score of  <dig>  to  <dig> .
interestingly, recombination of non-repaired models enabled the selection of a final model with a score close to  <dig> .
after backbone completion, the energy function was not able to distinguish between a good model derived from the best template, and an inferior model, produced by the closing algorithm.
case ii: t0311
in this case an all-helical protein with the length of  <dig> residues has been modelled.
the sequence search produced a list of  <dig> potential templates.
the top ranking alignment produced a model with a sc score of  <dig> .
the best, repaired, input model for the recombination had a sc score of  <dig> .
applying the standard ga selected a model of relatively poor quality with a sc score of  <dig> .
however, allowing two non-scored intermediate rounds improved the model beyond all initial input models and aided selecting the best member of the final ensemble, based on energy.
for this final model, which has a sc score of  <dig> , two helical secondary structure elements show improved positioning relative to the native structure.
the native structure and the best models for the standard ga and the two intermediate rounds ga are shown superimposed in figure  <dig>  clearly, the overall topology of the non-standard ga  is improved relative to the standard ga model , having several helical elements in the correct orientation.
general improvement along the modelling pipeline
the results for the different pre- and post-ga conditions are compared in figure  <dig>  here we compare several possible modelling pipelines.
the first pipeline considered, consisted of model selection without application of the ga. in this context the best final models were obtained using the fsf on minimised and repaired models.
this result seems to contradict our observations on ranking correlation; however, here the emphasis is on final model selection without further refinement of the population.
for this setup, an improvement of  <dig> % can be seen compared to the models derived by the initial alignment.
for all pipelines using the ga with or without aep, the unrepaired models were pre-ranked and the resulting model population was repaired before optimization.
since it was shown that un-repaired models are easier to rank, one might think that backbone completion should be the final step after recombination.
to test this we compared both possibilities and it can be seen that performing backbone completion before the ga rather than after provides an improvement of  <dig> % sc score; this can be explained by the additional sampling of the added secondary structure elements in the ga optimization.
the performances of the fine, the fsf, the combined and dfire energy score are compared for final structure selection.
the best results for the energy-driven ga were obtained using dfire which improves the average sc score by  <dig> % compared to the fine,  <dig> % compared to the fsf and  <dig> % compared to the combined energy score.
using dfire for the final model selection improved the average sc score for the normal ga from  <dig>  to  <dig> .
use of aep <dig> during recombination further improved the final models' quality from  <dig>  to  <dig>  in sc score using dfire for final selection.
dfire also performs better than the combined energy score for the final selection in aep <dig>  this shows the importance of using a final model selection scoring function that is not used for the optimization procedure.
overall, an improvement of  <dig> % is achieved for the optimum modelling pipeline  compared to the models derived from the first alignments.
comparing these results to the sc scores of all models produced by automatic servers during casp <dig> for our  <dig> targets, the normal ga would rank 12th and the aep <dig> 5th.
consistent selection of the best sc score model would enable a further improvement of up to 4%.
clustering, as described above, was used as an alternative selection protocol to identify the most representative models, but produced inferior results; on average  <dig> % lower than our optimal setup.
visual inspection of the final model ensembles indicated that the better structures are often isolated from the largest clusters.
overall, in  <dig> % of all targets, the best model of the final ensemble had a greater or equal sc score compared to the best model of the initial input population.
a similar trend was observed for the lowest energy models, where  <dig> % of the lowest energy model of the final population had greater or equal sc score compared to the lowest energy model of the initial population.
for  <dig> % of all cases the final lowest energy model was improved in sc score than the best model of the initial input population, thereby selecting or improving the best model.
finally, possible structural errors in the selected models were investigated, using the prosa web server  <cit>  which compares them to x-ray crystal and nmr structures.
as can be seen in figure  <dig>  all  <dig> models produced energy-related z-scores comparable to the scores of x-ray crystal structures.
furthermore, it has been shown for our high accuracy casp <dig> submissions that we ranked 8th of all submitting groups for the accurate prediction of the χ1/χ <dig> angles  <cit> .
the move set of the ga has not been changed for this work, and therefore these findings remain valid, indicating that the conformational sampling performed in internal coordinate space does not adversely affect the side chain quality.
however, as a further check of model quality, a subset of randomly selected models plus the final models selected with the best overall pipeline were also tested for stereo-chemical properties using the procheck  <cit>  software package.
these models showed a quality comparable to the other top-ranking models submitted to casp <dig>
CONCLUSIONS
in the present work we have performed a detailed analysis of the different steps that form the pipeline of our template based ga protein modelling approach.
the results shown here, clearly demonstrate that pre-ranking should be applied to unrepaired models before backbone completion.
ranking of these incomplete models using the novel fsf scoring scheme combined with minimization was shown to provide the best ranking .
this method could be used to pre-select a final model in protocols where rapid modelling from single templates is necessary.
dfire was found to be the most efficient way to select the top model .
the selectivity of dfire can be further improved by introducing alternative evolutionary pressure  to the ga protocol .
creating subtle movements in the protein models using aep, helps to select the better models by nudging them to lower energy states.
when using gas for protein modelling two different effects can be achieved.
first, gas can be applied to improve models beyond the best input structure.
second, gas can be used to ease the selection of the better protein models by lowering their energies.
however, lowering the model's energy does not necessarily improve the structural score.
in the approach used here, the ga was used to improve selection of good models.
improvement beyond or maintaining the best input model was seen for  <dig> % of all targets.
however, these models could only be identified in 25% for the normal ga, 31%, 40%, 30% for aep with 1– <dig> intermediate rounds respectively.
the application of all the above strategies improves the final average structural score by  <dig> % compared to the purely alignment-based pipeline.
this was achieved by a carefully balance between the number of sampled intermediate structures , the scoring functions used in the ga and the final selection of models with dfire.
overall, this pipeline would rank 5th, comparing these results to the scores of all models produced by automatic servers during casp <dig> for our  <dig> targets
further investigations and development need to be undertaken in order to make full use of the ga/aep conformational search engine.
other techniques will undoubtedly be required to further assist conformational search engines, such as gas, to recover from local minima.
in general more work is required to refine scoring schemes so that the best models can be consistently selected from the ensembles of structures.
competing interests
the authors declare that they have no competing interests.
authors' contributions
mno and pab devised the work.
mno carried out all computational work and wrote the initial manuscript draft.
alt and pab edited the paper.
all authors read and approved the final manuscript.
supplementary material
additional file 1
supplementary material.
this file provides a detailed description of the protein backbone repairing algorithm, detailed results of the ga recombination for structure and energy-guided optimizations and information on the 3d-jigsaw  <dig>  protein modelling web-server.
click here for file
acknowledgements
we thank all the members of the biomolecular modelling laboratory for many useful discussions and insights.
this work was funded by cancer research uk, the barbara mary hill memorial fund and a b'nai b'rith leo back lodge scholarship awarded to mno.

 impressive progress in protein structure modelling has been achieved over the last decade; however, improvement between subsequent rounds of the critical assessment of techniques for protein structure prediction is often considered to be modest. given the current accuracy, protein models are useful for qualitative analysis and decision-making in support of a wide range of experimental work. high accuracy modelling is essential for important applications such as, molecular replacement experiments, function predictions and virtual drug screening. modelling techniques, however, are still not accurate enough to close the gap between known protein sequences and solved protein structures. regardless of the current limitations of modelling, two very encouraging observations have been made from the casp results. first, the gap between the quality of fully automated and manual modelling techniques has narrowed and second, improvement beyond the best template is achieved more frequently. modern template-based modelling pipelines can be divided into a number of common steps. a typical pipeline starts with template identification and alignment construction. in the next step, models are built using single templates, multiple templates or template fragments. the resulting models are then often refined, and finally the models are ranked and the "best" model selected. template search and alignment algorithms are showing significant improvements in accuracy and are becoming increasingly efficient. well established sequence alignment algorithms such as fasta, blast and psi-blast are often replaced, or enhanced by more sensitive algorithms. these sensitive algorithms are based on multiple sequence alignments, sequence profiles or hidden markov models and take other information such as secondary structure prediction into account. the impact of better alignments on the final model's quality is substantial, as errors made at this stage are not likely to be recovered during the subsequent modelling process. once a single template or several templates have been selected and the alignments constructed, models can be built. it is common practice to search the conformational space in order to further refine the structures. several different approaches have been developed for this task, using conserved constraints, genetic algorithms, monte carlo sampling, molecular dynamics, principle components analysis or a combination of techniques. previous studies have shown that techniques combining several approaches perform best, when the different steps are carefully balanced. for quality control and to reduce computational costs, protein model ranking and filtering can be applied at almost any stage of a modelling pipeline. energy functions or statistical potentials are used to select a final model and usually form an integral part of the refinement method itself. given their importance, the ability to select the best model based on energy alone is still relatively poor. moreover, most energy scoring methods are optimised in the context of specific modelling approaches, and applying them in a different environment may produce less reliable results. model selection has become an important field of protein modelling, and a separate category has been introduced in the 4th critical assessment of fully automated structure prediction named model quality assessment programs. the importance of this field was further underlined by a category called quality assessment introduced in casp. several independent methods have been established and widely used to differentiate between models of high and poor quality. two different approaches can be distinguished; mqaps scoring models in the context of model ensembles and mqaps scoring single models independently. however, most of the top ranking mqaps are dependent on the information of model ensembles. despite all of the above efforts and improvement in protein model construction, ranking and selection, it is still not possible to consistently produce models of high quality. to further progress template-based modelling, it is necessary to carefully evaluate each step and minimise the accumulated errors. here, we describe a hierarchical modelling approach, where each step has been carefully evaluated, giving new insights into generating and selecting better models. a known limitation of genetic algorithm approaches in protein modelling is that models tend to end up in local minima, not exploring the conformational landscape enough to be able to find the global energy minima. as a way to alleviate this situation we have implemented the novel concept of alternating evolutionary pressure into our genetic algorithm search engine. in aep intermediate rounds of unrestricted linear growth are introduced, enabling the models to overcome small energy barriers. the aep approach is shown to promote greater sampling of the conformational landscape thereby enabling better structures to emerge, thus facilitating final model selection. methods in the following, the dataset, the algorithms applied and the pipeline of the modelling approach are described. the core of the method is an optimization protocol based on a genetic algorithm. this approach mimics the principles of evolution, combining and mutating protein model ensembles. details of the ga approach used to model and refine protein structures can be found in a previous publication. to assure maximum yield, each step in the modelling pipeline has been evaluated separately. for an overview of the pipeline see figure dataset as the main objective of this work is to highlight problems in protein modelling and to extract potential solutions, the performance of the approach was benchmarked against well established modelling methods. we considered all sequences from the seventh round of casp which were downloaded from the protein structure prediction center webpage. the protein sequences comprise single and multi-domain proteins. the final dataset consists of the targets out of the targets, for which reasonable templates could be identified. template identification, sequence alignments and initial model building templates were identified and sequence alignments constructed using the hidden markov model based algorithm hhpred in conjunction with psi-blast results and the pdb database, downloaded from the hhsearch webpage. standard values were used for hhsearch, as provided by the software distribution. in order to allow a fair and unbiased comparison with all template-based casp servers, the data set was restricted to the information available at the time of casp therefore, the psi-blast results, psipred secondary structure predictions and selected templates were created using time-stamped data. for the initial model building, all side chains were stripped off the templates, and the query sequence assigned to the backbone, according to the hhpred alignment. at this stage, neither insertions nor deletions were modelled or side chains added. for several targets, we were not able to identify the substantially better templates used by the hhpred servers. this might be due to the fact that only the pdb database is available for download; the hhpred server uses a combination of the pdb and pdb database. it seems reasonable that better templates can only be found once both databases are used in conjunction, especially in cases where single, isolated, templates of higher quality are available. repair algorithm due to deletions and insertions in the alignment, initial models are likely to be fractured. missing residues within β-strands are especially hard to insert since it is very likely that any closing process will disrupt the precise hydrogen-bond network. on the other hand, substantial progress can be made focusing on coil and helical regions. we have developed a novel protein model repair protocol, which enables the modelling of incomplete coil and helical secondary structure elements with the correct length, thereby helping to further break away from the initial templates. the new loop conformations are restricted to highly populated Φ/Ψ angles of the ramachandran plot and backbone clashes are not allowed. the ga conformational search engine is applied to all initial models after backbone completion; it is, therefore, not necessary to extensively sample conformational space at this stage. during processing, backbone bonds with non-standard length are first identified and fixed. for models with missing backbone elements, fragments are then created according to the psipred secondary structure prediction, using internal coordinates with standard bond and angle values. these fragments are spliced into the backbone of the incomplete model and subsequently adjusted using a mechanism for closure, based on the cyclic coordinate descent algorithm. the procedure is fully automated and only requires a protein model and the psipred prediction. all parameters used in the closing algorithm's procedure were derived from our ga algorithm, which was trained on the casp and cafasp datasets. for algorithmic details. model ranking several energy functions are used throughout the modelling pipeline. preliminary investigations indicated that poor quality input models are not selected in the ga optimization process. using model pre-ranking to remove these models at an early stage, allows more computational time to be spent on the better models. in the present approach, the best models are selected after optimization based on their energy scores. to quantify the ranking ability of our energy-scoring scheme for models, we calculate the pearson correlation-coefficient between the energy and sc scores, defined below. structure comparison score in order to assess the quality of the models generated, a measure describing the conformational similarity between models and the known native structure was required. we used a structural comparison scoring scheme, defined as the mean of the scaled tm, gdt and maxsub scores. since all three scores are scaled to the range, the final sc score also ranges from to sc= energy scoring schemes for each target all models were ranked using several different scoring schemes. a novel fast scoring function was used in the pre-ranking step. the fsf scoring scheme is composed of the following terms: sfsf = w1spp + w2scl + w3sss where spp, the residue-residue pair potentials, is a score of the internal packing according to an empirically derived mixed backbone atom-centroid potential, as described previously. scl, the clash penalty, clashes are counted between two residues if any two backbone-atoms from any pair of non-consecutive residues are closer than Å. sss, the secondary structure score, is a sum of psipred confidence scores for matches between predicted and assigned secondary structure. the weights for the fsf were selected using the simplex algorithm on the casp and cafasp datasets and are given by: w =, w = and w = -. for scoring of the model populations during the ga optimization, the coarse energy score is used to preselect the models and reduce the population size to models. subsequently the fine energy score is used to further reduce the population to models that are used for the next round. the coarse scoring scheme includes the following terms: scoarse = w1spp + w2scl + w3sss + w4shb + w3Δscomp where spp, scl and sss are defined as for the fsf. shb, the number of hydrogen bonds calculated using the software stride. Δscomp is the compactness reference score. the weights for the coarse scoring scheme are: w =, w =, w = -, w = - and w =. the fine scoring scheme includes the following terms: sfine = w1seef + w2sse + w3Δscomp where seef is calculated in the following way. scwrl is used to replace all the side chains for the energy calculation. all scored models are minimised and then scored using the effective energy function in charmm. sse, the solvent accessibility is calculated using the software pops_a and the solvation free energy. Δscomp, as previously defined. the weights for the fine scoring scheme are: w =, w = and w =. after optimization with the ga protocol, the final models are ranked with a combination of the fine and coarse energy scores, and an all atom pair-potential score, dfire. in the combined energy function the coarse and fine energy components are weighted according to the best template's sequence identity. sequence identities were binned into three ranges: –, –, –. for each of these ranges, the weights were optimised using the simplex algorithm on the casp dataset, see table weights for the final combined energy score assigned using the simplex algorithm, calculated from the casp and cafasp datasets. the higher the sequence identity, the less the coarse score is weighted. the most representative structure of the final ensemble, which is taken from the middle of the largest cluster, was used as a control. this was done to investigate how consistently the top ranked solutions are selected, compared to the most representative conformation of the final ensemble. energy minimization to investigate whether full cartesian space minimization facilitates protein model selection, all models, repaired and un-repaired, were minimised. the steepest descent and adopted basis newton-raphson methods were applied, as implemented in charmm until the value of the gradient dropped below kcal mol- Å - ga recombination we previously described an efficient move-set used to search conformational space of protein models. this move-set includes three global operators: the single and double crossover and the protein mutation operator; and two local operators: the helix and the coil mutation operators. a quick protein health check is performed during and after the application of the operators: Φ/Ψ angles must lie within the highly populated areas of the ramachandran plot and the change in energy-score is subject to a pseudo metropolis criterion. the protocol was optimised using the casp and cafasp datasets. the following modifications have been made to the protocol. firstly, the input models are clustered using the nearest neighbour method. the metric for this clustering approach is based upon overall protein model similarity weighted with the secondary structure scores. only the largest two clusters are used for further optimization, thereby removing a few, poor outliers. secondly, the range of movement for mutations has been changed, to allow finer movements. this was achieved by allowing all values within the highly populated areas of the ramachandran plot. after applying the closing algorithm to all selected models, the models were submitted to five parallel runs of the ga protocol. the optimization is run for at least five, and a maximum of rounds, dependant on the population convergence. running the ga for longer was found to increase the probability of ending in incorrect local minima conformations. alternating evolutionary pressure in a ga where alternating evolutionary pressure is applied, a number of non-scored rounds are allowed between each scored sampling round. in these non-scored rounds, the population grows linearly and the structures in the ensemble are allowed to sample energetically unfavourable states. although energy evaluation is not applied, to ensure reasonable sampling, basic protein health checks associated with the operators are still in place. four different setups were applied: a normal ga and a ga with one to three non-scored intermediate rounds. in each setup, fully ranked rounds were performed where the population was reduced to the top models. accessibility the complete modelling procedure can be accessed via a web server interface at:. the average running time for a protein model of residues is 6– hours for the standard ga and 15– hours if two intermediate non-scored rounds are used. for details of this server.  for this study we modelled diverse protein sequences from the casp dataset of the category template-based modelling. first, a novel backbone repair algorithm is introduced and compared to the performance of modeller. in the next step an optimal setup for pre-ranking is investigated. the resulting models are recombined using a ga and the improvement in model selection due to the introduction of aep is shown. finally, a summary is given showing the performance of several possible modelling pipelines. structural comparison of models before and after repair we have developed a novel algorithm for completing and closing protein backbones. coil and helical regions are completed and the length of incomplete helical secondary structure elements is adjusted to agree with the predicted secondary structure. in contrast to other loop modelling methods, only a single conformation is created for each added structural element. these conformations are further sampled once the model undergoes recombination using the ga. the distribution of improvement in sc score due to this repair process is shown in figure in 72% of all cases, completed structures show improvement in comparison to their initial score. the models improved by the repair algorithm show an average improvement of sc score and the best 25% of the population shows an improvement greater than sc score. analysing the sc score in terms of the assigned secondary structure, we found that approximately 80% of the improvement made for all models lies within helical secondary structure elements. in contrast to this, only approximately 20% improvement is gained completing coil regions. approximately 60% of the improvement is situated in the core region, defined here as the region between the n and c terminal secondary structure elements. the rest of the improvement is located within the termini. comparing the models derived from the top ten alignments of each target, with the equivalent models constructed with the automodel function in modeller, the repairing algorithm scores on average sc score compared to sc score for modeller. although the score for the closing algorithm is not significantly better, this method allows repair of models without the need for the alignment and/or template. initial ranking for modelling pipelines with extensive conformational search algorithms it is not obvious when to rank models. ranking can be applied at several stages, such as before insertions and deletions are dealt with, after backbone completion, after minimization or after refinement. intuitively, one might think that backbone completion is a minimum criterion to be fulfilled before further consideration on model quality can be made. to address this question, we analysed the effectiveness of the fsf and dfire scoring schemes before and after repair. on the other hand, ranking according to the alignments scores given by the alignments algorithms is normally not sufficient for model selection either. ranking purely based on the coverage dependant sequence identities of the alignments, produced a correlation score of, 12% smaller than the best fsf ranking. in figure it can be seen that the best ranking is obtained using the fsf on unrepaired, minimised structures, improving the correlation by 6% compared to the best dfire configuration and by 13% compared to ranking using the weighted sid. non-repaired models are generally easier to rank, this is valid both for dfire and the fsf. ranking unrepaired models results in an improved correlation coefficient of % for dfire, % for the fsf. minimization further facilitates ranking ability for unrepaired models by a further 1% using the fsf. recombination to further sample the conformational space and select a good final model, all repaired models were recombined and optimised using the ga. figure shows the median, first and third quantile of the different modelling populations pre/post ga. all ga runs only have ten fine energy scored rounds. applying more than ten fine energy rounds increases the probability of convergence of the model ensemble into a local minimum. running the ga optimization using the sc score to the native protein structure as the fitness function shows how much improvement can potentially be. after the application of the ga using the native structure as guidance, the model ensemble is very narrow and on average the final population is improved by 51% to an sc score of for non-repaired and for repaired models. interestingly even in this ideal scenario the move set is unable to produce better structure due to a lack of good quality templates, absence of secondary structure elements in the model population or insufficient sampling. it can also be seen that repaired models clearly improve the overall population, due to the models not missing secondary structure elements. longer sampling further increases the improvement, but for the purpose of comparison we limited the sampling to rounds. in practice we do not have access to the reference structure and have to rely on energy scores to drive the ga. figure also shows the results using different energy scores for final model selection. it can be seen that repairing structures does improve the top model, for all energy functions used. the best results can be obtained using the dfire energy function producing a % improvement compared to the fine score, a % improvement compared to the fsf score and a % improvement compared to the combined energy function. the difference between the average sc score for the best models created using the energy and sc score to drive the ga is only 7%. however, a further 4% improvement is lost when selecting the model with the lowest energy out of the final energy driven ensemble. alternating evolutionary pressure gas and other similar conformational search algorithms suffer from the problem that they tend to stay within local minima instead of exploring further afield and potentially finding a deeper minimum. we investigated whether alternating evolutionary pressure could facilitate energy based model selection, by gently pushing models over small energy barriers. this idea is illustrated in figure showing that small changes in protein structure, although insignificant in terms of the sc score, produce significantly better energies, hence facilitating protein model selection. two main elements dictate the success of ga approaches. one is the set of operators and the other is the fitness function. classically, gas operate for several generations, iteratively applying the conformational search engine and the fitness function until convergence is obtained. however, interesting results can be observed, once a series of conformational changes are applied, without intermediate population scoring and reduction. within these non-scored intermediate rounds the population grows linearly. the finer energy evaluation and the reduction of the model ensemble to the best members are not applied; however, during these rounds the basic protein health checks of the operators are still applied. a similar approach to aep was introduced by qian et al., where the refinement of protein structures was achieved using an iterative alternation of diversification and intensification steps. this approach combines ideas from tabu search and conformational space annealing, however, it is only applied if the lowest energy refined structures have not converged and show several variable and less reliable regions. in general, this methodology is different from classical gas where the optimization process is more variable, less directed and, therefore, convergence is only achieved after intensive sampling. for these classical gas, the principle of aep has only been used before to provide theoretical predictions of algorithm performance. here, we take this idea one step further by removing the ranking step for a number of intermediate rounds. we used four different setups: the standard ga and the ga with one, two and three non-scored intermediate rounds. the results presented in figure show that our ability to identify the better conformations varies strongly depending on the number of aep rounds used. for each ga setup, we calculated the percentage of targets for which the best model based on sc score also had the lowest energy score. using the standard ga protocol, the best model was identified in 25% of all targets using the dfire energy scores. similar results are produced with a single intermediate round. for the runs with two intermediate rounds the best model was identified in 40% of all targets. however, for three intermediate rounds the selectivity dropped to 30%. allowing two intermediate, un-scored ga rounds yield the best results for model selection in this analysis. in this setup, small energy barriers can be overcome, producing some very good individual models with low energies. however, once the evolutionary pressure is too low, as seen for the aep with three intermediate rounds, the whole population drifts away and the quality of the lowest energy model decreases. in figure we present the coarse energy distributions for two representative remote homology targets, t and t and fine energy distribution for two high homology targets, t and t energies for the normal ga and aep1- are shown for each distribution. generally it can be seen that the energy funnel is less well defined for lower homology modelling, t and t a known observation for energy-based model ranking. the advantage of the optimization process with aep is illustrated in t in this energy plot it can be seen that more sampling of higher quality models is found for aep. indeed, for 80% of all cases, including t t and t aep produced the best results in the final selection. t and t are examples where sampling with the standard ga is sufficient. in the cases of t t and t the problem of local minima for aep can be seen, where some individual models of poorer quality have very low energies compared to all other sampled models. sampling of the normal ga is often not as thorough as for aep, which can be seen in t and especially well in t two harder modelling targets. aep is sampling more space than the standard ga, but still less than aep and produces inferior results. in order to further understand the effects of aep we applied several "normal ga" runs with adapted parameters. first standard gas were run with the same number of rounds as given for aep1-3; these runs produced significantly inferior results compared to the standard ga. this effect can be explained by the oversampling of local minima, which could not be prevented even using statistically derived constraints. additionally, we increased the population size for normal gas to and models per round, which increased the computational costs but did not show any improvement in model quality for the lowest energy ranked models. in general, it seems that the aep protocol gives the better balance between sampling the variable regions without drifting too far away in regions that are more structurally conserved; models that undergo consecutive multiple mutations in the structurally conserved regions are less likely to survive. below, we present two cases; the first where there is no improvement using aep; the second where significant improvement could be achieved having two intermediate, un-scored, sampling rounds. case i: t0380 for t a β-strand mainly protein with residues had to be modelled. with the template search different templates were identified and, after backbone completion, the sc scores ranged from –. only one high quality template was identified for the starting population. the best member in all final ensembles had a sc score of approximately. for none of the four recombination-setups were we able to select a model close to the best input. the worst results were created using the aep all selected models ranged between a sc score of to. interestingly, recombination of non-repaired models enabled the selection of a final model with a score close to. after backbone completion, the energy function was not able to distinguish between a good model derived from the best template, and an inferior model, produced by the closing algorithm. case ii: t0311 in this case an all-helical protein with the length of residues has been modelled. the sequence search produced a list of potential templates. the top ranking alignment produced a model with a sc score of. the best, repaired, input model for the recombination had a sc score of. applying the standard ga selected a model of relatively poor quality with a sc score of. however, allowing two non-scored intermediate rounds improved the model beyond all initial input models and aided selecting the best member of the final ensemble, based on energy. for this final model, which has a sc score of, two helical secondary structure elements show improved positioning relative to the native structure. the native structure and the best models for the standard ga and the two intermediate rounds ga are shown superimposed in figure clearly, the overall topology of the non-standard ga is improved relative to the standard ga model, having several helical elements in the correct orientation. general improvement along the modelling pipeline the results for the different pre- and post-ga conditions are compared in figure here we compare several possible modelling pipelines. the first pipeline considered, consisted of model selection without application of the ga. in this context the best final models were obtained using the fsf on minimised and repaired models. this result seems to contradict our observations on ranking correlation; however, here the emphasis is on final model selection without further refinement of the population. for this setup, an improvement of % can be seen compared to the models derived by the initial alignment. for all pipelines using the ga with or without aep, the unrepaired models were pre-ranked and the resulting model population was repaired before optimization. since it was shown that un-repaired models are easier to rank, one might think that backbone completion should be the final step after recombination. to test this we compared both possibilities and it can be seen that performing backbone completion before the ga rather than after provides an improvement of % sc score; this can be explained by the additional sampling of the added secondary structure elements in the ga optimization. the performances of the fine, the fsf, the combined and dfire energy score are compared for final structure selection. the best results for the energy-driven ga were obtained using dfire which improves the average sc score by % compared to the fine, % compared to the fsf and % compared to the combined energy score. using dfire for the final model selection improved the average sc score for the normal ga from to. use of aep during recombination further improved the final models' quality from to in sc score using dfire for final selection. dfire also performs better than the combined energy score for the final selection in aep this shows the importance of using a final model selection scoring function that is not used for the optimization procedure. overall, an improvement of % is achieved for the optimum modelling pipeline compared to the models derived from the first alignments. comparing these results to the sc scores of all models produced by automatic servers during casp for our targets, the normal ga would rank 12th and the aep 5th. consistent selection of the best sc score model would enable a further improvement of up to 4%. clustering, as described above, was used as an alternative selection protocol to identify the most representative models, but produced inferior results; on average % lower than our optimal setup. visual inspection of the final model ensembles indicated that the better structures are often isolated from the largest clusters. overall, in % of all targets, the best model of the final ensemble had a greater or equal sc score compared to the best model of the initial input population. a similar trend was observed for the lowest energy models, where % of the lowest energy model of the final population had greater or equal sc score compared to the lowest energy model of the initial population. for % of all cases the final lowest energy model was improved in sc score than the best model of the initial input population, thereby selecting or improving the best model. finally, possible structural errors in the selected models were investigated, using the prosa web server which compares them to x-ray crystal and nmr structures. as can be seen in figure all models produced energy-related z-scores comparable to the scores of x-ray crystal structures. furthermore, it has been shown for our high accuracy casp submissions that we ranked 8th of all submitting groups for the accurate prediction of the χ1/χ angles. the move set of the ga has not been changed for this work, and therefore these findings remain valid, indicating that the conformational sampling performed in internal coordinate space does not adversely affect the side chain quality. however, as a further check of model quality, a subset of randomly selected models plus the final models selected with the best overall pipeline were also tested for stereo-chemical properties using the procheck software package. these models showed a quality comparable to the other top-ranking models submitted to casp  in the present work we have performed a detailed analysis of the different steps that form the pipeline of our template based ga protein modelling approach. the results shown here, clearly demonstrate that pre-ranking should be applied to unrepaired models before backbone completion. ranking of these incomplete models using the novel fsf scoring scheme combined with minimization was shown to provide the best ranking. this method could be used to pre-select a final model in protocols where rapid modelling from single templates is necessary. dfire was found to be the most efficient way to select the top model. the selectivity of dfire can be further improved by introducing alternative evolutionary pressure to the ga protocol. creating subtle movements in the protein models using aep, helps to select the better models by nudging them to lower energy states. when using gas for protein modelling two different effects can be achieved. first, gas can be applied to improve models beyond the best input structure. second, gas can be used to ease the selection of the better protein models by lowering their energies. however, lowering the model's energy does not necessarily improve the structural score. in the approach used here, the ga was used to improve selection of good models. improvement beyond or maintaining the best input model was seen for % of all targets. however, these models could only be identified in 25% for the normal ga, 31%, 40%, 30% for aep with 1– intermediate rounds respectively. the application of all the above strategies improves the final average structural score by % compared to the purely alignment-based pipeline. this was achieved by a carefully balance between the number of sampled intermediate structures, the scoring functions used in the ga and the final selection of models with dfire. overall, this pipeline would rank 5th, comparing these results to the scores of all models produced by automatic servers during casp for our targets further investigations and development need to be undertaken in order to make full use of the ga/aep conformational search engine. other techniques will undoubtedly be required to further assist conformational search engines, such as gas, to recover from local minima. in general more work is required to refine scoring schemes so that the best models can be consistently selected from the ensembles of structures. competing interests the authors declare that they have no competing interests. authors' contributions mno and pab devised the work. mno carried out all computational work and wrote the initial manuscript draft. alt and pab edited the paper. all authors read and approved the final manuscript. supplementary material additional file 1 supplementary material. this file provides a detailed description of the protein backbone repairing algorithm, detailed results of the ga recombination for structure and energy-guided optimizations and information on the 3d-jigsaw protein modelling web-server. click here for file acknowledgements we thank all the members of the biomolecular modelling laboratory for many useful discussions and insights. this work was funded by cancer research uk, the barbara mary hill memorial fund and a b'nai b'rith leo back lodge scholarship awarded to mno.
aep enabled the selection of the best models in 40% of all targets; compared to 25% for a normal ga. intermediate rounds within the ga runs where unrestrained, linear growth of the model populations is allowed. we apply the concept of alternating evolutionary pressure, i.e. nevertheless, there are still potential improvements to be made in template selection, refinement and protein model selection.
 automatic protein modelling pipelines are becoming ever more accurate; this has come hand in hand with an increasingly complicated interplay between all components involved. nevertheless, there are still potential improvements to be made in template selection, refinement and protein model selection.  in the context of an automatic modelling pipeline, we analysed each step separately, revealing several non-intuitive trends and explored a new strategy for protein conformation sampling using genetic algorithms. we apply the concept of alternating evolutionary pressure, i.e. intermediate rounds within the ga runs where unrestrained, linear growth of the model populations is allowed.  this approach improves the overall performance of the ga by allowing models to overcome local energy barriers. aep enabled the selection of the best models in 40% of all targets; compared to 25% for a normal ga. 