BACKGROUND
in silico modelling of cellular processes
this paper describes the construction and application of a logical model of known metabolic processes in bakers' yeast .
the model outlines the relationships between open reading frames , enzymes and reactions comprising the metabolic pathways in yeast, presenting the knowledge as facts in a restricted form of first order logic .
this representation allows the model to behave as a deductive  database, as well as a model for the study of metabolic behaviour.
a focus of our previous research has been the development and implementation of an automated system for the design, execution and interpretation of wet lab auxotrophic experiments: the robot scientist  <cit> .
in its first incarnation the scope of the robot scientist was limited to rediscovery of knowledge from a single metabolic pathway in yeast  biosynthesis pathway), where a logical model of this pathway  <cit>  acted as background knowledge from which the robot scientist was able to generate hypotheses.
the logical model presented here, "aber", is an updated and expanded version of this limited model .
the model has been expanded to include most of what is known about yeast metabolism.
the model has also been updated to a more explicitly relational data structure; replacing the single relation that included enzyme and metabolite details with separate relations, enabling a more natural and versatile knowledge representation.
to evaluate the correctness of the model we have used it to predict the results of empirical growth experiments of knockout mutants growing on defined minimal medium  <cit> .
although our model has been designed to be able to generate growth predictions on defined growth media, the model contains many essential genes, therefore a list of essential genes generated by the same gene-deletion study was used in conjunction with the minimal medium growth results to more accurately reflect the predictions of the model.
an assessment of the ability of the model to correctly predict gene essentiality was also performed; in this case the predictions of growth were compared to the list of essential genes alone, with no reference to the minimal medium experimental data.
roc analysis was used to compare this logical model to ind <dig>  <cit> , a state-of-the-art flux balance analysis  model  <cit>   and  <dig> represents the number of genes included in the model).
both models have also been compared with the predicted growth outcomes generated by a simple majority class classifier, as well as the probability that prediction success was purely random.
systems biology and the modelling of biochemical networks
systems biology  <cit>  represents a shift towards a synergistic approach to whole cell modelling, with the concentration on the interactions of many inter-related components rather than the behaviour of the individual components.
advances in mathematics and computer science have led to the development of diverse techniques and formalisms allowing the in silico modelling of these cell systems.
all computer models represent varying degrees of abstraction from the observable phenomena they represent, from coarse large scale models that capture only essential interactions and components of the system e.g.
kegg  <cit>  and ecocyc  <cit> , to higher fidelity representations of detailed functioning and interactions of a smaller set of components  <cit> .
there are two main groups of modelling techniques used to represent metabolic networks:
1) quantitative methods that aim to capture the changes in the quantities of metabolites and enzymes etc.
by representing cellular processes, e.g.
reaction kinetics, in detail
2) qualitative methods that aim simply model the presence of interactions, e.g.
reactions are modelled as chemical transformations, with no representation of reaction kinetics
the two most important quantitative methods are ode  and fba .
odes are the most established modelling representation in science.
in using odes to model metabolism the concentration of each metabolite is calculated by a single ode encapsulating all the reactions where the metabolite is synthesised or consumed, with fluxes determining the transformations to and from other metabolites in the network.
the use of ode models is the main technique of the quantitative sciences.
there are also now a number of specialised ode modelling packages for metabolism, e.g.
gepasi/copasi  <cit>  and e-cell  <cit> .
despite this, the current application of odes to modelling large-scale metabolism has a number of serious problems: we do not know, nor are likely to know soon, all the necessary parameters that such models demand; and there can be numerical analysis problems in solving ode metabolic models, resulting in quantitative and qualitative differences between simulators.
flux balance analysis   <cit>  is currently the most common approach to quantitatively modelling metabolism.
standard fba assumes a steady state model of cell metabolism; although more recent developments in fba  <cit>  have extended this to dynamic flux balance analysis that is capable of modelling cells with some state change.
cell reactions are modelled by two matrices: one corresponding to the stoichiometry of the reactions, the other containing the fluxes for the reactions.
typically many of these fluxes remain unknown and experimental measurement of the ranges for a small number of "control" fluxes and linear/nonlinear programming is used to determine the values of the unknown fluxes, based on an assumption of optimality.
fba models of the metabolism of a number of organisms exist, e.g.
s. cerevisiae  <cit>  and e. coli  <cit>  and recently the human metabolic network  <cit> .
the steady state assumption  and the inaccuracies of the unknown fluxes can lead to inaccuracies in the overall simulation  <cit> .
however, increased experimental evidence can decrease these inaccuracies.
a number of flux based alternatives to fba have been developed that address the changes in fluxes that occur after major environmental or other perturbations such as gene knockouts: moma   <cit>  and room   <cit> .
both make use of distance measures to determine a point in flux space that is closest to the wild type flux distribution, in keeping with a homeostasis hypothesis.
moma minimises the changes to each individual flux, thereby the overall network is as similar as possible to the wild type.
in contrast, room minimises the number of significant flux changes, thereby better approximating how short alternative pathways allow redundancy in metabolic networks.
in a predictive study using lethal/non-lethal knockouts in e. coli, room had an accuracy of 85%, which is comparable to the standard fba approach.
elementary mode analysis  <cit>  determines the set of smallest sub-networks of a larger metabolic network that still allow a metabolic steady state to be reached.
each elementary mode represents an alternative that the organism may use in conditions of perturbation.
stelling et al  <cit>  use elementary-mode analysis to determine lethal/non-lethal gene deletions in the central carbon metabolism of e. coli, where lethal mutants mostly have an empty set of elementary modes, and non-lethal mutants have at least one elementary mode with an overall positive growth rate.
stelling et al  <cit>  observed a 90% correct prediction rate for the  <dig> knockouts they studied.
however there can be a large number of elementary modes for even a moderately large metabolic network, indicating that there may be problems scaling this approach to whole metabolism networks.
logical and graph  based models  <cit>  are the commonest qualitative representations for modelling metabolism.
graph based models are used in metabolic databases e.g.
kegg  <cit> , ecocyc/metacyc  <cit> .
metabolic pathways are represented explicitly, each metabolite is a node in the graph and edges represent the chemical transformations found in the reactions comprising the pathway.
edges are further annotated by the enzyme that catalyse the reactions, and these are in turn are related to the gene that encode the enzymes.
lemke et al  <cit>  have developed a graph-based model of the metabolic network of e. coli and have used it to analyse how much damage the absence of each enzyme causes to the metabolic network.
metabolic damage is defined as the number of metabolites that can no longer be produced by the organism.
they show that only 9% of enzymes prevent production of  <dig> or more metabolites, but that more than 50% of essential enzymes are to be found in this group.
logical models may use computationally efficient forms of both propositional and fol  as their representation language  <cit> .
as in graph models, the reaction network is represented by a series of metabolite nodes and chemical transformation arcs, however the increased expressive power of logic can allow more accurate representations of the relationships between the genes, enzymes and gene products used as annotations to the reactions, as well as various cellular compartments.
for example, only enzyme names are used to annotate the reaction arcs in lemke et al  <cit> , unlike orf and ec number for the aber model.
metabolic modelling is done by keeping a tally of the metabolites added to the cell by each reaction .
the biocham system  <cit>  is a dedicated biochemical reasoning engine that uses a rule based temporal logic language to model and query all of the possible behaviours of a given biochemical model, and the mapk signal transduction cascades have been used as an example.
random boolean networks  <cit>  have also been used to model  <dig> of the genes comprising the yeast transcriptional network.
random boolean are useful for modelling systems where interactions are not known beforehand.
kauffman et al  <cit>  used this process to identify those networks that were most stable, thereby representing rules of biological relevance to the regulation of gene transcription.
in general the graph models are equivalent to propositional logic models and can adequately represent metabolic reactions as nodes and arcs.
however this representation is not expressive enough to adequately represent the relationships between orf/enzymes etc.
that control the reactions.
the increased expressivity of fol is required for this additional complexity.
the use of qualitative reasoning   <cit>  is an intermediate representation between quantitative and logical models, e.g.
king et al used qr to model the glycolysis pathway  <cit> .
rather than presence/absence of a metabolite in the cell as in the coarse representation of logical/graph models, or the quantitative concentration as in ode models, qr models record the qualitative state of each metabolite, e.g.
whether the concentration is increasing, decreasing or constant.
qualitative differential equations  <cit>  are used to calculate the change in quantity of the metabolites and enzymes, with a fol representation of the metabolite and enzyme components of the system.
a common task in qr is to generate the complete envisionment of the system, i.e.
all of the possible qualitative states that can be derived from an initial starting state.
this can be a computationally expensive task, king et al found  <dig>  possible states for the glycolysis model.
RESULTS
to test the utility of our "aber" model we compared it with the ind <dig> fba model, and a majority classifier that assigns each prediction to the most commonly occurring classification .
although ind <dig> and the aber model have both been constructed from iff <dig>  <cit> , subsequent development has led to a divergence of the number of orfs for which there are experimental results in giaever et al  <cit> : ind <dig> makes predictions for  <dig> orfs, while the aber model makes predictions for  <dig>  with  <dig> predictions shared by both models.
evaluation of the performance of the models examined the shared genes as well as the total number of genes in each model, so that the effect of adding newly categorised orfs to the original set can also be examined.
two different cases of gene knockout sensitivity on mmd+ura+hist+leu were considered, 1) genes found to be significantly sensitive after  <dig> generations and 2) genes found to be significantly sensitive after  <dig> generations that remain significantly sensitive after  <dig> generations.
as for ind <dig>  the metabolic network and the starting media definitions used by the aber model were altered to reflect the additional requirements for uracil, histidine and leucine.
an additional comparison was also made with the ind <dig> results where a preprocessing step was used to identify borderline cases of retarded growth in the sensitive/refractory data.
this preprocessing step used all of the sensitivity results for each orf, ignoring any refractory scores for orfs, therefore all of the sensitivity data is used rather than focusing only on orfs found to be sensitive in both giaever experiment sets .
the imbalance of prediction counts also results in a shortfall of experiment results from the ind <dig> preprocessing step .
in cases where no experimental results from comparison set c were available, results from set b were used.
definitions of the  <dig> sets of experimentally derived growth and no growth outcomes are given in table  <dig>
although the logical model uses mmd+ura+hist+leu as its set of starting compounds, comparison with the essential gene list is valid because knockouts that do not grow on rich ypd will also almost certainly not grow on a simple defined medium.
analysis of the model comparisons made use of performance metrics and statistical tests that evaluated the significance of any observed difference in model performance.
these are described below, followed by a discussion of the gene essentiality findings and the results of the minimal medium growth study
skewed performance metrics and significance tests
in an analysis of the use of roc space flach  <cit>  defines a number of metrics that use a skew ratio  to allow for bias introduced by classes of dissimilar size  these metrics can be used to both measure a classifier's performance and as conditions for evaluation of rules/tree nodes etc.
as classifiers are constructed.
the metrics are skewed accuracy and skewed precision.
table  <dig> shows the formulas for these two metrics as well as formulas for metrics representing intermediate steps.
in this table tp represents the number of true positive predictions, tn the number of true negatives, and fp and fn the number of true and false positives and negatives respectively.
pos corresponds to the total number of positive predictions and neg, the total number of negative predictions.
a mcnemar χ <dig> test was used to compare the performance of the aber model, ind <dig> and the majority classifier on each of the  <dig> comparison sets.
all three sets of orf sources  were used for comparison of the aber model and ind <dig>  the shared orfs and aber orf sources were used for the comparison of the aber model and the majority classifier.
the ind <dig> orfs and shared orfs sources were used for the comparison of ind <dig> and the majority classifier.
the mcnemar test analyses the proportions of mistakes for predictions common to both models.
to enable comparison of equivalent numbers the majority class was used to obtain predictions for orfs not found in either model .
significance for the mcnemar tests was determined for p >  <dig>  and the direction of the test result was collected .
the following equation defines the mcnemar test used, where b is the number of predictions found to be correct by model  <dig> and incorrect by model  <dig> and c is the converse .
a cumulative binomial test was also used to determine the probability of generating the predictions of the aber model by chance.
χ2=−1)2b+c
predicting gene essentiality
we use the ability of a metabolic network models to predict the essentiality of a gene as a measure of model quality.
gene essentiality results have been presented for iff <dig>  the prime source of knowledge for the aber model and the precursor to ind <dig>  to which the aber model is compared.
förster et al  <cit>  report a predictive accuracy of 85% for gene essentiality for iff <dig>  a model's prediction can be tested by comparison with the empirical "wet" experiment of growing the knockout strain.
the growth  of a knockout strain depends on both its genotype  have been removed) and the environment.
to use a model of metabolism to predict gene essentiality, it is therefore necessary to know the composition of the growth medium .
this makes the use of "wet" results using the most common growth medium for yeast, ypd, problematic because its exact composition is undefined.
this is why we have focused on using the giaever et al data  <cit>  for growth on a defined minimal medium.
comparing predicted experiment outcomes to growth outcomes obtained by a whole genome gene-deletion study and performance of ind750
estimating the probability of chance occurrence of model predictions
a cumulative binomial test was used to evaluate the probability that the predictions made by the logical model could have been made by chance.
this test calculates the significance of deviations from the binomial distribution of observations falling into two categories , assuming a random binomial model that assigns genes to essential/retarded growth and non essential/healthy growth categories by chance.
the probability is calculated for the range r..n, where r = 889: the number of predictions for growth and n is the total number of predictions made :
pbin=∑i=rn!⋅i!⋅pi⋅qq−i)
where p =  <dig> , the proportion of orfs that result in experimental growth and q =  <dig> - p .
this test resulted in a probability, pbin =  <dig>  × 10-32: a very low probability that the predictions were generated by chance.
discussion
comparison of our logical model with ind <dig> demonstrated that there is no significant difference in the performance of any of the models on the prediction of gene essentiality.
a significant increase in performance was observed for both the aber model and ind <dig> w.r.t the majority classifier on predicting growth outcome on defined growth medium, for all except the most stringent definition of growth from giaever et al  <cit> .
there was also no significant difference between the aber model and ind <dig> in the defined growth medium study.
the gene essentiality study indicates that both models are equally poor at predicting essential genes, indeed neither is an improvement on the simple majority classifier.
as is the case for the minimal medium experimental data, the essential gene data is highly skewed, reflecting the amount of redundancy in the yeast genome.
the improvement in performance of both models w.r.t the defined medium study is a result of the precise definition of the medium components .
the similarity of results for the aber model and ind <dig> indicate that the additional complexity of fba is not required for the prediction of essential genes or the determination of growth/retarded growth on defined media.
however the logical model is currently restricted to binary growth/no growth predictions and has no capability to predict growth rates unlike fba based techniques.
it is a case of horses for courses.
the logical model presented here is a useful addition to the currently existing models with an appropriate mix of expressive representation of metabolic concepts and a simple mechanism for determining predictive outcomes.
the high number of inaccuracies in the prediction of growth for all models indicates that much important information remains to be discovered about yeast metabolism.
using the logical model as background knowledge for active learning of hypotheses
a major motivation for our use of a logical formalism is that it enables the exploitation of techniques from the large research area of inductive logic programming  and related first-order learning methods  <cit> .
this is not possible with other formalisms such as fba.
in previous work  <cit> , a precursor of the aber logical model was used as a background theory in a discovery task involving the function of orfs from a single pathway  in yeast.
the task was implemented as an active learning loop, a machine learning technique based on abductive logic programming that mirrors the hypothetico-deductive process of hypothesis formation in scientific discovery.
in active learning a classifier is given a set of training examples incrementally, so that the most informative example  can be chosen at each iteration of the loop.
in scientific discovery training examples correspond to the outcome of experiments carried out automatically by a lab robot, and the active selection of a training example corresponds to choosing the experiment most likely to refute the largest number of hypotheses, given the outcome of the experiment .
figure  <dig> illustrates the how the active learning loop was applied to the functional genomics discovery task.
the formation of hypotheses from the outcomes of the selected experiments was performed by a limited form of abductive theory completion  <cit> , performed by progol, an ilp  <cit>  program that performs both abductive and inductive machine learning in a restricted form of fol.
in this task the logical model formed the incomplete background knowledge theory, to which the final hypothesis is added to "complete" or improve the theory, therefore adding to the current biological knowledge of yeast functional genomics.
the completion of the theory is analogous to identifying a new edge to be added to the metabolic network, as well as a new orf annotation for that edge.
this is described in figure  <dig>
the use of this restricted form of fol allows all components of the scientific discovery task to be described using the same knowledge representation, indeed the complexity of the entities and relationships required for accurate description of enzymes, reactions, experiments, and hypotheses etc can only be expressed by fol or an equivalently expressive knowledge representation .
abduction of hypotheses relating orfs, enzymes and reactions also requires that the machine learning program has fol as it's representation language, hence the use of programs from ilp.
the robot scientist concept therefore represents a close integration of logical inference techniques used for scientific discovery and an automated laboratory that can perform experiments.
the previous work on the aaa biosynthesis pathway was a proof of principle study.
recently the original robot hardware has been replaced by a state-of-the-art automated laboratory that substantially increases the throughput of the robot scientist  <cit>
CONCLUSIONS
this paper presents a logic based model of the metabolic processes in s. cerevisiae: "aber".
the core of the aber model is the fba model iff <dig>  <cit> .
added to this core are an additional  <dig> orfs, and also includes an additional  <dig> reactions taken from kegg  <cit> .
the aber model has been designed to enable automated reasoning about yeast metabolism.
use of the model to predict the essentiality of genes in knockout studies provides a mechanism which can be used to validate the model with respect to experimental evidence.
roc analysis and other statistical tests were carried out to evaluate the predictive ability of the model and to compare the performance with a state of the art fba model ; and to evaluate the effect of increasing the coverage of the model from iff <dig>  the results show that using a logical model rather that a fba based one results in no significant loss of performance, nor is any loss found by enlarging the coverage of the model to include information from kegg; and that both ind <dig> and our logical model represent an improvement over random predictions.

 in silico modelling of cellular processes this paper describes the construction and application of a logical model of known metabolic processes in bakers' yeast. the model outlines the relationships between open reading frames, enzymes and reactions comprising the metabolic pathways in yeast, presenting the knowledge as facts in a restricted form of first order logic. this representation allows the model to behave as a deductive database, as well as a model for the study of metabolic behaviour. a focus of our previous research has been the development and implementation of an automated system for the design, execution and interpretation of wet lab auxotrophic experiments: the robot scientist. in its first incarnation the scope of the robot scientist was limited to rediscovery of knowledge from a single metabolic pathway in yeast biosynthesis pathway), where a logical model of this pathway acted as background knowledge from which the robot scientist was able to generate hypotheses. the logical model presented here, "aber", is an updated and expanded version of this limited model. the model has been expanded to include most of what is known about yeast metabolism. the model has also been updated to a more explicitly relational data structure; replacing the single relation that included enzyme and metabolite details with separate relations, enabling a more natural and versatile knowledge representation. to evaluate the correctness of the model we have used it to predict the results of empirical growth experiments of knockout mutants growing on defined minimal medium. although our model has been designed to be able to generate growth predictions on defined growth media, the model contains many essential genes, therefore a list of essential genes generated by the same gene-deletion study was used in conjunction with the minimal medium growth results to more accurately reflect the predictions of the model. an assessment of the ability of the model to correctly predict gene essentiality was also performed; in this case the predictions of growth were compared to the list of essential genes alone, with no reference to the minimal medium experimental data. roc analysis was used to compare this logical model to ind, a state-of-the-art flux balance analysis model and represents the number of genes included in the model). both models have also been compared with the predicted growth outcomes generated by a simple majority class classifier, as well as the probability that prediction success was purely random. systems biology and the modelling of biochemical networks systems biology represents a shift towards a synergistic approach to whole cell modelling, with the concentration on the interactions of many inter-related components rather than the behaviour of the individual components. advances in mathematics and computer science have led to the development of diverse techniques and formalisms allowing the in silico modelling of these cell systems. all computer models represent varying degrees of abstraction from the observable phenomena they represent, from coarse large scale models that capture only essential interactions and components of the system e.g. kegg and ecocyc, to higher fidelity representations of detailed functioning and interactions of a smaller set of components. there are two main groups of modelling techniques used to represent metabolic networks: 1) quantitative methods that aim to capture the changes in the quantities of metabolites and enzymes etc. by representing cellular processes, e.g. reaction kinetics, in detail 2) qualitative methods that aim simply model the presence of interactions, e.g. reactions are modelled as chemical transformations, with no representation of reaction kinetics the two most important quantitative methods are ode and fba. odes are the most established modelling representation in science. in using odes to model metabolism the concentration of each metabolite is calculated by a single ode encapsulating all the reactions where the metabolite is synthesised or consumed, with fluxes determining the transformations to and from other metabolites in the network. the use of ode models is the main technique of the quantitative sciences. there are also now a number of specialised ode modelling packages for metabolism, e.g. gepasi/copasi and e-cell. despite this, the current application of odes to modelling large-scale metabolism has a number of serious problems: we do not know, nor are likely to know soon, all the necessary parameters that such models demand; and there can be numerical analysis problems in solving ode metabolic models, resulting in quantitative and qualitative differences between simulators. flux balance analysis is currently the most common approach to quantitatively modelling metabolism. standard fba assumes a steady state model of cell metabolism; although more recent developments in fba have extended this to dynamic flux balance analysis that is capable of modelling cells with some state change. cell reactions are modelled by two matrices: one corresponding to the stoichiometry of the reactions, the other containing the fluxes for the reactions. typically many of these fluxes remain unknown and experimental measurement of the ranges for a small number of "control" fluxes and linear/nonlinear programming is used to determine the values of the unknown fluxes, based on an assumption of optimality. fba models of the metabolism of a number of organisms exist, e.g. s. cerevisiae and e. coli and recently the human metabolic network. the steady state assumption and the inaccuracies of the unknown fluxes can lead to inaccuracies in the overall simulation. however, increased experimental evidence can decrease these inaccuracies. a number of flux based alternatives to fba have been developed that address the changes in fluxes that occur after major environmental or other perturbations such as gene knockouts: moma and room. both make use of distance measures to determine a point in flux space that is closest to the wild type flux distribution, in keeping with a homeostasis hypothesis. moma minimises the changes to each individual flux, thereby the overall network is as similar as possible to the wild type. in contrast, room minimises the number of significant flux changes, thereby better approximating how short alternative pathways allow redundancy in metabolic networks. in a predictive study using lethal/non-lethal knockouts in e. coli, room had an accuracy of 85%, which is comparable to the standard fba approach. elementary mode analysis determines the set of smallest sub-networks of a larger metabolic network that still allow a metabolic steady state to be reached. each elementary mode represents an alternative that the organism may use in conditions of perturbation. stelling et al use elementary-mode analysis to determine lethal/non-lethal gene deletions in the central carbon metabolism of e. coli, where lethal mutants mostly have an empty set of elementary modes, and non-lethal mutants have at least one elementary mode with an overall positive growth rate. stelling et al observed a 90% correct prediction rate for the knockouts they studied. however there can be a large number of elementary modes for even a moderately large metabolic network, indicating that there may be problems scaling this approach to whole metabolism networks. logical and graph based models are the commonest qualitative representations for modelling metabolism. graph based models are used in metabolic databases e.g. kegg, ecocyc/metacyc. metabolic pathways are represented explicitly, each metabolite is a node in the graph and edges represent the chemical transformations found in the reactions comprising the pathway. edges are further annotated by the enzyme that catalyse the reactions, and these are in turn are related to the gene that encode the enzymes. lemke et al have developed a graph-based model of the metabolic network of e. coli and have used it to analyse how much damage the absence of each enzyme causes to the metabolic network. metabolic damage is defined as the number of metabolites that can no longer be produced by the organism. they show that only 9% of enzymes prevent production of or more metabolites, but that more than 50% of essential enzymes are to be found in this group. logical models may use computationally efficient forms of both propositional and fol as their representation language. as in graph models, the reaction network is represented by a series of metabolite nodes and chemical transformation arcs, however the increased expressive power of logic can allow more accurate representations of the relationships between the genes, enzymes and gene products used as annotations to the reactions, as well as various cellular compartments. for example, only enzyme names are used to annotate the reaction arcs in lemke et al, unlike orf and ec number for the aber model. metabolic modelling is done by keeping a tally of the metabolites added to the cell by each reaction. the biocham system is a dedicated biochemical reasoning engine that uses a rule based temporal logic language to model and query all of the possible behaviours of a given biochemical model, and the mapk signal transduction cascades have been used as an example. random boolean networks have also been used to model of the genes comprising the yeast transcriptional network. random boolean are useful for modelling systems where interactions are not known beforehand. kauffman et al used this process to identify those networks that were most stable, thereby representing rules of biological relevance to the regulation of gene transcription. in general the graph models are equivalent to propositional logic models and can adequately represent metabolic reactions as nodes and arcs. however this representation is not expressive enough to adequately represent the relationships between orf/enzymes etc. that control the reactions. the increased expressivity of fol is required for this additional complexity. the use of qualitative reasoning is an intermediate representation between quantitative and logical models, e.g. king et al used qr to model the glycolysis pathway. rather than presence/absence of a metabolite in the cell as in the coarse representation of logical/graph models, or the quantitative concentration as in ode models, qr models record the qualitative state of each metabolite, e.g. whether the concentration is increasing, decreasing or constant. qualitative differential equations are used to calculate the change in quantity of the metabolites and enzymes, with a fol representation of the metabolite and enzyme components of the system. a common task in qr is to generate the complete envisionment of the system, i.e. all of the possible qualitative states that can be derived from an initial starting state. this can be a computationally expensive task, king et al found possible states for the glycolysis model.  to test the utility of our "aber" model we compared it with the ind fba model, and a majority classifier that assigns each prediction to the most commonly occurring classification. although ind and the aber model have both been constructed from iff, subsequent development has led to a divergence of the number of orfs for which there are experimental results in giaever et al: ind makes predictions for orfs, while the aber model makes predictions for with predictions shared by both models. evaluation of the performance of the models examined the shared genes as well as the total number of genes in each model, so that the effect of adding newly categorised orfs to the original set can also be examined. two different cases of gene knockout sensitivity on mmd+ura+hist+leu were considered, 1) genes found to be significantly sensitive after generations and 2) genes found to be significantly sensitive after generations that remain significantly sensitive after generations. as for ind the metabolic network and the starting media definitions used by the aber model were altered to reflect the additional requirements for uracil, histidine and leucine. an additional comparison was also made with the ind results where a preprocessing step was used to identify borderline cases of retarded growth in the sensitive/refractory data. this preprocessing step used all of the sensitivity results for each orf, ignoring any refractory scores for orfs, therefore all of the sensitivity data is used rather than focusing only on orfs found to be sensitive in both giaever experiment sets. the imbalance of prediction counts also results in a shortfall of experiment results from the ind preprocessing step. in cases where no experimental results from comparison set c were available, results from set b were used. definitions of the sets of experimentally derived growth and no growth outcomes are given in table although the logical model uses mmd+ura+hist+leu as its set of starting compounds, comparison with the essential gene list is valid because knockouts that do not grow on rich ypd will also almost certainly not grow on a simple defined medium. analysis of the model comparisons made use of performance metrics and statistical tests that evaluated the significance of any observed difference in model performance. these are described below, followed by a discussion of the gene essentiality findings and the results of the minimal medium growth study skewed performance metrics and significance tests in an analysis of the use of roc space flach defines a number of metrics that use a skew ratio to allow for bias introduced by classes of dissimilar size these metrics can be used to both measure a classifier's performance and as conditions for evaluation of rules/tree nodes etc. as classifiers are constructed. the metrics are skewed accuracy and skewed precision. table shows the formulas for these two metrics as well as formulas for metrics representing intermediate steps. in this table tp represents the number of true positive predictions, tn the number of true negatives, and fp and fn the number of true and false positives and negatives respectively. pos corresponds to the total number of positive predictions and neg, the total number of negative predictions. a mcnemar χ test was used to compare the performance of the aber model, ind and the majority classifier on each of the comparison sets. all three sets of orf sources were used for comparison of the aber model and ind the shared orfs and aber orf sources were used for the comparison of the aber model and the majority classifier. the ind orfs and shared orfs sources were used for the comparison of ind and the majority classifier. the mcnemar test analyses the proportions of mistakes for predictions common to both models. to enable comparison of equivalent numbers the majority class was used to obtain predictions for orfs not found in either model. significance for the mcnemar tests was determined for p > and the direction of the test result was collected. the following equation defines the mcnemar test used, where b is the number of predictions found to be correct by model and incorrect by model and c is the converse. a cumulative binomial test was also used to determine the probability of generating the predictions of the aber model by chance. χ2=−1)2b+c predicting gene essentiality we use the ability of a metabolic network models to predict the essentiality of a gene as a measure of model quality. gene essentiality results have been presented for iff the prime source of knowledge for the aber model and the precursor to ind to which the aber model is compared. förster et al report a predictive accuracy of 85% for gene essentiality for iff a model's prediction can be tested by comparison with the empirical "wet" experiment of growing the knockout strain. the growth of a knockout strain depends on both its genotype have been removed) and the environment. to use a model of metabolism to predict gene essentiality, it is therefore necessary to know the composition of the growth medium. this makes the use of "wet" results using the most common growth medium for yeast, ypd, problematic because its exact composition is undefined. this is why we have focused on using the giaever et al data for growth on a defined minimal medium. comparing predicted experiment outcomes to growth outcomes obtained by a whole genome gene-deletion study and performance of ind750 estimating the probability of chance occurrence of model predictions a cumulative binomial test was used to evaluate the probability that the predictions made by the logical model could have been made by chance. this test calculates the significance of deviations from the binomial distribution of observations falling into two categories, assuming a random binomial model that assigns genes to essential/retarded growth and non essential/healthy growth categories by chance. the probability is calculated for the range r..n, where r = 889: the number of predictions for growth and n is the total number of predictions made: pbin=∑i=rn!⋅i!⋅pi⋅qq−i) where p =, the proportion of orfs that result in experimental growth and q = - p. this test resulted in a probability, pbin = × 10-32: a very low probability that the predictions were generated by chance. discussion comparison of our logical model with ind demonstrated that there is no significant difference in the performance of any of the models on the prediction of gene essentiality. a significant increase in performance was observed for both the aber model and ind w.r.t the majority classifier on predicting growth outcome on defined growth medium, for all except the most stringent definition of growth from giaever et al. there was also no significant difference between the aber model and ind in the defined growth medium study. the gene essentiality study indicates that both models are equally poor at predicting essential genes, indeed neither is an improvement on the simple majority classifier. as is the case for the minimal medium experimental data, the essential gene data is highly skewed, reflecting the amount of redundancy in the yeast genome. the improvement in performance of both models w.r.t the defined medium study is a result of the precise definition of the medium components. the similarity of results for the aber model and ind indicate that the additional complexity of fba is not required for the prediction of essential genes or the determination of growth/retarded growth on defined media. however the logical model is currently restricted to binary growth/no growth predictions and has no capability to predict growth rates unlike fba based techniques. it is a case of horses for courses. the logical model presented here is a useful addition to the currently existing models with an appropriate mix of expressive representation of metabolic concepts and a simple mechanism for determining predictive outcomes. the high number of inaccuracies in the prediction of growth for all models indicates that much important information remains to be discovered about yeast metabolism. using the logical model as background knowledge for active learning of hypotheses a major motivation for our use of a logical formalism is that it enables the exploitation of techniques from the large research area of inductive logic programming and related first-order learning methods. this is not possible with other formalisms such as fba. in previous work, a precursor of the aber logical model was used as a background theory in a discovery task involving the function of orfs from a single pathway in yeast. the task was implemented as an active learning loop, a machine learning technique based on abductive logic programming that mirrors the hypothetico-deductive process of hypothesis formation in scientific discovery. in active learning a classifier is given a set of training examples incrementally, so that the most informative example can be chosen at each iteration of the loop. in scientific discovery training examples correspond to the outcome of experiments carried out automatically by a lab robot, and the active selection of a training example corresponds to choosing the experiment most likely to refute the largest number of hypotheses, given the outcome of the experiment. figure illustrates the how the active learning loop was applied to the functional genomics discovery task. the formation of hypotheses from the outcomes of the selected experiments was performed by a limited form of abductive theory completion, performed by progol, an ilp program that performs both abductive and inductive machine learning in a restricted form of fol. in this task the logical model formed the incomplete background knowledge theory, to which the final hypothesis is added to "complete" or improve the theory, therefore adding to the current biological knowledge of yeast functional genomics. the completion of the theory is analogous to identifying a new edge to be added to the metabolic network, as well as a new orf annotation for that edge. this is described in figure the use of this restricted form of fol allows all components of the scientific discovery task to be described using the same knowledge representation, indeed the complexity of the entities and relationships required for accurate description of enzymes, reactions, experiments, and hypotheses etc can only be expressed by fol or an equivalently expressive knowledge representation. abduction of hypotheses relating orfs, enzymes and reactions also requires that the machine learning program has fol as it's representation language, hence the use of programs from ilp. the robot scientist concept therefore represents a close integration of logical inference techniques used for scientific discovery and an automated laboratory that can perform experiments. the previous work on the aaa biosynthesis pathway was a proof of principle study. recently the original robot hardware has been replaced by a state-of-the-art automated laboratory that substantially increases the throughput of the robot scientist  this paper presents a logic based model of the metabolic processes in s. cerevisiae: "aber". the core of the aber model is the fba model iff. added to this core are an additional orfs, and also includes an additional reactions taken from kegg. the aber model has been designed to enable automated reasoning about yeast metabolism. use of the model to predict the essentiality of genes in knockout studies provides a mechanism which can be used to validate the model with respect to experimental evidence. roc analysis and other statistical tests were carried out to evaluate the predictive ability of the model and to compare the performance with a state of the art fba model; and to evaluate the effect of increasing the coverage of the model from iff the results show that using a logical model rather that a fba based one results in no significant loss of performance, nor is any loss found by enlarging the coverage of the model to include information from kegg; and that both ind and our logical model represent an improvement over random predictions. 
the use of predicate logic as the knowledge representation for modelling enables an explicit representation of the structure of the metabolic network, and enables logical inference techniques to be used for model identification/improvement. roc analysis and other statistical studies revealed that use of the simpler logical form and larger coverage results in no significant degradation of performance compared to ind a logical model of the known metabolic processes in s. cerevisiae was constructed from iff an existing flux balance analysis model, and augmented with information from the kegg online pathway database. compared to the fba model, the logical model has information on an additional putative genes and additional reactions.
 a logical model of the known metabolic processes in s. cerevisiae was constructed from iff an existing flux balance analysis model, and augmented with information from the kegg online pathway database. the use of predicate logic as the knowledge representation for modelling enables an explicit representation of the structure of the metabolic network, and enables logical inference techniques to be used for model identification/improvement.  compared to the fba model, the logical model has information on an additional putative genes and additional reactions. the correctness of this model was evaluated by comparison with ind by evaluating the performance of both models on predicting empirical minimal medium growth data/essential gene listings.  roc analysis and other statistical studies revealed that use of the simpler logical form and larger coverage results in no significant degradation of performance compared to ind 